Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 92 additions & 0 deletions apps/starry/pip-uv/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# pip + uv 离线测试 (4-arch)

在 StarryOS(CPython 3.14 musl rootfs)上验证 Python 包管理器 **pip 26.1.2** 与 **uv 0.11.19** 的常用操作。**全部离线**(本地 wheel / 本地 uv 二进制),覆盖 x86_64 / aarch64 / riscv64 / loongarch64 四个架构。

## 测试内容

`test_pipuv.sh` 分多个阶段,覆盖 pip 与 uv 的日常离线使用场景:

**pip(从 ensurepip 内置 wheel 自举 26.1.2)**
- `python3 --version`(3.14)/ `pip3 --version`(26.1.2)
- pip list / show / freeze / check
- `pip3 install --no-index --find-links /opt/wheels setuptools wheel`(本地 build backend)
- `pip3 wheel` / `pip3 download`(离线,针对本地 fixture 包)
- `python3 -m venv --without-pip` + 在 venv 内以同进程方式直装 pip / setuptools

**uv(本地二进制 0.11.19)**
- `uv --version`(0.11.19)/ `uv --help`
- `uv venv`(缓存放磁盘 `/root/.uvcache`,非 tmpfs)
- `uv pip install --no-index --find-links /opt/wheels ...` / `uv pip list`
- `uv run --no-project python3 -c ...`
- `uv run --no-project --script`(PEP 723 `# /// script` 内联元数据)

## 运行测试

```bash
# 单核 qemu, 逐架构 (x86_64 / aarch64 / riscv64 / loongarch64):
cargo xtask starry app qemu -t pip-uv --arch x86_64
```

(`cargo xtask starry app list` 可确认 `pip-uv` 被发现;本仓库无 `app run` 子命令。)

## 关于 test_pipuv.sh

`test_pipuv.sh` 是 **guest 内**脚本:由上面的 app 框架复制到 rootfs 的 `/usr/bin/`,在 QEMU 启动后由 guest 执行。它按 guest 环境写死路径(`/opt/wheels`、`/root/.uvcache`、`/root/v`)、设置 `PIP_BREAK_SYSTEM_PACKAGES=1` 并执行 `pip3 install` / `pip3 uninstall`,因此**不要在 Linux 主机上直接 `sh` 它**——在主机上要么失败、要么污染主机的 Python 环境。只想离机审阅逻辑请直接阅读该源文件。

## 资产准备

测试运行**全部离线**,但镜像构建(`prebuild.sh`)需要以下本地资产预先就位(下载日 2026-06-09 的各源最新版)。干净环境中若任一资产缺失,构建会在进入 guest 前以非零状态失败,并打印缺失的具体路径(见下表来源 URL 自行获取后放到默认路径,或用环境变量改指它处)。

| 资产 | 版本 | 默认路径 | 来源 URL |
|---|---|---|---|
| pip wheel(架构无关) | 26.1.2 | `$HOME/rcore/download/pip-uv/pip-26.1.2-py3-none-any.whl` | PyPI:`https://pypi.org/pypi/pip/26.1.2/json` → `files.pythonhosted.org` 上的 `.whl` |
| uv(x86_64,静态 musl) | 0.11.19 | `$HOME/rcore/download/pip-uv/uv-x86_64-unknown-linux-musl.tar.gz` | `https://github.com/astral-sh/uv/releases/download/0.11.19/uv-x86_64-unknown-linux-musl.tar.gz` |
| uv(aarch64,静态 musl) | 0.11.19 | `$HOME/rcore/download/pip-uv/uv-aarch64-unknown-linux-musl.tar.gz` | `https://github.com/astral-sh/uv/releases/download/0.11.19/uv-aarch64-unknown-linux-musl.tar.gz` |
| uv(riscv64,静态 musl) | 0.11.19 | `$HOME/rcore/download/pip-uv/uv-riscv64gc-unknown-linux-musl.tar.gz` | `https://github.com/astral-sh/uv/releases/download/0.11.19/uv-riscv64gc-unknown-linux-musl.tar.gz` |
| uv(loongarch64,动态 musl) | 0.11.19-r0 | `$HOME/rcore/download/pip-uv/uv-loongarch64-uv-0.11.19-r0.apk` | `https://dl-cdn.alpinelinux.org/alpine/edge/community/loongarch64/uv-0.11.19-r0.apk`(astral-sh 官方不发布 loong 二进制,故取 Alpine community 同版本原生构建) |
| build-backend wheels | setuptools 82.0.1 / wheel 0.47.0 / packaging / six | `$HOME/rcore/pipuv-work/offline-wheels/*.whl` | PyPI(`pip download setuptools wheel packaging six -d <dir>` 离线下载) |

可选 fast-path:若 `$HOME/rcore/pipuv-work/uvbins/uv-<arch>` 已有预解压的 per-arch uv 二进制,`prebuild.sh` 直接复制它,跳过解包 tarball/apk。

资产路径可用环境变量覆盖(不在默认位置时):

| 环境变量 | 默认值 | 含义 |
|---|---|---|
| `PIPUV_DOWNLOAD_DIR` | `$HOME/rcore/download/pip-uv` | pip wheel + per-arch uv tarball/apk |
| `PIPUV_WHEELS_DIR` | `$HOME/rcore/pipuv-work/offline-wheels` | 离线 build-backend wheels |
| `PIPUV_UVBINS_DIR` | `$HOME/rcore/pipuv-work/uvbins` | 预解压的 per-arch uv 二进制(fast-path,可选) |

校验(uv 静态二进制可在 host 直跑;loong 动态需在 rootfs 内验):

```bash
./uvbins/uv-x86_64 --version # uv 0.11.19 (x86_64-unknown-linux-musl)
strings <uv-bin> | grep -oE '0\.11\.19' # 4-arch 二进制均命中 0.11.19
python3 -c "import zipfile; print('pip-26.1.2' in zipfile.ZipFile('pip-26.1.2-py3-none-any.whl').namelist()[0])"
```

## 工作原理

1. `prebuild.sh` 被 xtask 框架调用,通过 `qemu-user-static` 在 staging rootfs 中执行原生 `apk` 安装 `python3`(使用 `--no-scripts` 跳过 busybox trigger),并用 `readelf` 解析运行时依赖复制到 overlay。
2. 将本地 **pip wheel**(`download/pip-uv/pip-26.1.2-py3-none-any.whl`)写入 rootfs 的 ensurepip 内置 wheel 目录与 `/opt/wheels`,使测试可纯离线自举 pip 26.1.2。
3. 将本地 **build-backend wheels**(`pipuv-work/offline-wheels/*.whl`:setuptools / wheel / packaging / six)复制到 rootfs 的 `/opt/wheels`。
4. 将 **按架构对应的 uv 0.11.19 二进制**复制到 rootfs 的 `/usr/local/bin/uv`(x86_64 / aarch64 / riscv64 取自 astral-sh 官方 musl tar.gz;loongarch64 取自 Alpine edge community apk,因 astral-sh 官方不发布 loong 二进制)。
5. `test_pipuv.sh` 被复制到 overlay 的 `/usr/bin/`。
6. overlay 被注入到 rootfs 镜像中,QEMU 启动后执行 `/usr/bin/test_pipuv.sh`。

## 判定标准

- **通过**: 输出匹配 `STARRY_PIPUV_TESTS_PASSED`
- **失败**: 输出匹配 `STARRY_PIPUV_STAGE_.*_FAILED`,或出现 panic / page fault / segmentation fault
- **超时**: x86_64 = 1800 秒;aarch64 / riscv64 = 3600 秒;loongarch64 = 5400 秒(TCG 下 uv 较慢)

## 测试范围:离线(stages 1–17)+ 在线(stages 18–20)

**stages 1–17(离线功能覆盖)**:pip / uv 的全命令族(install / list / show / freeze / check / wheel / download / uninstall / venv / `python3 -m pip` / `uv pip` / `uv run` / PEP 723)均以 `--no-index --find-links /opt/wheels`(pip)与本地二进制 + 本地 wheel(uv)的离线形式验证。

**stages 18–20(在线安装,真实 TCP,走本地 wheel 索引,无外网)**:额外覆盖在线安装的**完整真实网络路径**。测试框架按各 `qemu-*.toml` 的 `[host_http_server]` 配置,在宿主 `127.0.0.1:18390` 起一个静态 wheel 索引(目录 `online-index/`,仅含数个小的纯 Python wheel);guest 经 QEMU user-mode 网络(SLIRP)以 `10.0.2.2:18390` 直连宿主,真实经过 TCP 握手 + HTTP 下载 + 依赖解析 + 安装 + import:

- stage 18 `pip install`(markdown-it-py,真实拉取其依赖 mdurl)
- stage 19 `python3 -m pip install`(six,无依赖)
- stage 20 `uv pip install`(markdown-it-py,装入 uv venv)

该在线测试是**自包含(hermetic)**的:不依赖外网 / DNS / PyPI 可达性,在 CI 中确定性可复现。注意 pip 对纯 HTTP 索引需 `--trusted-host`;安装目标置于 ext4 磁盘(`/root`)而非 RAM 盘 `/tmp`(tmpfs 读回会损坏已安装文件)。
8 changes: 8 additions & 0 deletions apps/starry/pip-uv/build-aarch64-unknown-none-softfloat.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
target = "aarch64-unknown-none-softfloat"
env = { AX_IP = "10.0.2.15", AX_GW = "10.0.2.2" }
log = "Warn"
features = [
"qemu",
"ax-driver/virtio-blk",
"ax-driver/virtio-net",
]
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
target = "loongarch64-unknown-none-softfloat"
env = { AX_IP = "10.0.2.15", AX_GW = "10.0.2.2" }

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

可去除写死的值,通过dhcp 自动获取

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

同上,这部分建议在 rebase 到最新 rcore-os/dev 后一起处理。动态平台路径可用后这里不需要再写固定 AX_IP / AX_GW,删除 env = { AX_IP = ..., AX_GW = ... } 让 guest DHCP 获取地址即可。

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

感谢建议。这里说明一下:AX_IP = 10.0.2.15 / AX_GW = 10.0.2.2 并非任意写死,而正是 qemu SLIRP 给 guest 分配的那组地址 —— SLIRP 子网固定为 10.0.2.0/24,guest 唯一可分配地址 10.0.2.15、网关恒为 10.0.2.2。本批走 DHCP 的 app(如 node-lang)日志即为 eth0: DHCP acquired address 10.0.2.15/24 + DHCP router 10.0.2.2,与此处显式值完全一致。

显式静态设置的目的是让在线 pip/uv 安装阶段(stage 18–20 经 10.0.2.2:18390 真 HTTP)的网络在测试起跑前确定就绪、可复现,不受 DHCP lease 时序影响。我也实测了去掉这两行改走 DHCP:网络本身 OK(同样 DHCP acquired 10.0.2.15),但 offline 的 uv run 阶段出现卡顿(疑似 loong 上 uv run 偶发 pidfd 子进程路径,与网络无关,仍在确认),为稳妥暂保留显式配置。

地址既然与 DHCP 完全一致,若你倾向去掉走 DHCP,我可以切换 —— 请告知是否需要改为 DHCP。

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

切到动态平台后这里也不应继续写死 AX_IP / AX_GW;请删除固定地址配置,让 guest 通过 DHCP 获取网络参数,并用 current-head 的 loongarch64 app-QEMU 日志验证。

log = "Warn"
features = [
"ax-driver/serial",
"ax-driver/virtio-blk",
"ax-driver/virtio-net",
]
8 changes: 8 additions & 0 deletions apps/starry/pip-uv/build-riscv64gc-unknown-none-elf.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
target = "riscv64gc-unknown-none-elf"
env = { AX_IP = "10.0.2.15", AX_GW = "10.0.2.2" }
log = "Warn"
features = [
"ax-driver/serial",
"ax-driver/virtio-blk",
"ax-driver/virtio-net",
]
7 changes: 7 additions & 0 deletions apps/starry/pip-uv/build-x86_64-unknown-none.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
target = "x86_64-unknown-none"
env = { AX_IP = "10.0.2.15", AX_GW = "10.0.2.2" }
log = "Warn"
features = [
"ax-driver/virtio-blk",
"ax-driver/virtio-net",
]
Binary file not shown.
Binary file not shown.
Binary file not shown.
Loading