-
Notifications
You must be signed in to change notification settings - Fork 126
test(starry): cover UV 0.11 and subcommand behavior in functional tests. #1211
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
0136822
fix(axbuild): use locally-prepared app rootfs instead of re-pulling a…
eccee18
test(apps/starry): add online pip + uv install stages to pip-uv
Lfan-ke f419a99
test(starry): pip-uv — size guest RAM to 4G, stage loong libbz2, hard…
Lfan-ke e7ff776
Merge upstream/dev — resolve axbuild refactor for pip-uv app branch
Lfan-ke 34c7e95
ci: re-trigger fresh run (prior red = self-hosted/cancelled infra, un…
Lfan-ke 7901ce6
Merge remote-tracking branch 'origin/dev' into pipuv-app-rebased
Lfan-ke 04ff09b
test(pip-uv): switch loongarch64 to dynamic platform
Lfan-ke File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| # pip + uv 离线测试 (4-arch) | ||
|
|
||
| 在 StarryOS(CPython 3.14 musl rootfs)上验证 Python 包管理器 **pip 26.1.2** 与 **uv 0.11.19** 的常用操作。**全部离线**(本地 wheel / 本地 uv 二进制),覆盖 x86_64 / aarch64 / riscv64 / loongarch64 四个架构。 | ||
|
|
||
| ## 测试内容 | ||
|
|
||
| `test_pipuv.sh` 分多个阶段,覆盖 pip 与 uv 的日常离线使用场景: | ||
|
|
||
| **pip(从 ensurepip 内置 wheel 自举 26.1.2)** | ||
| - `python3 --version`(3.14)/ `pip3 --version`(26.1.2) | ||
| - pip list / show / freeze / check | ||
| - `pip3 install --no-index --find-links /opt/wheels setuptools wheel`(本地 build backend) | ||
| - `pip3 wheel` / `pip3 download`(离线,针对本地 fixture 包) | ||
| - `python3 -m venv --without-pip` + 在 venv 内以同进程方式直装 pip / setuptools | ||
|
|
||
| **uv(本地二进制 0.11.19)** | ||
| - `uv --version`(0.11.19)/ `uv --help` | ||
| - `uv venv`(缓存放磁盘 `/root/.uvcache`,非 tmpfs) | ||
| - `uv pip install --no-index --find-links /opt/wheels ...` / `uv pip list` | ||
| - `uv run --no-project python3 -c ...` | ||
| - `uv run --no-project --script`(PEP 723 `# /// script` 内联元数据) | ||
|
|
||
| ## 运行测试 | ||
|
|
||
| ```bash | ||
| # 单核 qemu, 逐架构 (x86_64 / aarch64 / riscv64 / loongarch64): | ||
| cargo xtask starry app qemu -t pip-uv --arch x86_64 | ||
| ``` | ||
|
|
||
| (`cargo xtask starry app list` 可确认 `pip-uv` 被发现;本仓库无 `app run` 子命令。) | ||
|
|
||
| ## 关于 test_pipuv.sh | ||
|
|
||
| `test_pipuv.sh` 是 **guest 内**脚本:由上面的 app 框架复制到 rootfs 的 `/usr/bin/`,在 QEMU 启动后由 guest 执行。它按 guest 环境写死路径(`/opt/wheels`、`/root/.uvcache`、`/root/v`)、设置 `PIP_BREAK_SYSTEM_PACKAGES=1` 并执行 `pip3 install` / `pip3 uninstall`,因此**不要在 Linux 主机上直接 `sh` 它**——在主机上要么失败、要么污染主机的 Python 环境。只想离机审阅逻辑请直接阅读该源文件。 | ||
|
|
||
| ## 资产准备 | ||
|
|
||
| 测试运行**全部离线**,但镜像构建(`prebuild.sh`)需要以下本地资产预先就位(下载日 2026-06-09 的各源最新版)。干净环境中若任一资产缺失,构建会在进入 guest 前以非零状态失败,并打印缺失的具体路径(见下表来源 URL 自行获取后放到默认路径,或用环境变量改指它处)。 | ||
|
|
||
| | 资产 | 版本 | 默认路径 | 来源 URL | | ||
| |---|---|---|---| | ||
| | pip wheel(架构无关) | 26.1.2 | `$HOME/rcore/download/pip-uv/pip-26.1.2-py3-none-any.whl` | PyPI:`https://pypi.org/pypi/pip/26.1.2/json` → `files.pythonhosted.org` 上的 `.whl` | | ||
| | uv(x86_64,静态 musl) | 0.11.19 | `$HOME/rcore/download/pip-uv/uv-x86_64-unknown-linux-musl.tar.gz` | `https://github.com/astral-sh/uv/releases/download/0.11.19/uv-x86_64-unknown-linux-musl.tar.gz` | | ||
| | uv(aarch64,静态 musl) | 0.11.19 | `$HOME/rcore/download/pip-uv/uv-aarch64-unknown-linux-musl.tar.gz` | `https://github.com/astral-sh/uv/releases/download/0.11.19/uv-aarch64-unknown-linux-musl.tar.gz` | | ||
| | uv(riscv64,静态 musl) | 0.11.19 | `$HOME/rcore/download/pip-uv/uv-riscv64gc-unknown-linux-musl.tar.gz` | `https://github.com/astral-sh/uv/releases/download/0.11.19/uv-riscv64gc-unknown-linux-musl.tar.gz` | | ||
| | uv(loongarch64,动态 musl) | 0.11.19-r0 | `$HOME/rcore/download/pip-uv/uv-loongarch64-uv-0.11.19-r0.apk` | `https://dl-cdn.alpinelinux.org/alpine/edge/community/loongarch64/uv-0.11.19-r0.apk`(astral-sh 官方不发布 loong 二进制,故取 Alpine community 同版本原生构建) | | ||
| | build-backend wheels | setuptools 82.0.1 / wheel 0.47.0 / packaging / six | `$HOME/rcore/pipuv-work/offline-wheels/*.whl` | PyPI(`pip download setuptools wheel packaging six -d <dir>` 离线下载) | | ||
|
|
||
| 可选 fast-path:若 `$HOME/rcore/pipuv-work/uvbins/uv-<arch>` 已有预解压的 per-arch uv 二进制,`prebuild.sh` 直接复制它,跳过解包 tarball/apk。 | ||
|
|
||
| 资产路径可用环境变量覆盖(不在默认位置时): | ||
|
|
||
| | 环境变量 | 默认值 | 含义 | | ||
| |---|---|---| | ||
| | `PIPUV_DOWNLOAD_DIR` | `$HOME/rcore/download/pip-uv` | pip wheel + per-arch uv tarball/apk | | ||
| | `PIPUV_WHEELS_DIR` | `$HOME/rcore/pipuv-work/offline-wheels` | 离线 build-backend wheels | | ||
| | `PIPUV_UVBINS_DIR` | `$HOME/rcore/pipuv-work/uvbins` | 预解压的 per-arch uv 二进制(fast-path,可选) | | ||
|
|
||
| 校验(uv 静态二进制可在 host 直跑;loong 动态需在 rootfs 内验): | ||
|
|
||
| ```bash | ||
| ./uvbins/uv-x86_64 --version # uv 0.11.19 (x86_64-unknown-linux-musl) | ||
| strings <uv-bin> | grep -oE '0\.11\.19' # 4-arch 二进制均命中 0.11.19 | ||
| python3 -c "import zipfile; print('pip-26.1.2' in zipfile.ZipFile('pip-26.1.2-py3-none-any.whl').namelist()[0])" | ||
| ``` | ||
|
|
||
| ## 工作原理 | ||
|
|
||
| 1. `prebuild.sh` 被 xtask 框架调用,通过 `qemu-user-static` 在 staging rootfs 中执行原生 `apk` 安装 `python3`(使用 `--no-scripts` 跳过 busybox trigger),并用 `readelf` 解析运行时依赖复制到 overlay。 | ||
| 2. 将本地 **pip wheel**(`download/pip-uv/pip-26.1.2-py3-none-any.whl`)写入 rootfs 的 ensurepip 内置 wheel 目录与 `/opt/wheels`,使测试可纯离线自举 pip 26.1.2。 | ||
| 3. 将本地 **build-backend wheels**(`pipuv-work/offline-wheels/*.whl`:setuptools / wheel / packaging / six)复制到 rootfs 的 `/opt/wheels`。 | ||
| 4. 将 **按架构对应的 uv 0.11.19 二进制**复制到 rootfs 的 `/usr/local/bin/uv`(x86_64 / aarch64 / riscv64 取自 astral-sh 官方 musl tar.gz;loongarch64 取自 Alpine edge community apk,因 astral-sh 官方不发布 loong 二进制)。 | ||
| 5. `test_pipuv.sh` 被复制到 overlay 的 `/usr/bin/`。 | ||
| 6. overlay 被注入到 rootfs 镜像中,QEMU 启动后执行 `/usr/bin/test_pipuv.sh`。 | ||
|
|
||
| ## 判定标准 | ||
|
|
||
| - **通过**: 输出匹配 `STARRY_PIPUV_TESTS_PASSED` | ||
| - **失败**: 输出匹配 `STARRY_PIPUV_STAGE_.*_FAILED`,或出现 panic / page fault / segmentation fault | ||
| - **超时**: x86_64 = 1800 秒;aarch64 / riscv64 = 3600 秒;loongarch64 = 5400 秒(TCG 下 uv 较慢) | ||
|
|
||
| ## 测试范围:离线(stages 1–17)+ 在线(stages 18–20) | ||
|
|
||
| **stages 1–17(离线功能覆盖)**:pip / uv 的全命令族(install / list / show / freeze / check / wheel / download / uninstall / venv / `python3 -m pip` / `uv pip` / `uv run` / PEP 723)均以 `--no-index --find-links /opt/wheels`(pip)与本地二进制 + 本地 wheel(uv)的离线形式验证。 | ||
|
|
||
| **stages 18–20(在线安装,真实 TCP,走本地 wheel 索引,无外网)**:额外覆盖在线安装的**完整真实网络路径**。测试框架按各 `qemu-*.toml` 的 `[host_http_server]` 配置,在宿主 `127.0.0.1:18390` 起一个静态 wheel 索引(目录 `online-index/`,仅含数个小的纯 Python wheel);guest 经 QEMU user-mode 网络(SLIRP)以 `10.0.2.2:18390` 直连宿主,真实经过 TCP 握手 + HTTP 下载 + 依赖解析 + 安装 + import: | ||
|
|
||
| - stage 18 `pip install`(markdown-it-py,真实拉取其依赖 mdurl) | ||
| - stage 19 `python3 -m pip install`(six,无依赖) | ||
| - stage 20 `uv pip install`(markdown-it-py,装入 uv venv) | ||
|
|
||
| 该在线测试是**自包含(hermetic)**的:不依赖外网 / DNS / PyPI 可达性,在 CI 中确定性可复现。注意 pip 对纯 HTTP 索引需 `--trusted-host`;安装目标置于 ext4 磁盘(`/root`)而非 RAM 盘 `/tmp`(tmpfs 读回会损坏已安装文件)。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| target = "aarch64-unknown-none-softfloat" | ||
| env = { AX_IP = "10.0.2.15", AX_GW = "10.0.2.2" } | ||
| log = "Warn" | ||
| features = [ | ||
| "qemu", | ||
| "ax-driver/virtio-blk", | ||
| "ax-driver/virtio-net", | ||
| ] |
8 changes: 8 additions & 0 deletions
8
apps/starry/pip-uv/build-loongarch64-unknown-none-softfloat.toml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| target = "loongarch64-unknown-none-softfloat" | ||
| env = { AX_IP = "10.0.2.15", AX_GW = "10.0.2.2" } | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 切到动态平台后这里也不应继续写死 |
||
| log = "Warn" | ||
| features = [ | ||
| "ax-driver/serial", | ||
| "ax-driver/virtio-blk", | ||
| "ax-driver/virtio-net", | ||
| ] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| target = "riscv64gc-unknown-none-elf" | ||
| env = { AX_IP = "10.0.2.15", AX_GW = "10.0.2.2" } | ||
| log = "Warn" | ||
| features = [ | ||
| "ax-driver/serial", | ||
| "ax-driver/virtio-blk", | ||
| "ax-driver/virtio-net", | ||
| ] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| target = "x86_64-unknown-none" | ||
| env = { AX_IP = "10.0.2.15", AX_GW = "10.0.2.2" } | ||
| log = "Warn" | ||
| features = [ | ||
| "ax-driver/virtio-blk", | ||
| "ax-driver/virtio-net", | ||
| ] |
Binary file added
BIN
+89.5 KB
apps/starry/pip-uv/online-index/markdown_it_py-4.2.0-py3-none-any.whl
Binary file not shown.
Binary file not shown.
Binary file not shown.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
可去除写死的值,通过dhcp 自动获取
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
同上,这部分建议在 rebase 到最新
rcore-os/dev后一起处理。动态平台路径可用后这里不需要再写固定AX_IP/AX_GW,删除env = { AX_IP = ..., AX_GW = ... }让 guest DHCP 获取地址即可。There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
感谢建议。这里说明一下:
AX_IP = 10.0.2.15/AX_GW = 10.0.2.2并非任意写死,而正是 qemu SLIRP 给 guest 分配的那组地址 —— SLIRP 子网固定为10.0.2.0/24,guest 唯一可分配地址10.0.2.15、网关恒为10.0.2.2。本批走 DHCP 的 app(如 node-lang)日志即为eth0: DHCP acquired address 10.0.2.15/24+DHCP router 10.0.2.2,与此处显式值完全一致。显式静态设置的目的是让在线 pip/uv 安装阶段(stage 18–20 经
10.0.2.2:18390真 HTTP)的网络在测试起跑前确定就绪、可复现,不受 DHCP lease 时序影响。我也实测了去掉这两行改走 DHCP:网络本身 OK(同样DHCP acquired 10.0.2.15),但 offline 的uv run阶段出现卡顿(疑似 loong 上uv run偶发 pidfd 子进程路径,与网络无关,仍在确认),为稳妥暂保留显式配置。地址既然与 DHCP 完全一致,若你倾向去掉走 DHCP,我可以切换 —— 请告知是否需要改为 DHCP。