Skip to content
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion apps/starry/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,19 @@ cargo xtask starry app run -t mariadb --arch x86_64
cargo xtask starry app run -t mariadb --arch riscv64
```

## MariaDB
Comment thread
ZR233 marked this conversation as resolved.
Outdated

The `mariadb` case is a QEMU app workflow that installs MariaDB in the guest,
initializes a fresh data directory, runs an InnoDB SQL workload, and checks that
the data survives a server restart.

```bash
cargo xtask starry app run -t mariadb --arch aarch64
cargo xtask starry app run -t mariadb --arch loongarch64
cargo xtask starry app run -t mariadb --arch x86_64
Comment thread
ZR233 marked this conversation as resolved.
cargo xtask starry app run -t mariadb --arch riscv64
```

## jcode
Comment thread
ZR233 marked this conversation as resolved.

The `jcode` case is an x86_64 QEMU app workflow that downloads the jcode AI coding
Expand All @@ -142,7 +155,7 @@ packages in a staging root during prebuild, injects runtime artifacts to the
app overlay, then runs nginx smoke tests inside StarryOS.

```bash
cargo xtask starry app run -t nginx --arch x86_64
cargo xtask starry app qemu -t nginx --arch x86_64
```

`apps/starry/nginx` maintains four directories: `smoke`, `phase`, `stress`, and
Expand Down
42 changes: 40 additions & 2 deletions apps/starry/nginx/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ This app organizes nginx tests into four directories:
Only smoke is connected:

```bash
cargo xtask starry app run -t nginx --arch riscv64
cargo xtask starry app qemu -t nginx --arch riscv64
```

`qemu-*.toml` starts `/usr/bin/nginx-smoke-tests.sh` only.
Expand All @@ -23,16 +23,54 @@ Unified local CLI script:

```bash
./apps/starry/nginx/nginx-cli-tests.sh smoke
./apps/starry/nginx/nginx-cli-tests.sh phase12
./apps/starry/nginx/nginx-cli-tests.sh phase1
./apps/starry/nginx/nginx-cli-tests.sh phase2
./apps/starry/nginx/nginx-cli-tests.sh all
```

## Phase QEMU Retest Entries

The app provides dedicated phase retest QEMU configs:

These entries are for local verification and are not wired into CI.

```bash
cargo xtask starry app qemu -t nginx --arch x86_64 --qemu-config apps/starry/nginx/qemu-x86_64-phase1.toml
cargo xtask starry app qemu -t nginx --arch riscv64 --qemu-config apps/starry/nginx/qemu-riscv64-phase1.toml
cargo xtask starry app qemu -t nginx --arch x86_64 --qemu-config apps/starry/nginx/qemu-x86_64-phase2.toml
cargo xtask starry app qemu -t nginx --arch riscv64 --qemu-config apps/starry/nginx/qemu-riscv64-phase2.toml
```

For lifecycle retest on 4 arches:

```bash
cargo xtask starry app qemu -t nginx --arch x86_64 --qemu-config apps/starry/nginx/qemu-x86_64-phase1-2.toml
cargo xtask starry app qemu -t nginx --arch riscv64 --qemu-config apps/starry/nginx/qemu-riscv64-phase1-2.toml
cargo xtask starry app qemu -t nginx --arch aarch64 --qemu-config apps/starry/nginx/qemu-aarch64-phase1-2.toml
cargo xtask starry app qemu -t nginx --arch loongarch64 --qemu-config apps/starry/nginx/qemu-loongarch64-phase1-2.toml
cargo xtask starry app qemu -t nginx --arch x86_64 --qemu-config apps/starry/nginx/qemu-x86_64-phase1-3.toml
cargo xtask starry app qemu -t nginx --arch riscv64 --qemu-config apps/starry/nginx/qemu-riscv64-phase1-3.toml
cargo xtask starry app qemu -t nginx --arch aarch64 --qemu-config apps/starry/nginx/qemu-aarch64-phase1-3.toml
cargo xtask starry app qemu -t nginx --arch loongarch64 --qemu-config apps/starry/nginx/qemu-loongarch64-phase1-3.toml
```

## Build/Prepare Logic

`prebuild.sh` injects only smoke entry and shared mirror helper into guest overlay:
`prebuild.sh` injects smoke/phase entries and shared mirror helper into guest overlay:

- `/usr/bin/nginx-smoke-tests.sh`
- `/usr/bin/nginx-phase12-tests.sh`
- `/usr/bin/nginx-phase1-tests.sh`
- `/usr/bin/nginx-phase2-tests.sh`
- `/usr/bin/nginx-alpine-mirror.sh`

Mirror helper: `apps/starry/nginx/nginx-alpine-mirror.sh`.

The mirror helper pins `apk` to the Alpine release branch of the running rootfs
(read from `/etc/alpine-release`, e.g. `v3.23`) instead of the moving
`latest-stable` alias. This keeps installed packages on the same musl/ABI as the
rootfs base; `latest-stable` can advance to a newer Alpine release whose binaries
need a newer musl (for example the `renameat2` symbol) and then fail to relocate
or crash on the current rootfs. Override the branch with `NGINX_APK_BRANCH` when
needed.
10 changes: 10 additions & 0 deletions apps/starry/nginx/build-aarch64-unknown-none-softfloat.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
target = "aarch64-unknown-none-softfloat"
log = "Warn"
features = [
"qemu",
"ax-driver/virtio-blk",
"ax-driver/virtio-net",
"ax-driver/virtio-gpu",
"ax-driver/virtio-input",
"ax-driver/virtio-socket",
]
14 changes: 14 additions & 0 deletions apps/starry/nginx/build-loongarch64-unknown-none-softfloat.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
target = "loongarch64-unknown-none-softfloat"
axconfig_overrides = ["plat.phys-memory-size=0x8000_0000"]
log = "Warn"
features = [
"ax-hal/loongarch64-qemu-virt",
"qemu",
"ax-driver/plat-static",
"ax-driver/virtio-blk",
"ax-driver/virtio-net",
"ax-driver/virtio-gpu",
"ax-driver/virtio-input",
"ax-driver/virtio-socket",
]
plat_dyn = false
49 changes: 49 additions & 0 deletions apps/starry/nginx/debug/ISSUE-004-bad-method-probe-instability.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# ISSUE-004: BAD Method Probe Instability

## Status

- State: deferred (temporarily bypassed in phase2)
- Scope: `BAD / HTTP/1.1` verification path in stage 2
- Impact: does not block phase2 main flow

## Symptom

- In some runs, `busybox nc` returns an empty response (`<empty>`) for both:
- `BAD / HTTP/1.1`
- `GET / HTTP/1.1`
- This can make a strict raw-probe assertion fail even when nginx behavior is normal.

## Confirmed Observations

- `curl -X BAD http://127.0.0.1:8080/...` stably returns `405`.
- `nc.openbsd` raw BAD request stably returns `HTTP/1.1 405 Not Allowed`.
- `busybox nc` may return `<empty>` under the same request pattern.

## Current Decision

- Keep phase2 BAD-method node as bypass (known issue log only).
- Do not use `busybox nc` as pass/fail oracle for this node.
- Continue tracking through debug scripts:
- `nginx-2-0-bad-method-debug.sh`
- `nginx-2-0-bad-method-matrix.sh`

## Repro Commands

```bash
cargo xtask starry app qemu -t nginx --arch x86_64 --qemu-config apps/starry/nginx/qemu-x86_64-bad-method-debug.toml
cargo xtask starry app qemu -t nginx --arch x86_64 --qemu-config apps/starry/nginx/qemu-x86_64-bad-method-matrix.toml
```

Optional phase2 retest:

```bash
cargo xtask starry app qemu -t nginx --arch riscv64 --qemu-config apps/starry/nginx/qemu-riscv64-phase2.toml
cargo xtask starry app qemu -t nginx --arch x86_64 --qemu-config apps/starry/nginx/qemu-x86_64-phase2.toml
```

## Next Follow-up

- Add a minimal socket-level probe outside nginx to isolate whether this is:
- `busybox nc` behavior under StarryOS, or
- StarryOS socket timing/EOF behavior exposed by `busybox nc`.
- After root cause is confirmed and fixed, restore strict assertion for the phase2 BAD-method node.
1 change: 1 addition & 0 deletions apps/starry/nginx/debug/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ This directory stores flexible debug scripts for single issue reproduction and d
Current scripts:

- `nginx-http-basic-tests.sh`: early HTTP basic script kept for issue-level debugging.
- `nginx-2-0-bad-method-debug.sh`: focused probe for stage 2.0 BAD method (`BAD / HTTP/1.1`) instability.

Rule:

Expand Down
110 changes: 110 additions & 0 deletions apps/starry/nginx/debug/nginx-2-0-bad-method-debug.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
#!/bin/sh
set -eu

BASE=/tmp/nginx-phase2-bad-method
CONF="$BASE/conf/http-basic.conf"
WWW="$BASE/www"
OUT="$BASE/out"
LOGDIR="$BASE/logs"
TIMEOUT_CMD=

log() { printf 'NGINX_BAD_METHOD_DEBUG_LOG: %s\n' "$*"; }
fail() { printf 'NGINX_BAD_METHOD_DEBUG_FAILED\n'; log "$*"; exit 1; }

init_timeout_cmd() {
if command -v timeout >/dev/null 2>&1; then TIMEOUT_CMD='timeout'; return; fi
if busybox timeout 2>&1 | grep -qi 'usage'; then TIMEOUT_CMD='busybox timeout'; return; fi
fail "timeout command not available"
}

run_with_timeout() { sec=$1; shift; $TIMEOUT_CMD "$sec" "$@"; }

cleanup_nginx() {
killall -q nginx 2>/dev/null || true
sleep 1
killall -q -9 nginx 2>/dev/null || true
}

prepare_packages() {
repo_file=/etc/apk/repositories
original_repos="$(cat "$repo_file")"
for mirror in https://mirrors.cernet.edu.cn/alpine https://dl-cdn.alpinelinux.org/alpine; do
printf '%s\n' "$original_repos" | sed "s#http://[^/]*/alpine/#$mirror/#g;s#https://[^/]*/alpine/#$mirror/#g" > "$repo_file"
rm -f /lib/apk/db/lock
if run_with_timeout 40 apk --timeout 40 update && run_with_timeout 40 apk --timeout 40 add nginx curl busybox-extras; then return 0; fi
done
return 1
}

prepare_tree() {
rm -rf "$BASE"
mkdir -p "$BASE/conf" "$WWW" "$LOGDIR" "$OUT"
printf 'BAD_METHOD_DEBUG_OK\n' > "$WWW/index.html"
cat > "$CONF" <<'EOF'
daemon off;
master_process off;
worker_processes 1;
error_log /tmp/nginx-phase2-bad-method/logs/error.log debug;
pid /tmp/nginx-phase2-bad-method/nginx.pid;
events { worker_connections 64; }
http { include /etc/nginx/mime.types; access_log /tmp/nginx-phase2-bad-method/logs/access.log; server { listen 127.0.0.1:8080; root /tmp/nginx-phase2-bad-method/www; location / { index index.html; } } }
EOF
}

start_nginx() {
nginx -t -c "$CONF" -p "$BASE/" || return 1
nginx -c "$CONF" -p "$BASE/" > "$LOGDIR/nginx-stdout.log" 2>&1 &
i=0
while [ "$i" -lt 6 ]; do
run_with_timeout 1 curl -fsS -o /dev/null http://127.0.0.1:8080/ >/dev/null 2>&1 && return 0
i=$((i + 1))
sleep 1
done
return 1
}

probe_bad_method() {
if command -v nc >/dev/null 2>&1; then
NC='nc'
elif busybox nc 2>&1 | grep -qi 'usage'; then
NC='busybox nc'
else
fail "nc not available"
fi
code=$(run_with_timeout 3 curl -sS -o /dev/null -w '%{http_code}' -X BAD http://127.0.0.1:8080/ || true)
[ -n "$code" ] || code='<empty>'
log "curl_bad_method_status=$code"

i=1
while [ "$i" -le 5 ]; do
out="$OUT/bad-$i.raw"
{ printf 'BAD / HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\n\r\n'; } | run_with_timeout 4 sh -c "$NC 127.0.0.1 8080" > "$out" || true
tr -d '\r' < "$out" > "$OUT/bad-$i.norm"
status=$(sed -n '1p' "$OUT/bad-$i.norm" || true)
[ -n "$status" ] || status='<empty>'
log "probe=$i status=$status"
i=$((i + 1))
done
if grep -Eq '^HTTP/1.1 (400|405)' "$OUT"/bad-*.norm; then
printf 'NGINX_BAD_METHOD_DEBUG_PASSED\n'
return 0
fi
log "no 400/405 hit in 5 probes"
return 1
}

init_timeout_cmd
( sleep 90; log "watchdog timeout"; kill -TERM $$ ) &
prepare_packages || fail "prepare packages"
prepare_tree || fail "prepare tree"
start_nginx || fail "start nginx"
probe_bad_method || {
if [ -f "$LOGDIR/error.log" ]; then
tail_line=$(sed -n '$p' "$LOGDIR/error.log" || true)
[ -n "$tail_line" ] || tail_line='<empty>'
log "error_log_tail=$tail_line"
fi
cleanup_nginx
fail "bad method probes"
}
cleanup_nginx
Loading