Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
131 changes: 131 additions & 0 deletions .claude/skills/board-linux-starry-debug/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
---
name: board-linux-starry-debug
description: Debug physical-board workflows that need Linux-side deployment or inspection before running StarryOS or ArceOS through the local board service. Use this skill when a board workload depends on files in the board Linux rootfs, when the user says to use `board connect`, when SSH/rsync deployment must happen while holding a board lease, when StarryOS reports a deployed binary as `not found`, or when diagnosing mismatches between Linux-visible files and StarryOS-visible files on OrangePi-5-Plus or similar boards.
---

# Board Linux Starry Debug

## Purpose

Use this workflow when one physical board must be kept leased while you:

1. Boot its normal Linux image.
2. Discover the board IP from serial or Linux commands.
3. SSH/rsync files into the Linux rootfs.
4. Flush the filesystem.
5. Release the board lease.
6. Reacquire the board for a StarryOS or ArceOS board run.

This avoids the common mistake of testing a newly copied binary on Linux, then rebooting into StarryOS before the ext4 state is safely visible to the next boot.

## Local Board Service

Prefer the repository's local board service unless the user explicitly gives a shared service endpoint.

```bash
cargo xtask board ls
cargo xtask board connect -b OrangePi-5-Plus
```

If the requested board type is missing, list first and use the exact local type. For example, a shared service may use `OrangePi-5-Plus-robot`, while the local service may expose `OrangePi-5-Plus`.

`board connect` holds the lease until the outer process exits. Logging out of the Linux shell inside serial does not necessarily release the board.

## Linux Deployment Flow

1. Keep `board connect` open until Linux reaches a login or shell prompt.
2. Get the IP from the boot banner or run:

```bash
ip -brief addr
```

3. In a separate host command, verify SSH:

```bash
ssh -o BatchMode=yes -o StrictHostKeyChecking=no -o ConnectTimeout=8 orangepi@<ip> \
'hostname; id; ip -brief addr'
```

4. Copy payloads to `/tmp`, then use sudo to replace the final path and flush:

```bash
rsync -az --delete <local-dir>/ orangepi@<ip>:/tmp/<name>/
ssh orangepi@<ip> '
set -e
printf "%s\n" orangepi | sudo -S rm -rf /target/path
printf "%s\n" orangepi | sudo -S mv /tmp/<name> /target/path
printf "%s\n" orangepi | sudo -S chown -R root:root /target/path
printf "%s\n" orangepi | sudo -S sync
ls -l /target/path
sync
'
```

Use a Linux-side smoke test after deployment when the workload can run on Linux. Capture its exact success markers or final result line.

## Releasing The Lease

Before starting the StarryOS or ArceOS board run, release the `board connect` process:

```bash
ps -ef | rg 'target/debug/tg-xtask board connect|cargo xtask board connect'
kill <pid>
sleep 2
cargo xtask board ls
```

Proceed only when the board is available again.

## StarryOS Board Run

Run the board workload through `cargo xtask`, using the local board type unless a remote endpoint is required:

```bash
cargo xtask starry app board -t <case> --board-config <config> -b OrangePi-5-Plus
```

If an app board config has its own `shell_init_cmd`, verify the runner honors it. If the observed command comes from the app's `init.sh` instead, inspect `scripts/axbuild/src/starry/mod.rs` and the app runner path before assuming the config is wrong.

## Diagnosing StarryOS `not found`

When StarryOS prints `<binary>: not found` for a path that exists on Linux, do not assume the binary build is broken. Check the StarryOS-visible rootfs.

Prefer a temporary board config outside the repository, or remove it before committing:

```toml
board_type = "OrangePi-5-Plus"
shell_prefix = "root@starry:/root #"
shell_init_cmd = '''
echo BOARD_DIAG_BEGIN
cd /target/path
pwd
ls -ld /target/path
ls -l /target/path
ls -l /lib/ld-linux-aarch64.so.1 /lib/aarch64-linux-gnu/ld-linux-aarch64.so.1 2>&1 || true
od -An -tx1 -N64 /target/path/<binary> 2>&1 || true
readelf -l /target/path/<binary> 2>&1 || true
echo BOARD_DIAG_DONE
'''
success_regex = ["(?m)^BOARD_DIAG_DONE$"]
fail_regex = ["(?i)\\bpanic(?:ked)?\\b"]
timeout = 120
```

Interpretation:

- Directory exists but binary is absent: redeploy from Linux and run `sync` before releasing the lease.
- Binary exists but PT_INTERP loader is absent: install or copy the dynamic loader and needed shared libraries, or rebuild for an available runtime.
- `readelf` is absent in the guest: use `od` and `ls` for minimal proof, and inspect ELF details from Linux or host-side `readelf`.

## Evidence To Keep

Keep the shortest useful evidence:

- Board type and whether local or shared service was used.
- Linux IP and deployment destination.
- Linux smoke result line or success marker.
- StarryOS final result line or success marker.
- Any root cause found, especially stale rootfs contents, missing loader, or runner command override behavior.

Delete temporary diagnostic configs before committing unless they are meant to become a maintained project test entry.
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
- Use `reassign-pr-reviewers` when the user wants to assign or rebalance GitHub PR reviewers for `rcore-os/tgoskits` from a discussion, ownership matrix, open PR scope, or existing review-request state, including preserving bot requests and handling collaborator permission limits.
- `board-uboot-fsck-repair`: project-local skill at `.claude/skills/board-uboot-fsck-repair/SKILL.md`
- Use `board-uboot-fsck-repair` when a physical board Linux rootfs needs ext4 recovery through U-Boot, initramfs fsck reports unrepaired corruption, OrangePi-5-Plus needs `extraboardargs=fsckfix`, or Starry board write tests must be bracketed by Linux fsck/boot checks.
- `board-linux-starry-debug`: project-local skill at `.claude/skills/board-linux-starry-debug/SKILL.md`
- Use `board-linux-starry-debug` when a physical-board workflow needs Linux-side deployment or inspection before running StarryOS or ArceOS, including `board connect` IP discovery, SSH/rsync while holding a board lease, explicit `sync` before rebooting into StarryOS, diagnosing StarryOS `not found` for files copied into the Linux rootfs, or comparing Linux-visible and StarryOS-visible board rootfs state.
- `crates-io-owner`: project-local skill at `.claude/skills/crates-io-owner/SKILL.md`
- Use `crates-io-owner` when the user wants to add or verify `github:rcore-os:crates-io` for branch-added crates, asks which new crates still need the crates.io team owner, or explicitly wants `cargo owner` used instead of `Cargo.toml` metadata.

Expand Down
99 changes: 95 additions & 4 deletions apps/starry/orangepi-5-plus-uvc-rknn/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,33 @@ The board rootfs must contain:
- `/usr/bin/uvc-fps`
- `/rknn_yolov8_image/rknn_yolov8_image`
- `/rknn_yolov8_image/rknn_yolov8_stream`
- `/rknn_yolov8_image/rknn_yolov8_bench`
- `/rknn_yolov8_image/lib/librknnrt.so`
- `/rknn_yolov8_image/lib/librga.so`
- `/rknn_yolov8_image/model/yolov8.rknn`
- `/rknn_yolov8_image/model/coco_80_labels_list.txt`

The fixed-image benchmark board config also expects a validation asset set under
`/rknn_yolov8_image/validation/`:

- `images.txt`: list of the three validation image paths, relative to
`/rknn_yolov8_image`;
- `expected.txt`: committed source asset with the expected detections for those
images;
- `tennis-ball-close.jpg`;
- `tennis-ball-black-box.jpg`;
- `tennis-ball-plant.jpg`.

`expected.txt` is consumed as a source asset during routine StarryOS board tests.
Regenerating it on Linux is only a maintenance step for intentional changes to
the model, image set, threshold, RKNN runtime, or postprocess behavior; routine
StarryOS tests should not regenerate it.

When updating `expected.txt`, use the committed validation images instead of
capturing or synthesizing replacements. The benchmark board config expects the
three image files and the matching committed `expected.txt` to be installed
together.

Build the image runner:

```bash
Expand Down Expand Up @@ -56,6 +78,38 @@ ssh orangepi@${BOARD_IP} '
'
```

Linux-side fixed-image validation should pass before the realtime UVC benchmark
when the validation assets are present:

```bash
ssh orangepi@${BOARD_IP} '
cd /rknn_yolov8_image &&
export LD_LIBRARY_PATH=/rknn_yolov8_image/lib:/usr/local/lib:/usr/lib/aarch64-linux-gnu:$LD_LIBRARY_PATH &&
printf "%s\n" orangepi | sudo -E -S \
./rknn_yolov8_bench --validate-list validation/images.txt --expected validation/expected.txt \
--min-confidence 25 --core-mask all --profile
'
```

The validation command must print:

```text
UVC_RKNN_VALIDATE_PASS images=3
```

Linux-side 60-second benchmark smoke can be shortened during setup:

```bash
ssh orangepi@${BOARD_IP} '
cd /rknn_yolov8_image &&
export LD_LIBRARY_PATH=/rknn_yolov8_image/lib:/usr/local/lib:/usr/lib/aarch64-linux-gnu:$LD_LIBRARY_PATH &&
printf "%s\n" orangepi | sudo -E -S \
./rknn_yolov8_bench --model model/yolov8.rknn --label model/coco_80_labels_list.txt \
--device 0 --width 320 --height 240 --fps 30 --duration-sec 8 --infer-every 1 \
--report-interval-sec 2 --min-confidence 25
'
```

For continuous manual testing with browser preview, use `--duration-sec 0
--max-inferences 0` and stop the program with `Ctrl+C`:

Expand Down Expand Up @@ -105,15 +159,52 @@ cargo xtask starry app board -t orangepi-5-plus-uvc-rknn \
--board-config configs/board-orangepi-5-plus-long-run.toml
```

For the current shared board, pass the concrete board lease endpoint:
For the local board service, pass the concrete board type:

```bash
cargo xtask starry app board -t orangepi-5-plus-uvc-rknn \
-b OrangePi-5-Plus-robot \
--server 10.30.12.60 \
--port 2999
-b OrangePi-5-Plus
```

Run the StarryOS benchmark example on the local board service:

```bash
cargo xtask starry app board -t orangepi-5-plus-uvc-rknn \
--board-config configs/board-orangepi-5-plus-bench.toml \
-b OrangePi-5-Plus
```

If the board is leased through a non-default shared service, add the matching
`--server` and `--port` values to either command.

The benchmark board config first runs fixed-image validation, then starts the
realtime UVC benchmark. The UVC benchmark does not start the HTTP stream. It
runs camera capture and RKNN inference for 60 seconds, then prints one
machine-readable summary line:

```text
UVC_RKNN_VALIDATE_PASS images=3
UVC_RKNN_BENCH_RESULT duration_sec=... captured=... capture_fps=... inferences=... infer_fps=... bytes=... throughput_mib_s=... dropped_latest=... decode_errors=... inference_errors=... decode_ms_avg=... decode_ms_p50=... decode_ms_p95=... infer_ms_avg=... infer_ms_p50=... infer_ms_p95=... detections=... vm_size_kb=... vm_rss_kb=... vm_hwm_kb=... mem_total_kb=... mem_free_kb=... mem_available_kb=...
UVC_RKNN_BENCH_DONE
```

To split the RKNN path into preprocessing, runtime, output, and postprocess
costs, enable profile mode:

```bash
RKNN_BENCH_PROFILE=1 ./rknn_yolov8_bench --duration-sec 8 --report-interval-sec 2
```

Profile mode adds one final summary line without changing the existing result
line:

```text
UVC_RKNN_BENCH_PROFILE_RESULT profile_samples=... perf_run_query_errors=... total_ms_avg=... letterbox_ms_avg=... inputs_set_ms_avg=... run_ms_avg=... outputs_get_ms_avg=... rknn_perf_run_ms_avg=... postprocess_ms_avg=...
```

Use `RKNN_BENCH_PROFILE_FRAMES=1` or `--profile-frames` for one
`RKNN_PROFILE` line per inference.

The same bounded smoke-test command is also stored in
`board-orangepi-5-plus.toml`, so this direct board command runs the default
example as well:
Expand Down
Loading
Loading