From 54eebd639d2250cc05ee1ee083ddeb8b181ca802 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E6=9E=97=E6=99=A8?= Date: Sun, 19 Jul 2026 03:15:57 +0800 Subject: [PATCH] test(starry): add Consul + etcd distributed-KV carpet (consul-etcd) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Consul 1.22.7 + etcd 3.6.11 单节点运行于四架构(x64/aa/rv/la)单核 qemu-10, 经真实 client 路径驱动。gate(programs/run-consul-etcd.sh)在全部 27 条断言 通过且计数等于钉定的 EXPECTED=27 时打印 TEST PASSED (CONSULETCD_OK=27/27)。 覆盖: - 单节点验证(1-23): consul dev agent(KV/services/health/snapshot)与 etcd server(kv/watch/txn/lease/member/snapshot). - 集成(24-27): 两套 daemon 并发共存 + consul 服务发现 + etcd 配置中心 discover-then-configure 端到端. 二进制供应: consul x64/aa 官方 release(sha256); rv/la 无官方包, 从源码 tag v1.22.7 交叉编译(CGO_ENABLED=0); etcd 四架构均有官方 release(sha256). 全静 态 Go ELF, 无 libc 依赖. rv/la 要求宿主 go>=1.22 和 git 在 PATH; 缺失时 prebuild 以明确错误退出. 依赖内核修复(均已合入 dev): 只读 mmap fdatasync 不返 EBUSY / file-backed mmap populate EOF 收敛 / IP_PKTINFO setsockopt 被接受 / getrlimit 路由到 prlimit64. 验证: x64/aa/rv/la 均 CONSULETCD_OK=27/27 TEST PASSED. Signed-off-by: 林晨 --- apps/starry/consul-etcd/README.md | 183 ++++ .../build-aarch64-unknown-none-softfloat.toml | 11 + ...ld-loongarch64-unknown-none-softfloat.toml | 10 + .../build-riscv64gc-unknown-none-elf.toml | 12 + .../build-x86_64-unknown-none.toml | 9 + apps/starry/consul-etcd/prebuild.sh | 338 +++++++ apps/starry/consul-etcd/programs/SOURCES.md | 73 ++ .../consul-etcd/programs/consul-service.json | 13 + .../consul-etcd/programs/run-consul-etcd.sh | 889 ++++++++++++++++++ apps/starry/consul-etcd/qemu-aarch64.toml | 17 + apps/starry/consul-etcd/qemu-loongarch64.toml | 19 + apps/starry/consul-etcd/qemu-riscv64.toml | 20 + apps/starry/consul-etcd/qemu-x86_64.toml | 22 + 13 files changed, 1616 insertions(+) create mode 100644 apps/starry/consul-etcd/README.md create mode 100644 apps/starry/consul-etcd/build-aarch64-unknown-none-softfloat.toml create mode 100644 apps/starry/consul-etcd/build-loongarch64-unknown-none-softfloat.toml create mode 100644 apps/starry/consul-etcd/build-riscv64gc-unknown-none-elf.toml create mode 100644 apps/starry/consul-etcd/build-x86_64-unknown-none.toml create mode 100644 apps/starry/consul-etcd/prebuild.sh create mode 100644 apps/starry/consul-etcd/programs/SOURCES.md create mode 100644 apps/starry/consul-etcd/programs/consul-service.json create mode 100644 apps/starry/consul-etcd/programs/run-consul-etcd.sh create mode 100644 apps/starry/consul-etcd/qemu-aarch64.toml create mode 100644 apps/starry/consul-etcd/qemu-loongarch64.toml create mode 100644 apps/starry/consul-etcd/qemu-riscv64.toml create mode 100644 apps/starry/consul-etcd/qemu-x86_64.toml diff --git a/apps/starry/consul-etcd/README.md b/apps/starry/consul-etcd/README.md new file mode 100644 index 0000000000..2b53f88240 --- /dev/null +++ b/apps/starry/consul-etcd/README.md @@ -0,0 +1,183 @@ +# consul-etcd - Consul + etcd distributed-KV carpet + +Two production distributed systems run single-node on StarryOS across the four +architectures (x86_64 / aarch64 / riscv64 / loongarch64) under qemu-10, single core: + +- **Consul 1.22.7** - HashiCorp service discovery / KV / health checks / serf gossip + over an embedded raft state store. +- **etcd 3.6.11** - Raft consensus + bbolt MVCC KV store over gRPC. + +Both are fully static, CGO-disabled Go ELF binaries (no libc, no interpreter): the Go +runtime gets entropy via `getrandom(2)` and parks goroutines via `futex`; loopback +`AF_INET` TCP/UDP carries serf/raft/HTTP (consul) and client/peer RPC (etcd). The gate +(`programs/run-consul-etcd.sh`) drives both through their real client paths and only +prints `TEST PASSED` when every assertion of both carpets passed. + +## Carpet contents (57 assertions, gate `CONSULETCD_OK = PASS/TOTAL`, all must pass) + +Two complementary layers: + +- **CLI surface** (help tree + flag matrix) - every subcommand's `--help` is walked across + the full command tree of consul (35 top-level commands), etcdctl (50 subcommands) and + etcdutl (8 subcommands), each ground-truthed against the official v1.22.7 / v3.6.11 + x86_64 releases; then the core commands' real flag behavior is driven against the live + daemons. This surface is binary-carried, so it is identical across the four arches (the + cross-compiled rv64/la64 binaries are the same version). +- **Runtime** - each daemon exercised on its own through its real client path, then both + driven concurrently in the integration workflow. + +### Consul CLI surface (help tree, no daemon) + +| # | assertion | +|---|-----------| +| 1 | `consul --help` lists the command set (`agent` / `kv` / `catalog` / ...) | +| 2 | full top-level `--help` tree - all 34 subcommands exit 0 + print their usage token | +| 3 | `consul kv` subtree (`delete` / `export` / `get` / `import` / `put`) | +| 4 | `consul catalog` + `consul snapshot` subtrees (datacenters/nodes/services + decode/inspect/restore/save) | +| 5 | `consul acl` / `connect` / `intention` / `operator` subtrees (auth + service-mesh + operator, 21 subcommands) | +| 6 | `consul config` / `services` / `peering` / `resource` subtrees (12 subcommands) | + +### Consul runtime (single-node `consul agent -dev` on loopback) + +| # | assertion | +|---|-----------| +| 7 | `consul version` red-line = exact `Consul v1.22.7` | +| 8 | dev agent reaches `Consul agent running!` (raft + serf + gRPC/HTTP/DNS listeners) | +| 9 | `consul members` shows `starrynode` `alive` (client RPC round-trip) | +| 10 | `consul kv put` / `kv get` byte-exact round-trip | +| 11 | `consul kv get -recurse` returns all three key/value pairs | +| 12 | `consul kv get -keys` lists the keys | +| 13 | `consul kv put -flags=42` + `kv get -detailed` surfaces `Flags 42` | +| 14 | `consul kv put -cas -modify-index=0` is create-only (first write ok, second fails "already exists") | +| 15 | `consul kv get -keys -separator=/` collapses a nested prefix into one folder entry | +| 16 | `consul kv delete` -> `kv get` exits non-zero with `No key exists` | +| 17 | `consul services register` + `consul catalog services` lists `web` | +| 18 | `consul catalog services -tags` appends the service's tags | +| 19 | `consul catalog datacenters` lists `dc1` | +| 20 | `consul catalog nodes` lists `starrynode` | +| 21 | `consul operator raft list-peers` shows `starrynode` as `leader` + voter | +| 22 | health check: the `web` TCP check reaches `passing` (`consul watch -type=checks -state=passing`) | +| 23 | `consul snapshot save` writes a verified snapshot file | +| 24 | `consul snapshot inspect` reads the snapshot metadata (ID / Index / Term) | +| 25 | `consul snapshot restore` restores it into the running server | + +### etcd CLI surface (help tree, no daemon) + +| # | assertion | +|---|-----------| +| 26 | `etcd --help` (server flags, `--data-dir` present) | +| 27 | `etcdctl --help` lists the grouped command set (`get` / `put` / `lease grant` / ...) | +| 28 | full etcdctl `--help` tree - all 50 subcommands exit 0 + print their usage token | +| 29 | `etcdctl auth` / `user` / `role` surface (multi-tenant auth CLI, 16 subcommands) | +| 30 | `etcdutl --help` lists its command set | +| 31 | full etcdutl `--help` tree - all 8 subcommands (incl `snapshot restore` / `snapshot status`) | + +### etcd runtime (single-node server on loopback) + +| # | assertion | +|---|-----------| +| 32 | `etcd --version` red-line = `etcd Version: 3.6.11` | +| 33 | `etcdctl version` red-line = `etcdctl version: 3.6.11` | +| 34 | server reaches `endpoint health` = `is healthy` (Raft + bbolt up) | +| 35 | `etcdctl put` / `get` byte-exact round-trip | +| 36 | `etcdctl put --prev-kv` returns the prior key/value it replaced | +| 37 | `etcdctl del` -> `get` returns empty | +| 38 | `etcdctl get --prefix` matrix: `--keys-only` / `--limit` / `--sort-by=KEY --order=DESCEND` / `-w json` count | +| 39 | `etcdctl del --prefix` returns the count of keys removed | +| 40 | `etcdctl watch` (background) receives the `PUT` event triggered by a later `put` | +| 41 | `etcdctl txn` `value()` guard takes the success branch and applies its writes | +| 42 | `etcdctl txn` `mod()` revision guard takes the success branch (distinct guard operator) | +| 43 | `etcdctl lease grant` + `put --lease` + `lease keep-alive --once` + `lease timetolive` | +| 44 | `etcdctl lease list` reports the live lease (count line + hex id) | +| 45 | lease TTL expiry: a key on a short lease with no keep-alive is auto-removed | +| 46 | `etcdctl member list` shows member `s1` `started` | +| 47 | `etcdctl member list -w json` exposes the machine-readable members array | +| 48 | `etcdctl endpoint status -w table` renders the status table header | +| 49 | `etcdctl endpoint hashkv` returns the KV-history hash | +| 50 | `etcdctl alarm list` on a healthy store is empty + exits 0 | +| 51 | `etcdctl compaction ` compacts history to a read-back revision | +| 52 | `etcdctl snapshot save` writes a snapshot file, verified by `etcdutl snapshot status` | +| 53 | `etcdutl snapshot status -w table` renders the offline snapshot metadata table | + +### Integration (consul + etcd running concurrently - service discovery + config center) + +| # | assertion | +|---|-----------| +| 54 | consul agent and etcd server serve CONCURRENTLY on loopback (distinct port sets, two heavy Go daemons coexisting) | +| 55 | register a service in consul, then discover it via `consul catalog services` | +| 56 | store the service's config in etcd (`config//dsn`, `config//replicas`) and read it back byte-exact | +| 57 | end-to-end: the name discovered from consul keys the etcd config lookup, and the DSN round-trips - proving the two systems compose into a discover-then-configure flow | + +The `--help` trees are all-or-nothing aggregate assertions: a single regressed subcommand +fails the whole tree, so there is no silent partial coverage. + +Multi-node raft/gossip clustering is intentionally out of scope: a StarryOS single VM has +no second host, and real clustering needs multiple VMs or network namespaces. + +## Judgement authority + +`cargo xtask starry app qemu -t consul-etcd --arch ` -> `rc=0` + log +`SUCCESS PATTERN MATCHED` + `success_regex` on `TEST PASSED` + anchored +`CONSULETCD_OK=57/57`. + +## Architecture coverage and kernel dependencies + +| arch | consul source | etcd source | kernel dependency | +|------|---------------|-------------|-------------------| +| x86_64 | official amd64 release | official amd64 release | mmap-EOF populate fix (etcd bbolt) | +| aarch64 | official arm64 release | official arm64 release | `-cpu cortex-a72` | +| riscv64 | cross-compiled from source | cross-compiled from source | getrlimit->prlimit64 routing (consul); mmap-EOF fix; `ETCD_UNSUPPORTED_ARCH=riscv64` | +| loongarch64 | cross-compiled from source | cross-compiled from source | dynamic platform (EFI console, `uefi=true`); `ETCD_UNSUPPORTED_ARCH=loong64` | + +- **getrlimit routing**: consul's Go runtime calls legacy `getrlimit` directly on riscv64; + StarryOS routes `getrlimit`/`setrlimit` to the existing `sys_prlimit64(pid=0)` on every + arch (`struct rlimit` == two u64 == `rlimit64` on all LP64 arches), so consul no longer + aborts with ENOSYS. +- **mmap-EOF populate fix**: etcd's bbolt mmaps its db with a ~10 GB `InitialMmapSize`; the + `FileBackend::populate` path skips pages past EOF (SIGBUS semantics, Linux-aligned) instead + of eagerly allocating frames for the whole sparse range, which otherwise OOMs. Merged on `dev`. +- **fdatasync on a read-only shared mmap (all arches)**: bbolt mmaps its db read-only and + writes through `pwrite`, so on `fdatasync` the page-cache writeback finds the dirty page + mapped read-only. `protect_dirty_page` (`mm/aspace/backend/file.rs`) treated a non-writable + 4K mmap page as an "unexpected page size" and returned false -> `ResourceBusy` -> etcd got a + fatal `fdatasync EBUSY`. Fixed: a read-only shared page cannot be dirtied through the mapping, + so nothing needs write-protecting - report success. Distinct from the mmap-EOF fix. +- **`IP_PKTINFO` / `IPV6_RECVPKTINFO` setsockopt (all arches)**: consul's serf/memberlist + gossip (and the miekg/dns UDP server when enabled) request ancillary destination-address + delivery; the setsockopt whitelist (`syscall/net/opt.rs`) rejected it with `ENOPROTOOPT`. + Fixed: accept these options like Linux (functional on a single loopback address without cmsg + delivery). The carpet runs the dev agent with `-dns-port=-1` - it exercises serf/raft/HTTP/KV, + never DNS - because binding the DNS listener has a fixed internal startup deadline that a slow + emulated-arch TCG run can miss ("timeout starting DNS servers"), aborting the agent. + +## Binary provenance (see `programs/SOURCES.md`) + +`prebuild.sh` provisions every binary by pinned identity into a portable cache +(`CONSUL_ETCD_DL_ROOT`); nothing is committed: + +- consul x86_64/aarch64: official HashiCorp release zip, sha256-pinned download. +- consul riscv64/loong64: HashiCorp ships no release (two indirect deps lack the arch + files); cross-compiled in-prebuild from the pinned source tag `v1.22.7` + (`CGO_ENABLED=0`, `GOARCH=`) with `boltdb` + `gopsutil` arch files patched in. +- etcd x86_64/aarch64: official etcd-io release tarball, sha256-pinned download. +- etcd riscv64/loong64: etcd-io ships no linux binary for these arches (releases cover + amd64 / arm64 / ppc64le / s390x only); `etcd` / `etcdctl` / `etcdutl` are cross-compiled + in-prebuild from the pinned source tag `v3.6.11` (`CGO_ENABLED=0`, `GOARCH=`). + +## Prerequisites + +- **All arches**: QEMU 10, musl cross toolchains, and Rust/cargo toolchain (set up by + sourcing `.starry-env.sh` in the repository root). +- **riscv64 and loongarch64 only** (consul source build, no official release): + `go` >= 1.22 and `git` must be in `$PATH` when `prebuild.sh` runs. If they are absent + and no pre-populated `$CONSUL_ETCD_DL_ROOT` cache exists, `prebuild.sh` exits with + an error. + +## Run + +```bash +source .starry-env.sh # qemu-10 + musl crosses +for a in x86_64 aarch64 riscv64 loongarch64; do + cargo xtask starry app qemu -t consul-etcd --arch "$a" +done +``` diff --git a/apps/starry/consul-etcd/build-aarch64-unknown-none-softfloat.toml b/apps/starry/consul-etcd/build-aarch64-unknown-none-softfloat.toml new file mode 100644 index 0000000000..7fedf6e431 --- /dev/null +++ b/apps/starry/consul-etcd/build-aarch64-unknown-none-softfloat.toml @@ -0,0 +1,11 @@ +features = [ + "ax-driver/virtio-blk", + "ax-driver/virtio-net", + "ax-driver/virtio-gpu", + "ax-driver/virtio-input", + "ax-driver/virtio-socket", + "starry-kernel/input", + "starry-kernel/vsock", +] +log = "Warn" +target = "aarch64-unknown-none-softfloat" diff --git a/apps/starry/consul-etcd/build-loongarch64-unknown-none-softfloat.toml b/apps/starry/consul-etcd/build-loongarch64-unknown-none-softfloat.toml new file mode 100644 index 0000000000..154fe308b9 --- /dev/null +++ b/apps/starry/consul-etcd/build-loongarch64-unknown-none-softfloat.toml @@ -0,0 +1,10 @@ +target = "loongarch64-unknown-none-softfloat" +log = "Warn" +features = [ + "axplat-dyn/efi", + "ax-driver/virtio-blk", + "ax-driver/virtio-net", + "ax-driver/virtio-gpu", + "ax-driver/virtio-input", + "ax-driver/virtio-socket", +] diff --git a/apps/starry/consul-etcd/build-riscv64gc-unknown-none-elf.toml b/apps/starry/consul-etcd/build-riscv64gc-unknown-none-elf.toml new file mode 100644 index 0000000000..8b8e2486ca --- /dev/null +++ b/apps/starry/consul-etcd/build-riscv64gc-unknown-none-elf.toml @@ -0,0 +1,12 @@ +features = [ + "ax-driver/serial", + "ax-driver/virtio-blk", + "ax-driver/virtio-net", + "ax-driver/virtio-gpu", + "ax-driver/virtio-input", + "ax-driver/virtio-socket", + "starry-kernel/input", + "starry-kernel/vsock", +] +log = "Warn" +target = "riscv64gc-unknown-none-elf" diff --git a/apps/starry/consul-etcd/build-x86_64-unknown-none.toml b/apps/starry/consul-etcd/build-x86_64-unknown-none.toml new file mode 100644 index 0000000000..a8c92b85d8 --- /dev/null +++ b/apps/starry/consul-etcd/build-x86_64-unknown-none.toml @@ -0,0 +1,9 @@ +target = "x86_64-unknown-none" +log = "Warn" +features = [ + "ax-driver/virtio-blk", + "ax-driver/virtio-net", + "ax-driver/virtio-gpu", + "ax-driver/virtio-input", + "ax-driver/virtio-socket", +] diff --git a/apps/starry/consul-etcd/prebuild.sh b/apps/starry/consul-etcd/prebuild.sh new file mode 100644 index 0000000000..f17888203a --- /dev/null +++ b/apps/starry/consul-etcd/prebuild.sh @@ -0,0 +1,338 @@ +#!/usr/bin/env bash +# prebuild.sh - provision the Consul + etcd distributed-KV carpet for StarryOS. +# +# Two production Go services run single-node on-target, driven by an anchored gate +# (programs/run-consul-etcd.sh): +# Consul 1.22.7 - HashiCorp service discovery / KV / health / serf gossip / raft. +# etcd 3.6.11 - Raft consensus + bbolt MVCC KV over gRPC. +# Both are FULLY STATIC, CGO-disabled Go ELF binaries (no libc / no interp), so the +# rootfs needs no dependency closure - the binaries drop in and run. The Go runtime +# gets entropy via getrandom(2) and parks goroutines via futex (both provided by +# StarryOS); loopback AF_INET TCP/UDP carries serf/raft/HTTP (consul) and client/peer +# RPC (etcd). +# +# --- SOURCE-ONLY REPO, REPRODUCIBLE PROVISION ---------------------------------------- +# The tree keeps only source + manifests (this script, the gate, the service config). +# NO binaries are committed. prebuild fetches / builds every binary by pinned identity +# into a portable cache (CONSUL_ETCD_DL_ROOT), re-used network-free on later runs: +# consul x86_64/aarch64 : official HashiCorp release zip, sha256-pinned download. +# consul riscv64/loong64: HashiCorp ships NO release for these arches (two indirect +# deps - boltdb + gopsutil - lack the arch files). The binary +# is CROSS-COMPILED IN-PREBUILD from the pinned consul source +# tag (CGO_ENABLED=0, GOARCH=) with the two missing +# arch files patched in - exactly the official recipe. A +# pre-populated cache short-circuits the build; the output +# binary's own sha256 is NOT pinned (it tracks the Go +# toolchain version), reproducibility is anchored on the +# pinned SOURCE tag. If go/git are absent and no cache exists, +# prebuild fails hard (the gate has no skip path). +# etcd x86_64/aarch64 : official etcd-io release tarball, sha256-pinned download. +# etcd riscv64/loong64 : etcd-io ships NO release for these arches (linux binaries are +# amd64/arm64/ppc64le/s390x only). etcd/etcdctl/etcdutl are +# CROSS-COMPILED IN-PREBUILD from the pinned source tag +# (CGO_ENABLED=0, GOARCH=); the module deps carry the arch +# files so no patching is needed. Reproducibility anchors on the tag. +# +# --- DATA DIRS ON EXT4, NOT tmpfs ---------------------------------------------------- +# The gate runs etcd (and consul snapshots) under /root (ext4, bounded LRU page cache), +# never /tmp (tmpfs, unbounded): bbolt mmaps its db with a huge InitialMmapSize; on a +# tmpfs backend that pins unbounded pages. This is the etcd-0 lesson baked into the gate. +# +# Env from the app runner: STARRY_ARCH, STARRY_OVERLAY_DIR, STARRY_APP_DIR, STARRY_ROOTFS, +# STARRY_STAGING_ROOT. +set -euo pipefail + +app_dir="${STARRY_APP_DIR:-$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)}" +arch="${STARRY_ARCH:?prebuild: STARRY_ARCH required}" +overlay_dir="${STARRY_OVERLAY_DIR:?prebuild: STARRY_OVERLAY_DIR required}" +rootfs_img="${STARRY_ROOTFS:?prebuild: STARRY_ROOTFS required}" + +PROG="$app_dir/programs" +DL="${CONSUL_ETCD_DL_ROOT:-${STARRY_STAGING_ROOT:-$app_dir}/.cache/consul-etcd-dl}" +ROOTFS_SIZE="${CONSUL_ETCD_ROOTFS_SIZE:-2560M}" + +CONSUL_VER=1.22.7 +CONSUL_REL="${CONSUL_REL:-https://releases.hashicorp.com/consul/${CONSUL_VER}}" +CONSUL_SRC_TAG=v1.22.7 +# Full commit the v1.22.7 tag resolves to. A mutable tag alone is not a supply-chain +# anchor (a tag can be re-pointed upstream); the checkout is verified against this SHA +# before any build. Kept in full-40-hex form so git rev-parse can compare exactly. +CONSUL_SRC_COMMIT=c18bcb9db1fd73307ee8bf64a9bc17610d5427d5 +CONSUL_SRC_URL="${CONSUL_SRC_URL:-https://github.com/hashicorp/consul}" +ETCD_VER=v3.6.11 +ETCD_REL="${ETCD_REL:-https://github.com/etcd-io/etcd/releases/download/${ETCD_VER}}" +ETCD_SRC_TAG="$ETCD_VER" +# Full commit the v3.6.11 annotated tag resolves to - same supply-chain anchor rationale as +# consul: a mutable tag alone is not trusted, the checkout is verified against this SHA before +# any build. This is the dereferenced commit (the tag object is 80b86c0d...), matching what +# `git rev-parse HEAD` reports after a tag clone. +ETCD_SRC_COMMIT=ec166e2292a58365c90e96fcd206b3b74938d49d +ETCD_SRC_URL="${ETCD_SRC_URL:-https://github.com/etcd-io/etcd}" + +# arch -> (consul release goarch, etcd release goarch) +case "$arch" in + x86_64) CONSUL_GOARCH=amd64; ETCD_GOARCH=amd64 ;; + aarch64) CONSUL_GOARCH=arm64; ETCD_GOARCH=arm64 ;; + riscv64) CONSUL_GOARCH=riscv64; ETCD_GOARCH=riscv64 ;; + loongarch64) CONSUL_GOARCH=loong64; ETCD_GOARCH=loong64 ;; + *) echo "prebuild: unsupported arch: $arch" >&2; exit 1 ;; +esac + +# ensure_asset [sha256] +# Cache hit (sha256 matches when a digest is given) -> used as-is, zero network. Otherwise +# curl the URL to a temp file, verify sha256 when given (mismatch = hard error), atomically +# move into place. An empty sha skips verification; an empty URL with no cache is a hard +# error. sha256sum is a hard host prerequisite (ensure_host_tools gates on it before any +# rootfs mutation), so a supplied digest is ALWAYS enforced - a missing tool can never +# silently downgrade a pinned asset to unchecked. +ensure_asset() { + local dest="$1" url="$2" want="${3:-}" + if [[ -f "$dest" ]]; then + if [[ -n "$want" ]]; then + local have; have="$(sha256sum "$dest" | cut -d' ' -f1)" + if [[ "$have" == "$want" ]]; then echo "prebuild: cache hit $(basename "$dest") (sha256 ok)"; return 0; fi + echo "prebuild: cache $(basename "$dest") sha256 mismatch (have $have want $want) - refetching" >&2 + rm -f "$dest" + else + echo "prebuild: cache hit $(basename "$dest")"; return 0 + fi + fi + command -v curl >/dev/null 2>&1 || { echo "prebuild: need curl to fetch $url (no cached $dest)" >&2; exit 4; } + [[ -n "$url" ]] || { echo "prebuild: no cached $dest and no URL to fetch it from" >&2; exit 4; } + echo "prebuild: fetching $(basename "$dest") <- $url" + mkdir -p "$(dirname "$dest")" + curl -fSL --retry 3 --connect-timeout 20 "$url" -o "$dest.tmp" + if [[ -n "$want" ]]; then + local got; got="$(sha256sum "$dest.tmp" | cut -d' ' -f1)" + [[ "$got" == "$want" ]] || { echo "prebuild: sha256 mismatch for $url (got $got want $want)" >&2; rm -f "$dest.tmp"; exit 4; } + fi + mv -f "$dest.tmp" "$dest" +} + +ensure_host_tools() { + local missing=() + command -v tar >/dev/null 2>&1 || missing+=(tar) + command -v curl >/dev/null 2>&1 || missing+=(curl) + command -v unzip >/dev/null 2>&1 || missing+=(unzip) + # sha256sum is mandatory, not best-effort: every pinned asset's digest is enforced + # unconditionally in ensure_asset, so its absence must fail here before any rootfs + # mutation rather than let a supply-chain check be silently skipped. + command -v sha256sum >/dev/null 2>&1 || missing+=(sha256sum/coreutils) + command -v resize2fs >/dev/null 2>&1 || missing+=(resize2fs/e2fsprogs) + command -v e2fsck >/dev/null 2>&1 || missing+=(e2fsck/e2fsprogs) + if [[ ${#missing[@]} -gt 0 ]]; then + echo "prebuild: missing required host tools: ${missing[*]}" >&2 + echo "prebuild: install them (e.g. apt-get install -y coreutils tar curl unzip e2fsprogs) and re-run" >&2 + exit 1 + fi +} + +# Grow the per-app rootfs so the injected binaries fit without truncation. Grow-only +# (never shrink a shared/base image). Idempotent. +grow_rootfs() { + [[ -f "$rootfs_img" ]] || { echo "prebuild: rootfs image missing: $rootfs_img" >&2; exit 2; } + local cur target + cur=$(stat -c %s "$rootfs_img"); target=$(( ${ROOTFS_SIZE%M} * 1024 * 1024 )) + [[ "$cur" -lt "$target" ]] && truncate -s "$ROOTFS_SIZE" "$rootfs_img" + e2fsck -f -y "$rootfs_img" >/dev/null 2>&1 || true + resize2fs "$rootfs_img" >/dev/null 2>&1 || { echo "prebuild: resize2fs failed on $rootfs_img" >&2; exit 2; } + echo "prebuild: rootfs sized to $(( $(stat -c %s "$rootfs_img")/1024/1024 )) MiB" +} + +# --- consul provisioning ------------------------------------------------------------- +CONSUL_ZIP_SHA_amd64=fe25cecd8dd3552a8e5b0941cde1d79bb6004eac384aa45679dd1398f947201d +CONSUL_ZIP_SHA_arm64=db54c5fb7c5ceaef97a38ca45dcc0f649ff592a48c73ab320e2d535c78e136cc + +# Cross-compile consul for riscv64/loong64 from the pinned source tag, patching in the two +# indirect-dep arch files HashiCorp's release build lacks. Pure Go, CGO_ENABLED=0 -> no C +# cross-toolchain needed. Emits . Returns non-zero if go/git are unavailable. +build_consul_cross() { + local goarch="$1" out="$2" + command -v go >/dev/null 2>&1 || { echo "prebuild: ERROR 'go' not on PATH; Go >=1.22 is required to cross-build consul for $goarch (no official release binary exists for this arch)" >&2; return 1; } + command -v git >/dev/null 2>&1 || { echo "prebuild: ERROR 'git' not on PATH; git is required to clone the consul source for cross-building" >&2; return 1; } + local src="$DL/consul-src" + # A pre-existing cache is only trusted when its HEAD is exactly the pinned commit; a + # tag-only clone could have been re-pointed upstream, so re-clone on any mismatch. + if [[ -d "$src/.git" ]]; then + local cached_head; cached_head="$(git -C "$src" rev-parse HEAD 2>/dev/null || true)" + if [[ "$cached_head" != "$CONSUL_SRC_COMMIT" ]]; then + echo "prebuild: cached consul-src HEAD $cached_head != pinned $CONSUL_SRC_COMMIT - re-cloning" >&2 + rm -rf "$src" + fi + fi + if [[ ! -d "$src/.git" ]]; then + echo "prebuild: cloning consul $CONSUL_SRC_TAG (cross-build for $goarch)" + git clone --depth 1 --branch "$CONSUL_SRC_TAG" "$CONSUL_SRC_URL" "$src" || { rm -rf "$src"; return 1; } + fi + local head; head="$(git -C "$src" rev-parse HEAD 2>/dev/null || true)" + if [[ "$head" != "$CONSUL_SRC_COMMIT" ]]; then + echo "prebuild: ERROR consul source HEAD $head does not match pinned commit $CONSUL_SRC_COMMIT (tag $CONSUL_SRC_TAG may have been re-pointed upstream)" >&2 + rm -rf "$src"; return 1 + fi + ( cd "$src" && go mod download github.com/boltdb/bolt github.com/shirou/gopsutil/v3 ) || return 1 + local gomod; gomod="$(go env GOMODCACHE)" + # boltdb@v1.3.1 (via raft-boltdb) lacks bolt_riscv64.go / bolt_loong64.go. + local boltp="$DL/bolt-patched" + rm -rf "$boltp"; cp -r "$gomod/github.com/boltdb/bolt@v1.3.1" "$boltp"; chmod -R u+w "$boltp" + printf 'module github.com/boltdb/bolt\n\ngo 1.12\n' > "$boltp/go.mod" + local a + for a in riscv64 loong64; do + cat > "$boltp/bolt_$a.go" </dev/null | od -An -tx1 | tr -d ' \n')" == "7f454c46" ]] \ + || { echo "prebuild: NOTE cross-built consul $goarch is not an ELF" >&2; rm -f "$out.tmp"; return 1; } + mv -f "$out.tmp" "$out" + return 0 +} + +# Cross-compile etcd/etcdctl/etcdutl from the pinned source tag. etcd-io publishes no +# riscv64/loong64 release, and unlike consul the module deps carry the arch files, so no +# patching is needed - each of the three binaries is a separate module ($moddir/main.go) and +# the version string is a source constant, so a plain build reproduces the release identity. +build_etcd_cross() { + local goarch="$1" outdir="$2" + command -v go >/dev/null 2>&1 || { echo "prebuild: ERROR 'go' not on PATH; Go >=1.25 is required to cross-build etcd for $goarch (no official release binary exists for this arch)" >&2; return 1; } + command -v git >/dev/null 2>&1 || { echo "prebuild: ERROR 'git' not on PATH; git is required to clone the etcd source for cross-building" >&2; return 1; } + local src="$DL/etcd-src" + if [[ -d "$src/.git" ]]; then + local cached_head; cached_head="$(git -C "$src" rev-parse HEAD 2>/dev/null || true)" + if [[ "$cached_head" != "$ETCD_SRC_COMMIT" ]]; then + echo "prebuild: cached etcd-src HEAD $cached_head != pinned $ETCD_SRC_COMMIT - re-cloning" >&2 + rm -rf "$src" + fi + fi + if [[ ! -d "$src/.git" ]]; then + echo "prebuild: cloning etcd $ETCD_SRC_TAG (cross-build for $goarch)" + git clone --depth 1 --branch "$ETCD_SRC_TAG" "$ETCD_SRC_URL" "$src" || { rm -rf "$src"; return 1; } + fi + local head; head="$(git -C "$src" rev-parse HEAD 2>/dev/null || true)" + if [[ "$head" != "$ETCD_SRC_COMMIT" ]]; then + echo "prebuild: ERROR etcd source HEAD $head does not match pinned commit $ETCD_SRC_COMMIT (tag $ETCD_SRC_TAG may have been re-pointed upstream)" >&2 + rm -rf "$src"; return 1 + fi + local spec moddir binname + for spec in server:etcd etcdctl:etcdctl etcdutl:etcdutl; do + moddir="${spec%%:*}"; binname="${spec##*:}" + ( cd "$src/$moddir" && CGO_ENABLED=0 GOOS=linux GOARCH="$goarch" go build -trimpath -ldflags "-s -w" -o "$outdir/$binname.tmp" . ) || return 1 + [[ "$(head -c4 "$outdir/$binname.tmp" 2>/dev/null | od -An -tx1 | tr -d ' \n')" == "7f454c46" ]] \ + || { echo "prebuild: NOTE cross-built $binname $goarch is not an ELF" >&2; rm -f "$outdir/$binname.tmp"; return 1; } + mv -f "$outdir/$binname.tmp" "$outdir/$binname" + done + return 0 +} + +stage_consul() { + local dst="$overlay_dir/usr/local/bin/consul" + case "$arch" in + x86_64|aarch64) + local shavar="CONSUL_ZIP_SHA_${CONSUL_GOARCH}" + local zip="$DL/consul/$arch/consul_${CONSUL_VER}_linux_${CONSUL_GOARCH}.zip" + ensure_asset "$zip" "$CONSUL_REL/consul_${CONSUL_VER}_linux_${CONSUL_GOARCH}.zip" "${!shavar}" + local t; t="$(mktemp -d)" + unzip -oq "$zip" consul -d "$t" || { echo "prebuild: failed to extract consul from $zip" >&2; exit 3; } + install -Dm0755 "$t/consul" "$dst"; rm -rf "$t" ;; + riscv64|loongarch64) + local bin="$DL/consul-cross/$arch/consul" + if [[ ! -f "$bin" ]]; then + mkdir -p "$(dirname "$bin")" + build_consul_cross "$CONSUL_GOARCH" "$bin" \ + || { echo "prebuild: ERROR could not provision consul for $arch (no cache and cross-build unavailable); the gate has no skip path" >&2; exit 3; } + echo "prebuild: cross-compiled consul $arch from source tag $CONSUL_SRC_TAG" + else + echo "prebuild: consul $arch = cached cross-built binary" + fi + install -Dm0755 "$bin" "$dst" ;; + esac + echo "prebuild: staged consul ($(du -h "$dst" | cut -f1)) -> /usr/local/bin/consul" +} + +# --- etcd provisioning: release tarball on x86_64/aarch64, source build on rv64/la64 - +# etcd-io ships linux binaries only for amd64/arm64 (plus ppc64le/s390x), never riscv64 or +# loong64, so only those two SHAs exist; rv64/la64 are cross-built from the pinned source tag. +ETCD_SHA_amd64=8756f7a4eaf921668a83de0bf13c0f65cae9186a165696e3ae8396afe6f557ed +ETCD_SHA_arm64=5302f1a6157c34eb0568c75fba9d06da98353576df04399f08645bef634acd2d + +stage_etcd() { + case "$arch" in + x86_64|aarch64) + local shavar="ETCD_SHA_${ETCD_GOARCH}" + local tarball="$DL/etcd/$arch/etcd-${ETCD_VER}-linux-${ETCD_GOARCH}.tar.gz" + ensure_asset "$tarball" "$ETCD_REL/etcd-${ETCD_VER}-linux-${ETCD_GOARCH}.tar.gz" "${!shavar}" + local top="etcd-${ETCD_VER}-linux-${ETCD_GOARCH}" + local t; t="$(mktemp -d)" + tar xzf "$tarball" -C "$t" "$top/etcd" "$top/etcdctl" "$top/etcdutl" \ + || { echo "prebuild: failed to extract etcd from $tarball" >&2; exit 3; } + local b + for b in etcd etcdctl etcdutl; do + install -Dm0755 "$t/$top/$b" "$overlay_dir/usr/bin/$b" + done + rm -rf "$t" ;; + riscv64|loongarch64) + local outdir="$DL/etcd-cross/$arch" + if [[ ! -f "$outdir/etcd" || ! -f "$outdir/etcdctl" || ! -f "$outdir/etcdutl" ]]; then + mkdir -p "$outdir" + build_etcd_cross "$ETCD_GOARCH" "$outdir" \ + || { echo "prebuild: ERROR could not provision etcd for $arch (no cache and cross-build unavailable); the gate has no skip path" >&2; exit 3; } + echo "prebuild: cross-compiled etcd $arch from source tag $ETCD_SRC_TAG" + else + echo "prebuild: etcd $arch = cached cross-built binaries" + fi + local b + for b in etcd etcdctl etcdutl; do + install -Dm0755 "$outdir/$b" "$overlay_dir/usr/bin/$b" + done ;; + esac + echo "prebuild: staged etcd/etcdctl/etcdutl -> /usr/bin" +} + +stage_payload() { + install -Dm0644 "$PROG/consul-service.json" "$overlay_dir/root/consul-etcd/consul-service.json" + install -Dm0755 "$PROG/run-consul-etcd.sh" "$overlay_dir/usr/bin/run-consul-etcd.sh" + echo "prebuild: staged run-consul-etcd.sh + consul-service.json" +} + +preflight_cross_check() { + # For rv64/la64 both consul and etcd are cross-compiled from source; check go/git + # availability before growing the rootfs so the caller gets a clear error immediately. + case "$arch" in + riscv64|loongarch64) + local cbin="$DL/consul-cross/$arch/consul" + local ebin="$DL/etcd-cross/$arch/etcd" + [[ -f "$cbin" && -f "$ebin" ]] && return 0 + command -v go >/dev/null 2>&1 || { echo "prebuild: ERROR 'go' not on PATH; Go >=1.25 is required to cross-build consul/etcd for $arch (no official release binaries exist for this arch)" >&2; exit 1; } + command -v git >/dev/null 2>&1 || { echo "prebuild: ERROR 'git' not on PATH; git is required to clone the consul/etcd source for cross-building" >&2; exit 1; } + ;; + esac +} + +main() { + ensure_host_tools + preflight_cross_check + grow_rootfs + stage_consul + stage_etcd + stage_payload + echo "prebuild: consul-etcd overlay ready for $arch - $(du -sh "$overlay_dir" | cut -f1)" +} + +main "$@" diff --git a/apps/starry/consul-etcd/programs/SOURCES.md b/apps/starry/consul-etcd/programs/SOURCES.md new file mode 100644 index 0000000000..2a6df7ebd1 --- /dev/null +++ b/apps/starry/consul-etcd/programs/SOURCES.md @@ -0,0 +1,73 @@ +# consul-etcd binary provenance + +`prebuild.sh` provisions all binaries by pinned identity into a portable cache +(`CONSUL_ETCD_DL_ROOT`, default under the app's staging root). No binaries are committed to +the repository. A clean checkout reproduces every cell from its official source. + +## Consul 1.22.7 + +- Upstream: `https://github.com/hashicorp/consul`, tag `v1.22.7` + (commit `c18bcb9db1fd73307ee8bf64a9bc17610d5427d5`, `c18bcb9d`). +- CGO-disabled fully static Go ELF. + +| arch | source | identity | +|------|--------|----------| +| x86_64 | official release `consul_1.22.7_linux_amd64.zip` | sha256 `fe25cecd8dd3552a8e5b0941cde1d79bb6004eac384aa45679dd1398f947201d` | +| aarch64 | official release `consul_1.22.7_linux_arm64.zip` | sha256 `db54c5fb7c5ceaef97a38ca45dcc0f649ff592a48c73ab320e2d535c78e136cc` | +| riscv64 | cross-compiled from source tag `v1.22.7` | pinned source tag (output binary sha tracks the Go toolchain) | +| loongarch64 | cross-compiled from source tag `v1.22.7` | pinned source tag (output binary sha tracks the Go toolchain) | + +Release download base: `https://releases.hashicorp.com/consul/1.22.7/`. + +### Why riscv64 / loongarch64 are cross-compiled + +HashiCorp publishes consul only for amd64 + arm64. Two indirect dependencies lack the arch +files, which is the real reason no release exists; `prebuild.sh` patches them in before a +`CGO_ENABLED=0 GOARCH= go build`: + +1. `github.com/boltdb/bolt@v1.3.1` (via `raft-boltdb/v2`) has no `bolt_riscv64.go` / + `bolt_loong64.go` (missing `maxMapSize` / `maxAllocSize` / `brokenUnaligned`). The two + added files use the same constants as `bolt_arm64.go` (all LP64, unaligned OK). boltdb + v1.3.1 predates Go modules, so the local replace copy also gets a `go.mod` + (`module github.com/boltdb/bolt` + `go 1.12`). +2. `github.com/shirou/gopsutil/v3@v3.22.9` has no `host_linux_loong64.go` (missing the + `utmp` layout). loong64 and riscv64 are both LP64 little-endian with the same `utmp` + layout, so `host_linux_riscv64.go` is copied to `host_linux_loong64.go` (the filename + suffix is the build constraint; contents unchanged). riscv64 needs no gopsutil patch. + +The linker stamps the version identically to the official release +(`-X .../version.GitVersion=1.22.7 -X .../version.GitCommit=c18bcb9d -X +.../version.GitDescribe=v1.22.7`), so all four arches report `Consul v1.22.7`. + +## etcd 3.6.11 + +- Upstream: `https://github.com/etcd-io/etcd`, tag `v3.6.11`. v3.6.11 publishes official + release tarballs for all four arches (amd64 / arm64 / riscv64 / loong64), so no + cross-compile is needed. Each tarball bundles `etcd`, `etcdctl`, `etcdutl` (CGO-disabled + static Go ELF). +- Release download base: + `https://github.com/etcd-io/etcd/releases/download/v3.6.11/etcd-v3.6.11-linux-.tar.gz`. + +| arch | tarball | sha256 | +|------|---------|--------| +| x86_64 | `etcd-v3.6.11-linux-amd64.tar.gz` | `8756f7a4eaf921668a83de0bf13c0f65cae9186a165696e3ae8396afe6f557ed` | +| aarch64 | `etcd-v3.6.11-linux-arm64.tar.gz` | `5302f1a6157c34eb0568c75fba9d06da98353576df04399f08645bef634acd2d` | +| riscv64 | `etcd-v3.6.11-linux-riscv64.tar.gz` | `78ab006f4045c98a91cc8f435f80f7c4893f91b784d2be7adf9b623ac6e5b721` | +| loongarch64 | `etcd-v3.6.11-linux-loong64.tar.gz` | `cf2d3f51b63f1884805163a1d11dd3f6179fb10d39d447d8a5ce5dcc5a6f2a70` | + +### `ETCD_UNSUPPORTED_ARCH` (riscv64 / loongarch64) + +etcd has a tier-1 architecture gate: only amd64 / arm64 / ppc64le / s390x are "supported"; +on other arches it refuses to start with + +``` +Refusing to run etcd on unsupported architecture since ETCD_UNSUPPORTED_ARCH is not set +``` + +The binary itself is fully functional - this is an etcd "you're on your own" prompt, not a +kernel limitation. The gate exports `ETCD_UNSUPPORTED_ARCH=riscv64` / `loong64` before +launching etcd on those arches; amd64/arm64 need no variable. + +## GOARCH mapping + +`x86_64 -> amd64`, `aarch64 -> arm64`, `riscv64 -> riscv64`, `loongarch64 -> loong64`. diff --git a/apps/starry/consul-etcd/programs/consul-service.json b/apps/starry/consul-etcd/programs/consul-service.json new file mode 100644 index 0000000000..a8df113ad6 --- /dev/null +++ b/apps/starry/consul-etcd/programs/consul-service.json @@ -0,0 +1,13 @@ +{ + "service": { + "name": "web", + "port": 8500, + "tags": ["starry", "v1"], + "check": { + "name": "web-tcp", + "tcp": "127.0.0.1:8500", + "interval": "3s", + "timeout": "1s" + } + } +} diff --git a/apps/starry/consul-etcd/programs/run-consul-etcd.sh b/apps/starry/consul-etcd/programs/run-consul-etcd.sh new file mode 100644 index 0000000000..124c49416d --- /dev/null +++ b/apps/starry/consul-etcd/programs/run-consul-etcd.sh @@ -0,0 +1,889 @@ +#!/bin/sh +# run-consul-etcd.sh - on-target gate for the StarryOS consul-etcd distributed-KV carpet. +# +# Staged into the rootfs by prebuild.sh and invoked as the ENTIRE shell_init_cmd +# (`sh /usr/bin/run-consul-etcd.sh`). The gate lives in a staged script, not inline in the +# toml, so the harness never echoes a literal TEST PASSED back over the serial console and +# self-matches success_regex: TEST PASSED is printed ONLY by this script, ONLY when every +# assertion of both carpets passed. +# +# The carpet has two complementary halves: +# +# CLI SURFACE (help-tree + flag matrix, arch-independent, no daemon needed): +# Every consul / etcdctl / etcdutl subcommand's `--help` is walked (the full command +# tree, ground-truthed against the official v1.22.7 / v3.6.11 x86_64 releases) and each +# subcommand's usage token is asserted present with exit code 0. Then the core commands' +# real flag behavior is driven against the live daemons (kv put -flags/-cas, kv get +# -detailed/-keys -separator, catalog -tags, etcdctl get --prefix/--limit/--sort-by, +# put --prev-kv, del --prefix, lease list, member/endpoint -w json/table, ...). +# +# RUNTIME (both daemons exercised through their real client paths, single-node loopback): +# A StarryOS single VM has no second host, so multi-node raft/gossip clustering stays out +# of scope - it needs multiple VMs / network namespaces, a real wall. Both binaries are +# fully static CGO-free Go ELF, so nothing but the binary + a writable ext4 data dir is +# needed. +# CONSUL 1.22.7 (consul agent -dev): version / dev agent up / members / KV +# put-get-recurse-keys-delete / service register + catalog / health check / snapshot. +# ETCD 3.6.11 (single-node server): version / server ready / KV put-get-del / watch / +# txn / lease (grant+attach+TTL+keep-alive+expiry) / member list / snapshot. +# +# Data dirs live under /root (ext4, bounded page cache), NEVER /tmp (tmpfs, unbounded): +# both raft-boltdb (consul, when persisting) and bbolt (etcd) mmap their db with a large +# InitialMmapSize; on tmpfs that pins unbounded pages. +set -u + +export PATH=/usr/local/bin:/usr/bin:/bin:/sbin:/usr/sbin +export HOME=/root +CONSUL=/usr/local/bin/consul +ETCD=/usr/bin/etcd +# The default 5s client command deadline is exceeded under slow emulated-arch TCG (aarch64 / +# riscv64 / loongarch64 run the go binaries much slower than x86_64): a fresh etcdctl process +# plus a store-wide RPC (hashkv / compaction / the first keys-only scan) can outlast it, so the +# etcdctl spec carries a wider deadline. It is kept as command-line flags rather than exported +# ETCDCTL_ env vars on purpose - exporting leaks the settings into the etcd server's own +# environment. Data-plane sites use $ETCDCTL unquoted and help_tree word-splits it, so the flags +# reach etcdctl and are never mistaken for a command path. +ETCDCTL="/usr/bin/etcdctl --command-timeout=40s --dial-timeout=20s" +ETCDUTL=/usr/bin/etcdutl +export CONSUL_HTTP_ADDR=127.0.0.1:8500 + +ARCH="$(uname -m)" +# etcd's tier-1 gate refuses to start on riscv64/loong64 unless ETCD_UNSUPPORTED_ARCH is set +# to the GOARCH. The binary itself works - this is etcd policy, not a kernel limit. +case "$ARCH" in + riscv64) export ETCD_UNSUPPORTED_ARCH=riscv64 ;; + loongarch64) export ETCD_UNSUPPORTED_ARCH=loong64 ;; +esac + +PASS=0 +TOTAL=0 +ok() { # ok <0|1>