iOS support: GPU-scene ABI, bounded memory, compose completeness - #34
Merged
Conversation
GPU text: label tiers, rotated depth-contour labels, and a HiDPI sprite atlas
ViewTiles clamped tile x at the world edge, so a view straddling ±180° rendered its far side empty (the Aleutian chain broke at the dateline in every consumer — pixel, PDF, and GPU-scene paths all walk ViewTiles). Longitude is cyclic: wrap the tile INDEX modulo the world width while the canvas origin keeps the unwrapped position, so the far side's tiles fetch and draw at their continuous spot in the view. y still clamps (mercator poles). Verified seamless against the NOAA whole-US library at 179.97°E.
- Forward --sysroot as a system include path to every C-compiling module (Zig bundles Apple libc headers only for macOS; lua/libtess2/svgraster all failed to find <stdlib.h> for -Dtarget=aarch64-ios*). - Stub Lua's l_system on iOS (system(3) is marked unavailable in the SDK): os.execute reports no shell; nothing in the portrayal path shells out. - Strip the static lib on iOS: std.debug's stack-trace machinery references _dyld_get_image_header_containing_address, which iOS' libdyld doesn't export. - Ignore per-platform install prefixes (zig-out-iphonesimulator etc.).
tile57_bake_tree now writes <out_dir>/_assets/ once per library — a single
3x sprite atlas (sprite@3x.png/.json) plus the resolution-independent SDF
label atlases (glyph{,-bold,-italic}.png/.json). They depend only on the
embedded catalogue, not any chart, so a GPU host loads them at open instead
of re-rasterizing ~1,500 symbols + the font faces every time (~1.3s at 1x,
several seconds at HiDPI). 3x is a single density that serves every device:
symbol size is cell_px/ppm (ratio-invariant), so the scene ratio only sets
the atlas's normalized-UV layout, and lower-density screens sample the
high-res atlas down. Private bakeAssetsDir; no new public API.
This reverts commit 125dc35.
nowNs() issued the raw Linux clock_gettime syscall, which dies with SIGSYS on macOS before the tool prints anything. Go through libc, which is what the engine itself does.
One line per considered face — tier, cell index, cscl, classify result, archive tile presence — while serving. Off (and free) without the env var. How the where-did-this-ground-go questions get answered.
renderComposeGpuScene dropped a tile on ANY per-tile build error (catch continue) and on a cache-store failure, leaving a tile-shaped NODATA hole with the neighbours' labels leaking over its edges — seen in the field on iOS, invisible in every desktop repro. Count the failures and print which error last struck, so the platform where it happens can say why. A healthy build prints nothing.
Field evidence from iOS shows 0-2 ms 'successful' builds where the Mac takes 200 ms warm — scenes coming out near-empty with nothing failing. An empty tile is legitimate over open ocean, so it is a count, not an error — but it is the first thing to read when tile-shaped NODATA holes appear over charted ground.
A truly-empty (open ocean) tile rebuilds for the cost of one partition classify; a TRANSIENTLY empty one — whatever emptied it — must not become a tile-shaped hole that sticks until eviction. Empties still contribute to the call that built them (their arenas outlive assemble), they just never enter g_geom.
Two bakes of one cell edition carry the same DSID name+date (the date is the cell's, not the bake's), so the ownership tie-break could not order them and fell through to input order — the host's directory enumeration. The same library could render differently on two machines, the stale twin's ground appearing only in the zoom window its older bake carried: cells 'showing up but not at the right zooms'. finishOpen now keeps exactly one of each twin set — the most capable archive by header (widest zoom span, then most addressed tiles, then most tile bytes) — a property of the archives, never of discovery order. Distinct dates are untouched: different editions already supersede cleanly via the newer-first clip order. Dropped twins are closed/unmapped in the owned-archive path and logged. Verified: a 7,217-cell library plus 10 duplicate archives composes to a scene byte-identical (tri hash) to the clean library.
Archives vs distinct cells, multi-edition groups (names + each edition's date and served zoom span), and archives starting above z0 (pre-fill- down bakes serve nothing coarse). Field reports get checked against these instead of inferred.
A Debug-mode zig build instruments C sources with UBSan whose runtime lives in Zig, not in the archive — an app linking libtile57.a then dies with undefined ___ubsan_handle_* symbols. Lua, libtess2 and the SVG rasterizer are vendored, pinned code; sanitizing them buys nothing and breaks external linking, so compile them clean in every mode.
'tile57 engine @ <describe --always --dirty>' — which engine a process ACTUALLY linked, stated by the process itself. Half of today went to a device running a morning engine while everyone reasoned about current code; build-log provenance helps only when the build log is read. The runtime line cannot be skipped.
The cache was bounded by ENTRY COUNT (1024) — gigabytes of tessellated tiles on a long session. On a memory-limited device (iOS jetsam) that grows the process to where allocations FAIL: scene assembly errors on exactly the widest views while smaller ones still fit, the host retries the identical failing build every frame, and the screen keeps a stale band's scene — cells at the wrong zooms, device-only, engine bytes identical to a healthy machine's (field-diagnosed via ok=false verts=0 build lines with warm-cache 7ms failures). - 160 MB byte budget beside the entry cap; LRU eviction honours a generation floor so a walk's own tiles are never evicted from under its parts (their arenas are referenced until assemble copies out). - geomDropHandle keeps the byte ledger; geomDropAll is the pressure valve. - A failed build drops the cache and retries once instead of failing identically forever.
A tile whose ground some cell OWNS but which serves nothing renders as a NODATA hole; renderComposeTileGpuScene now asks the compositor to explain it: one line naming every covering cell, its scale, its tier, and whether its archive has the tile. Legitimate empties (no owner — open ocean) stay silent; the print is budget-capped. This is how a library whose exact cells we cannot see (a field device) tells us which cell swallowed the ground and why.
An empty tile whose ground lies inside some cell's coverage yet is owned by NO face at the governing tier is a partition gap — the exact 'the partition map is wrong' failure — and it printed nothing. Now: UNOWNED-GAP names a witness cell whose coverage contains the tile. True ocean (no coverage at all) stays silent.
The two can DIFFER when the partition algorithm evolved since the bake (a fresh build assigns ground the bake never tiled → owned-but-empty holes), so which one a process used must be a printed fact, not an inference from timing.
faceTileBBox pushed every face ring point through lonLatToWorld — @tan/@cos/@log, i.e. the SYSTEM libm — then @floor'd into tile indices to decide which tiles a face covers. Transcendentals are not exactly rounded and macOS and iOS ship different libm builds: one ulp at a face edge flipped the floor and silently dropped an entire row of tiles from a big face. Whole-tile voids, one OS only, same code, same data, same sidecar — 'works everywhere but iOS'. The bbox is only a cull; the integer classifier (EdgeGrid over e7 coordinates, atan/sinh pure-Zig tile bounds) decides exactly. Widen the cull by one tile per side and the outcome no longer depends on whose libm rounded last. Mac output verified byte-identical (tri hash and PNG); engine tests green. (Also carries inert plumbing for a future cross-band overscale fallback: ownerTile/composeSeamTile take a deep_overscale flag, false at every call site today.)
A tile whose governing band serves NOTHING now walks down the tier stack and composes the best coarser band's chart, overscaled (the paper-chart / ECDIS behaviour the partition docs promised: 'the ground is owned by a coarser band, reached by querying that band's map' — no code ever did the walk). Ground between a band's cells no longer renders void at zooms its own band cannot serve; the US2GRLBC-class UNOWNED-GAPs at the library fit view now fill with water and features from the 1:700k chart instead of NODATA. Whole-tile fallback only — partial-tile band seams are a follow-up. True no-coverage ground (e.g. the Canadian half of Lake Huron) stays NODATA. Engine tests green; deep overscale capped at +8 zooms.
…, tap explains Three pieces, one goal — ground with ANY chart renders at EVERY zoom: - Partial-tile cross-band fill: whatever ground the governing band leaves bare in a tile (a coast crossing, a cell corner, or the whole tile) is filled from coarser bands, clipped to exactly the bare region via the exact integer booleans (never double-draws finer ground), served with deep overscale. Verified: Gulf of Maine ground covered at z7 but NODATA at z4-5 now renders chart at every zoom. - Antimeridian coverage split in toPlaneCells: unwrap rings, cut at +-180 into per-world-copy parts. A wrapped ring read flat is a world-spanning polygon that owns stripes of foreign ground. Changes the partition input key, so stale sidecars self-invalidate. - explainPoint on every cursor pick: tapping the chart logs the serving story of that spot — tile address, owner (or NOBODY) per tier, archive tile presence normal/deep, and what composeTile returns. Tap the hole, read the answer. - TILE57_COMPOSE_DEBUG lines carry the face bbox (mis-identifying cells by cscl cost a wrong theory today).
renderComposeTileGpuScene swallowed all three failure modes (catch {}),
turning a tile the compositor served — 143 KB in the field case — into
a silent empty: compositor provably healthy, scene provably missing the
tile, zero diagnostics in between. TILE LOST lines now name the stage
(compose/decode/replay), the error, and the served byte count.
…f-heal The fill ran exact booleans against RAW tier faces — a tier-0 face is a whole coastal cell, hundreds of thousands of points — and the resulting oversized allocation requests FAILED tile-by-tile on a memory-limited device (~1 GB footprint, so not the jetsam ceiling: single huge requests), at exactly the coarse zooms. TILE LOST (OutOfMemory) named it in the field on the first instrumented run. - rectClipRings: linear Sutherland–Hodgman cut of every boolean operand to the tile's classify box, so per-tile cost is bounded by the tile, never the face. - A per-tile OutOfMemory drops the tile-geometry cache and retries once (the whole-scene valve never saw per-tile failures). Mac output verified unchanged (verts/tri stable at the measured views); engine tests green.
…iles geomDropAll mid-walk freed cached scenes the walk's parts still reference; assemble then memcpy'd from freed arenas (device crash). geomDropCold honours the walk generation floor — everything colder is reclaimed, the walk's own tiles survive. The whole-scene valve keeps geomDropAll (it runs between walks).
…e midpoint fix - Every contribution to a composed tile — governing faces included — is rect-clipped to the tile BEFORE classify/projection/booleans, so per-tile time and memory are bounded by the tile, never by the face. Whole-cell faces at coarse tiers (hundreds of thousands of points) were projected wholesale per tile: per-tile arenas grew by hundreds of MB and their doubling requests failed on a memory-limited device — the field's persistent 'compose FAILED (OutOfMemory)'. composeSeamTile now takes one uniform contribution list (region + overscale flag). - Partition FORMAT_VERSION 3 and a documented rule: the version is the ALGORITHM generation. A sidecar carries bake-time faces VERBATIM and the input key never validates them — a field device rendered a Great Lakes cell owning Gulf-of-Mexico ground from exactly such a relic. Old sidecars now rebuild fresh and rewrite themselves. - Bake: archive-center midpoints computed in i64 — two e7 longitudes near the antimeridian overflowed i32 and panicked the bake.
'The bake is extremely slow' arrived with no way to answer WHERE from a field run. Every bake now prints total cell-work and the ten slowest cells with their source paths. Measured on a 120-cell mixed set: the cost is dominated by a handful of overview/Alaska cells at 20-37 s each (z0 fill-down over huge footprints — inherent work, unchanged since before the session: the whole bake pipeline diff vs then is the 4-line midpoint crash fix).
…ents - A failed shared-GPU-atlas build latched null atlases for the process: every symbol then TESSELLATES (black blobs, quads=0, 5-10x vertices), and the vertex bloat feeds the memory pressure that caused the failed build — a death spiral from one transient OutOfMemory. Failures now print, reclaim cold cache (walk-safe), and retry next scene. - composeSeamTile decodes each contributor in a resettable sub-arena and deep-dupes the clipped survivors' borrowed properties, so a many-cell coarse tile no longer holds every contributor's decode until encode. - Measured: with a LOADED partition, the worst coarse tiles compose in a few-hundred-MB envelope; the 2.5 GB peak that killed devices is the IN-MEMORY PARTITION BUILD over 7k cells — which the v3 sidecar gate now forces exactly once on devices with stale sidecars. Ship the bake's fresh sidecar with the library to avoid on-device builds.
…ment valve Moving the residual boolean chain between arenas cannot reduce PEAK — the chain accumulates hundreds of intermediate polygon sets before the fill ends. Ping-pong: every 16 steps the small surviving residual is copied to a drained twin arena and the other resets, bounding the chain to ~16 steps of scratch. Measured on the tiles that OOM'd every time on device: worst-tile process peak 2.5 GB -> ~360 MB (open included), compose transient now tens of MB, served bytes identical. TILE57_NO_FILL=1 disables the fill for A/B measurement.
Drops the per-tile GPU geometry pool (the largest reclaimable cache). Call between scene builds; lookout wires it to iOS memory warnings.
…er path A field CPU profile put ~2/3 of the render path's cycles in stbi_zlib_compress: the shared GPU-scene atlas build ran the FULL bake — composite the device-density pixel plane, then PNG-compress it — while its only consumer reads cell rects and dimensions. spriteMlnOpts want_pixels=false stops at layout; the bake path is unchanged. Every shared-atlas (re)build also logs itself with ratio and duration, so a rebuild loop can never again hide inside scene builds (measured: 2.16x build now 1.35 s once).
…nded scratch Three structural fixes from a malloc_history audit of the running app (652MB -> 230MB sim footprint at open, 1274MB -> ~780MB peak on the fattest Great Lakes sweep, tri hash byte-identical throughout): - gpa is now c_allocator everywhere, not smp_allocator: smp's per-CPU slab freelists never return pages to the OS, so the app's footprint ratcheted to the worst compose burst forever (and the pages were invisible to Instruments). libc malloc unmaps large blocks on free. Bake (ReleaseFast) is not slower: 39s vs the 54s smp baseline. - pmtiles.Reader decodes its root directory LAZILY on the first tile probe: a composed library opens thousands of archives, most never touched in a session; the eager decode cost ~80MB and seconds of open. Directory gunzip now runs through the arena's child allocator and is freed after decode, root and leaves both, so reader arenas hold only the decoded entries. - coverage decode takes an explicit scratch allocator: the gunzipped metadata JSON text and std.json parser scratch were retained in the coverage/compose arenas (~50MB across a full library); only the compact rings survive now. - GpuSurface.build and gpu.assemble grew their vertex/index/quad/range ArrayLists directly in the DESTINATION arena, stranding every outgrown copy — a cached tile scene (and the live view scene) paid ~2x its real bytes. Lists grow in scratch; final slices are duped.
The host measured its frame cap at coastal zooms as ENCODE cost: thousands of ranges, each one draw + one uniform write, because colour lived in the range. Three changes let contiguous ranges collapse: - tile57_gpu_vertex grows a straight-alpha RGBA (24 -> 28 B, in what was padding + 4). Every triangle vertex carries its op's resolved colour; Range.color remains as advisory metadata. ABI note in the header + layout test: a Metal host must read this struct with packed_float2 members or the 28-byte stride pads to 32 and every vertex after the first is garbage. - coalesce() no longer splits on colour (triangles carry it per-vertex now; quads always did): adjacent same-key ranges merge at build time. - assemble() re-lays the index and quad streams IN SORTED RANGE ORDER (stable paint_key sort), so ranges that draw identically sit contiguously and a host can merge whole paint bands into single calls. One extra linear copy on the build thread. Reference view: 4584 ranges -> 133 draw calls, rendered pixel-identical.
A host compiled against a newer tile57.h than the library it links renders garbage (sheared vertex stream: wrong colours, junk triangles, single-digit fps) with no error anywhere — exactly what a stale checkout produces the day the vertex struct grows. tile57_abi_gpu_layout() returns the packed sizeofs; a checking host turns skew into a refusal, and a new host against an old library dies at link, which is better still.
Adds the data a host needs to draw S-52 fills with hidden-surface removal instead of back-to-front overpainting (measured 8-18ms GPU on fat coastal views at phone density — pure overdraw of stacked opaque fills): - Vertex.depth + Quad.depth (f32, paint-order: LATER paint = SMALLER = closer), assigned per RANGE by build() and reassigned by assemble() after its global sort. Quads carry it too: linestyle-brick quads ride LOW paint bands and must lose to fills above them — the one content class the first cut missed (bisected to it via a dashed line surfacing through a DEPARE fill). - Range.flags bit 0 = OPAQUE (pattern-less triangles, alpha 255). coalesce() never colour-merges OPAQUE ranges: each keeps its own depth, so overlapping same-band fills of different colours resolve by depth exactly as painter's order did. Blended ranges still colour-merge. Contract: draw OPAQUE ranges front-to-back, depth LESS + write; then everything else in paint order, LESS, no write. Verified pixel-identical against single-phase painter's order on the Great Lakes reference.
The app's water shading NEVER responded to the mariner's contours: the GPU surface's fillArea dropped its DepthRange and pushed the bake-time colour token (baked at fixed defaults, because a baked tile must serve every mariner). The vector/pixel surfaces always re-resolved — which is why the test suite never noticed. A mariner setting a 7 ft safety contour watched exactly nothing happen to the water. - fillArea (gpu) re-resolves depth areas against the LIVE settings via seabedToken, like vector/pixel always did. - S-52 next-deeper snap: replayTile pre-scans the tile's contour ladder (DEPCN valdco + DEPARE drval1) and hands it to render surfaces via a new optional vtable slot; Surface.effectiveSafety picks the least available value >= the mariner's (else the deepest available). Shading, the danger-symbol swap, and the line emphasis all use the SAME snapped value, so the safe/unsafe split always coincides with a contour that exists in the data. - THE safety contour line (S-52 §10.5.5): the depth contour whose valdco matches the effective value draws 2.5x wide and solid on all render surfaces. - seabedToken (and its style-expression twin) normalize the ladder to shallow <= safety <= deep: un-normalized, a safety contour deeper than the deep contour let genuinely UNSAFE water shade in the white deep band. Chesapeake verification: safety 2 vs 20 moves 36.5% of pixels with the split landing on real chart contours. 326/326 tests pass.
zig's fetcher doesn't do git submodules, so a fetched tile57 package had no S-101 Portrayal Catalogue (and .paths shipped only src/). resolveCatalog now prefers the initialized submodule and falls back to a lazy s101_portrayal dependency pinned to the same upstream commit; build.zig.zon lists the real package contents (vendor/ subdir by subdir, keeping the submodules out). Lets lookout-core build libtile57.a from source as an auto-fetched dependency.
Restructure both API references into per-grouping pages (errors-lifecycle, bake, render, compose, assets, style), keeping c-api.md / zig-api.md as overviews under nested sidebar categories. The render pages are broken out per surface (PNG/PDF, host vector callbacks, draw-ready GPU scene, canvas, cross-view labels), and the Zig pages now use C-API-style signature blocks with prose instead of terse tables. - Document the draw-ready GPU scene (single-chart + composed) in both APIs, with a Zig usage walkthrough and a clear chart-opening (io/gpa) setup. - Expose GpuScene and compose.renderGpuScene in the public Zig surface so the C ABI stays a thin shim over it. - Fix the rendering-engine page's paint-order batching advice (it contradicted the corrected guidance) and point it at the GPU scene. - Add a Contributing page and soften the "AI-built" framing to an AI-First note.
The `tile57` CLI failed to cross-compile to x86_64-windows: compose_tile.zig's nowNs() called std.c.clock_gettime, whose clockid_t parameter is `void` on Windows — illegal under the win64 calling convention. Switch to std.Io.Clock.awake.now(io) (CLOCK_MONOTONIC), which is cross-platform and uses the io the command already holds. Windows now builds tile57.exe + tile57.lib.
compose.zig reads two debug-valve env vars (TILE57_COMPOSE_DEBUG, TILE57_NO_FILL)
via std.c.getenv. The shipped lib links libc, but addPkgTest omits it, so the
standalone compose test failed to compile on Linux ("dependency on libc must be
explicitly specified"). Link libc on both compose test binaries, matching what
bundle_test already does.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Everything the engine needed to run inside a long-lived mobile app process. 36 commits over main, in five threads.
GPU-scene ABI. Hosts upload once and draw: per-vertex colour turns whole paint bands into single draws, every vertex and quad carries paint-order depth (the overdraw contract), the sprite pack in the render path is layout-only (no pixels, no PNG decode), and the safety contour gets live shading with next-deeper snap and a bold line. tile57_abi_gpu_layout() exports the packed struct sizes so a host compiled against a newer header refuses a stale library instead of rendering sheared garbage. Includes the label-tier work merged via #33.
Memory. Allocation reworked for a process that never exits: c_allocator over arena-stranded scratch, per-contributor decode arenas, a byte-bounded tile-geometry cache, compose fill chains bounded by ping-pong compaction, per-tile OOM self-healing (plus a use-after-free fix in that valve), and tile57_trim_caches() -- the valve a host pulls on an iOS memory warning.
Composition. Tile-bounded serving end to end; cross-band overscale fallback so there is no void where any chart exists; complete cross-band fill including partial tiles and the antimeridian split; same-(name,date) twin archives collapse deterministically; and tile ownership no longer consults the OS's libm -- the fixed-point math is ours, so every platform partitions identically.
Apple toolchain + packaging. iOS cross-compilation of libtile57.a (sysroot forwarding, iOS strip, system(3) stub for Lua); a macOS nowNs() fix; and the repo is now a fetchable zig package -- real .paths, the S-101 Portrayal Catalogue as a lazy dependency pinned to the same commit as the submodule, and the archive exported as a named lazy path -- so hosts build the engine from source with no manual pre-build.
Diagnosability. A served tile that fails decode/replay/compose screams instead of leaving a silent hole; empty tiles are counted and never cached; explainEmpty names owned-but-empty and unowned ground; TILE57_COMPOSE_DEBUG traces per-tile owner decisions; opens print their composition set and which partition served; tile57_warmup logs the linked engine's commit; bakes profile themselves.
Verified by the unit suite, the pixel-parity goldens, and a whole-US ENC library rendered through the composed path on macOS and the iOS simulator.