Lighting primitives: a colour and brightness track alongside the choreography - #11
Open
yiyixuu wants to merge 23 commits into
Open
Lighting primitives: a colour and brightness track alongside the choreography#11yiyixuu wants to merge 23 commits into
yiyixuu wants to merge 23 commits into
Conversation
Twelve primitives over two orthogonal layers, colour and brightness, composed into a LightingTimeline with three read-outs: per-frame in the sim, baked cues for hardware, and a cue-list payload for the browser viewer. Spatial ordering throughout, so effects follow the formation rather than drone ids.
Merge lighting_primitives.py and lighting_compile.py into lighting.py, so the whole engine — config, selectors, waveforms, spreads, layers, timeline, the emittable primitives and the cue bake — reads top to bottom in one file. Cut the explanatory prose in lighting.py from 288 lines to 94, about the density of motion_primitives.py and backend.py. Docstring summaries and the Args/Returns/Raises blocks stay; the multi-paragraph justifications go, except where the code alone would mislead. Drop every reference to docs/specs and docs/plans across source and tests. Those directories are gitignored, so a fresh clone cannot resolve a "§8.5" or "(spec 7.3)" pointer; where a reference carried meaning it is now stated inline.
Motion primitives play forward from their own key, so a lighting key sharing that address resolved its selectors and spreads at the instant the formation began — freezing the formation the swarm was leaving for the whole look. On Fearless2's two-circle key that made `alternate_side` partition the preceding zig_zag, so a left/right effect read as an arbitrary up/down split. Sample at the end of the motion primitive covering the look instead, clamped to the look's own end. Defining "settled" by the primitive's own window rather than by swarm speed keeps it right for helix, twister and spiral_speed, where the drones never stop moving.
Four `positions` descriptions in lighting.py and the `response2lighting` summary still described the old sampling instant, which is now the one thing about that argument a reader most needs right.
Three tests assert a cos/sin ring is degenerate only to within a couple of ULPs, which is the case the engine's relative span tolerance exists for and one an exactly-equal fixture cannot reach. How much noise cos/sin actually leave is platform-dependent: linux-64 returns a bit-for-bit exact ring where osx-arm64 leaves ~2e-16, so the `0.0 < span` guard held locally and failed in CI. Nudge the largest coordinate of the guarded column two ULPs, via a shared conftest helper, so the fixtures stay degenerate-but-not-exact everywhere. The nudge lands on the largest coordinate rather than a fixed row because two ULPs of a coordinate sitting at zero is a denormal, which moves neither that column's span nor the point's radius.
Rendering a preset meant a `python -c` one-liner carrying two full paths. It now takes flags: a preset name or any substring matching exactly one, --preview for a quick 1280x720/30fps pass, -s to cut the show short, -o to override the output and --list to see what is available. The output path otherwise defaults to renders/<preset-slug>.mp4, so the common case is just the preset. Explicit --width/--height/--fps win over --preview so the two compose. Add a `render` pixi task so the interpreter and environment stay implicit too.
Four modes driving DroneSwarm.apply_colors directly, with no choreographer, axswarm or takeoff in the path: deck orientation, palette parade, generated hue wheel, brightness ramp. The ramp mirrors LightingTimeline.evaluate's b_min floor, bucket quantization and gamma, so the bench shows what the engine would send rather than a raw colour. --dry-run prints the WRGB tables without touching a radio.
The cf1xx entries were padding for the 100-drone demo and no hardware backs them. Leaves the 20 real drones on channels 10/20/30/40, all active.
…ency The mocap path builds a ROSConnector that asserts on an uninitialized rclpy, so the tool died before reaching a radio. Mirrors the guard deploy() already has. Reading drones.toml and settings.yaml directly also keeps the LLM and music-analysis stack out of a bench tool that only needs a URI per drone -- that import chain was what emitted the stray allin1 warning.
Docstrings are capped at three content lines: a summary plus, where the shape or a gotcha genuinely needs saying, a short note. Args/Returns sections go where the signature already says it; Raises sections go throughout. Inline comments that restated the code go, as do the section-divider banners in the test files. Comments explaining why, flagging a constraint or warning about a gotcha stay. Comment and docstring lines drop from 3506 to 1918 across 29 files, 22.9% of the tree to 14.0%. No code changed: every file's AST, with docstrings stripped, is identical to before. Also records the convention under a Commenting style heading in CLAUDE.md.
The 10 m half-extent was sized for the 100-drone sim swarm, which is gone. axswarm enforces these as hard position constraints and the offline collision check clamps against them, so leaving them at sim scale lets a 20-drone helix plan a ring wider than the room.
drone_ids is now one string of 1-indexed ids and inclusive ranges rather than an explicit array, matching DRONE_ID_SPEC_PATTERN and expand_drone_id_spec. The worked examples run 1..20, not 1..100. They previously named drones 31, 45-60 and 100 in the same block that tells the model every id must be in 1..num_drones, which renders as 1..20 for this swarm -- and every out-of-range id costs a reprompt.
The earlier comment pass covered *.py only, so this file kept a paragraph of rationale per constant: 62 comment lines against 15 of data. Drops every spec reference with them. docs/specs/ is gitignored, so the eighteen section numbers in here resolved to nothing for anyone else who clones the repo. The constraints they pointed at are now stated inline where they are load-bearing -- col_freq matching DroneSwarm's, channel_gain applying after the normalization and only to generated hues. Values are byte-identical; this is comments only.
GPT-5.x and o-series reject `temperature` with a 400, so gpt-5.6-luna could not run at all. They expose `reasoning.effort` in its place, which is also the better creativity knob here: choreography variety comes from which primitives land where, not from token-level jitter. Medium effort buys that plan-space exploration without max's latency. Reasoning tokens bill against max_output_tokens, so the 4096 ceiling sized for gpt-4o in dc20941 would truncate mid-think and surface as an empty response rather than a token error. Reasoning models get 32768. Older OpenAI models and Ollama tags keep temperature=0.0 and 4096.
At medium reasoning effort the details panel sat empty for the whole generation, because the prompt was only reachable after the LLM returned: initial_prompt formats and calls in one shot, so the conversation event could not fire any earlier. Both calls now take an on_prompt callback, fired as the request leaves, and the server emits it as prompt_sent. It carries history plus the new prompt, so a refine does not shrink the panel to its two new messages. Reasoning models also summarize their own thinking when asked; the raw tokens are never exposed. The summary rides in its own output item rather than output_text, so it needs pulling out separately and clearing per generation -- Ollama's native path produces none and would otherwise show the last model's. It renders above the response, not above the prompt. Note: requesting a summary may need a verified OpenAI org. If it 400s, drop "summary" from responses_model_kwargs.
…for top/bottom deck. also added more documentation and comments on design decisions of the lighting primitives
`stage_axis` was "+x", the direction the audience views *from*, so `left`, `right` and `alternate_side` split the swarm along depth instead of across the stage: a stage-left effect read as nothing at all. With the audience at +x and the show centred on x = 0, their right hand points along +y. Two things follow from fixing it. The camera's end pose was on -y, straight down the new left/right axis, which mirrors those effects in frame; it now stops at 45 degrees so the move stays inside the audience arc. And `_neighbour_ranks` sorts x first, so a chase enters from the furthest-upstage drone and comes forward -- correct, but world-frame and out of `stage_axis`'s reach, which is worth saying where the code is rather than leaving to be rediscovered.
The palette could not reach a desaturated warm or cool, and had no way to change colour over time, so a plan asking for gold or for a colour arriving with a phrase had to settle for amber snapping on the beat. Measured over 13 songs, those two were the largest single class of thing the system could not deliver: gold named 47 times, silver 15, violet 11. Values are derived the way `hue_to_wrgb` derives them -- normalize to a constant channel sum, then apply `channel_gain` -- and reproduce the shipped azure, blue, indigo and amber exactly, so the new entries match the wheel in apparent brightness. `violet` is a pure hue at 280. `gold` and `silver` are the first entries mixing the white LED with a hue, which `rainbow` cannot reach and `gradient` washes through; both facts are stated where an author will meet them. The white-mix fractions are chosen rather than measured and flagged UNMEASURED: run tools/calibrate_lighting.py before trusting them on hardware. `fade(sel, color_a, color_b, duration_beats, deck)` is the time counterpart of `gradient`, which is spatial and frozen. Layers evaluate at absolute show time, so a one-shot cannot locate itself; the build context now carries the look's own start. It holds color_b past the end rather than looping, because a look persists until the next key, and clamps below a cue tick where it would quantize to a cut.
The details panel only ever showed the attempt that survived every check, and only once it had. It is now a transcript the backend appends to as things happen: the prompt as it leaves, the model's reasoning summary, the raw answer as it lands, and the checker's rejection with the retry it triggers. `AppBackend.on_event` replaces the `on_prompt` callback, so `self_correct`'s reprompts report themselves too. - blink: widen a sub-tick lit window instead of leaving it to alias. A 0.1 duty over a period that is no multiple of the cue tick was lit on barely a third of its periods, at irregular times. The period is left alone -- blink runs at one shared phase, so stretching it would only move the stab off its beat. - chase: split into two schema variants so group_size can only pair with a spread that ranks the drones. The engine already rejected the other pairing, one whole generation round-trip after the schema accepted it. - even/odd select on the parity of the 1-indexed id, like every other way of naming a drone. `even` was the exact complement of `ids([2, 4, 6])`. - The structured-output prompt block now lists upper|lower alongside the rest. - sel fields are checked, not indexed: a bare KeyError escaped the reprompt path. - Trails are skipped entirely while TRAIL_RGBA is transparent, which it ships as.
ratheron
self-requested a review
August 24, 2026 09:59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a lighting track to the choreography the LLM authors: 12 primitives over colour and
brightness, on the same
s<seg>b<bar>t<beat>address space the motion track uses, compiled intoper-drone cue lists the Crazyflies replay from their own LED decks.
What is here
swarm_gpt/core/lighting.py— the engine. Looks compile to layers; colour layers resolveagainst a calibrated palette and brightness layers modulate them, with a documented resolution
order so two overlapping effects on the same drones is defined rather than accidental.
light_color,gradient,rainbow,light_on,light_off,pulse,blink,strobe_decay,chase,sweep,ripple_light,alternate_blink. Selectors coverall/ids/even/odd/first/left/right/upper/lower, and every primitive takes a deckso a drone can read two-tone.
swarm_gpt/data/lighting.toml— palette and calibration constants as data, withtools/calibrate_lighting.pyto measure them on the bench. Values still taken on trust areflagged
UNMEASUREDin the file rather than presented as verified.the engine's own tables, so a primitive added without a schema entry fails at schema-build time
instead of becoming a
KeyErrorat compile time.the quantization that will actually fly.
render.py— a scripted camera flythrough with audio, for recording a preset.Notes for review
build_lookconverts them with the song's BPM and clampsagainst the cue rate so a fast effect cannot alias.
the key falls, which is what keeps selectors like
upper/lowerresolving against the shape thelook was written for.
stage_axisis+y: the audience views from+x, so their right hand points along+y. Thelast commit fixes this and the camera end pose that mirrored it.
fade(sel, color_a, color_b, duration_beats, deck)is the time counterpart ofgradient, whichstays spatial and frozen.
gold,silverandvioletare added; gold and silver are the firstentries mixing the white LED with a hue, so
rainbowcannot reach them andgradientinterpolates through W as a wash. Their white-mix fractions are chosen rather than measured and
flagged UNMEASURED — run
tools/calibrate_lighting.pybefore trusting them on hardware.drones.tomlis left as-is on this branch.