diff --git a/README.md b/README.md
index c4d1345..6a0177a 100644
--- a/README.md
+++ b/README.md
@@ -10,6 +10,18 @@ theDAW also ships the first non-Mac port of Google's Magenta RealTime 2, which r
[User Guide](docs/USER_GUIDE.md) · [Windows Setup](docs/windows/setup-guide.md) · [Prompting](docs/guides/prompting.md)
+
+
+ 
+
+
+ ▶ Click to watch the full feature tour — also available vertical (9:16)
+
+
+
+ 
+
+
---
## Repository Structure
@@ -83,33 +95,75 @@ ARC checkpoints are post-trained for 8-step inference at `cfg_scale=1`. RF check
Every feature has a full reference in the [User Guide](docs/USER_GUIDE.md). This is the index.
### MAKE generation
+
+
+
One form drives text-to-audio, audio-to-audio, inpainting, and continuation. Supplied init audio, a text prompt, a painted inpaint region, and a Chimera stack all condition the same generation. The init audio accepts a voice recording from the built-in microphone recorder, an imported audio file, a clip from the media bucket or library, or a pattern rendered to a clip from the piano roll or step sequencer, and the init noise level sets how far the result departs from it. Controls cover duration, batch size, sampler steps, CFG, one-click seed reroll, and init noise level, and the model selector adjusts steps and CFG for RF variants on its own. Chimera blends several clips into one generation and beat-aligns them to a target tempo, automatic or fixed, under Start, Downbeat, or Phrase Weave alignment. Inpainting marks a region on a WaveSurfer preview. The Advanced Generation Panel holds output settings, and Quick Actions route a render to the editor, the init slot, or the inpaint module. Templates store full parameter sets, Saved Prompts keep a history, and the magic-prompt and sparkles buttons seed and optimize prompt text. The Spectrogram Viewer renders Mel, STFT, Chromagram, and CQT. The async job queue polls `/api/jobs/{id}`, supports a binary abort, and saves every render to the library.
### Generate, cloud and real-time
+
+
+
Suno cloud generation runs in the Aurora Cloud Console across simple, custom, cover, and mashup modes. The server-side key stays on the backend, finished tracks become library entries, and cover and mashup results write lineage edges. Magenta RealTime 2 provides text-to-music whenever its sidecar is running. theDAW vendors the first non-Mac MRT2 port in `sidecars/magenta-rt2-nvidia`, which removes the upstream macOS-only CMake guard so the model runs on Windows with WSL2 and NVIDIA, on native Linux, and on cloud GPUs. The extended sidecar also accepts MIDI-note and audio-style conditioning.
### EDIT multi-track editor
+
+
+
The timeline holds many tracks, and each clip computes and caches its own waveform peaks. Move drags clips along the timeline and between tracks, and Cut splits a clip while preserving source alignment. The snap grid offers Off, 1/4, 1/8, and 1/16 divisions, and zoom spans 5 to 400 px/s. Each track carries an editable name, mute, exclusive solo, volume, pan, and removal, and the live mixer applies those faders, pan, mute, and solo during playback. Each clip exposes left and right trim handles and fade-in and fade-out handles. Inpaint from editor crops the visible region and sends it to the backend. Commit Edit renders the audible tracks into one 44.1 kHz stereo WAV through `OfflineAudioContext`, applying fades, volume, and stereo pan, then saves and downloads the result. The status bar shows timecode and clip and track counts, and Preview auditions the selected clip.
### MIX effects, mastering, and the Edit Tool Stack
+
+
+
A chain of 24 FFmpeg effects covers a mastering chain, compression, highpass and lowpass filters, volume, tempo, vocal processing, lo-fi vinyl, stereo widening, reverb, delay, a sub exciter, phase isolation, parametric mid EQ, LUFS normalization, pitch shift, echo, fade, declick, silence removal, denoise, and export to FLAC, MP3, AAC, and Opus. Four macro sliders (Drive, Width, Air, Punch) map onto the active effect. Process history keeps the last eight runs, and any prior output promotes back to the current source. The Edit Tool Stack adds six module families under `/api/edit/*` (mastering, restoration, enhance, delivery, creative-fx, creative-neural) whose GUIs iframe into the effect stage.
### DJ performance console
+
+
+
Two decks run from a pro layout with jog wheels, a central mixer, scrolling waveform overviews, and a track browser, loading from the library, a saved set, or an online import. The engine handles octave-aware beatmatch sync with a continuous lock, key-lock, a 3-band EQ, a single-knob filter, and channel trim with auto-gain. Performance controls cover four hotcues, beat loops, momentary loop rolls, slip mode, beat jumps, and quantize. The FX rack adds a flanger, an impulse-response reverb, and a resonant wah per deck, with a master limiter on the DJ bus. Live stems ride on per-stem faders, and cue output pre-listens a deck through a headphone device chosen with `setSinkId`. Automix sequences and crossfades the set on its own, a ten-pad sampler bank fires one-shots, and a Next staging lane queues upcoming tracks. Design Mode turns the console into a hand-arranged layout that persists and exports.
### VJ visual engine
+
+
+
A 3D reactive visualizer renders a glowing spectrogram terrain with bloom, particles, fog, and shader effects, with several camera flight modes and color themes. The signal comes from the session audio, a microphone, MIDI, or a set sent from the DJ tab, and the visualizer pops out into a floating window for a second screen. The camera input accepts any camera the browser can open. It captures a local webcam or capture card, a phone or tablet camera on the same Wi-Fi through the LAN URL and QR code, and off-network devices including Quest 3 headsets through the Mobile Access External URL override (a Cloudflare tunnel or other public URL). HDMI capture cards, DSLRs in webcam mode, and virtual cameras all appear as selectable sources.
### TRAIN LoRA
+
+
+
Eight adapter types are available: `lora`, `dora-rows`, `dora-cols`, `bora`, and their `-xs` variants. Layer filtering runs through `--include` and `--exclude` with bracket-range expansion such as `layers[0-11]`. Inference exposes runtime strength, per-LoRA interval gating within a sigma range, and a per-LoRA layer filter, and adapters stack additively. Pre-computed SVD bases (`--svd_bases_path`) speed startup for the `-XS` variants, and `--base_precision bf16` lowers VRAM use. Some training endpoints return HTTP 501 today, and the UI shows a specific message for that status.
### LEARN genealogy graph and visualizations
+
+
+ 
+ 
+
+
Every track and the relationships between them render as an interactive force-directed graph in 3D and 2D through `react-force-graph` and three.js, alongside a layered SVG DAG. Edges trace how a piece descended from its sources, so a remix, an inpaint, a stem split, a Chimera blend, and a Suno cover each show their parentage. The camera flies through the graph, focuses a node, and opens any track from its node. theDAW carries other rich visualizations alongside it: the four-mode spectrogram viewer, the real-time spectral analyzer with oscilloscope, spectrum, and radial modes and RMS and peak meters, wavesurfer.js waveforms, a three.js and GLSL cymatics visualizer, and the DJ jog wheels and beatgrids.
### Library and Catalogue
+
+
+ 
+ 
+
+
The library lives on the backend, with audio on disk, metadata in `data/library.db`, and access over `/api/library/*`. Every render saves automatically with its prompt, model, duration, steps, CFG, seed, MIME type, and timestamp. List and grid views, full-text search, a favorites filter, and sorting by newest, duration, or title organize the collection, and each row plays inline with download, delete, favorite, and send-to-editor actions plus a details panel. The Catalogue view adds a cross-provider gallery with provider badges, an inspector with on-demand spectrograms, and a lineage panel, and it runs Suno cover and mashup from any entry.
### Bottom panel tools
+
+
+ 
+ 
+
+ 
+ 
+
+
The spectral analyzer shows oscilloscope, spectrum, and radial modes with RMS and peak meters, a LIVE indicator, and a fullscreen toggle. The piano roll edits MIDI-style notes, imports and exports MIDI, and renders to the editor. The step sequencer runs a 16-step drum machine at 40 to 240 BPM with five synthesized voices, random fill, and render-to-editor. The media bucket holds session audio in WAV, MP3, FLAC, OGG, AAC, M4A, and Opus. SLIDE presents a glass surface of faders and knobs synced with the VJ engine and the audio. Details shows the selected library entry.
### MIDI mapping and Controller Vision
@@ -165,6 +219,26 @@ Runtime LoRA strength changes through `set_lora_strength(model, 0.5, lora_index=
---
+## Credits
+
+theDAW was built by **[GANTASMO](https://github.com/gantasmo)** as part of the [Music Hackspace](https://musichackspace.org) Music Technology Hackathon at [Berklee College of Music](https://www.berklee.edu).
+
+Special thanks to [Music Hackspace](https://musichackspace.org), [Berklee College of Music](https://www.berklee.edu), and to Zack, CJ, Jordi, Zach, and Matt from [Stability AI](https://stability.ai) for their continued help and support.
+
+### Built with
+
+- **[Stability AI](https://stability.ai)** provides Stable Audio 3 and [stable-audio-tools](https://github.com/Stability-AI/stable-audio-tools), the diffusion model and pipeline at the core of theDAW.
+- **[Magenta](https://github.com/magenta)** RealTime by **[Google DeepMind](https://deepmind.google)** brings real-time music generation, running through theDAW's own [NVIDIA/CUDA port](sidecars/magenta-rt2-nvidia/), the first and only non-Mac port so far.
+- **[Suno](https://suno.com)** powers cloud music generation.
+- **[T5Gemma](https://huggingface.co/google/t5gemma-b-b-ul2)** by Google handles text conditioning.
+- **[Demucs](https://github.com/facebookresearch/demucs)** by Meta AI handles stem separation, and **[basic-pitch](https://github.com/spotify/basic-pitch)** by Spotify handles audio-to-MIDI transcription.
+- **[MLX](https://github.com/ml-explore/mlx)** by Apple is the inference core the Magenta port builds on, extended here with a CUDA backend.
+- **[PyTorch](https://pytorch.org)**, **[FFmpeg](https://ffmpeg.org)**, **[three.js](https://threejs.org)**, **[react-force-graph](https://github.com/vasturiano/react-force-graph)**, **[WaveSurfer.js](https://wavesurfer.xyz)**, **[React](https://react.dev)**, **[Vite](https://vitejs.dev)**, and **[Tailwind CSS](https://tailwindcss.com)** carry the rest, alongside the wider open-source community.
+
+Corrections and additions to this list are welcome through a GitHub issue.
+
+---
+
## License
Commercial use of these models is governed by the [Stability AI Community License](https://stability.ai/license).
diff --git a/docs/USER_GUIDE.md b/docs/USER_GUIDE.md
index c74cb5b..1b42a31 100644
--- a/docs/USER_GUIDE.md
+++ b/docs/USER_GUIDE.md
@@ -386,6 +386,8 @@ During the render, the COMMIT EDIT button shows an animated spinner and is disab
The MIX tab (`MixPanel`) is the single-screen effects and mastering workspace. It applies one or more of 24 FFmpeg-backed processors to a source file as an ordered chain. Processing is synchronous; the result is returned as binary audio and made available for inline playback, download, and routing back into the app.
+
+
### 8.1 Layout
MIX fills the viewport with no page scroll, in three bands:
@@ -399,6 +401,8 @@ The player footer is the process-chain transport.
Quick Master exposes the four parameters of a `mastering_chain` entry as live knobs (low boost, high boost, limiter ceiling, target LUFS). Applying it inserts a single `mastering_chain` entry into the chain once, then live-updates that entry's parameters as the knobs move.
+
+
### 8.3 Effect Catalog and Chain
The effect library is organized into categories (mastering, dynamics, EQ, tempo, cleanup, export) with list and tile view modes and color-coded groups. Add effects to the chain, toggle individual rows active or inactive, reorder rows, and clear the chain in one action. Each chain entry exposes bounded sliders and numeric fields sourced from `PARAM_BOUNDS` and `EFFECT_CATALOG`.
@@ -452,6 +456,8 @@ The last processing invocations are retained in the store. Any history item can
The DJ tab (`DJView`) is a live two-deck performance mixer built on the SLIDE control surface. It loads tracks from the Library or any source, beatmatches and key-locks them, mixes through a full center console, and hands sets off to the VJ for synchronized visuals. The Web Audio engine (`djEngine.ts`) runs the signal path per deck: source, optional time-stretch, delay compensation, trim, three-band EQ, filter, gain and crossfade, the DJ master bus, then the shared player master.
+
+
### 9.1 Waveform Hero and Decks
The two deck waveforms span the full width at the top, each with a beatgrid, hotcue markers, and a moving playhead. A thin per-deck header shows the title, detected BPM, musical key and Camelot value, and the SYNC control. Below the hero, Deck A and Deck B flank the center mixer.
@@ -464,6 +470,8 @@ Each deck provides play and cue, four hotcues, loops with loop-roll and slip mod
The mixer is the console between the decks. Each channel has a gain knob, a three-band EQ (high, mid, low), a single-knob filter that sweeps low-pass below center and high-pass above center, and a volume fader. The two pitch faders sit as the outer columns of the mixer. A crossfader runs along the bottom, and the toggle row (Quantize, Auto-gain, Limiter, MIDI) sits above the gain knobs. A master limiter protects the DJ master bus.
+
+
### 9.4 Live Stems and FX
Each deck has an FX and stems rack. Live stem faders ride the separated drums, bass, other, and vocals derived from Demucs, so a part can be pulled out or pushed up during playback. The FX rack adds per-deck effects on top of the EQ and filter.
@@ -1655,6 +1663,8 @@ Two offline helpers under `scripts/` bulk-import an existing SunoHarvester cache
The `magenta` module (`/api/magenta`) brings Google's **Magenta RealTime 2 (MRT2)** real-time music model into the Generate workspace as a text→music option. The model option appears only when the sidecar is reachable: the MAKE/Generate panel probes `GET /api/magenta/probe` and shows **"Magenta RT2 (text→music)"** when available.
+
+
### 27.1 The sidecar and conditioning
MRT2 runs as a sidecar (default `http://localhost:8777`, override with `STABLEDAW_MAGENTA_URL`) that loads `MagentaRT2Jax` once on the GPU. theDAW ships an **extended** sidecar (`sidecars/magenta/server.py`) that supersedes the upstream text-only studio server and exposes all three conditioning modes over one `POST /generate`:
@@ -1764,4 +1774,14 @@ Operational endpoints used by the Settings modal and shell.
---
+## Credits
+
+theDAW was built by **[GANTASMO](https://github.com/gantasmo)** as part of the [Music Hackspace](https://musichackspace.org) Music Technology Hackathon at [Berklee College of Music](https://www.berklee.edu).
+
+Special thanks to [Music Hackspace](https://musichackspace.org), [Berklee College of Music](https://www.berklee.edu), and to Zack, CJ, Jordi, Zach, and Matt from [Stability AI](https://stability.ai) for their continued help and support.
+
+**Built with:** [Stability AI](https://stability.ai) Stable Audio 3 and [stable-audio-tools](https://github.com/Stability-AI/stable-audio-tools) (the core diffusion model and pipeline); [Magenta](https://github.com/magenta) RealTime by [Google DeepMind](https://deepmind.google), running through theDAW's own [NVIDIA/CUDA port](../sidecars/magenta-rt2-nvidia/) (the first and only non-Mac port so far); [Suno](https://suno.com) (cloud generation); [T5Gemma](https://huggingface.co/google/t5gemma-b-b-ul2) by Google (text conditioning); [Demucs](https://github.com/facebookresearch/demucs) by Meta AI (stem separation); [basic-pitch](https://github.com/spotify/basic-pitch) by Spotify (audio-to-MIDI); [MLX](https://github.com/ml-explore/mlx) by Apple (the Magenta port's inference core, extended with a CUDA backend); and [PyTorch](https://pytorch.org), [FFmpeg](https://ffmpeg.org), [three.js](https://threejs.org), [react-force-graph](https://github.com/vasturiano/react-force-graph), [WaveSurfer.js](https://wavesurfer.xyz), [React](https://react.dev), [Vite](https://vitejs.dev), and [Tailwind CSS](https://tailwindcss.com), alongside the wider open-source community.
+
+---
+
*Maintained by the theDAW development team.*
diff --git a/docs/readme/showcase-poster.png b/docs/readme/showcase-poster.png
new file mode 100644
index 0000000..5593b37
Binary files /dev/null and b/docs/readme/showcase-poster.png differ
diff --git a/frontend/_capture_clips.mjs b/frontend/_capture_clips.mjs
index c1c4c5c..9ddaaeb 100644
--- a/frontend/_capture_clips.mjs
+++ b/frontend/_capture_clips.mjs
@@ -1,9 +1,10 @@
-// theDAW launch-video capture — ONE warm session.
+// theDAW launch-video capture — ONE warm session, built around ONE song.
//
-// Boots the app ONCE in a single headed (real-GPU) page, loads the hero audio /
-// stems / decks ONCE, then clicks + drives the Zustand stores tab-by-tab through
-// every scene while recording one continuous video. Each scene's stable hold is
-// timestamped, and the long recording is sliced into per-scene clips afterward.
+// Boots the app ONCE in a single headed (real-GPU) page, loads "Et Tu Machina"
+// (the hero song — imported, stem-separated, and MIDI-converted ahead of time),
+// then clicks + drives the Zustand stores tab-by-tab through every scene while
+// recording one continuous video. Each scene's stable hold is timestamped, and
+// the long recording is sliced into per-scene clips afterward.
//
// Why one session: spawning a fresh context per shot re-runs the boot splash and
// congests the single-worker backend (the splash then bleeds into the clip). With
@@ -19,24 +20,36 @@ import path from 'node:path';
import fs from 'node:fs';
const APP = 'http://localhost:5173';
-const OUT = path.resolve(process.cwd(), '..', 'showcase', 'clips-recorded');
+const ROOT = path.resolve(process.cwd(), '..');
+const OUT = path.resolve(ROOT, 'showcase', 'clips-recorded');
fs.mkdirSync(OUT, { recursive: true });
+// "Et Tu Machina" — the song the whole film is built around. Imported to the
+// library, separated into 4 stems, and converted to MIDI before capture, so the
+// EDIT timeline, MIX source, DJ deck, and piano roll all carry the same track.
const HERO = {
- heroUrl: '/api/library/audio/62a72211-bfa6-490b-8a06-b87031578b67_00',
- heroLabel: 'Dark-Wave Outrun Dubstep',
- heroId: '62a72211-bfa6-490b-8a06-b87031578b67_00',
+ heroUrl: '/api/library/audio/68006988e370427d9108e5c5d724a9f5',
+ heroLabel: 'Et Tu Machina',
+ heroId: '68006988e370427d9108e5c5d724a9f5',
};
-const DECK_B = { url: '/api/library/audio/81a3d137-2b6f-44fe-9e49-0f540d6ec3fc_00', label: 'Dark-Wave II' };
-const STEM_BASE = '2d196d72-edc4-4ab2-bccc-35721ad73fe0_00';
-const STEMS = ['drums', 'bass', 'vocals', 'guitar', 'piano', 'other'].map((n) => ({
+const DECK_B = { url: '/api/library/audio/81a3d137-2b6f-44fe-9e49-0f540d6ec3fc_00', label: 'Deck B' };
+const STEM_BASE = '68006988e370427d9108e5c5d724a9f5';
+const STEMS = ['drums', 'bass', 'vocals', 'other'].map((n) => ({
name: n, url: `/api/library/stems/${STEM_BASE}__${n}/audio`,
}));
-
-// Full coverage: every tab + every feature. Bottom-panel features are MAXIMIZED so
-// the feature fills the work area (the MAKE chrome + chimera orbs are hidden), and
-// the library is collapsed unless the scene is the catalogue itself. Lineage goes
-// fullscreen. Order groups by tab so the warm page transitions cleanly.
+// Et Tu Machina's MIDI, windowed to a clean 64-step piano-roll pattern (see _make_pianoroll.py).
+let ETU_PIANO = { bpm: 120, totalSteps: 64, notes: [] };
+try { ETU_PIANO = JSON.parse(fs.readFileSync(path.resolve(ROOT, 'showcase', '_etu_pianoroll.json'), 'utf8')); } catch (e) {}
+// The Magenta RT2 NVIDIA port — its standalone Studio UI + a repo/readme card (both static, file://).
+const fileUrl = (p) => 'file:///' + path.resolve(ROOT, p).replace(/\\/g, '/');
+const MAGENTA_STUDIO = fileUrl('sidecars/magenta-rt2-nvidia/port/oneclick/studio/index.html');
+const MAGENTA_CARD = fileUrl('showcase/_magenta_card.html');
+
+// Full coverage: every tab + every feature, with Et Tu Machina present throughout so
+// the film reads as one song being made. Bottom-panel features are MAXIMIZED so the
+// feature fills the work area; the library is collapsed unless the scene is the
+// catalogue itself. Lineage goes fullscreen. Order groups by tab so the warm page
+// transitions cleanly. nav-scenes (file://) navigate away and run strictly LAST.
const SCENES = [
// ── MAKE: generation + every sub-feature ──
{ id: '08_make', tab: 'make', play: true, hold: 6 },
@@ -46,18 +59,26 @@ const SCENES = [
{ id: '27_lora', tab: 'make', play: false, lora: true, hold: 6 },
{ id: '30_compare', tab: 'make', play: false, compare: true, hold: 6 },
{ id: '45_saved-prompts', tab: 'make', play: false, savedPrompts: true, hold: 6 },
- // ── EDIT: the multitrack editor in depth ──
+ { id: '58_spectrogram', tab: 'make', play: true, spectro: true, hold: 14 },
+ // ── cymatics + ferrofluid: ONE long fullscreen shot cycling all 4 modes ──
+ { id: '50_cymatics', tab: 'make', play: true, vizCycle: true, vizFull: true, hold: 26 },
+ // ── EDIT: the multitrack arrangement in depth (chopped / faded / staggered / mixed sources) ──
{ id: '03_edit-stems', tab: 'edit', play: true, buildStems: true, hold: 6 },
{ id: '31_edit-mix', tab: 'edit', play: true, buildStems: true, editMix: true, hold: 6 },
{ id: '22_cut-edit', tab: 'edit', play: false, buildStems: true, cutEdit: true, hold: 6 },
{ id: '40_inpaint-region', tab: 'edit', play: false, buildStems: true, inpaintRegion: true, hold: 6 },
{ id: '41_delete-clip', tab: 'edit', play: false, buildStems: true, deleteClip: true, hold: 6 },
- // ── MIX: effects browser + populated chain + the drag-to-arrange UI editor ──
+ { id: '59_commit-edit', tab: 'edit', play: false, buildStems: true, commitEdit: true, hold: 7 },
+ // ── MIX: effects browser + populated chain + EVERY effect on the real clip + the UI editor ──
{ id: '07_mix-effects', tab: 'mix', play: true, studioSource: true, hold: 6 },
{ id: '32_mix-chain', tab: 'mix', play: true, studioSource: true, mixChain: true, hold: 6 },
- { id: '39_design-mode', tab: 'mix', play: false, studioSource: true, designMode: true, hold: 7 },
- // ── DJ: console + sampler/side-list staged ──
+ { id: '57_mix-all-effects',tab: 'mix', play: true, studioSource: true, mixAll: true, hold: 13 },
+ { id: '39_design-mode', tab: 'mix', play: false, studioSource: true, designMode: true, designDrag: true, hold: 9 },
+ // ── DJ: console + performance (cue/loop/slip/fx) + automix + live stems + sampler ──
{ id: '02_dj-console', tab: 'dj', play: true, djDecks: true, levelAnim: true, hold: 7 },
+ { id: '54_dj-perform', tab: 'dj', play: true, djDecks: true, djPerf: true, hold: 12 },
+ { id: '55_dj-automix', tab: 'dj', play: true, djDecks: true, djAutomix: true, hold: 11 },
+ { id: '56_dj-stems', tab: 'dj', play: true, djDecks: true, djStems: true, hold: 9 },
{ id: '33_dj-sampler', tab: 'dj', play: true, djDecks: true, djSampler: true, hold: 6 },
// ── VJ / TRAIN ──
{ id: '19_vj-visualizer', tab: 'vj', play: true, vjClip: true, vjTour: true, hold: 12 },
@@ -84,12 +105,32 @@ const SCENES = [
{ id: '42_lib-actions', tab: 'make', play: false, catalogue: true, libContext: true, hold: 6 },
{ id: '43_stems-modal', tab: 'make', play: false, catalogue: true, stemsModal: true, hold: 6 },
{ id: '10_suno-cloud', tab: 'make', play: false, sunoModel: true, hold: 6 },
+ { id: '60_magenta-make', tab: 'make', play: false, magentaMake: true, hold: 6 },
{ id: '36_log', tab: 'make', play: true, logPanel: true, hold: 5 },
{ id: '37_docs', tab: 'make', play: false, docsModal: true, hold: 6 },
{ id: '38_settings', tab: 'make', play: false, settingsModal: true, hold: 6 },
+ // ── feature-coverage gap fills (from the showcase doc) ──
+ { id: '64_node-inspector', tab: 'learn', play: false, learn: '2d', lineageFull: true, nodeInspect: true, hold: 8 },
+ { id: '66_module-gui', tab: 'mix', play: true, studioSource: true, moduleGui: true, hold: 9 },
+ { id: '68_audio-to-midi', tab: 'make', play: false, catalogue: true, audioToMidi: true, hold: 7 },
+ { id: '67_prompt-enhance', tab: 'make', play: false, promptEnhance: true, hold: 6 },
+ { id: '70_vj-mobile', tab: 'vj', play: true, vjClip: true, vjMobile: true, hold: 8 },
+ { id: '71_dj-midi-learn', tab: 'dj', play: true, djDecks: true, djMidiLearn: true, hold: 7 },
+ { id: '69_controller-vision', tab: 'make', play: false, bottomTab: 'slide', maximizePanel: true, slideView: 'controller', controllerVision: true, hold: 7 },
+ // ── nav-scenes (file://) — Magenta RT2 NVIDIA port. MUST stay last (they navigate away). ──
+ { id: '63_magenta-studio-live', nav: 'http://localhost:8778', hold: 7 },
+ { id: '61_magenta-port-ui', nav: MAGENTA_STUDIO, hold: 7 },
+ { id: '62_magenta-card', nav: MAGENTA_CARD, hold: 7 },
];
-const SIZE = { width: 1920, height: 1080 };
+// Capture size / mode / supersample are env-configurable. A HEADED browser caps at the screen
+// width (1920 here); HEADLESS can exceed it (true higher-res so 9:16 punch-ins stay sharp).
+// DSF supersamples within the size. Fixed mouse coords scale by KX/KY off the 1920x1080 base.
+const VW = +(process.env.CAPW || 1920), VH = +(process.env.CAPH || 1080);
+const DSF = +(process.env.DSF || 1);
+const HEADLESS = process.env.HEADLESS === '1';
+const SIZE = { width: VW, height: VH };
+const CX = Math.round(VW / 2), CY = Math.round(VH / 2), KX = VW / 1920, KY = VH / 1080;
// Per-scene store driving. Heavy one-time builds (hero/stems/decks/studio/bucket)
// are guarded by window.__cap so they run once across the whole session. Every other
@@ -119,23 +160,56 @@ async function applyScene(spec) {
}
try { if (!player.getState().isPlaying) player.getState().play(); } catch (e) {}
- // ── one-time heavy builds, triggered the first time their scene arrives ──
+ // ── one-time heavy build: a REAL multitrack arrangement of Et Tu Machina ──
+ // Each stem becomes a track, clips are windowed + staggered for arrangement entrances,
+ // a couple are chopped, a few carry fades, and a 5th track holds a different source clip,
+ // so the timeline reads as a song being arranged rather than 6 bars stacked at t=0.
if (spec.buildStems && !C.stemsBuilt) {
- let first = true;
+ const palette = ['#06b6d4', '#f97316', '#ec4899', '#a855f7', '#10b981', '#facc15'];
+ // name → [startSec, durationSec, offsetIntoSource] (a meaty mid-song window)
+ const plan = { drums: [0, 22, 30], bass: [0, 22, 30], other: [3, 19, 35], vocals: [6, 16, 40] };
+ let idx = 0;
+ const firstIds = [];
for (const s of spec.stems) {
try {
const blob = await (await fetch(s.url)).blob();
const { peaks, duration } = await computePeaks(blob, 240);
+ const [st, dur, off] = plan[s.name] || [idx * 2, 18, 30];
const tracks = editor.getState().tracks;
let trackId;
- if (first && tracks.length && editor.getState().clips.filter((c) => c.trackId === tracks[0].id).length === 0) trackId = tracks[0].id;
- else trackId = editor.getState().addTrack({ name: s.name });
- first = false;
- const color = (editor.getState().tracks.find((t) => t.id === trackId) || {}).color || '#8b5cf6';
- const clipId = editor.getState().addClipToTrack({ trackId, label: s.name, audioBlob: blob, mimeType: 'audio/wav', sourceDuration: duration, offsetIntoSource: 0, durationSec: duration, startSec: 0, color });
+ if (idx === 0 && tracks.length && editor.getState().clips.filter((c) => c.trackId === tracks[0].id).length === 0) trackId = tracks[0].id;
+ else trackId = editor.getState().addTrack({ name: s.name, color: palette[idx % palette.length] });
+ editor.getState().updateTrack(trackId, { name: s.name });
+ const clipId = editor.getState().addClipToTrack({
+ trackId, label: s.name, audioBlob: blob, mimeType: 'audio/wav',
+ sourceDuration: duration, offsetIntoSource: Math.min(off, Math.max(0, duration - dur - 1)),
+ durationSec: Math.min(dur, duration), startSec: st, color: palette[idx % palette.length],
+ });
editor.getState().cachePeaks(clipId, peaks);
+ firstIds.push(clipId);
} catch (e) { log.push('stem ' + s.name + ' ' + e.message); }
+ idx++;
}
+ // a 5th track with a DIFFERENT source (an imported clip) for source variety
+ try {
+ const b2 = await (await fetch(spec.deckB.url)).blob();
+ const { peaks, duration } = await computePeaks(b2, 240);
+ const tId = editor.getState().addTrack({ name: 'import', color: palette[4] });
+ const cId = editor.getState().addClipToTrack({
+ trackId: tId, label: 'import', audioBlob: b2, mimeType: 'audio/wav',
+ sourceDuration: duration, offsetIntoSource: Math.min(20, Math.max(0, duration - 13)),
+ durationSec: Math.min(12, duration), startSec: 10, color: palette[4],
+ });
+ editor.getState().cachePeaks(cId, peaks);
+ editor.getState().updateClip(cId, { fadeOutSec: 2.0 });
+ } catch (e) { log.push('import-track ' + e.message); }
+ // chop a couple stems + apply fades so it reads as a worked arrangement
+ try {
+ if (firstIds[0]) { editor.getState().splitClipAt(firstIds[0], 8); editor.getState().splitClipAt(firstIds[0], 4); editor.getState().updateClip(firstIds[0], { fadeInSec: 0.8 }); }
+ if (firstIds[2]) editor.getState().splitClipAt(firstIds[2], 12);
+ if (firstIds[3]) editor.getState().updateClip(firstIds[3], { fadeInSec: 1.5 });
+ } catch (e) { log.push('arrange ' + e.message); }
+ try { editor.getState().setBpm(124); editor.getState().setSnap('1/8'); editor.getState().setZoom(34); editor.getState().setScrollSec(0); } catch (e) {}
C.stemsBuilt = true;
}
if (spec.djDecks && !C.decksLoaded) {
@@ -147,16 +221,24 @@ async function applyScene(spec) {
C.decksLoaded = true;
} catch (e) { log.push('dj ' + e.message); }
}
+ // DJ live stems: ride the separated stems on the deck's per-stem faders (uses the
+ // already-separated Et Tu Machina stems — no Demucs job needed at capture time).
+ if (spec.djStems && !C.djStemsLoaded) {
+ try { const dj = await imp('/src/state/djEngine.ts'); await dj.loadDeckStems('A', spec.stems.map((s) => ({ name: s.name, url: s.url }))); C.djStemsLoaded = true; } catch (e) { log.push('djstems ' + e.message); }
+ }
if (spec.studioSource && !C.studioSet) {
try {
- const studio = (await imp('/src/state/studioStore.ts')).useStudioStore;
const blob = await (await fetch(spec.heroUrl)).blob();
- studio.getState().setSourceFile(new File([blob], 'hero.wav', { type: 'audio/wav' }));
+ const f = new File([blob], 'et-tu-machina.wav', { type: 'audio/wav' });
+ (await imp('/src/state/studioStore.ts')).useStudioStore.getState().setSourceFile(f);
+ // The MIX INPUT field + processChain read the advanced-editor source — set it too so
+ // EVERY mix scene shows the imported track's waveform loaded, not an empty stage.
+ try { (await imp('/src/state/advancedEditorStore.ts')).useAdvancedEditorSourceStore.getState().setSource(f); } catch (e) {}
C.studioSet = true;
} catch (e) { log.push('studio ' + e.message); }
}
if (spec.fillBucket && !C.bucketFilled) {
- try { const mb = (await imp('/src/state/mediaBucketStore.ts')).useMediaBucketStore; const b = await (await fetch(spec.heroUrl)).blob(); mb.getState().add(new File([b], 'Dark-Wave.wav', { type: 'audio/wav' })); C.bucketFilled = true; } catch (e) { log.push('bucket ' + e.message); }
+ try { const mb = (await imp('/src/state/mediaBucketStore.ts')).useMediaBucketStore; const b = await (await fetch(spec.heroUrl)).blob(); mb.getState().add(new File([b], 'Et-Tu-Machina.wav', { type: 'audio/wav' })); C.bucketFilled = true; } catch (e) { log.push('bucket ' + e.message); }
}
// ── per-scene UI state (fully specified every time) ──
@@ -167,8 +249,8 @@ async function applyScene(spec) {
try {
const gp = (await imp('/src/state/generateParamsStore.ts')).useGenerateParamsStore;
- gp.getState().setField('model', spec.sunoModel ? 'suno' : 'medium');
- if (!C.srcFile) { const b = await (await fetch(spec.heroUrl)).blob(); C.srcFile = new File([b], 'source.wav', { type: 'audio/wav' }); }
+ gp.getState().setField('model', spec.sunoModel ? 'suno' : (spec.magentaMake ? 'magenta-small' : 'medium'));
+ if (!C.srcFile) { const b = await (await fetch(spec.heroUrl)).blob(); C.srcFile = new File([b], 'et-tu-machina.wav', { type: 'audio/wav' }); }
if (spec.inpaint) {
gp.getState().setField('initAudioFile', C.srcFile);
gp.getState().setField('inpaintAudioFile', C.srcFile);
@@ -180,9 +262,9 @@ async function applyScene(spec) {
} else {
gp.getState().setField('inpaintEnabled', false);
}
- // Build a CHIMERA stack — fold the hero + two stems into 3 chimera tracks.
+ // Build a CHIMERA stack — fold Et Tu Machina + two of its stems into 3 chimera tracks.
if (spec.chimeraBuild && !C.chimeraBuilt) {
- const srcs = [{ u: spec.heroUrl, l: 'Dark-Wave Outrun' }, { u: spec.stems[0].url, l: 'Drums' }, { u: spec.stems[2].url, l: 'Vocals' }];
+ const srcs = [{ u: spec.heroUrl, l: 'Et Tu Machina' }, { u: spec.stems[0].url, l: 'Drums' }, { u: spec.stems[2].url, l: 'Vocals' }];
for (const s of srcs) { try { const b = await (await fetch(s.u)).blob(); gp.getState().addChimeraClip({ blob: b, mimeType: 'audio/wav', label: s.l }); } catch (e) { log.push('chimera ' + e.message); } }
try { gp.getState().setChimeraField('targetBpm', 124); gp.getState().setChimeraField('alignMode', 'weave'); } catch (e) {}
C.chimeraBuilt = true;
@@ -191,6 +273,11 @@ async function applyScene(spec) {
}
} catch (e) {}
+ // Spectrogram viewer (COMPARE tab) renders from lastAudioUrl — point it at the song.
+ if (spec.spectro) {
+ try { const gs = (await imp('/src/state/generateStore.ts')).useGenerateStore; gs.setState({ lastAudioUrl: spec.heroUrl }); } catch (e) { log.push('spectro ' + e.message); }
+ }
+
// EDIT mixing: solo/mute/pan the stems so the per-track faders read as a live mix.
try {
if (spec.editMix) {
@@ -208,7 +295,17 @@ async function applyScene(spec) {
if (spec.mixChain && !C.mixChained) {
for (const fx of ['mastering_chain', 'reverb_delay', 'sub_exciter', 'stereo_widener', 'compression', 'lofi_vinyl']) ec.getState().addEffect(fx);
C.mixChained = true;
- } else if (!spec.mixChain) {
+ } else if (spec.mixAll && !C.mixAllBuilt) {
+ // EVERY effect, on the ACTUAL Et Tu Machina clip. The advanced-editor source store
+ // is what the MIX INPUT field + processChain() read (studioStore.sourceFile is not).
+ try {
+ const aes = (await imp('/src/state/advancedEditorStore.ts')).useAdvancedEditorSourceStore;
+ aes.getState().setSource(C.srcFile);
+ } catch (e) { log.push('mixall-src ' + e.message); }
+ ec.getState().clearChain();
+ for (const fx of ['mastering_chain', 'compression', 'eq_mid', 'sub_exciter', 'stereo_widener', 'reverb_delay', 'delay', 'echo', 'lofi_vinyl', 'phase_isolation', 'pitch_shift', 'loudnorm', 'vocal_processing', 'highpass']) ec.getState().addEffect(fx);
+ C.mixAllBuilt = true;
+ } else if (!spec.mixChain && !spec.mixAll) {
try { if (ec.getState().chain.length) ec.getState().clearChain(); } catch (e) {}
}
} catch (e) { log.push('mixchain ' + e.message); }
@@ -219,7 +316,7 @@ async function applyScene(spec) {
const samp = (await imp('/src/state/djSamplerStore.ts')).useDjSampler;
const labels = ['Kick', 'Snare', 'Hat', 'Stab', 'Vox', 'FX'];
for (let i = 0; i < labels.length; i++) { try { samp.getState().setPad(i, { entryId: spec.heroId, name: labels[i] }); } catch (e) {} }
- try { const sl = (await imp('/src/state/djSideListStore.ts')).useDjSideList; sl.getState().add({ entryId: spec.deckB.url, label: 'Dark-Wave II' }); sl.getState().add({ entryId: spec.heroId, label: 'Outrun Dubstep' }); } catch (e) {}
+ try { const sl = (await imp('/src/state/djSideListStore.ts')).useDjSideList; sl.getState().add({ entryId: spec.deckB.url, label: 'Deck B' }); sl.getState().add({ entryId: spec.heroId, label: 'Et Tu Machina' }); } catch (e) {}
C.djStaged = true;
}
} catch (e) { log.push('djsamp ' + e.message); }
@@ -249,21 +346,27 @@ async function applyScene(spec) {
}
} catch (e) { log.push('cut ' + e.message); }
- // PIANO ROLL: replace the seed with a dense, longer melody and start it playing so the
- // grid is full and the playhead moves.
+ // PIANO ROLL: load Et Tu Machina's own MIDI (windowed to 64 steps) and start it playing
+ // so the grid carries the actual song melody with the playhead moving.
try {
if (spec.pianoFill) {
const piano = (await imp('/src/state/pianoRollStore.ts')).usePianoRollStore;
- piano.getState().setTotalSteps(64);
- const scale = [60, 62, 63, 65, 67, 68, 70, 72]; // C minor-ish, two octaves of motion
- const notes = [];
- for (let step = 0; step < 64; step += 2) {
- const n = scale[(step / 2) % scale.length];
- notes.push({ note: n, step, length: 2, velocity: 80 + (step % 5) * 6 });
- if (step % 4 === 0) notes.push({ note: n - 12, step, length: 4, velocity: 70 }); // bass line
- if (step % 8 === 0) notes.push({ note: n + 7, step, length: 2, velocity: 64 }); // harmony
+ const etu = spec.etuPiano;
+ if (etu && etu.notes && etu.notes.length) {
+ piano.getState().setTotalSteps(etu.totalSteps || 64);
+ piano.getState().replaceAll(etu.notes.map((x) => ({ ...x })));
+ } else {
+ piano.getState().setTotalSteps(64);
+ const scale = [60, 62, 63, 65, 67, 68, 70, 72];
+ const notes = [];
+ for (let step = 0; step < 64; step += 2) {
+ const n = scale[(step / 2) % scale.length];
+ notes.push({ note: n, step, length: 2, velocity: 80 + (step % 5) * 6 });
+ if (step % 4 === 0) notes.push({ note: n - 12, step, length: 4, velocity: 70 });
+ if (step % 8 === 0) notes.push({ note: n + 7, step, length: 2, velocity: 64 });
+ }
+ piano.getState().replaceAll(notes.map((x) => ({ ...x })));
}
- piano.getState().replaceAll(notes.map((x) => ({ ...x })));
piano.getState().setPlaying(true);
}
} catch (e) { log.push('piano ' + e.message); }
@@ -298,6 +401,19 @@ const VJ_SOURCE = path.join(OUT, '_vjsource.mp4');
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
const clickByTitle = (page, re) => page.evaluate((src) => { const rx = new RegExp(src, 'i'); const b = [...document.querySelectorAll('button')].find((x) => rx.test(x.getAttribute('title') || '')); if (b) b.click(); }, re.source).catch(() => {});
const clickByText = (page, re) => page.evaluate((src) => { const rx = new RegExp(src, 'i'); const b = [...document.querySelectorAll('button')].find((x) => rx.test(x.textContent || '')); if (b) b.click(); }, re.source).catch(() => {});
+// A REAL mouse click (transient user activation) on the nth titled button — needed for
+// requestFullscreen() on the cymatics panels, which a synthetic .click() can't activate.
+async function realClickByTitle(page, re, nth = 0) {
+ const box = await page.evaluate(({ s, n }) => {
+ const rx = new RegExp(s, 'i');
+ const bs = [...document.querySelectorAll('button')].filter((x) => rx.test(x.getAttribute('title') || ''));
+ const b = bs[n]; if (!b) return null; const q = b.getBoundingClientRect();
+ if (q.width < 1 || q.height < 1) return null;
+ return { x: q.x + q.width / 2, y: q.y + q.height / 2 };
+ }, { s: re.source, n: nth }).catch(() => null);
+ if (box) { await page.mouse.click(box.x, box.y); return true; }
+ return false;
+}
const waitSplashGone = (page) => page.waitForFunction(() => {
// LoadingScreen unmounts (AnimatePresence) once the backend is ready; it is the only
@@ -343,8 +459,8 @@ async function sceneActions(page, scene) {
// When hovering, lit-count = the lineage size of that node. Densest = most lit.
scored.sort((a, b) => b.lit - a.lit);
const picks = [];
- for (const s of scored) { if (picks.every((p) => Math.hypot(p.cx - s.b.cx, p.cy - s.b.cy) > 60)) picks.push(s.b); if (picks.length >= 6) break; }
- scene._targets = picks;
+ for (const s of scored) { if (picks.every((p) => Math.hypot(p.cx - s.b.cx, p.cy - s.b.cy) > 36)) picks.push(s.b); if (picks.length >= 40) break; }
+ scene._targets = picks; // densest-first; the hold hovers a long parade of them
await page.mouse.move(960, 230);
if (!picks.length) err = '2d:no-nodes';
} catch (e) { err = '2d:' + e.message; }
@@ -365,6 +481,80 @@ async function sceneActions(page, scene) {
if (scene.compare) { await clickByText(page, /\bcompare\b/i); await sleep(800); }
if (scene.savedPrompts) { await clickByText(page, /saved\s*\(/i); await sleep(900); }
if (scene.designMode) { await clickByText(page, /edit layout/i); await sleep(1600); }
+ // Cymatics / ferrofluid: enter fullscreen on the orb; the hold cycles all 4 modes.
+ if (scene.vizCycle) {
+ await realClickByTitle(page, /ferrofluid orb/i, 0);
+ await sleep(500);
+ if (scene.vizFull) { await realClickByTitle(page, /^fullscreen$/i, 0); await sleep(1300); }
+ } else if (scene.vizMode) {
+ await realClickByTitle(page, new RegExp(scene.vizMode, 'i'), 0);
+ await sleep(600);
+ if (scene.vizFull) { await realClickByTitle(page, /^fullscreen$/i, 0); await sleep(1300); }
+ }
+ // Spectrogram viewer (COMPARE hero-tab): wait for /api/spectrogram, then toggle ON the Mel /
+ // STFT / Chroma / CQT layers so the STACK of spectrograms renders (default shows only the WF).
+ if (scene.spectro) {
+ await clickByText(page, /\bcompare\b/i);
+ await sleep(5200);
+ for (const m of ['Mel', 'STFT', 'Chroma', 'CQT']) { await page.evaluate((t) => { const b = [...document.querySelectorAll('button')].find((x) => (x.textContent || '').trim() === t); if (b) b.click(); }, m).catch(() => {}); await sleep(250); }
+ await sleep(1500);
+ }
+ // DJ automix: engage the hands-free auto-mixer (the crossfader then travels on its own).
+ if (scene.djAutomix) { await clickByText(page, /automix/i); await sleep(900); }
+ // Magenta panel (theDAW integrated): select the Magenta RT2 model option if the sidecar surfaced it.
+ if (scene.magentaMake) {
+ await page.evaluate(() => { const el = [...document.querySelectorAll('button,[role="option"],li,div,span')].find((x) => /magenta\s*rt2/i.test(x.textContent || '') && (x.textContent || '').length < 60); if (el) el.click(); }).catch(() => {});
+ await sleep(1000);
+ }
+ // EDIT commit: name the mixdown and click Commit Edit (offline 44.1k stereo render).
+ if (scene.commitEdit) {
+ try { await page.evaluate(() => { const i = document.querySelector('#editor-mixdown-name'); if (i) { i.value = 'Et Tu Machina (mixdown)'; i.dispatchEvent(new Event('input', { bubbles: true })); } }); } catch (e) {}
+ await clickByText(page, /commit\s*edit/i); await sleep(900);
+ }
+ // ── feature-coverage gap fills ──
+ // Node inspector: click the densest 2D node's actual element → its params + lineage panel.
+ if (scene.nodeInspect && scene._targets && scene._targets.length) {
+ const t = scene._targets[0];
+ await page.evaluate(({ x, y }) => { const el = document.elementFromPoint(x, y); const g = el && (el.closest('g') || el); if (g) g.dispatchEvent(new MouseEvent('click', { bubbles: true, clientX: x, clientY: y })); }, { x: t.cx, y: t.cy }).catch(() => {});
+ await page.mouse.click(t.cx, t.cy);
+ await sleep(1600);
+ }
+ // Edit Tool Stack: open a Studio Module's exact instrument GUI in the MIX effect stage.
+ if (scene.moduleGui) {
+ await page.evaluate(() => { const el = [...document.querySelectorAll('button')].find((x) => /studio/i.test(x.textContent || '') && (x.textContent || '').replace(/\s/g, '').length < 12); if (el) el.click(); }).catch(() => {});
+ await sleep(800);
+ for (const m of ['Imager', 'Maximizer', 'Exciter', 'Character FX', 'EQ']) { const ok = await page.evaluate((t) => { const b = [...document.querySelectorAll('button')].find((x) => { const s = (x.textContent || '').trim(); return new RegExp('^' + t.replace(/[.*+?^${}()|[\]\\]/g, '\\$&') + '\\b', 'i').test(s) && s.length > t.length; }); if (b) { b.click(); return true; } return false; }, m); if (ok) break; }
+ await sleep(2400);
+ }
+ // Audio-to-MIDI: right-click a catalogue track → show the menu with "Convert to MIDI" highlighted.
+ if (scene.audioToMidi) {
+ try {
+ await page.mouse.click(360, 130, { button: 'right' }); await sleep(800);
+ const box = await page.evaluate(() => { const b = [...document.querySelectorAll('*')].find((x) => /convert to midi/i.test(x.textContent || '') && x.children.length === 0); if (!b) return null; const r = b.getBoundingClientRect(); return { x: r.x + r.width / 2, y: r.y + r.height / 2 }; });
+ if (box) await page.mouse.move(box.x, box.y);
+ await sleep(900);
+ } catch (e) { err = (err || '') + ' a2m:' + e.message; }
+ }
+ // Prompt enhancement: type a prompt, hit the AI-enhance (wand) button.
+ if (scene.promptEnhance) {
+ try { await page.fill('textarea', 'lo-fi boom bap, dusty rhodes chords, vinyl crackle, 84 bpm'); } catch (e) {}
+ await sleep(300); await clickByTitle(page, /ai-enhance prompt/i); await sleep(1400);
+ }
+ // VJ mobile mirror: show the LAN QR + URL popover, then toggle the camera input.
+ if (scene.vjMobile) {
+ await clickByTitle(page, /mobile url/i); await sleep(1000);
+ await clickByTitle(page, /camera (on|off)/i); await sleep(700);
+ }
+ // DJ MIDI-learn: open the learn overlay and arm a binding.
+ if (scene.djMidiLearn) {
+ await clickByTitle(page, /^\s*midi\s*$/i); await sleep(800);
+ await clickByText(page, /^\s*learn\s*$/i); await sleep(1000);
+ }
+ // Controller Vision: Set up device → Identify by photo (AI) → the detect/identify modal.
+ if (scene.controllerVision) {
+ await clickByText(page, /set up device/i); await sleep(700);
+ await clickByText(page, /identify by photo/i); await sleep(1300);
+ }
if (scene.urlImport) {
try { await page.fill('input[placeholder*="YouTube" i]', 'https://www.youtube.com/watch?v=2Vv-BfVoq4g'); } catch (e) { err = (err || '') + ' url:' + e.message; }
await sleep(700);
@@ -434,25 +624,152 @@ async function vjTour(page, seconds) {
}
}
+// Full DJ performance workout: fire hotcues, toggle beat loops + a loop-roll, slip,
+// beat-jump, and sweep the per-deck FX (flanger / reverb / wah) + EQ + filter so every
+// pad and knob on the console is visibly working.
+async function djPerf(page, seconds) {
+ const t0 = Date.now();
+ const clickTitle = (rx) => page.evaluate((s) => { const r = new RegExp(s, 'i'); const b = [...document.querySelectorAll('button')].find((x) => r.test(x.getAttribute('title') || '')); if (b) b.click(); }, rx).catch(() => {});
+ const seq = ['cue 1', 'cue 2', '1-beat loop', 'jump \\+1 beat', 'cue 3', '½-beat loop', 'slip mode', 'jump -4 beats', 'cue 4', 'exit loop', '2-beat loop'];
+ let i = 0;
+ while ((Date.now() - t0) / 1000 < seconds) {
+ const p = (Date.now() - t0) / 1000;
+ await page.evaluate(async (ph) => {
+ try {
+ const dj = await import('/src/state/djEngine.ts');
+ if (dj.setDeckFx) { dj.setDeckFx('A', 'flanger', Math.sin(ph) * 0.5 + 0.5); dj.setDeckFx('A', 'reverb', Math.cos(ph * 0.7) * 0.5 + 0.5); dj.setDeckFx('B', 'wahwah', Math.sin(ph * 1.3) * 0.5 + 0.5); }
+ if (dj.setDeckEq) { dj.setDeckEq('A', 'low', Math.sin(ph) * 12); dj.setDeckEq('B', 'high', Math.cos(ph) * 12); }
+ if (dj.setDeckFilter) dj.setDeckFilter('A', Math.sin(ph * 0.8) * 0.7);
+ } catch (e) {}
+ }, p);
+ await clickTitle(seq[i % seq.length]);
+ i++;
+ await sleep(820);
+ }
+ // loop-roll flourish: press-and-hold a roll pad, then release
+ try {
+ const box = await page.evaluate(() => { const b = [...document.querySelectorAll('button')].find((x) => /loop-roll/i.test(x.getAttribute('title') || '')); if (!b) return null; const r = b.getBoundingClientRect(); return { x: r.x + r.width / 2, y: r.y + r.height / 2 }; });
+ if (box) { await page.mouse.move(box.x, box.y); await page.mouse.down(); await sleep(700); await page.mouse.up(); }
+ } catch (e) {}
+}
+
+// Automix is engaged (sceneActions clicked it); the crossfader auto-travels. Ride a gentle
+// EQ/filter sweep so console knobs also move while the auto-transition runs.
+async function djAutomix(page, seconds) {
+ const t0 = Date.now();
+ while ((Date.now() - t0) / 1000 < seconds) {
+ const p = (Date.now() - t0) / 1000;
+ await page.evaluate(async (ph) => { try { const dj = await import('/src/state/djEngine.ts'); if (dj.setDeckEq) { dj.setDeckEq('A', 'mid', Math.sin(ph) * 8); dj.setDeckEq('B', 'low', Math.cos(ph) * 8); } if (dj.setDeckFilter) dj.setDeckFilter('B', Math.sin(ph * 0.6) * 0.5); } catch (e) {} }, p);
+ await sleep(160);
+ }
+}
+
+// Ride the deck's live stem faders (drums/bass/vocals/other) up and down.
+async function djStems(page, seconds) {
+ const t0 = Date.now();
+ const names = ['drums', 'bass', 'vocals', 'other'];
+ while ((Date.now() - t0) / 1000 < seconds) {
+ const p = (Date.now() - t0) / 1000;
+ await page.evaluate(async ({ ph, nm }) => { try { const dj = await import('/src/state/djEngine.ts'); if (dj.setStemGain) nm.forEach((n, i) => dj.setStemGain('A', n, Math.abs(Math.sin(ph * 0.6 + i)))); } catch (e) {} }, { ph: p, nm: names });
+ await sleep(150);
+ }
+}
+
+// MIX: cycle the populated chain — click each chain row (its purple mono label) to open
+// that effect's instrument over the loaded Et Tu Machina waveform.
+async function mixCycle(page, seconds) {
+ const t0 = Date.now();
+ const rows = await page.evaluate(() => {
+ const out = [];
+ for (const s of document.querySelectorAll('span')) {
+ const c = (typeof s.className === 'string') ? s.className : '';
+ if (/font-mono/.test(c) && /text-purple-300/.test(c) && /font-semibold/.test(c)) {
+ const r = s.getBoundingClientRect();
+ if (r.width > 0 && r.height > 0) out.push({ x: r.x + r.width / 2, y: r.y + r.height / 2 });
+ }
+ }
+ return out;
+ }).catch(() => []);
+ let i = 0;
+ while ((Date.now() - t0) / 1000 < seconds) {
+ if (rows.length) { await page.mouse.click(rows[i % rows.length].x, rows[i % rows.length].y); }
+ i++;
+ await sleep(1050);
+ }
+}
+
// Undo cross-tab overlays so the next scene is clean (lineage fullscreen unmounts with
-// the tab; the assistant orb is global and must be toggled back off).
+// the tab; the assistant orb is global and must be toggled back off; cymatics fullscreen
+// must be Escaped).
async function sceneCleanup(page, scene) {
if (scene.openOrb) { try { await page.click('[aria-label="Toggle orb panel"]', { timeout: 1500 }); } catch (e) {} await sleep(250); }
+ if (scene.vizFull) { try { await page.keyboard.press('Escape'); } catch (e) {} await sleep(300); }
if (scene.settingsModal || scene.docsModal) { try { await page.keyboard.press('Escape'); } catch (e) {} await sleep(250); }
}
-// Slow, continuous drag over the 3D canvas → OrbitControls orbits the galaxy. Runs for
-// the whole hold so the recorded window is a live fly-around, not a static frame.
+// Design Mode (Edit Layout): grab panels / control groups and drag them to new positions so
+// the recording shows the INTERFACE BEING REARRANGED, not just the toggle being flipped.
+async function designDrag(page, seconds) {
+ const t0 = Date.now();
+ const click = (rx) => page.evaluate((s) => { const r = new RegExp(s, 'i'); const b = [...document.querySelectorAll('button')].find((x) => r.test(x.getAttribute('title') || '')); if (b) b.click(); }, rx).catch(() => {});
+ // The layout-action buttons — each visibly reflows the surface (scaling + padding shift).
+ const actions = ['balance the layout', 'mirror the left', 'mirror the right', 'add an empty panel', 'reset to the saved'];
+ let i = 0;
+ while ((Date.now() - t0) / 1000 < seconds) {
+ // DRAG A RESIZE EDGE (Splitter handle) — the two neighbouring panels scale + repad live.
+ const sp = await page.evaluate(() => {
+ const out = [];
+ for (const el of document.querySelectorAll('[class*="cursor-col-resize"],[class*="cursor-row-resize"]')) {
+ const r = el.getBoundingClientRect();
+ if (r.width > 0 && r.height > 0) out.push({ x: r.x + r.width / 2, y: r.y + r.height / 2, ax: /col-resize/.test(el.className) ? 'x' : 'y' });
+ }
+ return out;
+ }).catch(() => []);
+ if (sp.length) {
+ const s = sp[i % sp.length], d = (i % 2 ? 170 : -150);
+ await page.mouse.move(s.x, s.y); await page.mouse.down();
+ if (s.ax === 'x') { await page.mouse.move(s.x + d, s.y, { steps: 12 }); await page.mouse.move(s.x, s.y, { steps: 10 }); }
+ else { await page.mouse.move(s.x, s.y + d, { steps: 12 }); await page.mouse.move(s.x, s.y, { steps: 10 }); }
+ await page.mouse.up();
+ }
+ await sleep(350);
+ // PRESS A LAYOUT-ACTION BUTTON (center / mirror / add panel / reset) so its effect is shown.
+ await click(actions[i % actions.length]);
+ await sleep(750);
+ i++;
+ }
+}
+
+// Cymatics: ONE fullscreen shot that cycles all 4 modes (sceneActions started on the orb +
+// fullscreen). Switch mode every ~6s by clicking the panel's mode buttons while still fullscreen.
+async function vizCycle(page, seconds) {
+ const t0 = Date.now();
+ const modes = ['cymatic platform', 'liquid chrome', 'ferrofluid valley'];
+ let i = 0;
+ while ((Date.now() - t0) / 1000 < seconds) {
+ await sleep(6000);
+ if ((Date.now() - t0) / 1000 >= seconds) break;
+ if (i < modes.length) await realClickByTitle(page, new RegExp(modes[i], 'i'), 0);
+ i++;
+ }
+}
+
+// Fly the 3D genealogy galaxy. Mouse-drag orbits (always-on motion) PLUS the spaceship-flight
+// keys the app binds — WASD thrust + periodic hold-F FTL hyperspace warp — so the recording
+// shows the velocity flight + warp the showcase calls out, not just a slow pan.
async function flyAround3d(page, seconds) {
const cx = 960, cy = 540;
const t0 = Date.now();
+ await page.mouse.click(cx, cy); // focus the canvas so the flight keys register
await page.mouse.move(cx, cy);
await page.mouse.down();
let i = 0;
while ((Date.now() - t0) / 1000 < seconds) {
const a = i * 0.22;
await page.mouse.move(cx + Math.cos(a) * 300, cy + Math.sin(a) * 150, { steps: 10 });
- await sleep(110);
+ const k = ['w', 'd', 'w', 'a'][i % 4];
+ await page.keyboard.down(k); await sleep(110); await page.keyboard.up(k);
+ if (i % 6 === 5) { await page.keyboard.down('f'); await sleep(650); await page.keyboard.up('f'); } // FTL warp
i++;
}
await page.mouse.up();
@@ -465,16 +782,23 @@ async function holdAction(page, scene) {
const t0 = Date.now();
if (scene.flyAround) return flyAround3d(page, secs);
if (scene.vjTour) return vjTour(page, secs);
+ if (scene.djPerf) return djPerf(page, secs);
+ if (scene.djAutomix) return djAutomix(page, secs);
+ if (scene.djStems) return djStems(page, secs);
+ if (scene.mixAll) return mixCycle(page, secs);
+ if (scene.designDrag) return designDrag(page, secs);
+ if (scene.vizCycle) return vizCycle(page, secs);
if (scene.lineageHover) {
- // Hover the densest nodes one at a time; each lights its full family tree. ~equal time
- // per node so the recording walks through several big lineages.
+ // Walk a long PARADE of nodes, densest-first; each hover lights that node's full
+ // ancestry + descendant chain. Cycles the densest ~40 so the big lineages recur.
const tg = scene._targets || [];
if (!tg.length) { await sleep(secs * 1000); return; }
- const per = secs / tg.length;
- for (const t of tg) {
- await page.mouse.move(960, 230); await sleep(150);
- await page.mouse.move(t.cx, t.cy);
- await sleep(Math.max(500, per * 1000 - 150));
+ let i = 0;
+ while ((Date.now() - t0) / 1000 < secs) {
+ const t = tg[i % tg.length];
+ await page.mouse.move(t.cx, t.cy, { steps: 4 });
+ await sleep(1100);
+ i++;
}
return;
}
@@ -519,8 +843,11 @@ async function holdAction(page, scene) {
await sleep(secs * 1000);
}
-const browser = await chromium.launch({ headless: false, args: ['--autoplay-policy=no-user-gesture-required', '--start-maximized'] });
-const ctx = await browser.newContext({ viewport: SIZE, deviceScaleFactor: 1, recordVideo: { dir: OUT, size: SIZE } });
+const launchArgs = ['--autoplay-policy=no-user-gesture-required'];
+if (HEADLESS) launchArgs.push('--use-angle=gl', '--ignore-gpu-blocklist', '--enable-gpu', '--enable-webgl', '--enable-accelerated-2d-canvas');
+else launchArgs.push('--start-maximized');
+const browser = await chromium.launch({ headless: HEADLESS, args: launchArgs });
+const ctx = await browser.newContext({ viewport: SIZE, deviceScaleFactor: DSF, recordVideo: { dir: OUT, size: SIZE } });
const page = await ctx.newPage();
const tRec = Date.now(); // recording starts at context/page creation → this is video t=0.
// (Anchoring AFTER the splash wait shifts every slice earlier by
@@ -539,15 +866,25 @@ await page.mouse.click(8, 8);
for (const scene of scenes) {
let info = null, err = null;
try {
- info = await page.evaluate(applyScene, { ...HERO, deckB: DECK_B, stems: STEMS, ...scene });
- await sleep(1200); // let the tab transition + view settle
- err = await sceneActions(page, scene);
- await sleep(500);
- const start = (Date.now() - tRec) / 1000; // stable-hold window begins here
- await holdAction(page, scene);
- const end = (Date.now() - tRec) / 1000;
- marks.push({ id: scene.id, start, end });
- await sceneCleanup(page, scene);
+ if (scene.nav) {
+ // file:// scenes (the Magenta port UI + repo card) navigate away — they run last.
+ await page.goto(scene.nav, { waitUntil: 'domcontentloaded' }).catch((e) => { err = 'nav ' + e.message; });
+ await sleep(1600);
+ const start = (Date.now() - tRec) / 1000;
+ await holdAction(page, scene);
+ const end = (Date.now() - tRec) / 1000;
+ marks.push({ id: scene.id, start, end });
+ } else {
+ info = await page.evaluate(applyScene, { ...HERO, deckB: DECK_B, stems: STEMS, etuPiano: ETU_PIANO, ...scene });
+ await sleep(1200); // let the tab transition + view settle
+ err = await sceneActions(page, scene);
+ await sleep(500);
+ const start = (Date.now() - tRec) / 1000; // stable-hold window begins here
+ await holdAction(page, scene);
+ const end = (Date.now() - tRec) / 1000;
+ marks.push({ id: scene.id, start, end });
+ await sceneCleanup(page, scene);
+ }
} catch (e) { err = String(e).slice(0, 200); }
results.push({ id: scene.id, info, err });
console.log(`${scene.id}`, err ? ('ERR ' + err) : JSON.stringify(info));
@@ -567,7 +904,7 @@ for (const m of marks) {
const out = path.join(OUT, `${m.id}_h.mp4`);
try {
execFileSync('ffmpeg', ['-y', '-loglevel', 'error', '-ss', ss, '-t', dur, '-i', sessionWebm,
- '-an', '-vf', 'scale=1920:1080,fps=30,format=yuv420p', '-c:v', 'libx264', '-preset', 'veryfast', '-crf', '20', out]);
+ '-an', '-vf', `scale=${VW}:${VH},fps=30,format=yuv420p`, '-c:v', 'libx264', '-preset', 'medium', '-crf', '14', out]);
console.log(` sliced ${m.id}_h.mp4 (${dur}s @ ${ss}s)`);
} catch (e) { console.log(` slice FAIL ${m.id}`, e.message); }
}
diff --git a/frontend/public/USER_GUIDE.md b/frontend/public/USER_GUIDE.md
index c74cb5b..1b42a31 100644
--- a/frontend/public/USER_GUIDE.md
+++ b/frontend/public/USER_GUIDE.md
@@ -386,6 +386,8 @@ During the render, the COMMIT EDIT button shows an animated spinner and is disab
The MIX tab (`MixPanel`) is the single-screen effects and mastering workspace. It applies one or more of 24 FFmpeg-backed processors to a source file as an ordered chain. Processing is synchronous; the result is returned as binary audio and made available for inline playback, download, and routing back into the app.
+
+
### 8.1 Layout
MIX fills the viewport with no page scroll, in three bands:
@@ -399,6 +401,8 @@ The player footer is the process-chain transport.
Quick Master exposes the four parameters of a `mastering_chain` entry as live knobs (low boost, high boost, limiter ceiling, target LUFS). Applying it inserts a single `mastering_chain` entry into the chain once, then live-updates that entry's parameters as the knobs move.
+
+
### 8.3 Effect Catalog and Chain
The effect library is organized into categories (mastering, dynamics, EQ, tempo, cleanup, export) with list and tile view modes and color-coded groups. Add effects to the chain, toggle individual rows active or inactive, reorder rows, and clear the chain in one action. Each chain entry exposes bounded sliders and numeric fields sourced from `PARAM_BOUNDS` and `EFFECT_CATALOG`.
@@ -452,6 +456,8 @@ The last processing invocations are retained in the store. Any history item can
The DJ tab (`DJView`) is a live two-deck performance mixer built on the SLIDE control surface. It loads tracks from the Library or any source, beatmatches and key-locks them, mixes through a full center console, and hands sets off to the VJ for synchronized visuals. The Web Audio engine (`djEngine.ts`) runs the signal path per deck: source, optional time-stretch, delay compensation, trim, three-band EQ, filter, gain and crossfade, the DJ master bus, then the shared player master.
+
+
### 9.1 Waveform Hero and Decks
The two deck waveforms span the full width at the top, each with a beatgrid, hotcue markers, and a moving playhead. A thin per-deck header shows the title, detected BPM, musical key and Camelot value, and the SYNC control. Below the hero, Deck A and Deck B flank the center mixer.
@@ -464,6 +470,8 @@ Each deck provides play and cue, four hotcues, loops with loop-roll and slip mod
The mixer is the console between the decks. Each channel has a gain knob, a three-band EQ (high, mid, low), a single-knob filter that sweeps low-pass below center and high-pass above center, and a volume fader. The two pitch faders sit as the outer columns of the mixer. A crossfader runs along the bottom, and the toggle row (Quantize, Auto-gain, Limiter, MIDI) sits above the gain knobs. A master limiter protects the DJ master bus.
+
+
### 9.4 Live Stems and FX
Each deck has an FX and stems rack. Live stem faders ride the separated drums, bass, other, and vocals derived from Demucs, so a part can be pulled out or pushed up during playback. The FX rack adds per-deck effects on top of the EQ and filter.
@@ -1655,6 +1663,8 @@ Two offline helpers under `scripts/` bulk-import an existing SunoHarvester cache
The `magenta` module (`/api/magenta`) brings Google's **Magenta RealTime 2 (MRT2)** real-time music model into the Generate workspace as a text→music option. The model option appears only when the sidecar is reachable: the MAKE/Generate panel probes `GET /api/magenta/probe` and shows **"Magenta RT2 (text→music)"** when available.
+
+
### 27.1 The sidecar and conditioning
MRT2 runs as a sidecar (default `http://localhost:8777`, override with `STABLEDAW_MAGENTA_URL`) that loads `MagentaRT2Jax` once on the GPU. theDAW ships an **extended** sidecar (`sidecars/magenta/server.py`) that supersedes the upstream text-only studio server and exposes all three conditioning modes over one `POST /generate`:
@@ -1764,4 +1774,14 @@ Operational endpoints used by the Settings modal and shell.
---
+## Credits
+
+theDAW was built by **[GANTASMO](https://github.com/gantasmo)** as part of the [Music Hackspace](https://musichackspace.org) Music Technology Hackathon at [Berklee College of Music](https://www.berklee.edu).
+
+Special thanks to [Music Hackspace](https://musichackspace.org), [Berklee College of Music](https://www.berklee.edu), and to Zack, CJ, Jordi, Zach, and Matt from [Stability AI](https://stability.ai) for their continued help and support.
+
+**Built with:** [Stability AI](https://stability.ai) Stable Audio 3 and [stable-audio-tools](https://github.com/Stability-AI/stable-audio-tools) (the core diffusion model and pipeline); [Magenta](https://github.com/magenta) RealTime by [Google DeepMind](https://deepmind.google), running through theDAW's own [NVIDIA/CUDA port](../sidecars/magenta-rt2-nvidia/) (the first and only non-Mac port so far); [Suno](https://suno.com) (cloud generation); [T5Gemma](https://huggingface.co/google/t5gemma-b-b-ul2) by Google (text conditioning); [Demucs](https://github.com/facebookresearch/demucs) by Meta AI (stem separation); [basic-pitch](https://github.com/spotify/basic-pitch) by Spotify (audio-to-MIDI); [MLX](https://github.com/ml-explore/mlx) by Apple (the Magenta port's inference core, extended with a CUDA backend); and [PyTorch](https://pytorch.org), [FFmpeg](https://ffmpeg.org), [three.js](https://threejs.org), [react-force-graph](https://github.com/vasturiano/react-force-graph), [WaveSurfer.js](https://wavesurfer.xyz), [React](https://react.dev), [Vite](https://vitejs.dev), and [Tailwind CSS](https://tailwindcss.com), alongside the wider open-source community.
+
+---
+
*Maintained by the theDAW development team.*
diff --git a/showcase/build_roughcut.sh b/showcase/build_roughcut.sh
index a50bcfe..e4cf2da 100644
--- a/showcase/build_roughcut.sh
+++ b/showcase/build_roughcut.sh
@@ -17,82 +17,120 @@ FONT="showcase/.font.ttf"
# h-crop "full" or CW:CH:CX:CY ; v-crop is the 9:16 punch-in region ; start "" = clean tail.
# Order: a rapid-fire opening blast, then the full feature tour. Subtitles are lower-case
# and plain on purpose.
+# A "song being made in theDAW" arc, built around Et Tu Machina: hook -> make -> arrange ->
+# mix -> perform -> visualize -> lineage -> ecosystem. Order is the narrative; durations flex.
SCENES=(
- "03_edit-stems|1.5|full|608:1080:656:0||six stems from one track"
- "02_dj-console|1.5|full|540:960:690:90||a real two-deck dj console"
+ # hook + lineage
+ "50_cymatics|5.0|1680:945:120:70|608:1080:656:0|0.5|one machine, every instrument"
"01a_learn-2d-hero|1.9|full|608:1080:656:0|19.0|every track remembers its lineage"
+ # make the song
+ "08_make|1.4|full|608:1080:656:0||generate from a prompt"
+ "23_chimera|1.6|full|608:1080:656:0||fuse clips into a chimera"
+ "25_init-audio|1.4|full|608:1080:656:0||seed it with your own audio"
+ "21_inpaint|1.4|full|608:1080:656:0||repaint any section"
+ "45_saved-prompts|1.2|full|608:1080:656:0||save every prompt"
"05_sequencer|1.3|full|608:1080:656:0||a step sequencer"
- "08_make|1.4|full|608:1080:656:0||generate audio from a prompt"
+ "58_spectrogram|1.9|full|608:1080:656:0||mel · stft · chroma · cqt"
+ # arrange + edit
+ "03_edit-stems|1.9|full|608:1080:656:0||arrange the stems on a timeline"
+ "22_cut-edit|1.5|full|608:1080:656:0||cut and chop the clips"
+ "41_delete-clip|1.3|full|608:1080:656:0||edit like a daw"
+ "31_edit-mix|1.4|full|608:1080:656:0||mix the tracks live"
+ "40_inpaint-region|1.4|full|608:1080:656:0||paintbrush inpainting"
+ "59_commit-edit|1.5|full|608:1080:656:0||commit to a 44.1khz master"
+ # mix
+ "07_mix-effects|1.4|full|608:1080:1312:0||a studio effects rack"
+ "57_mix-all-effects|2.0|full|608:1080:400:0||every effect, on the track"
+ "66_module-gui|1.7|full|608:1080:1312:0||14 pro-grade instruments"
+ "32_mix-chain|1.4|full|608:1080:400:0||stack a mastering chain"
+ "27_lora|1.3|full|608:1080:1312:0||load loras"
+ "39_design-mode|1.6|full|608:1080:656:0||rearrange the interface"
+ # perform (dj)
+ "02_dj-console|1.5|full|540:960:690:90||a two-deck dj console"
+ "54_dj-perform|2.0|full|540:960:690:90||hotcues, loops, and fx"
+ "55_dj-automix|1.5|full|540:960:690:90||automix, hands-free"
+ "56_dj-stems|1.4|full|540:960:690:90||live stems on the deck"
+ "33_dj-sampler|1.4|full|608:1080:300:0||sampler + staging deck"
+ # visualize
+ "50_cymatics|5.0|1680:945:120:70|608:1080:656:0|6.8|cymatic platform"
+ "50_cymatics|5.0|1680:945:120:70|608:1080:656:0|12.5|liquid-chrome cymatics"
+ "50_cymatics|5.0|1680:945:120:70|608:1080:656:0|18.5|ferrofluid valley"
+ "12_slide-surface|1.5|full|608:1080:500:0||performance sliders"
+ "19_vj-visualizer|3.5|full|608:1080:750:0||drive live visuals"
+ "70_vj-mobile|1.6|full|608:1080:1312:0||mirror to a phone over wi-fi"
"04_visualize|1.2|full|608:1080:656:0||live spectrum"
- "32_mix-chain|1.5|full|608:1080:400:0||stack studio effects"
- "17_controller|1.4|full|608:1080:656:0||map any midi controller"
- "06_piano-roll|1.3|full|608:1080:300:0||a piano roll"
- "01b_learn-3d-hero|1.8|full|608:1080:656:0||fly through the 3d lineage"
- "23_chimera|1.7|full|608:1080:656:0||fuse clips into a chimera"
- "21_inpaint|1.5|full|608:1080:656:0||repaint any section"
- "25_init-audio|1.5|full|608:1080:656:0||seed it with your own audio"
- "22_cut-edit|1.9|full|608:1080:656:0||cut and arrange on a timeline"
- "41_delete-clip|1.5|full|608:1080:656:0||edit like a daw"
- "31_edit-mix|1.5|full|608:1080:656:0||mix the stems live"
- "07_mix-effects|1.5|full|608:1080:1312:0||a full studio effects rack"
- "39_design-mode|1.9|full|608:1080:656:0||rearrange the entire interface"
- "33_dj-sampler|1.6|full|608:1080:300:0||sampler and staging deck"
- "19_vj-visualizer|2.0|800:450:1120:230|608:1080:1312:0||drive live visuals"
- "20_train|1.7|full|608:1080:656:0||train your own model"
- "15_analyzer-scope|1.3|full|608:1080:656:0||oscilloscope"
+ "15_analyzer-scope|1.2|full|608:1080:656:0||oscilloscope"
"16_analyzer-radial|1.3|full|540:960:830:120||radial analyzer"
- "12_slide-surface|1.6|full|608:1080:500:0||performance sliders"
- "24_focus|1.5|full|608:1080:656:0||focus on one control"
- "40_inpaint-region|1.5|full|608:1080:656:0||paintbrush inpainting"
- "18_media-bucket|1.3|full|608:1080:120:0||drop in any file"
- "44_url-import|1.5|full|608:1080:120:0||or pull from a link"
- "13_details|1.4|full|608:1080:300:0||full metadata on everything"
- "42_lib-actions|1.7|full|608:1080:300:0||browse the whole catalogue"
- "11_assistant-orb|1.7|full|608:1080:0:0||an assistant that runs the app"
- "14_catalogue|1.7|full|608:1080:300:0||your entire library"
- "10_suno-cloud|1.7|full|608:1080:560:0||or render in the cloud"
- "27_lora|1.3|full|608:1080:1312:0||load loras"
- "45_saved-prompts|1.3|full|608:1080:656:0||save your prompts"
- "36_log|1.3|full|608:1080:1312:0||watch every job run"
- "37_docs|1.6|full|608:1080:656:0||documented from the inside"
+ # lineage + library + ecosystem
+ "65_3d-piano|6.5|full|608:1080:656:0|1.5|fly the lineage into the keys"
+ "48_galaxy-preset|1.3|full|608:1080:656:0||constellation view"
+ "14_catalogue|1.5|full|608:1080:300:0||your whole library"
+ "13_details|1.3|full|608:1080:300:0||full metadata on everything"
+ "24_focus|1.4|full|608:1080:656:0||focus on one control"
+ "17_controller|1.4|full|608:1080:656:0||map any midi controller"
+ "69_controller-vision|1.7|full|608:1080:656:0||identify a controller from a photo"
+ "20_train|1.7|full|608:1080:656:0||train it on your own sound"
+ "63_magenta-studio-live|1.8|full|608:1080:656:0||magenta rt2, on the local gpu"
+ "62_magenta-card|2.0|full|600:1080:540:0||the first non-mac magenta rt2 port"
+ "10_suno-cloud|1.4|full|608:1080:560:0||or render in the cloud"
+ "44_url-import|1.3|full|608:1080:120:0||pull audio from a link"
+ "18_media-bucket|1.2|full|608:1080:120:0||drop in any file"
+ "11_assistant-orb|1.6|full|608:1080:0:0||an assistant that runs the app"
+ "37_docs|1.5|full|608:1080:656:0||documented from the inside"
"38_settings|1.6|full|608:1080:656:0||modular, all of it"
+ "36_log|1.2|full|608:1080:1312:0||watch every job run"
)
-if [ "$ORI" = "v" ]; then W=1080; H=1920; SUBY="h-360"; SUBSZ=42; else W=1920; H=1080; SUBY="h-78"; SUBSZ=32; fi
+# Subtitles: bigger + raised off the bottom. Vertical sits higher (above the lower-third).
+if [ "$ORI" = "v" ]; then W=1080; H=1920; SUBY="h-470"; SUBSZ=58; else W=1920; H=1080; SUBY="h-128"; SUBSZ=46; fi
TMP="showcase/.roughtmp_${ORI}"
rm -rf "$TMP"; mkdir -p "$TMP"
-LIST="$TMP/list.txt"
-: > "$LIST"
-i=0
+CLIST="$TMP/clist.txt"; : > "$CLIST" # clean segments
+SLIST="$TMP/slist.txt"; : > "$SLIST" # subtitled segments
+SRT="$DIR/_showcase_${ORI}.srt"; : > "$SRT"
+
+# Crop coords in SCENES are authored in 1920x1080 space. Clips can be shot at different widths
+# across runs, so scale each clip's crop by ITS OWN source width / 1920.
+scale_crop() { echo "$1" | awk -F: -v s="$2" '{printf "%d:%d:%d:%d", 2*int($1*s/2+0.5), 2*int($2*s/2+0.5), int($3*s+0.5), int($4*s+0.5)}'; }
+srt_ts() { awk -v t="$1" 'BEGIN{h=int(t/3600); m=int((t-h*3600)/60); s=t-h*3600-m*60; printf "%02d:%02d:%06.3f", h, m, s}' | sed 's/\./,/'; }
+
+i=0; cum=0
for row in "${SCENES[@]}"; do
IFS='|' read -r id dur hcrop vcrop start sub <<< "$row"
f="$DIR/${id}_h.mp4"; [ -f "$f" ] || f="$DIR/${id}_h.webm"
if [ ! -f "$f" ]; then echo "skip (missing) ${id}"; continue; fi
if [ "$ORI" = "v" ]; then crop="$vcrop"; elif [ "$hcrop" = "full" ]; then crop=""; else crop="$hcrop"; fi
- if [ -n "$crop" ]; then geom="crop=${crop},scale=${W}:${H}"; else geom="scale=${W}:${H}:force_original_aspect_ratio=increase,crop=${W}:${H}"; fi
-
- subfile="$TMP/sub_$(printf '%02d' "$i").txt"
- printf '%s' "$sub" > "$subfile"
- draw="drawtext=fontfile=${FONT}:textfile=${subfile}:x=(w-text_w)/2:y=${SUBY}:fontsize=${SUBSZ}:fontcolor=white:box=1:boxcolor=black@0.5:boxborderw=14:line_spacing=6"
+ if [ -n "$crop" ]; then cw="$(ffprobe -v error -select_streams v:0 -show_entries stream=width -of default=nk=1:nw=1 "$f")"; sf="$(awk -v w="${cw:-1920}" 'BEGIN{printf "%.5f", w/1920}')"; crop="$(scale_crop "$crop" "$sf")"; geom="crop=${crop},scale=${W}:${H}:flags=lanczos"; else geom="scale=${W}:${H}:force_original_aspect_ratio=increase:flags=lanczos,crop=${W}:${H}"; fi
clip_dur="$(ffprobe -v error -show_entries format=duration -of default=nk=1:nw=1 "$f")"
if [ -z "$start" ]; then start="$(python -c "print(max(0.0, ${clip_dur} - ${dur} - 0.3))")"; fi
- seg="$(printf 'seg_%02d.mp4' "$i")"
- ffmpeg -y -loglevel error -ss "$start" -t "$dur" -i "$f" -an \
- -vf "${geom},${draw},fps=30,format=yuv420p" -c:v libx264 -preset veryfast -crf 20 "$TMP/$seg"
- printf "file '%s'\n" "$seg" >> "$LIST"
+
+ subfile="$TMP/sub_$(printf '%02d' "$i").txt"; printf '%s' "$sub" > "$subfile"
+ draw="drawtext=fontfile=${FONT}:textfile=${subfile}:x=(w-text_w)/2:y=${SUBY}:fontsize=${SUBSZ}:fontcolor=white:box=1:boxcolor=black@0.5:boxborderw=16:line_spacing=6"
+ cseg="$(printf 'cseg_%02d.mp4' "$i")"; sseg="$(printf 'sseg_%02d.mp4' "$i")"
+ # ONE high-quality encode each (clean + subtitled) — no second re-encode pass.
+ ffmpeg -y -loglevel error -ss "$start" -t "$dur" -i "$f" -an -vf "${geom},fps=30,format=yuv420p" -c:v libx264 -preset slow -crf 16 -pix_fmt yuv420p "$TMP/$cseg"
+ ffmpeg -y -loglevel error -ss "$start" -t "$dur" -i "$f" -an -vf "${geom},${draw},fps=30,format=yuv420p" -c:v libx264 -preset slow -crf 16 -pix_fmt yuv420p "$TMP/$sseg"
+ printf "file '%s'\n" "$cseg" >> "$CLIST"
+ printf "file '%s'\n" "$sseg" >> "$SLIST"
+
+ st="$cum"; en="$(awk -v a="$cum" -v b="$dur" 'BEGIN{printf "%.3f", a+b}')"
+ printf '%d\n%s --> %s\n%s\n\n' "$((i + 1))" "$(srt_ts "$st")" "$(srt_ts "$en")" "$sub" >> "$SRT"
+ cum="$en"
echo " + ${id} (${dur}s) — ${sub}"
i=$((i + 1))
done
-SILENT="$TMP/cat.mp4"
-ffmpeg -y -loglevel error -f concat -safe 0 -i "$LIST" -c copy "$SILENT"
-VLEN="$(ffprobe -v error -show_entries format=duration -of default=nk=1:nw=1 "$SILENT")"
+# concat (stream copy — NO extra generation) then mux the song (copy)
+CLEAN_SILENT="$TMP/clean.mp4"; ffmpeg -y -loglevel error -f concat -safe 0 -i "$CLIST" -c copy "$CLEAN_SILENT"
+SUB_SILENT="$TMP/subbed.mp4"; ffmpeg -y -loglevel error -f concat -safe 0 -i "$SLIST" -c copy "$SUB_SILENT"
+VLEN="$(ffprobe -v error -show_entries format=duration -of default=nk=1:nw=1 "$CLEAN_SILENT")"
+CLEAN_OUT="$DIR/_showcase_${ORI}_clean.mp4"
+ffmpeg -y -loglevel error -i "$CLEAN_SILENT" -i "$SONG" -map 0:v -map 1:a -t "$VLEN" -c:v copy -c:a aac -b:a 256k -shortest "$CLEAN_OUT"
OUT="$DIR/_showcase_${ORI}.mp4"
-ffmpeg -y -loglevel error -i "$SILENT" -i "$SONG" -map 0:v -map 1:a -t "$VLEN" \
- -c:v copy -c:a aac -b:a 256k -shortest "$OUT"
+ffmpeg -y -loglevel error -i "$SUB_SILENT" -i "$SONG" -map 0:v -map 1:a -t "$VLEN" -c:v copy -c:a aac -b:a 256k -shortest "$OUT"
rm -rf "$TMP"
-echo "wrote $OUT (${VLEN}s, ${W}x${H})"
+echo "wrote $OUT + $CLEAN_OUT + $SRT (${VLEN}s, ${W}x${H})"
diff --git a/showcase/clips-recorded/_showcase_h.mp4 b/showcase/clips-recorded/_showcase_h.mp4
index 97ac3ca..ba032fd 100644
Binary files a/showcase/clips-recorded/_showcase_h.mp4 and b/showcase/clips-recorded/_showcase_h.mp4 differ
diff --git a/showcase/clips-recorded/_showcase_v.mp4 b/showcase/clips-recorded/_showcase_v.mp4
index 8dabf91..164e268 100644
Binary files a/showcase/clips-recorded/_showcase_v.mp4 and b/showcase/clips-recorded/_showcase_v.mp4 differ