Skip to content

Add Tone3000 integration#165

Draft
lfagundes wants to merge 14 commits into
masterfrom
tone3000
Draft

Add Tone3000 integration#165
lfagundes wants to merge 14 commits into
masterfrom
tone3000

Conversation

@lfagundes

Copy link
Copy Markdown
Contributor
  • New T3K tab in UI
  • A popup window to navigate in T3K
  • Downloading a tone creates a folder inside NAM Models folder
  • Models list on running plugin instances is updated, with new files on top
  • A frontend test suite, plus backend tests for the upload

lfagundes and others added 14 commits July 9, 2026 13:06
pytest + tornado AsyncHTTPTestCase harness running against the real
web.Application with a faked audio backend (no JACK, mod-host or
hardware needed). Covers /files/list behavior (type mapping, extension
filtering, subfolder recursion, ordering, per-request freshness), the
index page render, and the CORS header conventions.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CORS: split the hello tests into one allowed-origin base class with the
origin as a class property (subclassed per allowed origin variant) and a
separate foreign-origin class.

Conftest: point MOD_USER_PEDALBOARDS_DIR/MOD_USER_PLUGINS_DIR at the temp
tree (defaults are the real ~/.pedalboards and ~/.lv2), abort the run if
any writable mod.settings path escapes the test root, and document the
routes tests must never call.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Covers favorites, config/set, save_user_id, tokens, banks, system
info/prefs, hello, ping, template loaders, static-file and JSON handler
header conventions. Pins current behavior, including 500s on /auth/nonce
without device credentials, /tokens/save without expires_in_days, and
missing templates, plus the banks.json rewrite side effect of GET /banks/.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Save/info/remove/load_bundle round trip against the sandboxed pedalboards
dir under the fake host, plus snapshot list/name/save/saveas behavior
(including the Default snapshot seeded by SESSION.reset).

Key finding recorded in the conftest never-call list: /pedalboard/list and
/banks/ segfault (lilv_new_uri on the uninitialized global lilv world) as
soon as a real pedalboard bundle exists on disk; both are only safe against
an empty pedalboards dir, so the round trip verifies bundle presence via
the filesystem and only lists once the dir is empty again. Conftest also
gains an autouse fixture resetting the pedalboards dir between tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pins webserver handler behavior under FakeHost/FakeHMI (not the production
mod-host protocol): connect/disconnect always true, parameter set
short-circuits on uninitialized HMI, reset, buffersize {ok:false,size:0}
in dev, xruns, midi device shapes (500 on missing body keys), truebypass
false, transport sync modes, cv port add 500 without plugin instances.

/effect/remove/<instance> is added to the never-call list: for any
unregistered instance the KeyError from mapper.get_id_without_creating is
swallowed by the gen.coroutine future in Host.remove_plugin, the callback
never fires, and the request hangs forever.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pins the connect-time report_current_state burst pushed to every socket
(sys_stats, stats, transport, truebypass, loading_start, size, then the
loading_end ready marker), payload shapes for transport/truebypass/size,
the identical burst on a second concurrent connection, and clean client
close tolerance. Prefixes and arg counts only; volatile payloads unpinned.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
/effect/* endpoints need the global lilv world initialized or they
segfault, so these tests are excluded from the default run via markers
(lilv: empty-world behavior; lilv_fixture: a binaryless .lv2 fixture
bundle). LV2_PATH points at an empty sandbox dir for determinism.

Pins: /effect/list -> [], /effect/get on unknown uri -> 404 HTML,
/effect/bulk skips unknown uris and 501s without a JSON content type,
/effect/add on unknown uri mutates host state before failing late with
404. The two marker sets must never run in one process: a second
modtools.utils.init() corrupts lilv's namespace singleton and segfaults
at interpreter exit (documented in pytest.ini and both test modules).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
mod-ui lists the user-files tree (GET /files/list) but has never written
to it: uploads belong to the separate file-manager service on port 8081,
which is a human-facing iframe a page cannot drive programmatically. The
upcoming Tone3000 integration downloads model files in the browser and
needs exactly one server-side capability -- writing the received bytes to
the device's disk -- so add the narrowest route that does that.

POST /files/upload/<filetype>?folder=X&name=Y with an
application/octet-stream body writes USER_FILES_DIR/<category>/X/Y. The
folder/extension policy is reused from
FilesList._get_dir_and_extensions_for_filetype, so only filetypes that
map to a real category folder are accepted, with their extension
enforced. folder and name must each be a single path component --
anything os.path.basename would rewrite is refused rather than silently
rewritten -- and the content-type gate keeps every cross-origin POST
behind a CORS preflight that mod-ui never answers. The response carries
the same fullname string /files/list builds by walking, so a caller can
find its own upload in the list.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TONE3000 (tone3000.com) hosts NAM amp-capture models -- the files the
Neural Amp Modeler plugin loads from the NAM Models user-files folder.
This adds the entry point for browsing it from the device UI: a new icon
in the bottom-left #main-menu cluster and its full-screen overlay panel,
cloned from the File Manager precedent (trigger icon + #*-library panel +
JqueryClass box + Desktop wiring + statusTooltip).

The panel itself only introduces the tab and explains what it is for;
the way into the catalog lands next.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The tab cannot embed tone3000.com in an iframe: TONE3000's sign-in cookie
is SameSite=Lax, so it is never sent on a cross-site iframe subrequest --
the OTP verifies, the cookie comes back, the next request drops it, and
the user silently lands on the login screen again, in every browser. Only
TONE3000 could change that. A popup is a top-level browsing context, so
its cookies are first-party and none of this applies.

So the Open button starts TONE3000's OAuth 2.0 + PKCE "Select" flow in a
popup sized and placed to cover the panel's content area: mint the
verifier/state pair into sessionStorage, open the window synchronously
(while we still hold the user activation from the click), then navigate
it to the authorize URL, restricted to format=nam since that is what this
device's NAM plugin can load. The client id is the OAuth publishable key
-- a public value that grants no data access on its own -- configured via
MOD_TONE3000_CLIENT_ID and handed to the page through the index template.

The popup deliberately survives tab switches (no windowclose handler):
closing it whenever the window manager raises another panel would throw
away the user's place in the catalog. Only leaving the page closes it.

Note crypto.subtle only exists in secure contexts. localhost is one; the
real device, on plain http:// over a LAN IP, is not -- a pure-JS sha256
will be needed before this ships to hardware.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Once the user picks a tone, TONE3000 redirects the popup to
/tone3000-callback.html with an authorization code. That page is served
from mod-ui's own origin, so it is same-origin with the window that
opened it and reads the PKCE verifier and CSRF state live through
window.opener -- which means the whole tail of the flow can run right
there in the popup: verify state, exchange the code, read the tone and
its models, and land the files on the device.

A tone is a group of models -- the same capture at several sizes and
architectures -- so the selection gives a tone_id, not a file. The page
reads the tone, pages through its models, keeps the .nam ones, and
downloads them one by one with the Bearer token (every TONE3000 data
endpoint, the model_url file included, requires it; /api/v1/* sends
Access-Control-Allow-Origin: * so this works from our origin and the
token never reaches the device). Each file is handed to our
/files/upload route, into a per-tone folder under NAM Models named
"<title> (<tone id>)", with filenames kept human-readable and
deduplicated only when they actually collide.

The page mirrors what the user will find afterwards in the File Manager:
the tone's folder with one row per model, each row rendered up front and
lighting up in place -- downloading, saved, or failed with the reason on
hover -- as the chain reaches it. Since the authorization code is
single-use, "Continue browsing" mints a fresh authorize URL through the
opener and reuses the popup for it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…n top

A plugin's file dropdown is filled once, from the /files/list snapshot
taken while its GUI was being built, and nothing ever re-reads it -- so a
file downloaded afterwards stays invisible until the plugin is removed
and re-added. The backend scan is always live; the staleness is entirely
client-side.

Give GUI a refreshFileTypesLists(fileType, hoist): re-fetch the lists,
re-render the plugin's own icon/settings templates off-DOM, and swap in
just the file-list widgets, re-registering each through
assignControlFunctionality so the current selection is restored from
parameter.value. Going through the template is what makes this work for
any plugin -- we never build the option nodes ourselves, so we never
assume how the author nested them. Re-rendering the whole icon is not an
option: its port elements carry the jsPlumb endpoints the connection
manager holds on to, and replacing them would cut every cable into the
plugin. The expand-button binding moves into assignFileListExpand so the
swapped-in widget gets it re-applied -- a list that was short enough at
build time may not be after a download.

The callback page hands the just-written paths to every NAM plugin on
the board through tone3000RefreshFileLists; each refreshed dropdown
lifts those to the front, so the tone the user just downloaded is the
first thing in the list instead of buried wherever its name happens to
sort. Only for as long as that GUI lives -- a reload takes the list
straight from /files/list again, in its own order.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
For someone whose whole session is tone-hunting, the intermediate panel
is a click that says nothing new. Add an opt-in checkbox that opens the
popup straight from selecting the tab: windowopen fires inside the click
that selected it, so the user activation window.open needs is still
held -- any later and the browser blocks it as unsolicited.

The preference is stored browser-side (localStorage, guarded -- it
throws outright when storage is disabled, rather than degrading): it is
a preference about this browser's popup, not device state, and there is
no user-settings route to put it behind.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Tone3000 work leans on two pieces of frontend behavior that nothing
exercised: the modgui file-list pipeline (including the new in-place
refresh, selection restore and hoist-to-top) and the tone3000Box wiring
(the auto-open preference, and the popup being focused -- never re-opened
or re-navigated -- when the tab is selected again).

The app itself is browser-global JS served from html/, so the harness
builds a jsdom window, loads the real html/js files into it, and stubs
only what jsdom cannot provide (window.open). package.json exists solely
to run this suite under Node's built-in test runner; the modgui tests
drive a vendored fixture template that reproduces the one property they
need from a real plugin icon -- file options nested inside a wrapping
element -- so the suite runs in a standalone clone. A jstest workflow
runs it in CI next to pylint.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant