Add Ampache media server backend that supports Ampache's native JSON API (API8) - #810
Add Ampache media server backend that supports Ampache's native JSON API (API8)#810lachlan-00 wants to merge 27 commits into
Conversation
…n setup/migration UI, add provider-contract tests
There was a problem hiding this comment.
Hi,
this PR missed a big chunck of functionality, I added but I don't have Ampache so please do a VERY DETAILED check of all the functionality (online and offline) with a focus on:
- Setup Wizard: Try to install from scratch AudioMuse-AI and select Ampache as proivder;
- Multiple provider: Still in the setup wizard, try to add a second music provider.
- Provider Migration: try to migrate to ampache to another and vice versa;
Also please explain why the default Navidrome API, that implement Open Subosnic API, doesn't work for Ampache.
Here a details of the fix that I made that I also ask you to doublecheck:
-
create_or_replace_playlistcrashed on every call because Ampache accepted 2 parameters while the dispatcher passed 3. This caused aTypeErrorduring Sonic Fingerprint cron and Alchemy Radio runs. Fix: addeduser_creds=Noneand passed it toget_playlist_by_name. -
Instant playlists had the wrong name because Ampache did not append
_instantlike the other backends. This made instant playlists collide with manually created playlists. Fix: append_instantwhen creating the playlist. -
Failed downloads were written to disk as audio because the HTTP status was not checked before saving the stream. An error response could therefore be saved as an MP3 file. Fix: added
raise_for_status()before returning the stream. -
Sonic Fingerprint used ratings instead of play counts because
get_top_played_songsused thehighestfilter. Ampache defineshighestas highest rated andfrequentas most played. Fix: changedhighesttofrequent. -
Ampache was missing from both server type lists in the setup wizard. Fix: added the Ampache option and credential hint.
-
Ampache credential fields were not displayed in the setup wizard because
static/setup.jshad no Ampache configuration. The password was also not marked as secret, and values were lost when changing server type. Fix: added the URL, username, and password fields and preserved their values. -
Ampache could not be added as a second server because
static/music_servers_admin.jshad no Ampache credential definition. Fix: added URL, username, and password fields. -
Ampache could not be selected as a Provider Migration target because the option and credential fields were missing from the page. Fix: added both and updated the page introduction.
-
Migration from Ampache failed because
_current_provider_credshad no Ampache branch. Fix: replaced the hardcoded provider logic with the mappings already defined inconfig.py. This also supports future providers automatically. -
BASIC_SERVER_FIELDSdid not include the Ampache configuration keys. Fix: added the threeAMPACHE_*keys. -
Ampache parameters were missing from
docs/PARAMETERS.md, the README, andMULTI_SERVER.md. Fix: added the new parameters and updated the supported provider lists. The tested version badge was left unchanged because no Ampache version was verified. -
The new
ampache.pyfile broke CI because it used the wrong license header and had noMain Features:section. Fix: added the project AGPL header, the required section, and two missing# noqa: TRY400comments. -
The 422-line Ampache backend had no direct test coverage. Fix: added
test_mediaserver_ampache.pywith 18 tests. -
There was no test to detect incomplete provider registration across Python, JavaScript, HTML, and documentation. Fix: added
test_provider_registration_contract.pywith 50 tests.
- Library picker works.
list_librariesreturns lower caseid/namelike the
other backends. Before, the wizard showed no catalogs and the admin page saved
[object Object], so the server returned zero songs. - API key only installs can be saved.
AMPACHE_USERis now optional, as the docs
always said. Before, every save path rejected an empty username. - Playlist creation returns a dictionary, not a string. Fixes the crash in the
Sonic Fingerprint cron and the false 500 error in the chat playlist endpoint. - Broken downloads are detected. Ampache sends API errors with HTTP 200 and JSON,
which was written to disk as an audio file. An expired session now retries. - Recent albums respect the library filter. The backend pages until it has enough.
Before, an install could stall and never analyse anything. - Full library fetch filters on the server instead of downloading everything and
dropping most of it. Tracks keep only the 11 fields consumers read. - The password is no longer sent in clear text as a fallback API key in the URL.
- A changed password no longer passes the connection test. The token cache now
includes the password, so a rotated one cannot reuse the old session. - Playlist failures are visible. An empty playlist is not reported as created, and
a failed delete stops the replace instead of creating a duplicate name. - Playlist lookup uses the caller credentials, not the default server.
- The lyrics timeout also limits the login handshake, so a slow server cannot block
the worker for 60 seconds. - Connection test warns on relative paths, and dead or duplicated code was removed.
…th, playlist return contract, catalogue/album filtering, and corrupt-download detection.
|
I can do all that testing no problem! I have integrated develop.ampache.dev which is a small library but for my large home library the scanning very slow process Is there a way to add more workers or speed that up? It's still got a long way to go. |
|
Please keep in mind that I just did a code review with CLAUDE, my expectations is that you finalize the review and you do real test because I don’t have Ampache installed at all. Also please give a check to SonarCloud issue. About analysis speed up there is two important way:
|
|
I've set up my tester a bit better with better worker split and postgres config that should speed it up. I will run through it all again and sonarcube as well and keep updating the pr, haven't reviewed what's changed yet just getting it set up for now so will take me a bit to complete all that |
* add docs/AMPACHE.md * default to https in messaging and hints * use _log_error function in to redact URL from logs * catalog id's are not in responses (yet) but the albums call can filter on conditions (e.g. cond=catalog,32) so we can filter on api6 and 8 with that
Every provider reports the path it holds a track at, so an install whose library is mounted into the container can read the file directly: no HTTP round trip, no second copy of the bytes, and no load on the media server. Provider-agnostic - it lives in the dispatcher, and all six backends populate Path/FilePath. Off by default. LOCAL_FILE_ACCESS enables it, LOCAL_FILE_ROOTS allowlists the directories a track may come from, and LOCAL_FILE_PATH_MAP rewrites the server's prefix onto this container's mount point. Two properties the implementation is built around: * The pipeline DELETES whatever download_track returns (the finally in tasks/analysis/album.py), so the library file is never returned. What the caller gets is a link inside TEMP_DIR - a symlink, or a hardlink where symlinks are not permitted - and removing it never touches the target. * The path comes from the media server and is therefore untrusted. The REAL location is resolved first, so a symlink planted in the library cannot escape, and anything outside LOCAL_FILE_ROOTS is refused. With no root configured nothing is read at all. Every failure returns None and the track is downloaded as before, so local access is an optimisation that cannot become a new way for analysis to fail. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This reverts commit e928514.
|
Even with all the mitigations for speed I've put in it's going to take more than 6 months to scan my home server. I'll do the testing with a smaller library by skipping my biggest catalog. I've added 3 scripts that i'll squash down that i'm using as evidence to build on. and will report when done |
This reverts commit a217afb.
|



testplan.md
report.md
AS-IS
Does not exist. AudioMuse-AI supports Jellyfin, Emby, Navidrome, Lyrion and Plex.
Ampache can be connected today via the navidrome backend, because Ampache also serves
the Subsonic API - but that path only sees what Subsonic exposes, and indexes tracks
under Subsonic's prefixed id form rather than Ampache's own ids.
TO-BE
Adds an
ampachemedia server backend that speaks Ampache's native JSON API.tasks/mediaserver/ampache.pyimplements the full provider surface:get_all_songs,get_recent_albums,get_tracks_from_album,get_album_track_ids,search_albums,download_track,list_libraries,test_connection, the playlist functions,get_top_played_songsandget_lyrics.Requires Ampache API 8 or newer (
_MIN_API_MAJOR = 8). Album browsing depends onthe
catalogid being present on album objects and on thecondbrowse filter.test_connectionwarns when a server reports an older API, so the setup wizard saysso rather than an analysis quietly finding nothing.
Notes on a few choices:
Gatekeeper::getAuth()readsAuthorization: Bearerbefore it looks at the query string, andApiHandlerresolves the user via
findByApiKeyand opens the session itself. For a key-shapedsecret (32+ hex) the backend settles this with a single bearer
pingper credentialset, then every request carries the header and omits
authentirely - so no secretreaches the URL or the web server's access log. A refusal is remembered and falls
back to the handshake; a network blip is deliberately not remembered.
findByApiKeywould only refuse it, and beingrefused means having put the password on the wire for nothing. A password handshakes
and sends a time-salted hash, never the password itself.
Session::extend()resets the expiry on every authenticated call, so the cached window is slid forward
on each accepted request and re-issued only once it has genuinely lapsed. The real
window comes from the server's
session_expire. The 4701 retry remains the backstop.MUSIC_LIBRARIESis pushed into the query, resolved to Ampache catalog ids andsent as
cond=catalog,<id>- one browse per catalogue, sincecondconditionscombine rather than alternate. It is still enforced locally, because Ampache
ignores a
condit does not understand instead of refusing it; a browse whoserows report another catalogue abandons the per-catalogue plan for one unfiltered walk
filtered locally. A failed page is retried once with the filter intact, and a
catalogue that cannot be completed is logged as
INCOMPLETErather than returned asif whole - cleaning prunes against that list.
AMPACHE_PAGE_SIZE(default 500) sets rows per browse page. The cost isserver-side per-song work, not network, so raising it cuts request count and not the
work; values above roughly 2300 cannot complete inside the 60s request timeout.
needs ids and a count, so it uses
browse(Catalog::get_name_array) instead ofalbum_songs, which skips hydrating every song's rating, art, album and artist. Anyanswer that cannot be trusted - unknown catalogue id, a
total_countthat disagreeswith the rows returned - falls back to
album_songs, because a short list would makean unfinished album look complete. This is an optional dispatcher capability
(
get_album_track_ids); the other five providers are untouched and fall backautomatically.
single
songand analbum_songsrow through the sameJson8_Data::songs_array, sothe per-track
songcall was re-paying that row's whole hydration cost to re-read onefield.
download, notstream, so analysis sees the original file rather thana transcode, and a JSON error body arriving under HTTP 200 is refused rather than
saved as audio.
get_last_played_timereturnsNone- Ampache exposes no per-track last-playedtimestamp, so recency-weighted callers fall back to play counts.
keyed differently from the same library through navidrome. Both dedupe to the same
fingerprint, so a library can be registered twice with no re-analysis.
Registration is the usual set:
config.MEDIASERVER_FIELDS_BY_TYPE/MEDIASERVER_CRED_KEY_BY_FIELD/AMPACHE_*globals,_PROVIDER_NAMES,_SUPPORTED_TYPES,_SUPPORTED_PROVIDERSin the migration probe,_SUPPORTED_TARGETSin provider migration, and
AMPACHE_PASSWORDinSECRET_FIELDSso the wizard masks it.The setup wizard picks the fields up automatically from
MEDIASERVER_FIELDS_BY_TYPE.Test
Unit:
pytest test/unit/ -q-> 3067 passed, 1 skipped. 84 cases intest/unit/test_mediaserver_ampache.py, plustest/unit/test_provider_registration_contract.py, which enforces that every backendbinds to every dispatcher call site and that Ampache is registered in config, the
dispatcher, both JavaScript files, both HTML dropdowns and
docs/PARAMETERS.md.Initial functional pass, against Ampache 8.0.0 (develop) in Docker with
AudioMuse-AI from
deployment/docker-compose.yaml:POST /api/servers/testwithserver_type: ampache->{"ok": true, "sample_count": 2, "path_format": "absolute"}POST /api/servers, then ranPOST /api/analysis/startscoreandtrack_server_mapGET /api/similar_tracks?item_id=1returned a neighbour with a distance scorein
track_server_mapagainst the sameitem_idfingerprints -1/3001(ampache,match_tier=path) andso-1/so-3001(navidrome,match_tier=fingerprint) -confirming dedup treats them as one track and no re-analysis happens.
Verified end to end through Ampache's OpenSubsonic
getSonicSimilarTracks, whichreturned the expected neighbour at the expected similarity with either backend as
default.
Live-server pass, on a production Ampache 8 install with two servers registered
(the default connected as navidrome, the second through this connector). Evidence taken
from the Apache access log for the Ampache vhost, filtered to
json.server.phpso thevhost's own web-UI traffic is excluded:
action=albums, paged rather than probed per albumcond=catalog,<id>present on the albums browseping, 0handshake, noauth=in any query stringaction=songacross the run withLYRICS_ENABLED=truebrowse(type=album)vs 246album_songs- the remainingalbum_songsare the album jobs fetching real metadata, one per album analysedAMPACHE ... FETCH FAILED/INCOMPLETE/RETURNED NO ALBUMSin worker logsNot claimed, as each needs deliberate fault injection or a different configuration:
session invalidated mid-analysis, an injected page failure, a short
total_count, anignored
cond, a refused API key, and a password-configured run.Other useful information
Ampache's side of this is a plugin implementing a new sonic-analysis plugin type, which
backs the OpenSubsonic
sonicSimilarityextension (getSonicSimilarTracks,findSonicPath). It tries both id forms, so it works with this backend or withnavidrome and needs no configuration either way.
One upstream detail worth flagging:
/api/find_pathreturns no per-hop distance, only asingle
total_distance. Consumers that assume a per-hop score will read a missing keyas
0- i.e. a perfect match.Checklist
Type of change:
Tested on architecture:
Tested on media server:
Updated:
docs/AMPACHE.md,docs/PARAMETERS.md)Other:
NOTES
get_last_played_timeis coming to Ampache 8.Page-size behaviour was measured on a 659,593-song Ampache 8 install: roughly 38
songs/second and 1.7 MB per 500-row response, with the rate not degrading as offset
grows, so cost is linear in library size. A full-library analysis run on that install is
still to come, along with demo connections via https://develop.ampache.dev
(https://demo.ampache.dev after the Ampache 8 release).
This will be released as part of Ampache 8 / API 8.