Skip to content

Add Ampache media server backend that supports Ampache's native JSON API (API8) - #810

Open
lachlan-00 wants to merge 27 commits into
NeptuneHub:mainfrom
lachlan-00:AmpacheConnector
Open

Add Ampache media server backend that supports Ampache's native JSON API (API8)#810
lachlan-00 wants to merge 27 commits into
NeptuneHub:mainfrom
lachlan-00:AmpacheConnector

Conversation

@lachlan-00

@lachlan-00 lachlan-00 commented Jul 27, 2026

Copy link
Copy Markdown

testplan.md
report.md

AS-IS

Does not exist. AudioMuse-AI supports Jellyfin, Emby, Navidrome, Lyrion and Plex.

Ampache can be connected today via the navidrome backend, because Ampache also serves
the Subsonic API - but that path only sees what Subsonic exposes, and indexes tracks
under Subsonic's prefixed id form rather than Ampache's own ids.

TO-BE

Adds an ampache media server backend that speaks Ampache's native JSON API.

tasks/mediaserver/ampache.py implements the full provider surface: get_all_songs,
get_recent_albums, get_tracks_from_album, get_album_track_ids, search_albums,
download_track, list_libraries, test_connection, the playlist functions,
get_top_played_songs and get_lyrics.

Requires Ampache API 8 or newer (_MIN_API_MAJOR = 8). Album browsing depends on
the catalog id being present on album objects and on the cond browse filter.
test_connection warns when a server reports an older API, so the setup wizard says
so rather than an analysis quietly finding nothing.

Notes on a few choices:

  • An API key needs no handshake at all. Gatekeeper::getAuth() reads
    Authorization: Bearer before it looks at the query string, and ApiHandler
    resolves the user via findByApiKey and opens the session itself. For a key-shaped
    secret (32+ hex) the backend settles this with a single bearer ping per credential
    set, then every request carries the header and omits auth entirely - so no secret
    reaches the URL or the web server's access log. A refusal is remembered and falls
    back to the handshake; a network blip is deliberately not remembered.
  • A password never takes that path. findByApiKey would only refuse it, and being
    refused means having put the password on the wire for nothing. A password handshakes
    and sends a time-salted hash, never the password itself.
  • Sessions slide rather than expiring on a timer. Ampache's Session::extend()
    resets the expiry on every authenticated call, so the cached window is slid forward
    on each accepted request and re-issued only once it has genuinely lapsed. The real
    window comes from the server's session_expire. The 4701 retry remains the backstop.
  • MUSIC_LIBRARIES is pushed into the query, resolved to Ampache catalog ids and
    sent as cond=catalog,<id> - one browse per catalogue, since cond conditions
    combine rather than alternate. It is still enforced locally, because Ampache
    ignores a cond it does not understand instead of refusing it; a browse whose
    rows report another catalogue abandons the per-catalogue plan for one unfiltered walk
    filtered locally. A failed page is retried once with the filter intact, and a
    catalogue that cannot be completed is logged as INCOMPLETE rather than returned as
    if whole - cleaning prunes against that list.
  • AMPACHE_PAGE_SIZE (default 500) sets rows per browse page. The cost is
    server-side per-song work, not network, so raising it cuts request count and not the
    work; values above roughly 2300 cannot complete inside the 60s request timeout.
  • The dispatch loop asks for ids, not track metadata. Deciding what to enqueue only
    needs ids and a count, so it uses browse (Catalog::get_name_array) instead of
    album_songs, which skips hydrating every song's rating, art, album and artist. Any
    answer that cannot be trusted - unknown catalogue id, a total_count that disagrees
    with the rows returned - falls back to album_songs, because a short list would make
    an unfinished album look complete. This is an optional dispatcher capability
    (get_album_track_ids); the other five providers are untouched and fall back
    automatically.
  • Lyrics come from the album fetch that already returned them. Ampache serialises a
    single song and an album_songs row through the same Json8_Data::songs_array, so
    the per-track song call was re-paying that row's whole hydration cost to re-read one
    field.
  • Downloads use download, not stream, so analysis sees the original file rather than
    a transcode, and a JSON error body arriving under HTTP 200 is refused rather than
    saved as audio.
  • get_last_played_time returns None - Ampache exposes no per-track last-played
    timestamp, so recency-weighted callers fall back to play counts.
  • Track ids are Ampache's own row ids, so a library analysed through this backend is
    keyed differently from the same library through navidrome. Both dedupe to the same
    fingerprint, so a library can be registered twice with no re-analysis.

Registration is the usual set: config.MEDIASERVER_FIELDS_BY_TYPE /
MEDIASERVER_CRED_KEY_BY_FIELD / AMPACHE_* globals, _PROVIDER_NAMES,
_SUPPORTED_TYPES, _SUPPORTED_PROVIDERS in the migration probe, _SUPPORTED_TARGETS
in provider migration, and AMPACHE_PASSWORD in SECRET_FIELDS so the wizard masks it.
The setup wizard picks the fields up automatically from MEDIASERVER_FIELDS_BY_TYPE.

Test

Unit: pytest test/unit/ -q -> 3067 passed, 1 skipped. 84 cases in
test/unit/test_mediaserver_ampache.py, plus
test/unit/test_provider_registration_contract.py, which enforces that every backend
binds to every dispatcher call site and that Ampache is registered in config, the
dispatcher, both JavaScript files, both HTML dropdowns and docs/PARAMETERS.md.

Initial functional pass, against Ampache 8.0.0 (develop) in Docker with
AudioMuse-AI from deployment/docker-compose.yaml:

  1. POST /api/servers/test with server_type: ampache ->
    {"ok": true, "sample_count": 2, "path_format": "absolute"}
  2. Registered it as default via POST /api/servers, then ran POST /api/analysis/start
  3. Analysis completed; tracks downloaded and analysed, rows written to score and
    track_server_map
  4. GET /api/similar_tracks?item_id=1 returned a neighbour with a distance score
  5. Registered the same library a second time through the navidrome backend. Both appear
    in track_server_map against the same item_id fingerprints - 1/3001 (ampache,
    match_tier=path) and so-1/so-3001 (navidrome, match_tier=fingerprint) -
    confirming dedup treats them as one track and no re-analysis happens.

Verified end to end through Ampache's OpenSubsonic getSonicSimilarTracks, which
returned the expected neighbour at the expected similarity with either backend as
default.

Live-server pass, on a production Ampache 8 install with two servers registered
(the default connected as navidrome, the second through this connector). Evidence taken
from the Apache access log for the Ampache vhost, filtered to json.server.php so the
vhost's own web-UI traffic is excluded:

Check Result Evidence
Album discovery PASS 2 action=albums, paged rather than probed per album
Library filter applied server-side PASS cond=catalog,<id> present on the albums browse
API key uses the bearer header PASS 258 ping, 0 handshake, no auth= in any query string
No secret in any log PASS literal secret: 0 hits in worker/flask logs, 0 in Apache logs, 0 in the Ampache log
Lyrics reuse the album fetch PASS 0 action=song across the run with LYRICS_ENABLED=true
Dispatch loop browses for ids PASS 234 browse(type=album) vs 246 album_songs - the remaining album_songs are the album jobs fetching real metadata, one per album analysed
No silent incompleteness PASS no AMPACHE ... FETCH FAILED / INCOMPLETE / RETURNED NO ALBUMS in worker logs
Ampache-side errors PASS no server errors in the Ampache log for the window
Throughput PASS 3236 tracks downloaded and analysed in one run, no aborted pages

Not claimed, as each needs deliberate fault injection or a different configuration:
session invalidated mid-analysis, an injected page failure, a short total_count, an
ignored cond, a refused API key, and a password-configured run.

Other useful information

Ampache's side of this is a plugin implementing a new sonic-analysis plugin type, which
backs the OpenSubsonic sonicSimilarity extension (getSonicSimilarTracks,
findSonicPath). It tries both id forms, so it works with this backend or with
navidrome and needs no configuration either way.

One upstream detail worth flagging: /api/find_path returns no per-hop distance, only a
single total_distance. Consumers that assume a per-hop score will read a missing key
as 0 - i.e. a perfect match.

Checklist

Type of change:

  • New feature

Tested on architecture:

  • Intel

Tested on media server:

  • Ampache 8.0.0 (develop)
  • Navidrome (regression check - same library through both backends)
  • Jellyfin
  • Emby
  • Lyrion

Updated:

  • Documentation (docs/AMPACHE.md, docs/PARAMETERS.md)
  • Unit test
  • Integration test

Other:

  • CONTRIBUTING.md read and accepted
  • Checked performance on a big library (> 150k songs) works without issues

NOTES

get_last_played_time is coming to Ampache 8.

Page-size behaviour was measured on a 659,593-song Ampache 8 install: roughly 38
songs/second and 1.7 MB per 500-row response, with the rate not degrading as offset
grows, so cost is linear in library size. A full-library analysis run on that install is
still to come, along with demo connections via https://develop.ampache.dev
(https://demo.ampache.dev after the Ampache 8 release).

This will be released as part of Ampache 8 / API 8.

Copilot AI review requested due to automatic review settings July 27, 2026 06:13

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@NeptuneHub NeptuneHub left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi,
this PR missed a big chunck of functionality, I added but I don't have Ampache so please do a VERY DETAILED check of all the functionality (online and offline) with a focus on:

  • Setup Wizard: Try to install from scratch AudioMuse-AI and select Ampache as proivder;
  • Multiple provider: Still in the setup wizard, try to add a second music provider.
  • Provider Migration: try to migrate to ampache to another and vice versa;

Also please explain why the default Navidrome API, that implement Open Subosnic API, doesn't work for Ampache.

Here a details of the fix that I made that I also ask you to doublecheck:

  • create_or_replace_playlist crashed on every call because Ampache accepted 2 parameters while the dispatcher passed 3. This caused a TypeError during Sonic Fingerprint cron and Alchemy Radio runs. Fix: added user_creds=None and passed it to get_playlist_by_name.

  • Instant playlists had the wrong name because Ampache did not append _instant like the other backends. This made instant playlists collide with manually created playlists. Fix: append _instant when creating the playlist.

  • Failed downloads were written to disk as audio because the HTTP status was not checked before saving the stream. An error response could therefore be saved as an MP3 file. Fix: added raise_for_status() before returning the stream.

  • Sonic Fingerprint used ratings instead of play counts because get_top_played_songs used the highest filter. Ampache defines highest as highest rated and frequent as most played. Fix: changed highest to frequent.

  • Ampache was missing from both server type lists in the setup wizard. Fix: added the Ampache option and credential hint.

  • Ampache credential fields were not displayed in the setup wizard because static/setup.js had no Ampache configuration. The password was also not marked as secret, and values were lost when changing server type. Fix: added the URL, username, and password fields and preserved their values.

  • Ampache could not be added as a second server because static/music_servers_admin.js had no Ampache credential definition. Fix: added URL, username, and password fields.

  • Ampache could not be selected as a Provider Migration target because the option and credential fields were missing from the page. Fix: added both and updated the page introduction.

  • Migration from Ampache failed because _current_provider_creds had no Ampache branch. Fix: replaced the hardcoded provider logic with the mappings already defined in config.py. This also supports future providers automatically.

  • BASIC_SERVER_FIELDS did not include the Ampache configuration keys. Fix: added the three AMPACHE_* keys.

  • Ampache parameters were missing from docs/PARAMETERS.md, the README, and MULTI_SERVER.md. Fix: added the new parameters and updated the supported provider lists. The tested version badge was left unchanged because no Ampache version was verified.

  • The new ampache.py file broke CI because it used the wrong license header and had no Main Features: section. Fix: added the project AGPL header, the required section, and two missing # noqa: TRY400 comments.

  • The 422-line Ampache backend had no direct test coverage. Fix: added test_mediaserver_ampache.py with 18 tests.

  • There was no test to detect incomplete provider registration across Python, JavaScript, HTML, and documentation. Fix: added test_provider_registration_contract.py with 50 tests.

  • Library picker works. list_libraries returns lower case id/name like the
    other backends. Before, the wizard showed no catalogs and the admin page saved
    [object Object], so the server returned zero songs.
  • API key only installs can be saved. AMPACHE_USER is now optional, as the docs
    always said. Before, every save path rejected an empty username.
  • Playlist creation returns a dictionary, not a string. Fixes the crash in the
    Sonic Fingerprint cron and the false 500 error in the chat playlist endpoint.
  • Broken downloads are detected. Ampache sends API errors with HTTP 200 and JSON,
    which was written to disk as an audio file. An expired session now retries.
  • Recent albums respect the library filter. The backend pages until it has enough.
    Before, an install could stall and never analyse anything.
  • Full library fetch filters on the server instead of downloading everything and
    dropping most of it. Tracks keep only the 11 fields consumers read.
  • The password is no longer sent in clear text as a fallback API key in the URL.
  • A changed password no longer passes the connection test. The token cache now
    includes the password, so a rotated one cannot reuse the old session.
  • Playlist failures are visible. An empty playlist is not reported as created, and
    a failed delete stops the replace instead of creating a duplicate name.
  • Playlist lookup uses the caller credentials, not the default server.
  • The lyrics timeout also limits the login handshake, so a slow server cannot block
    the worker for 60 seconds.
  • Connection test warns on relative paths, and dead or duplicated code was removed.

@lachlan-00

Copy link
Copy Markdown
Author

I can do all that testing no problem!

I have integrated develop.ampache.dev which is a small library but for my large home library the scanning very slow process

Is there a way to add more workers or speed that up? It's still got a long way to go.

@NeptuneHub

Copy link
Copy Markdown
Owner

Please keep in mind that I just did a code review with CLAUDE, my expectations is that you finalize the review and you do real test because I don’t have Ampache installed at all. Also please give a check to SonarCloud issue.

About analysis speed up there is two important way:

  • raise a new worker, you can deploy on another machine just be sure it reach Redis, PostgreSQL and music server. In docs/architecture.md it explained a bit and in /deployment/test you have an example of docker compose of worker only;
  • be sure to have configured lyrics API OR that the music server return lyrics. This because ASR is the slowest thing, doing on few tracks as fallback is ok, if you had to do on thousand song it can take very slow.

@lachlan-00

Copy link
Copy Markdown
Author

I've set up my tester a bit better with better worker split and postgres config that should speed it up.

I will run through it all again and sonarcube as well and keep updating the pr, haven't reviewed what's changed yet just getting it set up for now so will take me a bit to complete all that

lachlan-00 and others added 17 commits July 30, 2026 08:44
* add docs/AMPACHE.md
* default to https in messaging and hints
* use _log_error function in to redact URL from logs
* catalog id's are not in responses (yet) but the albums call can filter on conditions (e.g. cond=catalog,32) so we can filter on api6 and 8 with that
Every provider reports the path it holds a track at, so an install whose
library is mounted into the container can read the file directly: no HTTP
round trip, no second copy of the bytes, and no load on the media server.
Provider-agnostic - it lives in the dispatcher, and all six backends
populate Path/FilePath.

Off by default. LOCAL_FILE_ACCESS enables it, LOCAL_FILE_ROOTS allowlists
the directories a track may come from, and LOCAL_FILE_PATH_MAP rewrites the
server's prefix onto this container's mount point.

Two properties the implementation is built around:

* The pipeline DELETES whatever download_track returns (the finally in
  tasks/analysis/album.py), so the library file is never returned. What the
  caller gets is a link inside TEMP_DIR - a symlink, or a hardlink where
  symlinks are not permitted - and removing it never touches the target.
* The path comes from the media server and is therefore untrusted. The
  REAL location is resolved first, so a symlink planted in the library
  cannot escape, and anything outside LOCAL_FILE_ROOTS is refused. With no
  root configured nothing is read at all.

Every failure returns None and the track is downloaded as before, so local
access is an optimisation that cannot become a new way for analysis to fail.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@lachlan-00

Copy link
Copy Markdown
Author

Even with all the mitigations for speed I've put in it's going to take more than 6 months to scan my home server.

I'll do the testing with a smaller library by skipping my biggest catalog.

I've added 3 scripts that i'll squash down that i'm using as evidence to build on. and will report when done

@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants