feat: semantic paths for cross-platform Windows/Wine save portability#614
feat: semantic paths for cross-platform Windows/Wine save portability#614thedavidweng wants to merge 7 commits into
Conversation
Test fix: semantic_paths default changeCommit What changedIn Fix: Changed to Also added missing |
Add semantic path support so a Windows game's saves can be backed up and restored across Windows and Wine/Proton without per-game redirects. Saves are identified by their portable meaning (e.g. <winDocuments>/...) instead of the source machine's username or Wine prefix location. Core module (src/semantic): - SemanticBase/SemanticPath: parse, serialize, storage-path encoding, and case-aware equality for Windows known-folder bases and drive roots - convert: physical<->semantic for native Windows paths and Wine-prefix paths (lexical, symlink-safe), plus manifest-origin derivation - prefix: Wine prefix validation and Wine-user detection - materialize: semantic->physical for the current Windows user or a selected Wine prefix, with drive-mapping fallback and long-path checks - conflict/signals/preview: duplicate-key detection, foreign-platform comparison signals, and dry-run preview analysis Integration: - scan: derive a semantic key per file (manifest origin first, then reverse mapping), keeping the physical path as the copy source - layout: store keys under a versioned `semantic-v1` format with a reserved `__ludusavi_semantic__` storage namespace; force a new full backup when a legacy chain first switches to semantic; materialize keys on restore and surface a per-file restore error instead of writing an invalid path - config: opt-in `backup.semanticPaths` (default off), per-game preferred Wine prefixes, a global restore `winePrefix`, and `driveMappings` - cli/api: `--wine-prefix` for restore with CLI-vs-per-game conflict detection - gui: PORTABLE / NEW FULL BACKUP / CONFLICT / INVALID PREFIX badges, with lazily cached conflict detection Scope is intentionally limited to Windows<->Wine/Proton. Steam userdata and native Linux paths keep their existing absolute-path behavior.
- proptest round-trips: parse/serialize, storage-path invariants, and materialize->re-derive stability; username and Wine-prefix changes do not change the semantic key - criterion benchmark for physical<->semantic conversion across many paths - test fixture Wine prefix marker file Adds proptest, criterion, and tempfile as dev-dependencies.
Add Fluent keys (English source plus fallbacks) for the portable / new-full-backup / conflict / invalid-prefix badges, the Wine prefix conflict and missing-drive errors, and the semantic preview notice.
- cross-platform-sync-plan.md: design and implementation plan, scoped to Windows<->Wine/Proton; XDG bases, Steam userdata identity, and cross-platform registry translation are listed as explicitly out of scope - help: explain semantic vs physical paths, the new config fields, and the Windows/Wine transfer support level - schema: document `backup.semanticPaths`, restore `winePrefix`/`driveMappings`, and per-game preferred prefixes
|
Hi! Thanks for your PR. Before I take a closer look at the implementation, let's start with some high-level questions:
|
df8122c to
06bb7d7
Compare
|
Hi @mtkennerly thank you for reviewing.
I think
This PR didn't solved it. I think Native Linux path to a native Windows path can be a future task after the manifest is updated, right now a file only gets a semantic key if it's a recognized Windows location or it's found inside a validated Wine prefix. The Linux Dustforce path matches neither, so it stays on legacy paths. I've scoped this to #156 described.
This PR didn't solved it as well. I think your #156 comment is the right mode I'd like to build after we agreed on the plan, on restore, when a game has no resolved prefix, prompt the user to choose one and remember it per-game (the config already has preferredWinePrefixes for this), and warn when a saved prefix no longer exists. I deliberately haven't built the GUI for this yet.
My plan is to use the same per-game selection described above, discovery picks a prefix, and the user can override/pin it via preferredWinePrefixes or --wine-prefix. The genuinely-different-saves-per-prefix scenario (e.g. two installs you keep separate) is out of scope, the semantic key alone can't tell them apart. My main goal is to make Steam Deck (Wine/Proton) ↔ Windows two-way backup/restore work cleanly, the backend for both directions is in place, I'd like to confirm you agree with the overall direction. |
Let me take a closer look at the code first before I make a decision on that. I'm open to Also, not directly related to what we're solving here, but I do like the idea of
Makes sense; I agree we should start with Windows <-> Wine 👍
I think solving that edge case would be a requirement to merge this; it's important to me that a user can back up and restore on the same system and it "just works" without any ambiguity. If the semantic key alone isn't enough, then we'll need some other metadata to track the original Wine prefix. (That was part of my idea with the |
…disambiguation Implement per-file Wine prefix resolution during restore using backup metadata (pathContexts) so that same-system multi-prefix restores "just works" without ambiguity errors. Core changes: - Add MappingPathKey enum (Semantic, SemanticWithContext, Legacy) with parse/serialize/storage_path for context-aware mapping keys - Add PathContext struct storing source prefix metadata (prefix_path, wine_user, drive_mappings) in FullBackup.path_contexts - Add ResolvedMaterializeTarget that owns its data (no lifetime constraints) - Refactor resolve_wine_prefix_for_game with full priority chain: CLI → per-game preferred → source context → custom game winePrefix → launcher-discovered → global restore.winePrefix → root discovery - Add build_context_targets and materialize_and_fixup shared helpers used by CLI, API, and GUI restore paths Backup scan: - Track which Wine prefix each file matched during backup scan - Assign deterministic context IDs by canonical (prefix_path, wine_user) order - Store path_contexts even for single-prefix backups (for cross-machine restore) - Use context-aware mapping keys only when >1 prefix for the same game Restore flow (all three paths: CLI, API, GUI): - Phase 1: scan_for_restoration(..., None) reads backup metadata including pathContexts - Phase 2: build per-context targets from pathContexts, materialize semantic paths, recalculate restore state (redirected, ignored, change) - Single-prefix backups use the stored context as fallback for non-contextual files, preventing ambiguity on multi-prefix restore machines - Context path invalid on current machine → resolver fallback with source_context parameter Conflict detection: files with same semantic key but different context IDs are NOT conflicts. Validation: mixed semantic + legacy keys in SemanticV1 backups now validate correctly by falling back to legacy path handling on parse failure. Fixes: - GUI: Wine prefix ambiguity errors now surface in the error popup instead of being silently logged - CLI/API: safe .get(name) for manifest reads (no panic on missing games) - Generic launcher: use game name as fallback candidate directory - prelude.rs: gate Arc/AtomicBool imports behind #[cfg(feature = "app")] for --no-default-features compatibility Tests: 335 passing (302 lib + 26 main + 7 property)
Add `decide_prefix_resolution()` as a testable decision seam that classifies Wine prefix resolution into explicit `ResolutionOutcome` variants (Resolved, NoCandidate, Ambiguous, AmbiguousUser, Conflict, StalePreference). Wire it into both the CLI and GUI restore flows, replacing inline resolution and ad-hoc error matching. CLI: replace inline `resolve_wine_prefix_for_game` with decision function + `format_cli_prefix_message` for actionable error output. GUI: replace inline `resolve_wine_prefix_without_cli` with decision function in background task; replace inline Error matching with `error_to_resolution_outcome` + `outcome_to_selection_request`. Also: `--wine-user`/`--persist-wine-prefix` CLI flags, interactive prefix selection modals, stale preference detection, per-game prefix persistence, 18 unit tests + 3 e2e tests + 2 persistence tests, CHANGELOG, and end-user documentation. Refs mtkennerly#614
I have implemented this in 1a68391 On backup: Each Wine prefix gets its own PathContext (stored in the backup metadata) with its prefix path, wine user, and drive mappings. Every save file is tagged with the context ID it came from. On restore: build_context_targets reads those context IDs and builds a map: context 0 → prefix A target, context 1 → prefix B target. The materializer then routes each file back to the prefix it came from. Files from prefix A go to prefix A, files from prefix B go to prefix B. For files without a context tag (non-contextual semantic files): if there's exactly one context, they go there as a fallback. If there are multiple contexts, the general resolver (decide_prefix_resolution) picks one, or the user is prompted to choose via the interactive GUI modal or --wine-prefix CLI flag. This is similar to the approach you described in #194 (comment) The two layers:
I think it's good to use both ideas because they all solved part of the problem. |
|
Okay, I had some time to look at this more closely and think through some options. While I do think there's merit to
There's another approach I'd like us to explore before significantly changing the backup format. I think we can generate redirects dynamically:
The main challenge would be, when making a new backup, how to detect whether the live files are equivalent to the latest existing backup, accounting for the Windows/Wine redirects. For now, we can start simple and consider the "preferred Wine prefix" to be the first Then the question is what info we need to add to the backup schema to support this. Technically, since Windows lets you move special folders around, we would need to tag every special folder to be sure what it really represents, but again, we can start simple and incrementally add more support. For an MVP, I think we can just add a field like backups:
- name: "."
os: linux
files:
"/home/user-1/Games/prefixname/drive_c/users/wine-1/AppData/Local/save.dat": {}
semantics:
directories:
"/home/user-1/Games/prefixname":
kind: wineThat's just a more general version of the semantics:
directories:
"C:/users/user-1/AppData/Local":
kind: winLocalAppData...so that we don't have to rely on a heuristic of the path containing What do you think? |
Strip ~8,500 lines of old semantic path machinery (materialize, conflict, preview, signals, restore_prompt) and replace with a minimal dynamic redirect approach: - BackupSemantics metadata on FullBackup stores Wine prefix directories - WineRedirectContext built from custom game config at backup/restore time - generate_restore_redirect handles Wine↔Windows path conversion - detect_windows_special_folder heuristic for Windows→Wine restores - materialize_to_wine for Wine→Wine with username remapping - config.scan.redirect_wine toggle (default false) - Incremental backup check now uses WineRedirectContext - Real-path integration tests using Steam Deck/Proton backup data - All clippy warnings resolved, dead storage_path() removed
Makes sense. I refactored the PR to align with your vision. |
Summary
This adds optional semantic paths so that a Windows game's saves can be
backed up and restored across Windows and Wine/Proton without setting up a
per-game redirect for each one.
The idea is to key a backup by what a save location means rather than where
it happened to live on the machine that created the backup. A Wine save found on
Linux currently embeds the Linux username, the launcher's prefix layout, the
game-specific folder, and the Wine username:
None of that is meaningful when restoring on Windows or into a different prefix.
With semantic paths, the same save is keyed as:
Scope
This PR is deliberately limited to Windows ↔ Wine/Proton portability, which
is the case that can be solved reliably today (a Windows game running under Wine
stores its saves in Windows locations inside the prefix).
It does not attempt native Windows ↔ native Linux equivalence, Steam Cloud
userdataremapping, or registry translation. Those keep their existingbehavior (see "Out of scope" below).
It is off by default (
backup.semanticPaths: false), so nothing changes forexisting users unless they opt in.
How it works
Semantic key format
Keys are stored in
mapping.yamlas<baseName>/relative/path, for example:The recognized bases are the common Windows locations:
winHome,winDocuments,winAppData,winLocalAppData,winLocalAppDataLow,winSavedGames,winPublic,winProgramData,winDir, andwinDrive-<letter>for genuinely drive-rooted paths. Comparison is case-insensitive (Windows
convention); the key carries no username, prefix path, or OS-specific separator.
Backup
During a scan, each file gets a semantic key derived in priority order:
<winDocuments>entry), thenIf none apply, the file keeps its existing absolute-path (legacy) behavior. The
physical path is always preserved as the copy source.
Restore
When a backup is marked
pathFormat: semantic-v1, each key is materialized to areal path on the current machine — to the current user's known folders on
Windows, or into a selected Wine prefix on Linux. If a key cannot be
materialized (e.g. no prefix is available), that file is reported as a restore
error rather than written to an invalid location.
The target prefix can come from
--wine-prefix, a per-game preference, orlauncher discovery; a conflict between an explicit
--wine-prefixand a savedper-game preference fails with a clear message.
Configuration
All new fields are opt-in:
backup.semanticPaths(defaultfalse) — enable semantic keys for new backupsrestore.winePrefix— global Wine prefix used as a fallback on Linuxrestore.preferredWinePrefixes— per-game Wine prefix (and optional Wine user / drive mappings)restore.driveMappings— fallback for<winDrive-*>keys when a prefix has no matchingdosdevicessymlinkBackward compatibility
Legacy; no migration is performed.mapping.yamlfiles withoutpathFormatare read asLegacy, and theLegacyvalue is not serialized.so a chain never mixes the two formats.
Out of scope
These are intentionally left for separate work and keep their current behavior:
relationship data — see
docs/cross-platform-sync-plan.md, Phase 6).userdatacross-account remapping (a native Windows/Linux concern,not Wine/Proton); these saves continue to use absolute paths.
separate save streams.
Tests
detection, and preview analysis.
invariant that changing the username or Wine prefix location does not change
the semantic key.
Related issues