Skip to content

feat: semantic paths for cross-platform Windows/Wine save portability#614

Open
thedavidweng wants to merge 7 commits into
mtkennerly:masterfrom
thedavidweng:feat/semantic-paths
Open

feat: semantic paths for cross-platform Windows/Wine save portability#614
thedavidweng wants to merge 7 commits into
mtkennerly:masterfrom
thedavidweng:feat/semantic-paths

Conversation

@thedavidweng

@thedavidweng thedavidweng commented May 29, 2026

Copy link
Copy Markdown

Summary

This adds optional semantic paths so that a Windows game's saves can be
backed up and restored across Windows and Wine/Proton without setting up a
per-game redirect for each one.

The idea is to key a backup by what a save location means rather than where
it happened to live on the machine that created the backup. A Wine save found on
Linux currently embeds the Linux username, the launcher's prefix layout, the
game-specific folder, and the Wine username:

/home/deck/Games/Heroic/Prefixes/Alan Wake/drive_c/users/steamuser/Documents/Remedy/Alan Wake/save.dat

None of that is meaningful when restoring on Windows or into a different prefix.
With semantic paths, the same save is keyed as:

<winDocuments>/Remedy/Alan Wake/save.dat

Scope

This PR is deliberately limited to Windows ↔ Wine/Proton portability, which
is the case that can be solved reliably today (a Windows game running under Wine
stores its saves in Windows locations inside the prefix).

It does not attempt native Windows ↔ native Linux equivalence, Steam Cloud
userdata remapping, or registry translation. Those keep their existing
behavior (see "Out of scope" below).

It is off by default (backup.semanticPaths: false), so nothing changes for
existing users unless they opt in.

How it works

Semantic key format

Keys are stored in mapping.yaml as <baseName>/relative/path, for example:

<winDocuments>/Remedy/Alan Wake/save.dat
<winAppData>/Publisher/Game/settings.cfg
<winDrive-d>/Games/Title/save.dat

The recognized bases are the common Windows locations: winHome,
winDocuments, winAppData, winLocalAppData, winLocalAppDataLow,
winSavedGames, winPublic, winProgramData, winDir, and winDrive-<letter>
for genuinely drive-rooted paths. Comparison is case-insensitive (Windows
convention); the key carries no username, prefix path, or OS-specific separator.

Backup

During a scan, each file gets a semantic key derived in priority order:

  1. the matched manifest placeholder (e.g. a <winDocuments> entry), then
  2. reverse mapping from the current Windows known folders, then
  3. reverse mapping from a validated Wine prefix.

If none apply, the file keeps its existing absolute-path (legacy) behavior. The
physical path is always preserved as the copy source.

Restore

When a backup is marked pathFormat: semantic-v1, each key is materialized to a
real path on the current machine — to the current user's known folders on
Windows, or into a selected Wine prefix on Linux. If a key cannot be
materialized (e.g. no prefix is available), that file is reported as a restore
error rather than written to an invalid location.

The target prefix can come from --wine-prefix, a per-game preference, or
launcher discovery; a conflict between an explicit --wine-prefix and a saved
per-game preference fails with a clear message.

Configuration

All new fields are opt-in:

  • backup.semanticPaths (default false) — enable semantic keys for new backups
  • restore.winePrefix — global Wine prefix used as a fallback on Linux
  • restore.preferredWinePrefixes — per-game Wine prefix (and optional Wine user / drive mappings)
  • restore.driveMappings — fallback for <winDrive-*> keys when a prefix has no matching dosdevices symlink

Backward compatibility

  • Existing backups stay Legacy; no migration is performed.
  • Old mapping.yaml files without pathFormat are read as Legacy, and the
    Legacy value is not serialized.
  • The first semantic backup after a legacy chain is forced to be a full backup,
    so a chain never mixes the two formats.
  • Legacy restores are completely unchanged.

Out of scope

These are intentionally left for separate work and keep their current behavior:

  • Native Windows ↔ native Linux equivalence without Wine (needs manifest
    relationship data — see docs/cross-platform-sync-plan.md, Phase 6).
  • Steam Cloud userdata cross-account remapping (a native Windows/Linux concern,
    not Wine/Proton); these saves continue to use absolute paths.
  • Cross-platform registry translation; registry data keeps its existing format.
  • Multiple users in one backup, and multiple installs of the same game with
    separate save streams.

Tests

  • Unit tests for path conversion, prefix validation, materialization, conflict
    detection, and preview analysis.
  • Property tests (proptest) for parse/serialize round-trips and for the
    invariant that changing the username or Wine prefix location does not change
    the semantic key.
  • A criterion benchmark for conversion throughput.

Related issues

@thedavidweng

thedavidweng commented May 29, 2026

Copy link
Copy Markdown
Author

Test fix: semantic_paths default change

Commit e07816b fixes test failures caused by semantic_paths now defaulting to true.

What changed

In src/scan.rs, 19 test calls to scan_game_for_backup passed semantic_paths_enabled: true. Since the config default is now true, these tests started deriving semantic keys from test data (files under the repo path get mapped to WinHome), which didn't match the existing assertions (expected semantic_key: None).

Fix: Changed to false in the generic scan tests. These cover file matching, toggling, filtering, redirects, globs etc. — not semantic path derivation. The dedicated semantic path suite (82 tests in src/semantic/ + tests/semantic_properties.rs) already thoroughly covers the semantic conversion and materialization logic.

Also added missing drive_c/windows/system.reg to the Wine prefix test fixture.

@thedavidweng thedavidweng marked this pull request as ready for review May 29, 2026 02:44
@thedavidweng thedavidweng marked this pull request as draft May 29, 2026 02:46
Add semantic path support so a Windows game's saves can be backed up and
restored across Windows and Wine/Proton without per-game redirects. Saves are
identified by their portable meaning (e.g. <winDocuments>/...) instead of the
source machine's username or Wine prefix location.

Core module (src/semantic):
- SemanticBase/SemanticPath: parse, serialize, storage-path encoding, and
  case-aware equality for Windows known-folder bases and drive roots
- convert: physical<->semantic for native Windows paths and Wine-prefix paths
  (lexical, symlink-safe), plus manifest-origin derivation
- prefix: Wine prefix validation and Wine-user detection
- materialize: semantic->physical for the current Windows user or a selected
  Wine prefix, with drive-mapping fallback and long-path checks
- conflict/signals/preview: duplicate-key detection, foreign-platform
  comparison signals, and dry-run preview analysis

Integration:
- scan: derive a semantic key per file (manifest origin first, then reverse
  mapping), keeping the physical path as the copy source
- layout: store keys under a versioned `semantic-v1` format with a reserved
  `__ludusavi_semantic__` storage namespace; force a new full backup when a
  legacy chain first switches to semantic; materialize keys on restore and
  surface a per-file restore error instead of writing an invalid path
- config: opt-in `backup.semanticPaths` (default off), per-game preferred Wine
  prefixes, a global restore `winePrefix`, and `driveMappings`
- cli/api: `--wine-prefix` for restore with CLI-vs-per-game conflict detection
- gui: PORTABLE / NEW FULL BACKUP / CONFLICT / INVALID PREFIX badges, with
  lazily cached conflict detection

Scope is intentionally limited to Windows<->Wine/Proton. Steam userdata and
native Linux paths keep their existing absolute-path behavior.
- proptest round-trips: parse/serialize, storage-path invariants, and
  materialize->re-derive stability; username and Wine-prefix changes do not
  change the semantic key
- criterion benchmark for physical<->semantic conversion across many paths
- test fixture Wine prefix marker file

Adds proptest, criterion, and tempfile as dev-dependencies.
Add Fluent keys (English source plus fallbacks) for the portable /
new-full-backup / conflict / invalid-prefix badges, the Wine prefix conflict
and missing-drive errors, and the semantic preview notice.
- cross-platform-sync-plan.md: design and implementation plan, scoped to
  Windows<->Wine/Proton; XDG bases, Steam userdata identity, and cross-platform
  registry translation are listed as explicitly out of scope
- help: explain semantic vs physical paths, the new config fields, and the
  Windows/Wine transfer support level
- schema: document `backup.semanticPaths`, restore `winePrefix`/`driveMappings`,
  and per-game preferred prefixes
@mtkennerly

mtkennerly commented May 29, 2026

Copy link
Copy Markdown
Owner

Hi! Thanks for your PR. Before I take a closer look at the implementation, let's start with some high-level questions:

  • Adding a PathFormat mode may make sense, but that's quite different than the approach I described in Add option to translate Wine prefixes across OSes #194 (comment) . Why didn't you take that approach, or if you tried it, what problems did you run into?
  • How have you addressed the concerns I raised in this comment regarding relative path ambiguity? Add option to translate Wine prefixes across OSes #194 (comment) . I believe the approach I described using os and wine_prefixes metadata in the backup, while still storing absolute paths as we do today, would avoid those concerns.
  • What happens if you back up on Windows and want to restore on Linux, but there's no Wine prefix for the game yet?
  • Just on Linux (ignoring the Windows translations), what happens if you have two Wine prefixes for the same game? How do we know which files to restore into which prefix?

@thedavidweng thedavidweng force-pushed the feat/semantic-paths branch from df8122c to 06bb7d7 Compare May 29, 2026 07:01
@thedavidweng thedavidweng marked this pull request as ready for review May 29, 2026 07:19
@thedavidweng thedavidweng marked this pull request as draft May 29, 2026 07:28
@thedavidweng

Copy link
Copy Markdown
Author

Hi @mtkennerly thank you for reviewing.

I think PathFormat is more clear than the metadata approach because keeping absolute paths leaves the backup identity tied to the source machine, which I don't think completely solves #310
I want the backup identity itself to be portable, and PathFormat seems like the easy way to do cross-machine sync/deduplication and fits the need for most users. But the translation table in convert.rs is the same logic either design needs. If you think the metadata route is better for compatibility reasons, I'm happy to switch.

  • How have you addressed the concerns I raised in this comment regarding relative path ambiguity? Add option to translate Wine prefixes across OSes #194 (comment) . I believe the approach I described using os and wine_prefixes metadata in the backup, while still storing absolute paths as we do today, would avoid those concerns.

This PR didn't solved it. I think Native Linux path to a native Windows path can be a future task after the manifest is updated, right now a file only gets a semantic key if it's a recognized Windows location or it's found inside a validated Wine prefix. The Linux Dustforce path matches neither, so it stays on legacy paths. I've scoped this to #156 described.

  • What happens if you back up on Windows and want to restore on Linux, but there's no Wine prefix for the game yet?

This PR didn't solved it as well. I think your #156 comment is the right mode I'd like to build after we agreed on the plan, on restore, when a game has no resolved prefix, prompt the user to choose one and remember it per-game (the config already has preferredWinePrefixes for this), and warn when a saved prefix no longer exists. I deliberately haven't built the GUI for this yet.

  • Just on Linux (ignoring the Windows translations), what happens if you have two Wine prefixes for the same game? How do we know which files to restore into which prefix?

My plan is to use the same per-game selection described above, discovery picks a prefix, and the user can override/pin it via preferredWinePrefixes or --wine-prefix.

The genuinely-different-saves-per-prefix scenario (e.g. two installs you keep separate) is out of scope, the semantic key alone can't tell them apart.

My main goal is to make Steam Deck (Wine/Proton) ↔ Windows two-way backup/restore work cleanly, the backend for both directions is in place, I'd like to confirm you agree with the overall direction.

@thedavidweng thedavidweng marked this pull request as ready for review May 29, 2026 08:33
@mtkennerly

mtkennerly commented May 29, 2026

Copy link
Copy Markdown
Owner

If you think the metadata route is better for compatibility reasons, I'm happy to switch.

Let me take a closer look at the code first before I make a decision on that. I'm open to PathFormat in theory as long as we can solve the edge cases and the user can select which mode they prefer. I'll be busy for a few days, but I'll try to make some time within the next week to read through the diff thoroughly.

Also, not directly related to what we're solving here, but I do like the idea of PathFormat::Semantic as a way to facilitate backup exports in the future, to make it easier to share a backup with other people.

This PR didn't solved it. I think Native Linux path to a native Windows path can be a future task after the manifest is updated, right now a file only gets a semantic key if it's a recognized Windows location or it's found inside a validated Wine prefix.

Makes sense; I agree we should start with Windows <-> Wine 👍

The genuinely-different-saves-per-prefix scenario (e.g. two installs you keep separate) is out of scope, the semantic key alone can't tell them apart.

I think solving that edge case would be a requirement to merge this; it's important to me that a user can back up and restore on the same system and it "just works" without any ambiguity. If the semantic key alone isn't enough, then we'll need some other metadata to track the original Wine prefix. (That was part of my idea with the wine_prefixes metadata, but I'm open to other approaches that solve the same problem.)

…disambiguation

Implement per-file Wine prefix resolution during restore using backup metadata
(pathContexts) so that same-system multi-prefix restores "just works" without
ambiguity errors.

Core changes:
- Add MappingPathKey enum (Semantic, SemanticWithContext, Legacy) with
  parse/serialize/storage_path for context-aware mapping keys
- Add PathContext struct storing source prefix metadata (prefix_path,
  wine_user, drive_mappings) in FullBackup.path_contexts
- Add ResolvedMaterializeTarget that owns its data (no lifetime constraints)
- Refactor resolve_wine_prefix_for_game with full priority chain:
  CLI → per-game preferred → source context → custom game winePrefix →
  launcher-discovered → global restore.winePrefix → root discovery
- Add build_context_targets and materialize_and_fixup shared helpers
  used by CLI, API, and GUI restore paths

Backup scan:
- Track which Wine prefix each file matched during backup scan
- Assign deterministic context IDs by canonical (prefix_path, wine_user) order
- Store path_contexts even for single-prefix backups (for cross-machine restore)
- Use context-aware mapping keys only when >1 prefix for the same game

Restore flow (all three paths: CLI, API, GUI):
- Phase 1: scan_for_restoration(..., None) reads backup metadata including
  pathContexts
- Phase 2: build per-context targets from pathContexts, materialize semantic
  paths, recalculate restore state (redirected, ignored, change)
- Single-prefix backups use the stored context as fallback for non-contextual
  files, preventing ambiguity on multi-prefix restore machines
- Context path invalid on current machine → resolver fallback with
  source_context parameter

Conflict detection: files with same semantic key but different context IDs
are NOT conflicts.

Validation: mixed semantic + legacy keys in SemanticV1 backups now validate
correctly by falling back to legacy path handling on parse failure.

Fixes:
- GUI: Wine prefix ambiguity errors now surface in the error popup instead
  of being silently logged
- CLI/API: safe .get(name) for manifest reads (no panic on missing games)
- Generic launcher: use game name as fallback candidate directory
- prelude.rs: gate Arc/AtomicBool imports behind #[cfg(feature = "app")]
  for --no-default-features compatibility

Tests: 335 passing (302 lib + 26 main + 7 property)
Add `decide_prefix_resolution()` as a testable decision seam that
classifies Wine prefix resolution into explicit `ResolutionOutcome`
variants (Resolved, NoCandidate, Ambiguous, AmbiguousUser, Conflict,
StalePreference). Wire it into both the CLI and GUI restore flows,
replacing inline resolution and ad-hoc error matching.

CLI: replace inline `resolve_wine_prefix_for_game` with decision
function + `format_cli_prefix_message` for actionable error output.
GUI: replace inline `resolve_wine_prefix_without_cli` with decision
function in background task; replace inline Error matching with
`error_to_resolution_outcome` + `outcome_to_selection_request`.

Also: `--wine-user`/`--persist-wine-prefix` CLI flags, interactive
prefix selection modals, stale preference detection, per-game prefix
persistence, 18 unit tests + 3 e2e tests + 2 persistence tests,
CHANGELOG, and end-user documentation.

Refs mtkennerly#614
@thedavidweng

Copy link
Copy Markdown
Author

I think solving that edge case would be a requirement to merge this; it's important to me that a user can back up and restore on the same system and it "just works" without any ambiguity. If the semantic key alone isn't enough, then we'll need some other metadata to track the original Wine prefix. (That was part of my idea with the wine_prefixes metadata, but I'm open to other approaches that solve the same problem.)

I have implemented this in 1a68391

On backup: Each Wine prefix gets its own PathContext (stored in the backup metadata) with its prefix path, wine user, and drive mappings. Every save file is tagged with the context ID it came from.

On restore: build_context_targets reads those context IDs and builds a map: context 0 → prefix A target, context 1 → prefix B target. The materializer then routes each file back to the prefix it came from. Files from prefix A go to prefix A, files from prefix B go to prefix B.

For files without a context tag (non-contextual semantic files): if there's exactly one context, they go there as a fallback. If there are multiple contexts, the general resolver (decide_prefix_resolution) picks one, or the user is prompted to choose via the interactive GUI modal or --wine-prefix CLI flag.

This is similar to the approach you described in #194 (comment)

The two layers:

I think it's good to use both ideas because they all solved part of the problem.

@mtkennerly

Copy link
Copy Markdown
Owner

Okay, I had some time to look at this more closely and think through some options. While I do think there's merit to PathFormat::Semantic, I still have some concerns with the complexity it adds:

  • It's hard to make sure we're preserving enough context for restores on the same system. 1a68391 disambiguates multiple Wine prefixes, but what about multiple users on a Windows system? We can add handling for all of these edge cases, of course, but that adds more complexity. We could also just say those cases aren't compatible with the new format, but that leads into the next point.
  • Users have to choose the new path format and evaluate the trade-offs, rather than just getting the new functionality "for free".
  • The proposed implementation makes the GUI/CLI layers more aware of the semantic logic than I think they should be ideally.

There's another approach I'd like us to explore before significantly changing the backup format. I think we can generate redirects dynamically:

  • Add a config option scan.redirectWine (bool).
  • Limit the updated logic to fn game_file_target, which is used by fn scan_game_for_backup and fn scan_for_restoration to set the redirect info on a ScannedFile (all other GUI/CLI logic would work as-is based on that info).
  • The updated logic in fn game_file_target would be explicitly "best effort", so there's no need for the proposed errors like WinePrefixConflict and WineUserAmbiguity. If a game doesn't already have a preferred Wine prefix, if that prefix doesn't already exist or has multiple users, if a path is on a complex UNC drive instead of a simple drive letter, etc, then we just don't generate redirects for those paths and they would yield a restore error like they do today.
  • Updated documentation would just need to be in docs\help\redirects.md and docs\help\transfer-between-operating-systems.md (but should be commented out until release time, since non-technical users may not understand that they're reading docs for the latest unreleased code).

The main challenge would be, when making a new backup, how to detect whether the live files are equivalent to the latest existing backup, accounting for the Windows/Wine redirects.

For now, we can start simple and consider the "preferred Wine prefix" to be the first wine_prefix entry from a matching custom game. I'm open to other options, but mainly I'd like to keep this first stage focused on the redirect logic rather than the config/UI side.

Then the question is what info we need to add to the backup schema to support this. Technically, since Windows lets you move special folders around, we would need to tag every special folder to be sure what it really represents, but again, we can start simple and incrementally add more support. For an MVP, I think we can just add a field like semantics to each backup (since the semantics may change between backups) that includes kind: wine for Wine prefixes:

backups:
  - name: "."
    os: linux
    files:
      "/home/user-1/Games/prefixname/drive_c/users/wine-1/AppData/Local/save.dat": {}
    semantics:
      directories:
        "/home/user-1/Games/prefixname":
          kind: wine

That's just a more general version of the wine_prefixes field I proposed in #194, but leaving room to add other kind variants. For example, we may want:

semantics:
  directories:
    "C:/users/user-1/AppData/Local":
      kind: winLocalAppData

...so that we don't have to rely on a heuristic of the path containing AppData/Local and can handle a relocated version of the folder. But for an MVP, a heuristic may be sufficient, especially if it means we can avoid adding complexity to the backup format for now.

What do you think?

Strip ~8,500 lines of old semantic path machinery (materialize, conflict,
preview, signals, restore_prompt) and replace with a minimal dynamic
redirect approach:

- BackupSemantics metadata on FullBackup stores Wine prefix directories
- WineRedirectContext built from custom game config at backup/restore time
- generate_restore_redirect handles Wine↔Windows path conversion
- detect_windows_special_folder heuristic for Windows→Wine restores
- materialize_to_wine for Wine→Wine with username remapping
- config.scan.redirect_wine toggle (default false)
- Incremental backup check now uses WineRedirectContext
- Real-path integration tests using Steam Deck/Proton backup data
- All clippy warnings resolved, dead storage_path() removed
@thedavidweng

Copy link
Copy Markdown
Author

Okay, I had some time to look at this more closely and think through some options. While I do think there's merit to PathFormat::Semantic, I still have some concerns with the complexity it adds:

  • It's hard to make sure we're preserving enough context for restores on the same system. 1a68391 disambiguates multiple Wine prefixes, but what about multiple users on a Windows system? We can add handling for all of these edge cases, of course, but that adds more complexity. We could also just say those cases aren't compatible with the new format, but that leads into the next point.

  • Users have to choose the new path format and evaluate the trade-offs, rather than just getting the new functionality "for free".

  • The proposed implementation makes the GUI/CLI layers more aware of the semantic logic than I think they should be ideally.

There's another approach I'd like us to explore before significantly changing the backup format. I think we can generate redirects dynamically:

  • Add a config option scan.redirectWine (bool).

  • Limit the updated logic to fn game_file_target, which is used by fn scan_game_for_backup and fn scan_for_restoration to set the redirect info on a ScannedFile (all other GUI/CLI logic would work as-is based on that info).

  • The updated logic in fn game_file_target would be explicitly "best effort", so there's no need for the proposed errors like WinePrefixConflict and WineUserAmbiguity. If a game doesn't already have a preferred Wine prefix, if that prefix doesn't already exist or has multiple users, if a path is on a complex UNC drive instead of a simple drive letter, etc, then we just don't generate redirects for those paths and they would yield a restore error like they do today.

  • Updated documentation would just need to be in docs\help\redirects.md and docs\help\transfer-between-operating-systems.md (but should be commented out until release time, since non-technical users may not understand that they're reading docs for the latest unreleased code).

The main challenge would be, when making a new backup, how to detect whether the live files are equivalent to the latest existing backup, accounting for the Windows/Wine redirects.

For now, we can start simple and consider the "preferred Wine prefix" to be the first wine_prefix entry from a matching custom game. I'm open to other options, but mainly I'd like to keep this first stage focused on the redirect logic rather than the config/UI side.

Then the question is what info we need to add to the backup schema to support this. Technically, since Windows lets you move special folders around, we would need to tag every special folder to be sure what it really represents, but again, we can start simple and incrementally add more support. For an MVP, I think we can just add a field like semantics to each backup (since the semantics may change between backups) that includes kind: wine for Wine prefixes:

backups:

  - name: "."

    os: linux

    files:

      "/home/user-1/Games/prefixname/drive_c/users/wine-1/AppData/Local/save.dat": {}

    semantics:

      directories:

        "/home/user-1/Games/prefixname":

          kind: wine

That's just a more general version of the wine_prefixes field I proposed in #194, but leaving room to add other kind variants. For example, we may want:

semantics:

  directories:

    "C:/users/user-1/AppData/Local":

      kind: winLocalAppData

...so that we don't have to rely on a heuristic of the path containing AppData/Local and can handle a relocated version of the folder. But for an MVP, a heuristic may be sufficient, especially if it means we can avoid adding complexity to the backup format for now.

What do you think?

Makes sense. I refactored the PR to align with your vision.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants