You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 9b9b120
Browse filesBrowse the repository at this point in the historyBrowse files
fix(node): make the repo dedup key did:key-aware so distinct DID methods don't collapse
The dedup grouping key took the last ':' segment of owner_did
(split_part / rsplit), so two repos owned by did:key:X and
did:gitlawb:X with the same name collapsed into one logical repo on
the list, paged, count, stats, and GraphQL surfaces. That is the exact
cross-method collision the codebase already guards against in
did_matches.
Replace it with a did:key-aware key that strips a did:key: prefix only
when the remainder is a bare id (no ':'), otherwise keeps the full DID,
reproducing did_matches/key_id as an equivalence relation: did:key:X
and a bare mirror X still collapse, while distinct methods never merge.
Applied byte-identically across DEDUP_CTE (DISTINCT ON / PARTITION BY /
ORDER BY), count_repos_deduped, the empty-page count fallback, and the
in-memory dedupe_canonical_repos, so the SQL and Rust paths agree.
The backing index lived in the already-released migration v1, so v1 is
left untouched and a new migration v7 drops idx_repos_owner_short_name
and builds idx_repos_owner_key_name on the matching expression; the
CASE must stay byte-identical to the queries or Postgres stops using it.
Tests cover both the in-memory and SQL paths: distinct methods stay
separate, bare-id and did:key forms collapse, and the residual-colon
guard keeps a malformed did:key:did:gitlawb:X distinct from the bare
method DID. 216 pass.
"ALTER TABLE agents ADD COLUMN IF NOT EXISTS deactivated_at TEXT",
768
768
],
769
769
},
770
+
Migration{
771
+
version:7,
772
+
name:"repo_owner_dedup_key_didkey_aware",
773
+
stmts:&[
774
+
// The dedup grouping key moved from the last `:` segment to a
775
+
// did:key-aware key (strip `did:key:`, leave any other DID method
776
+
// whole) so `did:key:X` and `did:gitlawb:X` no longer collapse. Swap
777
+
// the index that backs it: drop the last-segment one from v1 and build
778
+
// the matching expression index. The CASE must stay byte-for-byte in
779
+
// sync with DEDUP_CTE / count_repos_deduped or Postgres won't use it.
780
+
"DROP INDEX IF EXISTS idx_repos_owner_short_name",
781
+
"CREATE INDEX IF NOT EXISTS idx_repos_owner_key_name ON repos ((CASE WHEN owner_did LIKE 'did:key:%' AND position(':' in substr(owner_did, 9)) = 0 THEN substr(owner_did, 9) ELSE owner_did END), name)",
/// Shared dedup CTE: collapses the mirror row and the canonical row of one
898
912
/// logical repo into a single survivor. `$1` is an optional owner filter
899
-
/// (NULL = all rows). Grouping is `(split_part(owner_did,':',-1), name)`; the
900
-
/// canonical row wins (mirror rows carry a slash-form `id` written only by
913
+
/// (NULL = all rows). Grouping collapses on a did:key-aware owner key: strip a
914
+
/// `did:key:` prefix (8 chars, so `substr(owner_did, 9)`) only when the
915
+
/// remainder is a bare id with no `:`, otherwise keep the full DID. That is the
916
+
/// exact normalization in `crate::api::did_matches`, so `did:key:X` and a bare
917
+
/// `X` collapse while distinct DID methods (`did:gitlawb:X`) never merge. The
918
+
/// CASE is repeated verbatim in `count_repos_deduped` and the v7 index and must
919
+
/// stay byte-identical or Postgres stops using the index.
920
+
/// The canonical row wins (mirror rows carry a slash-form `id` written only by
901
921
/// `upsert_mirror_repo`), ties broken by earliest `created_at` then `id` so a
902
922
/// full tie still picks a deterministic survivor. `list_all_repos_paged`,
903
923
/// `list_all_repos_deduped`, and the marker logic in
904
924
/// `crate::api::repos::dedupe_canonical_repos` must stay in sync.
905
925
constDEDUP_CTE:&'staticstr = "WITH deduped AS (
906
-
SELECT DISTINCT ON (split_part(owner_did, ':', -1), name)
926
+
SELECT DISTINCT ON (CASE WHEN owner_did LIKE 'did:key:%' AND position(':' in substr(owner_did, 9)) = 0 THEN substr(owner_did, 9) ELSE owner_did END, name)
-- group MAX, not the canonical row's own value: pushes that
910
930
-- arrive via gossip touch only the mirror row, so the
911
931
-- canonical updated_at goes stale
912
932
MAX(updated_at) OVER (
913
-
PARTITION BY split_part(owner_did, ':', -1), name
933
+
PARTITION BY CASE WHEN owner_did LIKE 'did:key:%' AND position(':' in substr(owner_did, 9)) = 0 THEN substr(owner_did, 9) ELSE owner_did END, name
914
934
) AS updated_at,
915
935
disk_path, forked_from, machine_id
916
936
FROM repos
917
937
WHERE ($1::text IS NULL OR owner_did = $1 OR owner_did LIKE '%:' || $1)
918
-
ORDER BY split_part(owner_did, ':', -1), name,
938
+
ORDER BY CASE WHEN owner_did LIKE 'did:key:%' AND position(':' in substr(owner_did, 9)) = 0 THEN substr(owner_did, 9) ELSE owner_did END, name,
919
939
-- mirror rows carry a slash-form id (\"{owner_short}/{name}\"),
920
940
-- written only by upsert_mirror_repo; canonical ids are UUIDs.
921
941
-- Rank canonical (no slash) ahead of the mirror in each group,
@@ -967,7 +987,7 @@ impl Db {
967
987
968
988
let total = if out.is_empty(){
969
989
let row = sqlx::query(
970
-
"SELECT COUNT(DISTINCT (split_part(owner_did, ':', -1), name)) AS cnt
990
+
"SELECT COUNT(DISTINCT (CASE WHEN owner_did LIKE 'did:key:%' AND position(':' in substr(owner_did, 9)) = 0 THEN substr(owner_did, 9) ELSE owner_did END, name)) AS cnt
971
991
FROM repos
972
992
WHERE ($1::text IS NULL OR owner_did = $1 OR owner_did LIKE '%:' || $1)",
973
993
)
@@ -1005,12 +1025,12 @@ impl Db {
1005
1025
}
1006
1026
1007
1027
/// Count of distinct logical repos (mirror + canonical collapsed), for
1008
-
/// `/api/v1/stats`. Uses the same `(split_part(owner_did,':',-1), name)`
1009
-
/// grouping as `DEDUP_CTE`; the marker/tiebreak only decide which row would
1010
-
/// survive, not the group count, so they are not needed here.
1028
+
/// `/api/v1/stats`. Uses the same did:key-aware owner-key grouping as
1029
+
/// `DEDUP_CTE` (the CASE must stay byte-identical); the marker/tiebreak only
1030
+
/// decide which row would survive, not the group count, so they are not needed here.
"SELECT COUNT(DISTINCT (split_part(owner_did, ':', -1), name)) AS cnt FROM repos",
1033
+
"SELECT COUNT(DISTINCT (CASE WHEN owner_did LIKE 'did:key:%' AND position(':' in substr(owner_did, 9)) = 0 THEN substr(owner_did, 9) ELSE owner_did END, name)) AS cnt FROM repos",
0 commit comments