GC gets stuck when deleted chunk files are already missing

[Ramblurr]

I am using attic commit 7a19204 with SQLite and local storage.

The db has millions of chunks with state D. The first 500 rows reference files that no longer exist. File deletion returns ENOENT, so the database rows remain.

Every GC run retries the same 500 rows and never reaches the remaining chunks.

It seems to me that missing files should be treated as already deleted, allowing their database rows to be removed.

Related: #65, #116, #266.

Here is how I made the conclusion..

-- Chunk counts by state and recorded compressed size. `D` means deleted or pending deletion; `V` means valid.
SELECT state, count(*) AS rows,
       sum(coalesce(file_size, 0)) AS compressed_bytes
FROM chunk
GROUP BY state
ORDER BY state;

=> D | 4479031 | 172262219232
=> V | 1099351 | 38019341845

-- All chunks pending deletion have zero holders.
SELECT holders_count, count(*) AS rows
FROM chunk
WHERE state = 'D'
GROUP BY holders_count;

=> 0 | 4479031

-- None of the chunks pending deletion are referenced by a NAR.
SELECT count(DISTINCT c.id)
FROM chunk AS c
JOIN chunkref AS cr ON cr.chunk_id = c.id
WHERE c.state = 'D';

=> 0

-- No valid chunks were waiting to be marked as orphaned when this query was run.
SELECT count(*)
FROM chunk AS c
LEFT JOIN chunkref AS cr ON cr.chunk_id = c.id
WHERE c.state = 'V'
  AND c.holders_count = 0
  AND cr.id IS NULL;

=> 0

Then I use this little script to examine the first 500 rows selected from state D. Every referenced file was already missing.

import json
import sqlite3
from pathlib import Path

server_db = Path("/mnt/fast/attic/server.db")
storage = server_db.parent / "storage"

with sqlite3.connect(f"file:{server_db}?mode=ro", uri=True) as db:
    rows = db.execute(
        "SELECT remote_file FROM chunk WHERE state = ? LIMIT 500",
        ("D",),
    ).fetchall()

files = []
for (remote_file,) in rows:
    name = json.loads(remote_file)["Local"]["name"]
    files.append(storage / name[0] / name[:2] / name)

existing = sum(path.exists() for path in files)
print(f"checked={len(files)} existing={existing} missing={len(files) - existing}")

# => checked=500 existing=0 missing=500