[markusressel]

Implements the owner decision "AI embeddings to catch crops: yes" for the experimental matching pipeline (py_image_dedup/matching/, not wired in).

Changes

• matching/embeddings.py
  • OnnxEmbeddingExtractor: SSCD (sscd_disc_mixup, Meta, MIT) via onnxruntime on the CPU. onnxruntime is a new optional extra (embeddings) and is imported lazily, the package works without it.
  • ensure_model(): downloads the model on demand into $PY_IMAGE_DEDUP_MODEL_CACHE (default ~/.cache/py-image-dedup/models) and verifies its SHA-256. A model can also be given by path. The model is not bundled in the repo or the Docker image.
  • EmbeddingStore: SQLite cache, invalidated by size, mtime and model id.
  • find_similar_pairs(): brute force cosine similarity search with numpy, block-wise.
  • EmbeddingStage: pipeline option, default off. It matches only pairs the hash stages did not match, as MatchReason.EMBEDDING with the note crop/embedding similarity 0.xx and confidence 0.50 .. 0.75. The burst veto applies.
  • Crop relations per group (DuplicateGroup.crops, has_crops): the largest member is the reference. Members that reach it through same-framing hash matches show the full frame. All other members are crops. This is needed because a pairwise "fewer pixels" check confuses a crop with a downscaled messenger copy. rank_crops_first() is the default ranking hint. Groups with crops are review-only.
• models.py / pipeline.py: minimal hooks (MatchReason.EMBEDDING, Match.embedding_similarity, CropRelation, MatchingPipeline(embedding_stage=...), annotate_crops after clustering).
• tools/export_sscd_onnx.py: exports the official TorchScript release to ONNX. Needs PyTorch in a throw-away venv. It checks the SHA-256 and compares the onnxruntime output with the PyTorch output.
• matching/crop_benchmark.py: generates crops of tests/images in a temp dir (none are committed) and compares the pipeline with and without the embedding stage.
• Docs section "Crop detection (embeddings, opt-in)" in docs/matching-prototype.md, plus updates to TODO.md and AGENTS.md.

Model source

The default download is nutszebra/sscd_onnx on Hugging Face, pinned to revision 0f03b67 with its sha256. It is a third-party ONNX export of Meta's official file (dl.fbaipublicfiles.com, sha256 9f26bd4c...). I verified that its embeddings match my own export from the official file (max abs diff 1e-7). That repo has no license tag, but the weights themselves are MIT. For production, prefer a local export with tools/export_sscd_onnx.py and pass the file by path.

Benchmark (tests/images + 21 generated crops: 10/25/40 %, square, 16:9, portrait, corner crop as a messenger copy)

	Precision	Recall	Crop pair recall
hashes only	100 %	21.8 %	12.2 %
hashes + embeddings (>= 0.6)	100 %	100 %	100 %

• Cosine similarity of negatives: max 0.13. Best match per crop: 0.73 .. 0.94.
• Speed: 156 ms/file with 1 thread, 40 ms/file wall with 16 threads.
• Model: 98 MB file, +240 MB RSS when loaded, 560-590 MB peak RSS while embedding.
• Search: 40k images in 6.6 s, 100k in 44 s.

Tests

• Unit tests use a fake extractor and need no model or network. The real-model test only runs when onnxruntime is installed and the model is already cached; otherwise it is skipped. I ran it locally and it passes.
• poetry run pytest: 188 passed, 1 skipped. The Docker build passes; the image does not include the extra.

🤖 Generated with Claude Code