feat: expose per-search distance evaluation statistics by wxyucs · Pull Request #2545 · antgroup/vsag

wxyucs · 2026-07-27T13:07:27Z

Implements the maintained-v1 per-search logical distance-evaluation statistics contract for HGraph, BruteForce, IVF, Pyramid, SINDI, and SIMQ.

Adds additive C++ JSON statistics with phase/backend breakdowns, completeness, saturating uint64 aggregation, Python opt-in results, and C opaque statistics accessors. Legacy HNSW and DiskANN are explicit non-goals.

Fixes: #2544

vsag-bot · 2026-07-27T13:07:31Z

/label status/waiting-for-review
/waiting-on reviewer
/request-review @jiaweizone
/request-review @inabao

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

vsag-bot · 2026-07-27T13:07:40Z

Automated pull request review completed.

Review effort: high (925 changed lines across 41 files).

Submitted 2 inline comments.
Review: #2545 (review)

mergify · 2026-07-27T13:08:04Z

Merge Protections

🟢 All 3 merge protections satisfied — ready to merge.

Show 3 satisfied protections

🟢 Require kind label

label~=^kind/

🟢 Require version label

label~=^version/

🟢 Require linked issue for feature/bug PRs

body~=(?im)(?:^|[\s\-\*])(?:close[sd]?|fix(?:e[sd])?|resolve[sd]?)\s*:?\s+(?:#\d+|[\w.\-]+/[\w.\-]+#\d+|https?://github\.com/[\w.\-]+/[\w.\-]+/issues/\d+)

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

vsag-bot

Automated inline review completed.

Review effort: high (520 changed lines across 20 files).
Submitted 8 inline comments.

Reviewed commit c2020bb.

vsag-bot

Automated inline review completed.

Review effort: high (673 changed lines across 30 files).
Submitted 8 inline comments.

Reviewed commit f1ec40e.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

LHT129

Code Review Summary

Commit a4bbf900 addresses the issues flagged in the previous review round:

✅ P1 & P2 Issues Resolved

P1 (attach_statistics reading other_result from legacy callers): Fixed by introducing a statistics_requests set keyed by SearchResult_t*, eliminating the dependency on other_result.
P2 (Sparse term counting sums posting-list lengths): Fixed by using UnorderedSet<uint16_t> to count unique document IDs in SparseTermDataCell::Query().
P2 (make_empty_search_result only handles BruteForce): Fixed by expanding to a switch covering all index types.

Observations

sparse_dmq_datacell.h: backend_ is hardcoded to SPARSE_SQ8 while all other datacells derive it from the quantizer name via QuantizerDistanceBackend or BackendFromName. This is acceptable since SparseDMQDataCell is a concrete class with a fixed quantizer, but worth noting for consistency.
The BackendFromName function ordering correctly handles substring overlaps (e.g., "fp16" before "fp32", "bf16" before "fp16", "sq8_uniform" before "sq8", "pq_fastscan" before "pq").

Overall

The implementation is clean, well-tested, and the critical issues from the previous review have been properly addressed. No new blocking issues found.

vsag-bot

Automated inline review completed.

Review effort: high (788 changed lines across 36 files).
Submitted 4 inline comments.

Reviewed commit a4bbf90.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

LHT129

Code Review Summary

I have thoroughly reviewed all 30 changed files in this PR. The implementation adds per-search distance evaluation statistics across all maintained index types (HGraph, BruteForce, IVF, Pyramid, SINDI, SIMQ) with JSON output, phase/backend breakdowns, and C/Python API accessors.

Prior Review Rounds

All issues identified in previous review rounds by LHT129 and vsag-bot have been properly addressed:

SaturatingAdd logic: Fixed to report saturation only after successful CAS with simplified overflow condition.
BackendName hot-path allocation: Replaced with enum-based DistanceEvaluationBackend cached at data cell construction time.
SINDI backend selection: Extracted into a clean switch-based sparse_backend() helper.
GNO-IMI routing counter: Instrumented the actual ClassifyDatasForSearch path.
PQ FastScan classification: pqfs quantizer name now recognized before generic pq branch.
Route-only IVF counting: Now counts each valid returned centroid distance evaluation.
Empty BruteForce statistics: Zero statistics contract added to all empty result paths.
C API opt-in lifecycle: Statistics are explicitly opt-in via enable_statistics/get_statistics/destroy_statistics, tracked externally without reading uninitialized other_result.
SINDI sparse complexity: Replaced full-window sweep with SparseEvaluationTracker that counts unique candidate IDs via generation-based deduplication.
Missing datacell counters: Added AddDistance calls to RaBitQSplitDataCell, SparseVectorDataCell, SparseDmqDataCell, and MultiVectorDataCell Query overrides.
BruteForce parallel atomics: Worker threads set local_query_context.stats = nullptr to avoid per-candidate CAS contention, aggregating once per thread after the loop.
Empty index contract: make_empty_search_result now covers all six index types (HGraph, IVF, Pyramid, BruteForce, SINDI, SIMQ).

Current Assessment

The code is well-structured and consistent. Key design decisions are sound:

Phase tracking: QueryContext::distance_phase flows through the search pipeline, with flatten_reorder.cpp temporarily switching to RERANK and restoring the previous phase.
Backend caching: FlattenInterface::backend_ is set once at construction via BackendFromName(quantizer->Name()), avoiding string lookups in hot paths.
Thread safety: All SearchStatistics counters use std::atomic with memory_order_relaxed; AddDistance uses CAS loops with saturating arithmetic.
Test coverage: Unit tests cover distance contract, unknown backend incompleteness, saturation, backend name classification, empty result contract, and C API statistics lifecycle.

No new issues found. The implementation is complete and correct.

LHT129

Code Review Summary — PR #2545

This PR adds per-search distance evaluation statistics across all index algorithms (HGraph, BruteForce, IVF, Pyramid, SINDI, SIMQ), with JSON output, phase/backend breakdowns, and C/Python API accessors. The current push (commit fdd1a2b) addresses all issues raised in prior review rounds by LHT129 and vsag-bot.

Prior Issues — All Resolved ✓

#	Issue	Status
1	WORKBOARD.md committed	Removed
2	SaturatingAdd return value subtlety	Fixed — returns void, caller reads counter directly
3	BackendName called in hot path	Fixed — backend_ cached at construction in FlattenDataCell, BucketDataCell, etc.
4	Sparse backend ternary chain readability	Fixed — extracted `sparse_backend()` helper with switch
5	GNO-IMI search routing never reaches counter	Fixed — `inner_joint_classify_datas` now adds routing statistics
6	PQ FastScan classified as ordinary PQ	Fixed — `PQ_FASTSCAN` enum value added
7	Route-only IVF omits centroid distance evaluations	Fixed — route-only path counts centroid evaluations in routing phase
8	Empty BruteForce searches return no statistics	Fixed — `make_empty_search_result` covers all 6 index types
9	FlattenDataCell backend_ not set	Fixed — set in constructor via `SearchStatistics::BackendFromName`
10	SparseEvaluationTracker generation overflow	Fixed — `BeginWindow()` increments generation with uint16_t wrap
11	SparseTermDataCell query not using tracker	Fixed — uses `evaluation_tracker.Mark()` and returns `evaluation_tracker.Count()`
12	SINDI mutable path missing stats	Fixed — both mutable and immutable paths instrumented

Overall Assessment

Correctness: The statistics pipeline is well-designed with proper opt-in semantics, thread-safe CAS-based counting, and consistent phase/backend tracking across all index types.
Performance: Backend names are cached at construction time to avoid hot-path string allocation. Per-thread local aggregation in BruteForce avoids CAS contention.
Test coverage: New unit tests cover distance contract, saturation, backend name classification, SparseEvaluationTracker, C API lifecycle, and empty result statistics.
Maintainability: Clean separation between QueryContext (pipeline state) and SearchStatistics (aggregation). JSON output format is extensible.

Verdict: Approve. All prior concerns have been addressed. No new issues found.

LHT129

Test review from LHT129

LHT129

Test review from LHT129 via API

LHT129

Code Review Summary — PR #2545

This PR adds per-search distance evaluation statistics across all index algorithms (HGraph, BruteForce, IVF, Pyramid, SINDI, SIMQ), with JSON output, phase/backend breakdowns, and C/Python API accessors. The current push (commit fdd1a2b) addresses all issues raised in prior review rounds by LHT129 and vsag-bot.

Prior Issues — All Resolved ✓

#	Issue	Status
1	WORKBOARD.md committed	Removed
2	SaturatingAdd return value subtlety	Fixed — returns void, caller reads counter directly
3	BackendName called in hot path	Fixed — backend_ cached at construction in FlattenDataCell, BucketDataCell, etc.
4	Sparse backend ternary chain readability	Fixed — extracted `sparse_backend()` helper with switch
5	GNO-IMI search routing never reaches counter	Fixed — `inner_joint_classify_datas` now adds routing statistics
6	PQ FastScan classified as ordinary PQ	Fixed — `PQ_FASTSCAN` enum value added
7	Route-only IVF omits centroid distance evaluations	Fixed — route-only path counts centroid evaluations in routing phase
8	Empty BruteForce searches return no statistics	Fixed — `make_empty_search_result` covers all 6 index types
9	FlattenDataCell backend_ not set	Fixed — set in constructor via `SearchStatistics::BackendFromName`
10	SparseEvaluationTracker generation overflow	Fixed — `BeginWindow()` increments generation with uint16_t wrap
11	SparseTermDataCell query not using tracker	Fixed — uses `evaluation_tracker.Mark()` and returns `evaluation_tracker.Count()`
12	SINDI mutable path missing stats	Fixed — both mutable and immutable paths instrumented

Overall Assessment

Correctness: The statistics pipeline is well-designed with proper opt-in semantics, thread-safe CAS-based counting, and consistent phase/backend tracking across all index types.
Performance: Backend names are cached at construction time to avoid hot-path string allocation. Per-thread local aggregation in BruteForce avoids CAS contention.
Test coverage: New unit tests cover distance contract, saturation, backend name classification, SparseEvaluationTracker, C API lifecycle, and empty result statistics.
Maintainability: Clean separation between QueryContext (pipeline state) and SearchStatistics (aggregation). JSON output format is extensible.

Verdict: Approve. All prior concerns have been addressed. No new issues found.

Signed-off-by: Xiangyu Wang <wxy407827@antgroup.com> Assisted-by: OpenAI Codex:gpt-5

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

vsag-bot

Automated inline review completed.

Review effort: high (925 changed lines across 41 files).
Submitted 2 inline comments.

Reviewed commit 8467218.

vsag-bot · 2026-07-29T11:42:01Z

    uint64_t coarse_probe_count = 0;
    auto coarse_results = coarse_search(
        query_mvs[0].vectors_, query_mvs[0].len_, coarse_k, &coarse_dist_cmp, &coarse_probe_count);
+    stats.AddDistance(SearchStatistics::DistancePhase::ROUTING, "fp32", coarse_dist_cmp);


[P2] Use nested HGraph distance evaluations for SIMQ routing

coarse_dist_cmp is populated from the nested HGraph's legacy dist_cmp, but that counter increments only for visited neighbors and omits the entry-point distance computed at the start of every graph traversal. For example, a one-centroid HGraph can evaluate its entry point while returning dist_cmp == 0, so SIMQ reports zero routing evaluations despite doing one. Preserve the legacy SIMQ field, but aggregate the nested result's new distance_evaluations value into this contract.

vsag-bot · 2026-07-29T11:42:01Z

            return make_empty_dataset_with_stats();
        }
        if (iter_filter_ctx->IsFirstUsed()) {
+            ctx.distance_phase = DistanceEvaluationPhase::ROUTING;


[P2] Preserve statistics for empty iterator results

In this iterator-search path, a reject-all filter can still perform routing/base distance evaluations but leave search_result empty. The function then returns DatasetImpl::MakeEmptyDataset() at the earlier empty-result branch before the statistics attachment, so the result has an empty statistics string and loses all counts produced under this phase. Attach stats.Dump() to that empty result just as the non-iterator path does.

LHT129 · 2026-07-29T11:48:51Z

+            case IndexType::SINDI:
+            case IndexType::SIMQ: {
+                SearchStatistics statistics;
+                result->Statistics(statistics.Dump());


[note] Pyramid is listed in the documentation and make_empty_search_result() as a maintained index with statistics support, but no search-path instrumentation exists for Pyramid (unlike HGraph, BruteForce, IVF, SINDI, and SIMQ which all have explicit AddDistance calls in their search implementations). Non-empty Pyramid searches will return an empty statistics string rather than the documented contract. Either add instrumentation to the Pyramid search path or remove Pyramid from the documented supported list.

Copilot AI review requested due to automatic review settings July 27, 2026 13:07

wxyucs requested review from LHT129, inabao and jiaweizone as code owners July 27, 2026 13:07

pull-request-size Bot added the size/L label Jul 27, 2026

Copilot AI reviewed Jul 27, 2026

vsag-bot assigned wxyucs Jul 27, 2026

vsag-bot self-requested a review July 27, 2026 13:07

wxyucs added kind/feature Brand-new functionality or capabilities 引入全新的功能、新特性或新能力 version/1.1 module/api module/index module/datacell module/python module/docs labels Jul 27, 2026

wxyucs force-pushed the issue-2544-distance-stats branch from b503d2f to cfca2cb Compare July 27, 2026 13:10

Copilot AI review requested due to automatic review settings July 27, 2026 13:10

Copilot AI reviewed Jul 27, 2026

wxyucs force-pushed the issue-2544-distance-stats branch from cfca2cb to 2be5988 Compare July 28, 2026 03:27

Copilot AI review requested due to automatic review settings July 28, 2026 03:27

wxyucs force-pushed the issue-2544-distance-stats branch from 2be5988 to c2020bb Compare July 28, 2026 03:27

pull-request-size Bot added size/XL and removed size/L labels Jul 28, 2026

Copilot AI reviewed Jul 28, 2026

LHT129 reviewed Jul 28, 2026

View reviewed changes

Comment thread WORKBOARD.md Outdated

Comment thread src/query_context.h

Comment thread src/query_context.h

Comment thread src/datacell/flatten_datacell.h Outdated

Comment thread src/algorithm/sindi/sindi.cpp

vsag-bot reviewed Jul 28, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings July 28, 2026 08:31

vsag-bot reviewed Jul 28, 2026

View reviewed changes

LHT129 reviewed Jul 28, 2026

View reviewed changes

Comment thread src/algorithm/ivf/flat_bucket_searcher.cpp Outdated

LHT129 reviewed Jul 28, 2026

View reviewed changes

Comment thread src/query_context.h Outdated

wxyucs mentioned this pull request Jul 28, 2026

ci: CircleCI setup can time out waiting for Ubuntu package mirrors #2558

Open

Copilot AI review requested due to automatic review settings July 28, 2026 10:43

wxyucs force-pushed the issue-2544-distance-stats branch from f1ec40e to a4bbf90 Compare July 28, 2026 10:43

Copilot AI reviewed Jul 28, 2026

LHT129 reviewed Jul 28, 2026

View reviewed changes

vsag-bot self-requested a review July 28, 2026 11:02

vsag-bot reviewed Jul 28, 2026

View reviewed changes

Comment thread src/algorithm/sindi/sindi.cpp Outdated

Comment thread src/datacell/sparse_term_datacell.cpp Outdated

Comment thread src/index/index_impl.h

Comment thread src/query_context.h

Copilot AI review requested due to automatic review settings July 29, 2026 06:34

wxyucs force-pushed the issue-2544-distance-stats branch from a4bbf90 to fdd1a2b Compare July 29, 2026 06:34

vsag-bot self-requested a review July 29, 2026 06:34

Copilot AI reviewed Jul 29, 2026

Copilot AI review requested due to automatic review settings July 29, 2026 06:38

wxyucs force-pushed the issue-2544-distance-stats branch from fdd1a2b to 7f7dd05 Compare July 29, 2026 06:38

Copilot AI reviewed Jul 29, 2026

LHT129 reviewed Jul 29, 2026

View reviewed changes

Comment thread src/datacell/bucket_datacell.h

wxyucs mentioned this pull request Jul 29, 2026

[bug](ci): IVF random-allocator continuation test segfaults after allocation failures #2547

Open

feat: expose per-search distance evaluation statistics

8467218

Signed-off-by: Xiangyu Wang <wxy407827@antgroup.com> Assisted-by: OpenAI Codex:gpt-5

wxyucs force-pushed the issue-2544-distance-stats branch from 7f7dd05 to 8467218 Compare July 29, 2026 11:32

Copilot AI review requested due to automatic review settings July 29, 2026 11:32

Copilot AI reviewed Jul 29, 2026

vsag-bot reviewed Jul 29, 2026

View reviewed changes

LHT129 reviewed Jul 29, 2026

View reviewed changes

Uh oh!

Conversation

wxyucs commented Jul 27, 2026

Uh oh!

vsag-bot commented Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

vsag-bot commented Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mergify Bot commented Jul 27, 2026

Merge Protections

🟢 Require kind label

🟢 Require version label

🟢 Require linked issue for feature/bug PRs

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vsag-bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vsag-bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

LHT129 left a comment

Choose a reason for hiding this comment

Code Review Summary

✅ P1 & P2 Issues Resolved

Observations

Overall

Uh oh!

vsag-bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

LHT129 left a comment

Choose a reason for hiding this comment

Code Review Summary

vsag-bot commented Jul 27, 2026 •

edited

Loading

vsag-bot commented Jul 27, 2026 •

edited

Loading