[WIP]fix(eval): correct peak memory measurement

[jac0626]

Change Type

• Bug fix
• New feature
• Improvement/Refactor
• Documentation
• CI/Build/Infra

Linked Issue

• Fixes: eval_performance Reports Incorrect Memory Usage (16777216.00 TB) #1109

What Changed

• Replace the underflow-prone, point-in-time RSS calculation with a restartable background sampler that reads process RSS every 5 ms on Linux and macOS.
• Preserve memory_peak(build/search) while adding raw baseline, absolute peak, delta, availability, sample-count, and error fields; clamp negative deltas to zero.
• Measure only Index::Build() during build and the latency/QPS query pass during search; run KNN statistics outside monitored intervals.
• Reuse one evaluation dataset in build,search mode and release the build index before creating/deserializing the search index.
• Report index-owned memory through index_memory(B) in addition to process RSS and existing memory details.
• Add regression tests, macOS eval-test coverage, and synchronized English/Chinese documentation.

Root Cause

The previous monitor subtracted two uint64_t RSS page counts without validating reads or ordering. If the final RSS was below the constructor-time baseline, subtraction wrapped near UINT64_MAX, producing 16777216.00 TB. Build also sampled only after completion, search used a later intrusive query pass, and build,search retained duplicate datasets and index objects.

Test Evidence

• make fmt
• make lint
• make test
• make cov, run tests, and collect coverage
• Other (described below)

Test details:

cmake --build build --target unittests eval_performance --parallel 6
./build/tests/unittests '[eval]' --allow-running-no-tests
# All tests passed: 64 assertions in 11 test cases

clang-format-15 --dry-run --Werror <changed C++ files>
# Ubuntu clang-format version 15.0.7

python3 -c "import yaml; yaml.safe_load(open('.github/workflows/pr-ci.yml'))"
git diff --check

The eval tests also passed 50 consecutive repetitions. A temporary 20,000-vector HDF5 build,search end-to-end run completed successfully and produced sane byte/RSS fields without the oversized TB value.

Compatibility Impact

• API/ABI compatibility: none; changes are confined to the eval tool, tests, CI, and documentation.
• Behavior changes: memory output now represents sampled process-RSS peak delta for the measured phase. Existing human-readable memory_peak(build/search) keys remain available; additional diagnostics are emitted in JSON.

Performance and Concurrency Impact

• Performance impact: no change to VSAG's Index::KnnSearch() implementation. When memory and latency are enabled together, one 5 ms RSS sampling thread runs concurrently and may add small benchmark noise; KNN statistics no longer contaminate latency/QPS measurements.
• Concurrency/thread-safety impact: adds a phase-scoped sampler thread guarded by private synchronization; it does not take index or per-query locks.

Documentation Impact

• No docs update needed
• Updated docs:
  • README.md
  • DEVELOPMENT.md
  • CONTRIBUTING.md
  • Other: docs/docs/en/src/resources/eval.md, docs/docs/zh/src/resources/eval.md

Risk and Rollback

• Risk level: medium; the change updates eval measurement boundaries and output diagnostics but does not modify the library API or index algorithms.
• Rollback plan: revert commit af9b6b1e to restore the previous eval monitor and pass lifecycle.

Checklist

• I have linked the relevant issue.
• I have added/updated tests for the bug fix.
• I have considered API compatibility impact.
• I have updated docs for the behavior change.
• My commit message follows project conventions.

[vsag-bot]

/label status/blocked
/waiting-on author
/request-review @jiaweizone
/request-review @wxyucs
/request-review @inabao
/request-review @LHT129
/request-review @vsag-bot

[vsag-bot]

Automated pull request review failed.

Review effort: high (1328 changed lines across 18 files).

git exited with 128: Cloning into '/tmp/vsag-pull-request-reviews/f4b26dd0-a029-46c5-8cd1-b1edc7bd7277/source'...
fatal: unable to access 'https://github.com/antgroup/vsag.git/': Received HTTP code 503 from proxy after CONNECT

No GitHub review was submitted.

[mergify]

Merge Protections

🟢 All 3 merge protections satisfied — ready to merge.

Show 3 satisfied protections

🟢 Require kind label

• label~=^kind/

🟢 Require version label

• label~=^version/

🟢 Require linked issue for feature/bug PRs

• body~=(?im)(?:^|[\s\-\*])(?:close[sd]?|fix(?:e[sd])?|resolve[sd]?)\s*:?\s+(?:#\d+|[\w.\-]+/[\w.\-]+#\d+|https?://github\.com/[\w.\-]+/[\w.\-]+/issues/\d+)

[Copilot]

Pull request overview

Fixes incorrect eval_performance peak memory reporting by replacing the underflow-prone RSS delta calculation with a phase-scoped background RSS sampler, while also tightening evaluation phase boundaries and enriching JSON diagnostics. This improves correctness of memory metrics (notably avoiding the multi-terabyte underflow artifact) without changing VSAG core APIs/ABI.

Changes:

• Reworked MemoryPeakMonitor to sample process RSS in a restartable background thread, clamp negative deltas to zero, and emit detailed JSON diagnostics while preserving memory_peak(build/search).
• Refactored search evaluation passes to isolate monitored query phases from KNN statistics collection via SearchPassRunner.
• Added unit/regression coverage for the new monitor/pass scheduling and expanded CI/docs to reflect the updated metrics.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tools/eval/monitor/memory_peak_monitor.h	Redesigns the monitor API/state for a background RSS sampler and richer result fields.
tools/eval/monitor/memory_peak_monitor.cpp	Implements platform RSS readers (Linux/macOS), sampling loop, clamped deltas, and expanded JSON output.
tools/eval/monitor/memory_peak_monitor_test.cpp	Adds regression/unit tests for baseline/peak/delta behavior, restartability, and formatting.
tools/eval/CMakeLists.txt	Adds search_pass_runner.cpp to the eval tool build sources.
tools/eval/case/search_pass_runner.h	Introduces a helper to run monitored search passes with optional memory sampling coordination.
tools/eval/case/search_pass_runner.cpp	Implements pass scheduling to ensure memory sampling aligns with the intended query pass boundaries.
tools/eval/case/search_pass_runner_test.cpp	Adds tests validating pass order, memory-on/off behavior, and exception unwinding semantics.
tools/eval/case/search_eval_case.h	Extends constructor to optionally reuse an already-loaded dataset; adds helpers for pass separation.
tools/eval/case/search_eval_case.cpp	Uses SearchPassRunner to separate monitored query passes from statistics collection; adds index_memory(B) reporting.
tools/eval/case/eval_case.h	Extends factory/ctor to accept an optional shared dataset instance.
tools/eval/case/eval_case.cpp	Implements dataset reuse through MakeInstance and the base EvalCase constructor.
tools/eval/case/build_search_eval_case.h	Reuses one dataset across build+search and releases build index before creating/deserializing the search index.
tools/eval/case/build_eval_case.h	Extends build case constructor to accept an optional shared dataset.
tools/eval/case/build_eval_case.cpp	Tightens build measurement boundary around Index::Build() and adds index_memory(B) reporting.
tests/CMakeLists.txt	Adds eval monitor/pass sources to unittests (plus Threads/json linkage) to run new [eval] tests.
docs/docs/en/src/resources/eval.md	Documents the new RSS sampling semantics, phase boundaries, and additional JSON fields.
docs/docs/zh/src/resources/eval.md	Chinese documentation parity for the new memory metrics and boundaries.
.github/workflows/pr-ci.yml	Ensures tools/** triggers C++ CI and runs eval-tagged tests on macOS ASan builds.

[wxyucs]

/version 1.1