Skip to content

MOD-15579 Lazy initialization of the shared SVS thread pool#971

Open
dor-forer wants to merge 1 commit into
mainfrom
dor-forer-MOD-15579-svs-thpool-lazy
Open

MOD-15579 Lazy initialization of the shared SVS thread pool#971
dor-forer wants to merge 1 commit into
mainfrom
dor-forer-MOD-15579-svs-thpool-lazy

Conversation

@dor-forer
Copy link
Copy Markdown
Collaborator

@dor-forer dor-forer commented May 26, 2026

Make initialization of the shared SVS thread pool lazy: OS worker threads are spawned only when the first SVS index is created, not when VecSim_UpdateThreadPoolSize is called. This eliminates the cost of pre-allocated SVS workers in deployments that configure RediSearch workers but never create an SVS index (today, every Redis with WORKERS=N configured spawns N-1 SVS worker threads at module init regardless of whether SVS is ever used).

The change extends the existing deferred-resize protocol to cover a second "safe point". VecSimSVSThreadPoolImpl::resize() now defers application in two cases:

  • No SVS index attached yet → recorded; applied on first onIndexAttached() (lazy thread spawn).
  • Shrink while jobs are in flight (pending_jobs_ > 0) → recorded; applied when endScheduledJob() drops the counter to 0 (existing behaviour).
  • Otherwise → applied immediately via resize_locked().

The two deferral cases share deferred_size_ and never overlap (no jobs can be in flight before the first index attaches). One new gate field, has_attached_index_, distinguishes the two states. It is set on the first onIndexAttached() call from the per-index VecSimSVSThreadPool ctor.

VecSim_UpdateThreadPoolSize continues to set the write mode (InPlace/Async) eagerly — only the SVS pool sizing is now deferred. The C API surface is unchanged.

RediSearch integration impact: none required — VecSim_UpdateThreadPoolSize(N) is still the single entry point for both module init and CONFIG SET search-workers M. Pre-index calls now just record the size; post-index calls behave exactly as before.

Main objects this PR modified

  1. VecSimSVSThreadPoolImpl::resize() — now lazy until the first index attaches.
  2. VecSimSVSThreadPoolImpl::onIndexAttached() — new entry point called from the per-index VecSimSVSThreadPool ctor; flips has_attached_index_ and applies any deferred size.
  3. VecSimSVSThreadPoolImpl::has_attached_index_ — new member field gating spawn vs. defer.
  4. VecSimSVSThreadPoolImpl::resetForTest() — new BUILD_TESTS-only hook that restores the singleton to a clean baseline. Asserts pending_jobs_ == 0 so misuse fails loudly instead of corrupting the deferred-protocol bookkeeping.
  5. VecSim_UpdateThreadPoolSize() — comment updated; behaviour now lazy via the new resize() semantics.

Tests

  • SVSTest.ThreadPoolLazyInit (new) — verifies VecSim_UpdateThreadPoolSize does not allocate worker threads before any SVS index exists, and that the first SVS index creation triggers the deferred spawn at the previously requested size. Post-VecSimIndex_New checks use ASSERT_* so a regression in the lazy-spawn contract halts the test on the first failure.
  • SVSThreadPoolTest.UpdateThreadPoolSizeModeTransitions — validates the lazy → eager transition (pool size stays at 1 until an index attaches; subsequent transitions apply immediately).
  • SVSThreadPoolTest fixture — SetUp() calls onIndexAttached() so the pool-isolation tests retain eager resize() semantics.
  • SVSTest.NumThreadsParamIgnored — calls onIndexAttached() up front (resizes before constructing an index) and restores the singleton via resetForTest() so it does not leak has_attached_index_ to later tests.
  • SVSTest.sharedMemoryTracksThreadPoolResize — calls onIndexAttached() up front so the resize-without-index path applies eagerly and the grow/shrink memory deltas are observable.

Mark if applicable

  • This PR introduces API changes
  • This PR introduces serialization changes

Supersedes #969 — ownership transferred to @dor-forer. Builds on #972 (MOD-15578), now merged to main.


Note

Medium Risk
Changes shared thread-pool lifecycle and when OS threads are created, but the public C API is unchanged and deferred-shrink while jobs are in flight is preserved; risk is mainly regressions in worker count or timing around module init vs first SVS index.

Overview
Lazy SVS thread pool sizing defers spawning OS worker threads until the first SVS index is created. VecSim_UpdateThreadPoolSize still switches write mode (InPlace / Async) immediately; only pool sizing is deferred.

VecSimSVSThreadPoolImpl::resize() now records the requested size in deferred_size_ when no index has attached (has_attached_index_ is false) instead of calling resize_locked() right away. The first per-index VecSimSVSThreadPool construction calls onIndexAttached(), which sets the flag and applies any deferred size—this is where workers are created on the lazy path. After attach, resize behaves as before (immediate grow/shrink, with shrink still deferred while pending_jobs_ > 0).

A BUILD_TESTS-only resetForTest() clears slots, deferred_size_, and has_attached_index_ so unit tests get a clean singleton. Tests add ThreadPoolLazyInit, call onIndexAttached() where tests resize without a real index, and extend UpdateThreadPoolSizeModeTransitions for the lazy → eager transition.

Reviewed by Cursor Bugbot for commit 6f073fb. Bugbot is set up for automated code reviews on this repo. Configure here.

@jit-ci
Copy link
Copy Markdown

jit-ci Bot commented May 26, 2026

🛡️ Jit Security Scan Results

CRITICAL HIGH MEDIUM

✅ No security findings were detected in this PR


Security scan by Jit

@dor-forer dor-forer changed the base branch from meiravg_svs_thpool_alloactor to dor-forer-MOD-15578-track-svs-thpool-memory May 26, 2026 14:26
@codecov
Copy link
Copy Markdown

codecov Bot commented May 26, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.09%. Comparing base (ca937db) to head (6f073fb).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #971      +/-   ##
==========================================
- Coverage   97.11%   97.09%   -0.02%     
==========================================
  Files         141      141              
  Lines        8144     8160      +16     
==========================================
+ Hits         7909     7923      +14     
- Misses        235      237       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@dor-forer dor-forer force-pushed the dor-forer-MOD-15578-track-svs-thpool-memory branch from 31c0e0a to 0adca87 Compare May 26, 2026 14:51
@dor-forer dor-forer force-pushed the dor-forer-MOD-15579-svs-thpool-lazy branch from 6b4b160 to 8fe69ac Compare May 26, 2026 15:04
@dor-forer dor-forer force-pushed the dor-forer-MOD-15578-track-svs-thpool-memory branch from 0adca87 to 9d082f0 Compare May 26, 2026 15:23
@dor-forer dor-forer force-pushed the dor-forer-MOD-15579-svs-thpool-lazy branch from 8fe69ac to 760524d Compare May 26, 2026 15:23
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 760524d. Configure here.

Comment thread src/VecSim/algorithms/svs/svs_utils.h Outdated
@dor-forer dor-forer force-pushed the dor-forer-MOD-15579-svs-thpool-lazy branch 2 times, most recently from 75228a3 to f278e8d Compare May 27, 2026 08:39
@dor-forer dor-forer force-pushed the dor-forer-MOD-15579-svs-thpool-lazy branch from f278e8d to 09d0e83 Compare June 3, 2026 15:06
Base automatically changed from dor-forer-MOD-15578-track-svs-thpool-memory to main June 4, 2026 17:06
@dor-forer dor-forer force-pushed the dor-forer-MOD-15579-svs-thpool-lazy branch from 09d0e83 to 33f7564 Compare June 7, 2026 06:28
Defer creation of the shared SVS thread pool's OS worker threads until
the first SVS index is created, instead of spawning them eagerly on
`VecSim_UpdateThreadPoolSize()` at module init. Today every Redis with
`WORKERS=N` configured spawns N-1 SVS worker threads at startup even in
deployments that never create an SVS index; this eliminates that cost.

The change extends the existing deferred-resize protocol to a second
safe point. `VecSimSVSThreadPoolImpl::resize()` now defers in two cases:

* No SVS index attached yet  -> record the requested size; apply it on
  the first `onIndexAttached()` call (lazy thread spawn).
* Shrink while jobs are in flight (`pending_jobs_ > 0`) -> record; apply
  when `endScheduledJob()` drops the counter to 0 (existing behaviour).
* Otherwise -> apply immediately via `resize_locked()`.

The two deferral cases share `deferred_size_` and never overlap (no jobs
can be in flight before the first index attaches). A new gate field
`has_attached_index_` distinguishes the states; it is set on the first
`onIndexAttached()` call from the per-index `VecSimSVSThreadPool` ctor.

`VecSim_UpdateThreadPoolSize` still sets the write mode eagerly — only
the SVS pool sizing is deferred. The C API surface is unchanged, and no
RediSearch integration changes are required.

Test-only `resetForTest()` (under BUILD_TESTS) restores the singleton to
a clean baseline; it asserts `pending_jobs_ == 0` so misuse fails loudly
instead of corrupting the deferred-protocol bookkeeping.

Tests:
* SVSTest.ThreadPoolLazyInit (new) — verifies UpdateThreadPoolSize does
  not allocate worker threads before any SVS index exists, and that the
  first index creation triggers the deferred spawn at the requested size.
* SVSThreadPoolTest.UpdateThreadPoolSizeModeTransitions — validates the
  lazy -> eager transition (size stays 1 until an index attaches).
* SVSThreadPoolTest fixture / SVSTest.NumThreadsParamIgnored /
  SVSTest.sharedMemoryTracksThreadPoolResize — call onIndexAttached()
  up front so resizes apply eagerly where the test needs it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dor-forer dor-forer force-pushed the dor-forer-MOD-15579-svs-thpool-lazy branch from 33f7564 to 6f073fb Compare June 7, 2026 06:55
@dor-forer dor-forer requested a review from alonre24 June 7, 2026 07:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant