-
Notifications
You must be signed in to change notification settings - Fork 99
feat: expose split RaBitQ parameters for Pyramid #2457
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
0ed782c
db6f812
f3bfdd3
8f9322b
56d5e33
92a23ec
b3debc7
2a77660
3c3a50f
568d6d1
27ceb32
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -93,7 +93,9 @@ Build-time parameters live under `index_param`. | |
| | `neighbor_sample_rate` | float | — | ODescent neighbor sampling rate. | | ||
| | `no_build_levels` | int[] | `[]` | Tree levels that skip graph construction (0-indexed from the root). | | ||
| | `use_reorder` | bool | `false` | Keep a high-precision copy for rescoring. | | ||
| | `precise_quantization_type` | string | `"fp32"` | Quantizer for reordering. | | ||
| | `precise_quantization_type` | string | `"fp32"` | Quantizer for reordering. Use `"rabitq"` with `rabitq_bits_per_dim_precise` to enable RaBitQ x+y split reorder from base storage. | | ||
| | `rabitq_bits_per_dim_base` | int | `1` | RaBitQ stored-code bits. In x+y split mode, this is `x`, the filter bits used during graph traversal; allowed range is `[1, 8]`. | | ||
| | `rabitq_bits_per_dim_precise` | int | unset | RaBitQ split `y` bits. When set with `base_quantization_type: "rabitq"` and `precise_quantization_type: "rabitq"`, Pyramid uses split storage; `rabitq_bits_per_dim_base` remains `x`, and `x + y <= 8`. | | ||
| | `fast_encode_rabitq` | bool | `true` | Use the fast multi-bit RaBitQ encoder for RaBitQ base or precise storage; set to `false` for the exact encoder. | | ||
| | `fast_encode_rabitq_rounds` | int | `6` | Fast RaBitQ refinement rounds in `[1, 32]`. | | ||
| | `base_io_type` / `precise_io_type` | string | `"block_memory_io"` | Base and reorder storage backends; `uring_io` is available in builds with liburing. | | ||
|
|
@@ -103,6 +105,20 @@ Build-time parameters live under `index_param`. | |
| | `build_thread_count` | int | `1` | Threads used for parallel build. | | ||
| | `hierarchies` | array | `[]` | Named hierarchy definitions. Each element is either a string (inherits all top-level params) or an object with `name` and optional overrides (`max_degree`, `ef_construction`, `alpha`, `no_build_levels`, `index_min_size`). When present, multi-hierarchy mode is activated and each hierarchy maintains its own independent path tree. | | ||
|
|
||
| ### RaBitQ split configuration | ||
|
|
||
| Set all five parameters together to enable RaBitQ x+y split storage and reordering: | ||
|
|
||
| ```json | ||
| { | ||
| "use_reorder": true, | ||
| "base_quantization_type": "rabitq", | ||
| "precise_quantization_type": "rabitq", | ||
| "rabitq_bits_per_dim_base": 3, | ||
| "rabitq_bits_per_dim_precise": 5 | ||
| } | ||
| ``` | ||
|
|
||
| ## Search parameters | ||
|
|
||
| Search-time parameters live under the `pyramid` sub-object: | ||
|
|
@@ -113,6 +129,7 @@ Search-time parameters live under the `pyramid` sub-object: | |
| | `subindex_ef_search` | int | `50` | Candidate list size used when traversing intermediate sub-graphs on the path. | | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [suggestion]
Consider adding a row like: |
||
| | `hierarchies` | string[] | `[]` | Select which hierarchy to search. Empty means use the default (unnamed) hierarchy. | | ||
| | `hierarchy_op` | string | `"single"` | How to combine results across hierarchies: `single` (search one hierarchy), `union`, or `intersection`. **Note:** `union` and `intersection` are not yet implemented — setting them will cause `KnnSearch`/`RangeSearch` to return an error. | | ||
| | `rabitq_error_rate` | float | `1.9` | Positive lower-bound error multiplier for this search. The default `1.9` is relatively large; increasing it improves accuracy but slows down search. | | ||
|
|
||
| ```cpp | ||
| auto result = index->KnnSearch( | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.