perf(hgraph): fuse RaBitQ split storage and search for x=1..4

docs/docs/en/src/quantization/rabitq_split.md

@@ -46,6 +46,7 @@ The relevant parameters are:
 | `rabitq_bits_per_dim_query` | Must be `32` for split storage. |
 | `rabitq_error_rate` | Default positive multiplier applied to the lower-bound error term. |
 | `use_reorder` | Should be `true` so candidates are ranked with the `x+y` distance. |
+| `rabitq_fused_datacell` | HGraph only; enables fused graph/code layout. Default: `false`. |
 
 The constraints are:
 
@@ -58,6 +59,29 @@ x + y <= 8
 If `rabitq_bits_per_dim_precise` is omitted, HGraph and Pyramid use the standard RaBitQ
 path instead of split storage.
 
+### HGraph fused in-memory layout
+
+For HGraph only, set `rabitq_fused_datacell` to `true` to store each
+bottom-layer node's neighbors, cluster id, label, x-bit code, and y-bit
+supplement in one cache-line-aligned record. Pyramid uses ordinary split
+storage; `rabitq_fused_datacell` is not a Pyramid parameter. The specialized
+HGraph search loop reads the record directly and prefetches graph links and
+quantized codes together. The codec uses 16 reproducibly trained residual
+clusters.
+
+The fused layout is opt-in and has stricter constraints than ordinary split
+storage:
+
+- `1 <= x <= 4`, `y >= 1`, and `x + y <= 8`.
+- The metric must be L2 or inner product.
+- The graph, filter codes, and supplement codes must all use memory IO.
+- MCI, `deduplicate_storage`, and force remove must be disabled.
+- PCA is not supported in fused v1; omit `rabitq_pca_dim` or set it to `0`.
+- The legacy v0.14 serialization format is not supported.
+
+Indexes created without this option keep their existing layout, behavior, and
+serialization format.
+
 Enable the filter/lower-bound search path with:
 
 ```json
@@ -71,11 +95,25 @@ Enable the filter/lower-bound search path with:
 }
 ```
 
+For Pyramid, put the equivalent search controls under `pyramid`:
+
+```json
+{
+    "pyramid": {
+        "ef_search": 200,
+        "rabitq_one_bit_search": true,
+        "rabitq_error_rate": 1.9
+    }
+}
+```
+
 The external search key is named `rabitq_one_bit_search`, but on a split index
 it uses all `x` filter bits configured by `rabitq_bits_per_dim_base`.
-`hgraph.rabitq_error_rate` overrides the index default for that search. It can
-be swept without rebuilding because the stored record contains the geometric
-error scale before this multiplier is applied.
+`hgraph.rabitq_error_rate` and `pyramid.rabitq_error_rate` override the index
+default for their respective searches without requiring a rebuild. Native
+fused HGraph records store the geometric error scale before this multiplier is
+applied. HNSW-compatible fused `1+7` records retain metadata scaled by the
+canonical default and apply an override as a ratio at query time.
 
 ## Search pipeline
 
@@ -260,19 +298,29 @@ sum_i q_i * u_i
       + sum_i q_i * s_i
 ```
 
-For L2 with an x-bit lookup filter, HGraph and Pyramid pass the previously computed
-filter distance to reorder as a hint. `ComputeDistWithSplitCodeAndFilterDist`
-recovers the first term from that hint and computes only the second term from
-the y supplement planes:
+For `x >= 2`, the canonical HGraph and Pyramid graph-search path carries the
+exact x-bit filter inner product from traversal to reorder. Ordinary split
+storage exposes it through `QueryWithDistanceLowerBoundAndFilterIP`, and
+reorder consumes it through `QueryWithFilterIPHint` and
+`ComputeDistWithSplitCodeAndFilterIP`. Fused HGraph uses the same exact hint
+semantics while reading the code directly from the node record. These
+canonical paths do not recover the inner product from a distance, and full
+rerank computes only the second term from the y supplement planes:
 
 ```text
 full contribution = shifted filter contribution + supplement contribution
 ```
 
-Thus a `3+5` index reuses the 3-bit filter result and scans only 5 new bit
-planes for each reordered candidate. If the hint is unavailable or cannot be
-used, the code falls back to `ComputeDistWithSplitCode`, which computes the
-same final distance directly from both split records.
+Thus ordinary and fused `2+y`, `3+y`, and `4+y` indexes reuse the exact x-bit
+filter inner product and scan only the y supplement planes for each reranked
+candidate. `QueryWithDistanceHint` and
+`ComputeDistWithSplitCodeAndFilterDist` remain compatibility APIs for callers
+that only have a filter distance; they are not the canonical graph-search
+pipeline. The fused `1+y` traversal uses a four-bit query bit-plane and
+popcount approximation; precise reranking recomputes its exact one-bit
+contribution because the approximate value is not an exact full-distance
+hint. If a usable hint is unavailable, the code computes the same final
+distance directly from both split records.
 
 ## Memory, disk, and hybrid IO
 
@@ -338,30 +386,46 @@ The split datacell serializes, in order:
 Create the destination index with parameters compatible with the serialized
 index, especially `dim`, `metric_type`, x/y bit widths, and query bits.
 Changing an encoded parameter requires rebuilding the index. Tuning only the
-search-time `hgraph.rabitq_error_rate` does not.
+search-time `hgraph.rabitq_error_rate` or `pyramid.rabitq_error_rate` does not.
+
+For a fused index, the codec model is serialized with the split datacell and
+the per-node codes are serialized once as part of the bottom-graph slab.
+Ordinary and streaming round trips preserve this layout without creating a
+second count-scaled copy of the split codes.
 
 ## Implementation map
 
 | Area | File / entry point |
 | --- | --- |
-| External x/y parameter mapping | `src/algorithm/hgraph/hgraph_param_mapping.cpp` |
+| External x/y parameter mapping | `hgraph_param_mapping.cpp`, `pyramid.cpp` |
 | Split record ownership and IO | `src/datacell/rabitq_split_datacell.h` |
 | Plane layout and code splitting | `RaBitQuantizer::StoredPlaneIndex`, `SplitCode` |
 | Filter estimate and lower bound | `ComputeDistWithOneBitLowerBound` |
 | Direct split distance | `ComputeDistWithSplitCode` |
-| Reorder using the filter hint | `ComputeDistWithSplitCodeAndFilterDist` |
+| Reorder using the filter hint | `ComputeDistWithSplitCodeAndFilterDist`, `ComputeDistWithSplitCodeAndFilterIP` |
 | SIMD dispatch | `src/simd/rabitq_simd.cpp` |
 | AVX2 / AVX512 lookup kernels | `src/simd/avx2.cpp`, `src/simd/avx512.cpp` |
 | Runnable memory/disk/hybrid example | `examples/cpp/323_index_hgraph_rabitq_split.cpp` |
 
 ## Operational notes
 
-- Split storage is currently available on HGraph and Pyramid and requires fp32 query codes. Pyramid enables the one-bit split search path by default for split indexes; pass `rabitq_one_bit_search: false` under `pyramid` to force the standard search path.
-- `l2`, `ip`, and `cosine` are supported. The filter-hint reorder shortcut is
-  currently specialized for L2.
+- Split storage is currently available on HGraph and Pyramid and requires fp32
+  query codes. Pyramid enables the one-bit split search path by default for
+  split indexes; pass `rabitq_one_bit_search: false` under `pyramid` to force
+  the standard search path.
+- `l2`, `ip`, and `cosine` are supported. For `x >= 2`, the canonical ordinary
+  split and fused HGraph paths directly reuse the exact filter inner product
+  for L2 and inner product. Other cases safely compute the full split distance.
+- The fused datacell supports only L2 and inner product and only the in-memory
+  configuration described above.
+- With `support_duplicate: true`, duplicate build probes and alias-expanding
+  queries use the canonical HGraph searcher; the fused slab remains the code
+  and graph storage.
 - Keep `use_reorder: true` unless x-bit traversal accuracy alone has been
   validated for the dataset.
 - Changing x, y, metric, or transform parameters requires rebuilding the
-  index. A search-time `hgraph.rabitq_error_rate` override does not.
+  index. A search-time `hgraph.rabitq_error_rate` or
+  `pyramid.rabitq_error_rate` override does not.
 - Use [RaBitQ](rabitq.md) for the general quantizer description and
-  [HGraph](../indexes/hgraph.md) and [Pyramid](../indexes/pyramid.md) for the complete index parameter tables.
+  [HGraph](../indexes/hgraph.md) and [Pyramid](../indexes/pyramid.md) for the
+  complete index parameter tables.

docs/docs/zh/src/quantization/rabitq_split.md

@@ -45,6 +45,7 @@ RaBitQ x+y split 是 HGraph 和 Pyramid 面向低比特底库码的存储与搜
 | `rabitq_bits_per_dim_query` | split storage 必须使用 `32`。 |
 | `rabitq_error_rate` | lower-bound 误差项的默认正数倍率。 |
 | `use_reorder` | 建议设为 `true`，使用 `x+y` 距离排序候选。 |
+| `rabitq_fused_datacell` | 仅用于 HGraph；启用融合布局，默认值为 `false`。 |
 
 参数约束为：
 
@@ -57,6 +58,25 @@ x + y <= 8
 如果不配置 `rabitq_bits_per_dim_precise`，HGraph 和 Pyramid 使用 standard RaBitQ 路径，
 不会创建 split storage。
 
+### HGraph 融合内存布局
+
+仅对 HGraph，将 `rabitq_fused_datacell` 设为 `true` 后，底层节点的邻居、
+cluster id、label、x-bit code 和 y-bit supplement 会存入同一个 cache-line
+对齐的 record。Pyramid 使用普通 split storage；`rabitq_fused_datacell` 不是
+Pyramid 参数。HGraph 专用搜索循环直接读取该 record，并联合预取图邻居和
+量化码。codec 使用固定随机种子可复现训练的 16 个 residual clusters。
+
+融合布局是显式启用的，并且比普通 split storage 有更严格的约束：
+
+- `1 <= x <= 4`、`y >= 1` 且 `x + y <= 8`。
+- metric 必须是 L2 或内积。
+- graph、filter code 和 supplement code 必须全部使用内存 IO。
+- 必须关闭 MCI、`deduplicate_storage` 和 force remove。
+- fused v1 不支持 PCA；请省略 `rabitq_pca_dim` 或将其设为 `0`。
+- 不支持旧版 v0.14 序列化格式。
+
+未启用该参数的索引保持原有布局、行为和序列化格式。
+
 使用以下搜索参数启用 filter/lower-bound 搜索路径：
 
 ```json
@@ -70,10 +90,24 @@ x + y <= 8
 }
 ```
 
+Pyramid 需要把对应搜索参数放在 `pyramid` 下：
+
+```json
+{
+    "pyramid": {
+        "ef_search": 200,
+        "rabitq_one_bit_search": true,
+        "rabitq_error_rate": 1.9
+    }
+}
+```
+
 外部搜索参数仍命名为 `rabitq_one_bit_search`，但对 split 索引，它会使用
 `rabitq_bits_per_dim_base` 配置的全部 `x` 个 filter bits。
-`hgraph.rabitq_error_rate` 可以为单次搜索覆盖索引默认值。record 中保存的是乘倍率前的
-几何误差尺度，因此 sweep 这个搜索参数不需要重建索引。
+`hgraph.rabitq_error_rate` 和 `pyramid.rabitq_error_rate` 可以分别为对应索引的
+单次搜索覆盖默认值，且不需要重建索引。原生 HGraph fused record 保存
+乘倍率前的几何误差尺度；HNSW-compatible fused `1+7` record 保留按规范
+默认值缩放的 metadata，并在查询时按相对该默认值的倍率应用 override。
 
 ## 搜索流程
 
@@ -252,17 +286,25 @@ sum_i q_i * u_i
       + sum_i q_i * s_i
 ```
 
-对使用 x-bit lookup filter 的 L2 搜索，HGraph 和 Pyramid 会把之前计算的 filter distance
-作为 hint 传给 reorder。`ComputeDistWithSplitCodeAndFilterDist` 从 hint 恢复第一项，
-只从 y 个 supplement planes 计算第二项：
+当 `x >= 2` 时，HGraph 和 Pyramid 的 canonical graph-search 路径会把遍历阶段
+算出的精确 x-bit filter inner product 直接传给 reorder。普通 split storage
+通过 `QueryWithDistanceLowerBoundAndFilterIP` 输出该值，reorder 再通过
+`QueryWithFilterIPHint` 和 `ComputeDistWithSplitCodeAndFilterIP` 直接消费。
+HGraph fused 路径从 node record 读取 code，但使用相同的精确 hint 语义。
+这些路径无需从 distance 恢复 inner product，full rerank 只计算 y supplement
+planes 对应的第二项：
 
 ```text
 full contribution = shifted filter contribution + supplement contribution
 ```
 
-因此 `3+5` 索引会复用 3-bit filter 结果，每个重排候选只扫描 5 个新的 bit-plane。
-如果 hint 不存在或不能使用，代码会回退到 `ComputeDistWithSplitCode`，直接从两个
-split records 计算相同的最终距离。
+因此普通和 fused `2+y`、`3+y`、`4+y` 都会复用精确的 x-bit filter inner
+product，每个重排候选只扫描 y 个 supplement planes。
+`QueryWithDistanceHint` 和 `ComputeDistWithSplitCodeAndFilterDist` 仍作为兼容 API，
+供只有 filter distance 的调用方使用，但它们不是 canonical graph-search 路径。
+fused `1+y` 的遍历使用 4-bit query bit-plane 与 popcount 近似值；精确重排会
+重新计算它的 1-bit 精确贡献，因为该近似值不能作为精确 full-distance hint。
+如果没有可用 hint，代码会直接从两个 split records 计算相同的最终距离。
 
 ## 内存、磁盘和混合 IO
 
@@ -324,29 +366,40 @@ split datacell 按以下顺序序列化：
 
 创建目标索引时必须使用与序列化索引兼容的参数，尤其是 `dim`、`metric_type`、
 x/y bit 数和 query bits。修改编码参数需要重建索引；只调整搜索参数
-`hgraph.rabitq_error_rate` 不需要。
+`hgraph.rabitq_error_rate` 或 `pyramid.rabitq_error_rate` 不需要。
+
+对于 fused 索引，codec model 随 split datacell 序列化，每个节点的 code 只在
+bottom-graph slab 中序列化一次。普通和 streaming 往返都会保留该布局，
+不会再生成一份随节点数增长的 split code 副本。
 
 ## 实现位置
 
 | 模块 | 文件 / 入口 |
 | --- | --- |
-| 外部 x/y 参数映射 | `src/algorithm/hgraph/hgraph_param_mapping.cpp` |
+| 外部 x/y 参数映射 | `hgraph_param_mapping.cpp`、`pyramid.cpp` |
 | split record 和 IO | `src/datacell/rabitq_split_datacell.h` |
 | plane 布局和 code 拆分 | `RaBitQuantizer::StoredPlaneIndex`、`SplitCode` |
 | filter 距离和 lower bound | `ComputeDistWithOneBitLowerBound` |
 | 直接计算 split distance | `ComputeDistWithSplitCode` |
-| 使用 filter hint 的 reorder | `ComputeDistWithSplitCodeAndFilterDist` |
+| 使用 filter hint 的 reorder | `ComputeDistWithSplitCodeAndFilterDist`、`ComputeDistWithSplitCodeAndFilterIP` |
 | SIMD dispatch | `src/simd/rabitq_simd.cpp` |
 | AVX2 / AVX512 lookup kernel | `src/simd/avx2.cpp`、`src/simd/avx512.cpp` |
 | 内存/磁盘/混合 IO 示例 | `examples/cpp/323_index_hgraph_rabitq_split.cpp` |
 
 ## 使用注意
 
-- split storage 当前可用于 HGraph 和 Pyramid，并且要求 fp32 query code。Pyramid 的 split 索引默认启用 one-bit split 搜索路径；如需强制使用普通搜索路径，可以在 `pyramid` 搜索参数下传 `rabitq_one_bit_search: false`。
-- 支持 `l2`、`ip` 和 `cosine`；利用 filter hint 的 reorder 快速路径当前针对 L2。
+- split storage 当前可用于 HGraph 和 Pyramid，并且要求 fp32 query code。
+  Pyramid 的 split 索引默认启用 one-bit split 搜索路径；如需强制使用普通搜索路径，
+  可以在 `pyramid` 搜索参数下传 `rabitq_one_bit_search: false`。
+- 支持 `l2`、`ip` 和 `cosine`。当 `x >= 2` 时，canonical 普通 split 路径和
+  HGraph fused 路径会为 L2 和内积直接复用精确 filter inner product；其他情况
+  会安全地计算完整 split distance。
+- fused datacell 只支持 L2、内积以及上文所述的纯内存配置。
+- 启用 `support_duplicate: true` 时，重复向量 build probe 和展开 alias 的查询使用
+  HGraph canonical searcher；fused slab 仍负责保存 code 和 graph。
 - 除非已经验证仅靠 x-bit 遍历距离能满足召回要求，否则应保持
   `use_reorder: true`。
 - 修改 x、y、metric 或 transform 参数后必须重建索引；在搜索参数中覆盖
-  `hgraph.rabitq_error_rate` 不需要重建。
+  `hgraph.rabitq_error_rate` 或 `pyramid.rabitq_error_rate` 不需要重建。
 - RaBitQ 通用说明见 [RaBitQ](rabitq.md)，完整 HGraph 参数见
-  [HGraph 索引](../indexes/hgraph.md)和 [Pyramid 索引](../indexes/pyramid.md)。
+  [HGraph 索引](../indexes/hgraph.md) 和 [Pyramid 索引](../indexes/pyramid.md)。

include/vsag/constants.h

@@ -237,6 +237,7 @@ extern const char* const HGRAPH_PRECISE_DIRECT_READ;
 extern const char* const HGRAPH_PARAMETER_EF_RUNTIME;
 extern const char* const HGRAPH_PARAMETER_HOPS_LIMIT;
 extern const char* const HGRAPH_PARAMETER_RABITQ_ONE_BIT_SEARCH;
+extern const char* const HGRAPH_RABITQ_FUSED_DATACELL;
 extern const char* const HGRAPH_PARAMETER_BRUTE_FORCE_THRESHOLD;
 extern const char* const HGRAPH_USE_MCI;
 extern const char* const HGRAPH_MCI_MCS;