diff --git a/docs/docs/en/src/api/search.md b/docs/docs/en/src/api/search.md index 7924ee544f..a6bd210ec7 100644 --- a/docs/docs/en/src/api/search.md +++ b/docs/docs/en/src/api/search.md @@ -41,6 +41,44 @@ enum class SearchMode { | `limited_size_` | `int64_t` | `-1` | Cap on range results; `-1` means no limit. | | `params_str_` | `std::string` | `""` | Algorithm-specific search params as JSON (e.g. `ef_search`). | +### Custom query distance callback + +`distance_batch_func_` optionally supplies query-to-vector scores from application code. It receives +stable external vector IDs, so a closure can keep the query and query-specific state outside VSAG: + +```cpp +request.distance_batch_size_ = 32; +request.distance_batch_func_ = [query](const int64_t* ids, uint64_t count, float* scores) { + for (uint64_t i = 0; i < count; ++i) { + scores[i] = Score(query, ids[i]); + } +}; +``` + +| Field | Type | Default | Meaning | +|-------|------|---------|---------| +| `distance_batch_func_` | `SearchDistanceBatchFunc` | `nullptr` | Fills one score for each input external ID, in input order. Smaller finite scores are better. | +| `distance_batch_size_` | `uint64_t` | `1` | Maximum IDs passed to one callback invocation. Set this to the scorer's efficient batch width for batched inference or external data access. Must be positive when a callback is set. | + +The callback is request-scoped and is not serialized. It may capture the query. `hgraph` permits a +null `query_` in callback mode, while `ivf` still requires one query vector for bucket routing. The +callback must be stable for one request, thread-safe if the caller enables parallel execution, +non-throwing, and must write only finite scores. + +`brute_force` uses the callback for exact KNN and range search. `hgraph` supports KNN only; the +callback drives graph traversal and final ordering, while the graph remains built with its configured +built-in metric. Therefore recall under the callback score is not guaranteed. HGraph may score +filtered traversal nodes to preserve graph connectivity, but filtered nodes are never returned. +Callback HGraph does not support parallel search or `brute_force_threshold`; those configurations +are rejected rather than silently changing search semantics. + +`ivf` supports callback KNN with a non-null single query vector. IVF uses its configured built-in +metric to select `scan_buckets_count` buckets, then applies the callback to candidates in those +buckets. The callback controls candidate ranking but cannot recover vectors outside the selected +buckets; increase `scan_buckets_count` to improve recall. Callback IVF does not support range search, +`disable_bucket_scan`, bucket-graph search, or parallel search. Those configurations are rejected. +Reordering is automatically disabled in callback mode. Other indexes do not support the callback. + ### IVF bucket routing IVF accepts `{"ivf":{"scan_buckets_count":N,"disable_bucket_scan":true}}` through diff --git a/docs/docs/zh/src/api/search.md b/docs/docs/zh/src/api/search.md index c32e3974fe..ce1a3233ef 100644 --- a/docs/docs/zh/src/api/search.md +++ b/docs/docs/zh/src/api/search.md @@ -41,6 +41,39 @@ enum class SearchMode { | `limited_size_` | `int64_t` | `-1` | 范围结果的上限；`-1` 表示不限。 | | `params_str_` | `std::string` | `""` | 算法特有的搜索参数 JSON（如 `ef_search`）。 | +### 自定义查询距离回调 + +`distance_batch_func_` 可选地让应用提供 query 到向量的分数。它接收稳定的外部向量 ID， +因此闭包可以在 VSAG 之外持有 query 和该 query 的上下文： + +```cpp +request.distance_batch_size_ = 32; +request.distance_batch_func_ = [query](const int64_t* ids, uint64_t count, float* scores) { + for (uint64_t i = 0; i < count; ++i) { + scores[i] = Score(query, ids[i]); + } +}; +``` + +| 字段 | 类型 | 默认值 | 含义 | +|------|------|--------|------| +| `distance_batch_func_` | `SearchDistanceBatchFunc` | `nullptr` | 按输入外部 ID 的顺序填写对应分数。分数越小越好，且必须是有限值。 | +| `distance_batch_size_` | `uint64_t` | `1` | 单次回调的最大 ID 数。批量推理或外部数据访问应设置为 scorer 的高效批大小；设置回调时必须为正。 | + +回调仅属于当前请求，不参与序列化。它可以捕获 query。`hgraph` 的回调模式允许 `query_` 为 null， +但 `ivf` 仍需要一个查询向量用于桶路由。回调在一次请求内必须稳定；若调用方启用并行执行则必须 +线程安全；不得抛异常，且只能写入有限分数。 + +`brute_force` 用该回调执行精确 KNN 和范围搜索。`hgraph` 仅支持 KNN：回调驱动图遍历和最终 +排序，但图仍由配置的内置 metric 构建，因此无法保证该回调分数下的 recall。HGraph 可能为保持 +图连通性而计算被过滤的遍历节点，但不会返回这些节点。 +回调 HGraph 不支持并行检索或 `brute_force_threshold`；这些配置会被拒绝，而不会静默改变检索语义。 + +`ivf` 支持带回调的 KNN，但必须提供一个非空的单查询向量。IVF 先以配置的内置 metric 选出 +`scan_buckets_count` 个桶，再对这些桶中的候选调用回调。回调决定候选排序，但无法召回未被选中的 +桶内向量；增大 `scan_buckets_count` 可提高 recall。回调 IVF 不支持范围搜索、`disable_bucket_scan`、 +bucket graph 或并行搜索，这些配置会被拒绝。回调模式会自动关闭精排。其他索引不支持该回调。 + ### IVF 桶路由 IVF 可通过 `params_str_` 接收 diff --git a/include/vsag/search_request.h b/include/vsag/search_request.h index 705cd3b99d..9eb3f106b2 100644 --- a/include/vsag/search_request.h +++ b/include/vsag/search_request.h @@ -15,6 +15,7 @@ #pragma once #include +#include #include #include @@ -26,6 +27,9 @@ namespace vsag { +using SearchDistanceBatchFunc = + std::function; + enum class SearchMode { KNN_SEARCH = 1, RANGE_SEARCH = 2, @@ -83,6 +87,25 @@ class SearchRequest { */ std::string params_str_{}; + /** + * @brief Optional request-scoped callback for custom query scoring. + * + * Receives stable external IDs and writes one lower-is-better score per ID. + * When non-null, supported indexes use this callback for search traversal and + * result ordering instead of their built-in vector metric. Graph traversal can + * score filtered IDs to keep the graph connected, but filtered IDs are never + * returned as results. + */ + SearchDistanceBatchFunc distance_batch_func_{nullptr}; + + /** + * @brief Maximum number of IDs submitted to distance_batch_func_ per invocation. + * + * Defaults to 1 for scalar scorers. Set this to the scorer's efficient batch + * width when it benefits from batched inference or data access. + */ + uint64_t distance_batch_size_{1}; + // for attribute filter /** * @brief Flag to enable attribute-based filtering during search diff --git a/src/algorithm/bruteforce/bruteforce.cpp b/src/algorithm/bruteforce/bruteforce.cpp index 87936f50e7..46b5b50b56 100644 --- a/src/algorithm/bruteforce/bruteforce.cpp +++ b/src/algorithm/bruteforce/bruteforce.cpp @@ -15,7 +15,9 @@ #include "bruteforce.h" #include +#include #include +#include #include #include #include @@ -367,15 +369,30 @@ DatasetPtr BruteForce::SearchWithRequest(const SearchRequest& request) const { std::shared_lock read_lock(this->global_mutex_); - auto computer = this->make_search_computer(request.query_); + const bool use_custom_distance = request.distance_batch_func_ != nullptr; + if (use_custom_distance) { + CHECK_ARGUMENT(request.distance_batch_size_ > 0, + "distance_batch_size must be greater than 0"); + } + + ComputerInterfacePtr computer = nullptr; + if (not use_custom_distance) { + computer = this->make_search_computer(request.query_); + } bool is_range = (request.mode_ == SearchMode::RANGE_SEARCH); if (is_range) { - if (not is_multi_vector_) { + if (use_custom_distance) { + CHECK_ARGUMENT(std::isfinite(request.radius_), "radius must be finite"); + CHECK_ARGUMENT(request.radius_ >= 0.0F, "radius must be non-negative"); + CHECK_ARGUMENT(request.limited_size_ != 0, "limited_size must not be 0"); + } else if (not is_multi_vector_) { this->validate_range_args(request.query_, request.radius_, request.limited_size_); } } else { - if (not is_multi_vector_) { + if (use_custom_distance) { + CHECK_ARGUMENT(request.topk_ > 0, "topk must be greater than 0"); + } else if (not is_multi_vector_) { this->validate_knn_args(request.query_, request.topk_); } } @@ -434,7 +451,13 @@ BruteForce::SearchWithRequest(const SearchRequest& request) const { for (const auto& pair : label_to_inner_id) { float dist = 0.0F; const auto inner_id = pair.second; - this->inner_codes_->Query(&dist, computer, &inner_id, 1); + if (use_custom_distance) { + const auto label = this->label_table_->GetLabelById(inner_id); + request.distance_batch_func_(&label, 1, &dist); + CHECK_ARGUMENT(std::isfinite(dist), "distance callback must return finite scores"); + } else { + this->inner_codes_->Query(&dist, computer, &inner_id, 1); + } reasoning_ctx->SetTrueDistance(inner_id, dist); } } @@ -450,8 +473,39 @@ BruteForce::SearchWithRequest(const SearchRequest& request) const { auto search_func = [&](InnerIdType start, InnerIdType end, const DistHeapPtr& cur_heap) { uint32_t dist_cmp_local = 0; + std::vector custom_inner_ids; + std::vector custom_labels; + std::vector custom_dists; + if (use_custom_distance) { + const uint64_t batch_capacity = + std::min(request.distance_batch_size_, end - start); + custom_inner_ids.reserve(batch_capacity); + custom_labels.reserve(batch_capacity); + custom_dists.resize(batch_capacity); + } + + auto flush_custom_batch = [&]() { + if (custom_inner_ids.empty()) { + return; + } + request.distance_batch_func_( + custom_labels.data(), custom_labels.size(), custom_dists.data()); + for (uint64_t j = 0; j < custom_inner_ids.size(); ++j) { + const float dist = custom_dists[j]; + CHECK_ARGUMENT(std::isfinite(dist), "distance callback must return finite scores"); + if (reasoning != nullptr) { + reasoning->RecordVisit(custom_inner_ids[j], dist, 0); + } + if (not is_range || dist <= radius) { + cur_heap->Push(dist, custom_inner_ids[j]); + } + } + dist_cmp_local += static_cast(custom_inner_ids.size()); + custom_inner_ids.clear(); + custom_labels.clear(); + }; + for (InnerIdType i = start; i < end; ++i) { - float dist = 0.0F; if (attr_filter != nullptr and not attr_filter->CheckValid(i)) { if (reasoning != nullptr) { reasoning->RecordFilterReject(i); @@ -459,21 +513,31 @@ BruteForce::SearchWithRequest(const SearchRequest& request) const { continue; } if (ft == nullptr or ft->CheckValid(i)) { - inner_codes_->Query(&dist, computer, &i, 1); - ++dist_cmp_local; - if (reasoning != nullptr) { - reasoning->RecordVisit(i, dist, 0); - } - if (is_range and dist > radius) { - continue; + if (use_custom_distance) { + custom_inner_ids.push_back(i); + custom_labels.push_back(this->label_table_->GetLabelById(i)); + if (custom_inner_ids.size() == request.distance_batch_size_) { + flush_custom_batch(); + } + } else { + float dist = 0.0F; + inner_codes_->Query(&dist, computer, &i, 1); + ++dist_cmp_local; + if (reasoning != nullptr) { + reasoning->RecordVisit(i, dist, 0); + } + if (is_range and dist > radius) { + continue; + } + cur_heap->Push(dist, i); } - cur_heap->Push(dist, i); } else { if (reasoning != nullptr) { reasoning->RecordFilterReject(i); } } } + flush_custom_batch(); dist_cmp.fetch_add(dist_cmp_local, std::memory_order_relaxed); }; @@ -491,11 +555,24 @@ BruteForce::SearchWithRequest(const SearchRequest& request) const { for (auto i = 0; i < parallel_count; ++i) { auto start = i * chunk_size; auto end = std::min(start + chunk_size, count); + if (start >= end) { + continue; + } auto future = this->thread_pool_->GeneralEnqueue(search_func, start, end, heaps[i]); futures.emplace_back(std::move(future)); } + std::exception_ptr first_error = nullptr; for (auto& future : futures) { - future.get(); + try { + future.get(); + } catch (...) { + if (first_error == nullptr) { + first_error = std::current_exception(); + } + } + } + if (first_error != nullptr) { + std::rethrow_exception(first_error); } heap = heaps[0]; for (auto i = 1; i < parallel_count; ++i) { diff --git a/src/algorithm/hgraph/hgraph_search.cpp b/src/algorithm/hgraph/hgraph_search.cpp index 384ad2d2e6..7abd62ee8b 100644 --- a/src/algorithm/hgraph/hgraph_search.cpp +++ b/src/algorithm/hgraph/hgraph_search.cpp @@ -380,16 +380,36 @@ HGraph::SearchWithRequest(const SearchRequest& request) const { const auto& query = request.query_; bool is_range = (request.mode_ == SearchMode::RANGE_SEARCH); auto k = request.topk_; + const bool use_custom_distance = request.distance_batch_func_ != nullptr; + + if (use_custom_distance) { + CHECK_ARGUMENT(request.distance_batch_size_ > 0, + "distance_batch_size must be greater than 0"); + CHECK_ARGUMENT(not is_range, "HGraph custom distance only supports KNN search"); + } if (is_range) { - this->validate_range_args(query, request.radius_, request.limited_size_); + if (not use_custom_distance) { + this->validate_range_args(query, request.radius_, request.limited_size_); + } } else { - this->validate_knn_args(query, k); + if (not use_custom_distance) { + this->validate_knn_args(query, k); + } else { + CHECK_ARGUMENT(k > 0, "topk must be greater than 0"); + } } auto params = HGraphSearchParameters::FromJson(request.params_str_); ctx.rabitq_error_rate = params.rabitq_error_rate; + if (use_custom_distance) { + CHECK_ARGUMENT(params.parallel_search_thread_count == 1, + "HGraph custom query distance does not support parallel search"); + CHECK_ARGUMENT(params.brute_force_threshold <= 0.0F, + "HGraph custom query distance does not support brute_force_threshold"); + } + CHECK_ARGUMENT( // NOLINT params.ef_search >= 1, fmt::format("ef_search({}) must be at least 1", params.ef_search)); @@ -412,7 +432,8 @@ HGraph::SearchWithRequest(const SearchRequest& request) const { std::shared_ptr reasoning_ctx; if (not is_range and not request.expected_labels_.empty()) { reasoning_ctx = std::make_shared(this->allocator_); - reasoning_ctx->SetSearchParams(k, "HGraph", use_reorder_, request.filter_ != nullptr); + reasoning_ctx->SetSearchParams( + k, "HGraph", use_custom_distance ? false : use_reorder_, request.filter_ != nullptr); UnorderedMap label_to_inner_id(this->allocator_); for (const auto& label : request.expected_labels_) { @@ -426,19 +447,28 @@ HGraph::SearchWithRequest(const SearchRequest& request) const { request.expected_labels_.begin(), request.expected_labels_.end(), this->allocator_); reasoning_ctx->InitializeExpectedTargets(expected_labels_vec, label_to_inner_id); - const auto* const query_vector = get_data(query); - auto precise_flatten = this->basic_flatten_codes_; - if (use_reorder_) { - precise_flatten = this->high_precise_codes_; - } - if (create_new_raw_vector_) { - precise_flatten = this->raw_vector_; + FlattenInterfacePtr precise_flatten = nullptr; + ComputerInterfacePtr computer = nullptr; + if (not use_custom_distance) { + precise_flatten = this->basic_flatten_codes_; + if (use_reorder_) { + precise_flatten = this->high_precise_codes_; + } + if (create_new_raw_vector_) { + precise_flatten = this->raw_vector_; + } + computer = precise_flatten->FactoryComputer(get_data(query)); } - auto computer = precise_flatten->FactoryComputer(query_vector); for (const auto& pair : label_to_inner_id) { float dist = 0.0F; const auto inner_id = pair.second; - precise_flatten->Query(&dist, computer, &inner_id, 1); + if (use_custom_distance) { + const auto label = this->label_table_->GetLabelById(inner_id); + request.distance_batch_func_(&label, 1, &dist); + CHECK_ARGUMENT(std::isfinite(dist), "distance callback must return finite scores"); + } else { + precise_flatten->Query(&dist, computer, &inner_id, 1); + } reasoning_ctx->SetTrueDistance(inner_id, dist); } ctx.reasoning_ctx = reasoning_ctx.get(); @@ -449,15 +479,35 @@ HGraph::SearchWithRequest(const SearchRequest& request) const { search_param.topk = 1; search_param.ef = 1; search_param.is_inner_id_allowed = nullptr; - search_param.enable_rabitq_one_bit_search = params.rabitq_one_bit_search; + search_param.enable_rabitq_one_bit_search = + use_custom_distance ? false : params.rabitq_one_bit_search; + search_param.distance_batch_func = request.distance_batch_func_; + search_param.distance_batch_size = request.distance_batch_size_; if (search_param.ep == INVALID_ENTRY_POINT) { return make_empty_dataset_with_stats(); } - auto vt = this->pool_->TakeOne(); + struct visited_list_guard { + std::shared_ptr pool; + VisitedListPtr visited_list; + + void + Release() { + if (visited_list != nullptr) { + pool->ReturnOne(visited_list); + visited_list.reset(); + } + } + + ~visited_list_guard() { + Release(); + } + }; + visited_list_guard vt_guard{this->pool_, this->pool_->TakeOne()}; + auto& vt = vt_guard.visited_list; - const auto* raw_query = get_data(query); + const auto* raw_query = use_custom_distance ? nullptr : get_data(query); for (auto i = static_cast(this->route_graphs_.size() - 1); i >= 0; --i) { auto result = this->search_one_graph( raw_query, this->route_graphs_[i], this->basic_flatten_codes_, search_param, vt, &ctx); @@ -482,8 +532,9 @@ HGraph::SearchWithRequest(const SearchRequest& request) const { search_param.consider_duplicate = true; search_param.range_search_limit_size = static_cast(request.limited_size_); search_param.parallel_search_thread_count = params.parallel_search_thread_count; - search_param.enable_reorder = params.enable_reorder; - search_param.enable_rabitq_one_bit_search = params.rabitq_one_bit_search; + search_param.enable_reorder = use_custom_distance ? false : params.enable_reorder; + search_param.enable_rabitq_one_bit_search = + use_custom_distance ? false : params.rabitq_one_bit_search; } else { search_param.ef = std::max(params.ef_search, k); search_param.is_inner_id_allowed = ft; @@ -493,9 +544,10 @@ HGraph::SearchWithRequest(const SearchRequest& request) const { std::min(search_param.topk, static_cast(static_cast(k) * params.topk_factor)); } - search_param.enable_reorder = params.enable_reorder; + search_param.enable_reorder = use_custom_distance ? false : params.enable_reorder; search_param.consider_duplicate = true; - search_param.enable_rabitq_one_bit_search = params.rabitq_one_bit_search; + search_param.enable_rabitq_one_bit_search = + use_custom_distance ? false : params.rabitq_one_bit_search; if (params.enable_time_record) { search_param.time_cost = std::make_shared(); search_param.time_cost->SetThreshold(params.timeout_ms); @@ -529,33 +581,42 @@ HGraph::SearchWithRequest(const SearchRequest& request) const { DistHeapPtr search_result; bool brute_force_used = false; MCIHybridSearchResult mci_result(params, ft); - if (params.brute_force_threshold > 0.0F and - mci_result.valid_ratio <= params.brute_force_threshold) { - if (is_range) { - search_result = this->brute_force_search( - raw_query, ft, request.limited_size_, request.radius_, &ctx); + if (not use_custom_distance) { + if (params.brute_force_threshold > 0.0F and + mci_result.valid_ratio <= params.brute_force_threshold) { + if (is_range) { + search_result = this->brute_force_search( + raw_query, ft, request.limited_size_, request.radius_, &ctx); + } else { + search_result = this->brute_force_search( + raw_query, ft, k, 0.0F, &ctx); + } + brute_force_used = true; + mci_result.route = "brute_force"; } else { - search_result = - this->brute_force_search(raw_query, ft, k, 0.0F, &ctx); + mci_result = this->try_mci_search(request, params, ft, raw_query, search_param, &ctx); + if (mci_result.route == "mci") { + search_result = std::move(mci_result.result); + } else { + search_result = this->search_one_graph(raw_query, + this->bottom_graph_, + this->basic_flatten_codes_, + search_param, + vt, + &ctx, + rabitq_lower_bound_candidates_ptr); + } } - brute_force_used = true; - mci_result.route = "brute_force"; } else { - mci_result = this->try_mci_search(request, params, ft, raw_query, search_param, &ctx); - if (mci_result.route == "mci") { - search_result = std::move(mci_result.result); - } else { - search_result = this->search_one_graph(raw_query, - this->bottom_graph_, - this->basic_flatten_codes_, - search_param, - vt, - &ctx, - rabitq_lower_bound_candidates_ptr); - } + search_result = this->search_one_graph(raw_query, + this->bottom_graph_, + this->basic_flatten_codes_, + search_param, + vt, + &ctx, + rabitq_lower_bound_candidates_ptr); } - - this->pool_->ReturnOne(vt); + vt_guard.Release(); // Reorder if (mci_result.route != "mci" and not brute_force_used and use_reorder_ and diff --git a/src/algorithm/ivf/ivf.cpp b/src/algorithm/ivf/ivf.cpp index 46c45dd838..050dc4c277 100644 --- a/src/algorithm/ivf/ivf.cpp +++ b/src/algorithm/ivf/ivf.cpp @@ -17,9 +17,11 @@ #include #include +#include #include #include #include +#include #include "algorithm/inner_index_interface.h" #include "attr/argparse.h" @@ -1709,6 +1711,151 @@ IVF::search(const DatasetPtr& query, return search_result; } +DistHeapPtr +IVF::search_with_custom_distance(const DatasetPtr& query, + const SearchRequest& request, + const InnerSearchParam& param, + QueryContext& ctx, + ReasoningContext* reasoning_ctx) const { + const auto* query_data = query->GetFloat32Vectors(); + auto candidate_buckets = + partition_strategy_->ClassifyDatasForSearch(query_data, 1, param, &ctx); + if (reasoning_ctx != nullptr) { + reasoning_ctx->RecordBucketSelection(candidate_buckets); + } + + int64_t topk = request.topk_; + const int64_t origin_topk = topk; + if (buckets_per_data_ > 1) { + CHECK_ARGUMENT(topk <= std::numeric_limits::max() / buckets_per_data_, + "topk is too large for multi-bucket IVF search"); + topk *= buckets_per_data_; + } + + auto search_result = DistanceHeap::MakeInstanceBySize(this->allocator_, topk); + const auto& filter = param.is_inner_id_allowed; + Filter* attr_filter = nullptr; + + Vector candidate_ids(this->allocator_); + Vector candidate_labels(this->allocator_); + Vector scores(this->allocator_); + const uint64_t batch_capacity = std::min( + request.distance_batch_size_, std::max(1, this->GetNumElements())); + candidate_ids.reserve(batch_capacity); + candidate_labels.reserve(batch_capacity); + scores.resize(batch_capacity); + + auto is_timed_out = [&]() { + if (param.time_cost == nullptr or not param.time_cost->CheckOvertime()) { + return false; + } + if (ctx.stats != nullptr) { + ctx.stats->is_timeout.store(true, std::memory_order_relaxed); + } + return true; + }; + + auto submit_batch = [&]() { + if (candidate_ids.empty()) { + return true; + } + if (is_timed_out()) { + return false; + } + request.distance_batch_func_( + candidate_labels.data(), candidate_labels.size(), scores.data()); + for (uint64_t i = 0; i < candidate_ids.size(); ++i) { + CHECK_ARGUMENT(std::isfinite(scores[i]), + "custom query distance callback must return finite scores"); + if (reasoning_ctx != nullptr) { + reasoning_ctx->RecordVisit(candidate_ids[i] / buckets_per_data_, scores[i], 0); + } + search_result->Push(scores[i], candidate_ids[i]); + while (search_result->Size() > static_cast(topk)) { + if (reasoning_ctx != nullptr) { + reasoning_ctx->RecordEviction(search_result->Top().second / buckets_per_data_, + 0); + } + search_result->Pop(); + } + } + candidate_ids.clear(); + candidate_labels.clear(); + return true; + }; + + bool timed_out = false; + for (const auto bucket_id : candidate_buckets) { + if (is_timed_out()) { + timed_out = true; + break; + } + if (bucket_id == INVALID_BUCKET_ID) { + continue; + } + if (not param.executors.empty()) { + param.executors[0]->Clear(); + attr_filter = param.executors[0]->Run(bucket_id); + } + const auto bucket_size = bucket_->GetBucketSize(bucket_id); + const auto* ids = bucket_->GetInnerIds(bucket_id); + for (InnerIdType offset = 0; offset < bucket_size; ++offset) { + const auto inner_id = ids[offset]; + if (inner_id == std::numeric_limits::max()) { + continue; + } + const auto origin_id = inner_id / buckets_per_data_; + if (attr_filter != nullptr and not attr_filter->CheckValid(offset)) { + if (reasoning_ctx != nullptr) { + reasoning_ctx->RecordFilterReject(origin_id); + } + continue; + } + if (filter != nullptr and not filter->CheckValid(origin_id)) { + if (reasoning_ctx != nullptr) { + reasoning_ctx->RecordFilterReject(origin_id); + } + continue; + } + candidate_ids.push_back(inner_id); + candidate_labels.push_back(label_table_->GetLabelById(origin_id)); + if (candidate_ids.size() == batch_capacity and not submit_batch()) { + timed_out = true; + break; + } + } + if (timed_out) { + break; + } + } + if (not timed_out) { + submit_batch(); + } + + if (buckets_per_data_ == 1) { + return search_result; + } + + std::unordered_map id_to_min_score; + while (not search_result->Empty()) { + const auto& [score, inner_id] = search_result->Top(); + const auto origin_id = inner_id / buckets_per_data_; + auto iter = id_to_min_score.find(origin_id); + if (iter == id_to_min_score.end() or score < iter->second) { + id_to_min_score[origin_id] = score; + } + search_result->Pop(); + } + + for (const auto& [origin_id, score] : id_to_min_score) { + search_result->Push(score, origin_id); + if (search_result->Size() > static_cast(origin_topk)) { + search_result->Pop(); + } + } + return search_result; +} + void IVF::merge_one_unit(const MergeUnit& unit) { check_merge_illegal(unit); @@ -1771,8 +1918,28 @@ IVF::SearchWithRequest(const SearchRequest& request) const { bool is_range = (request.mode_ == SearchMode::RANGE_SEARCH); auto param = this->create_search_param(request.params_str_, request.filter_); + const bool use_custom_distance = request.distance_batch_func_ != nullptr; + if (use_custom_distance) { + CHECK_ARGUMENT(request.distance_batch_size_ > 0, + "custom query distance batch size must be greater than 0"); + CHECK_ARGUMENT(not is_range, "IVF custom query distance only supports KNN search"); + CHECK_ARGUMENT(not param.disable_bucket_scan, + "IVF custom query distance does not support disable_bucket_scan"); + CHECK_ARGUMENT(request.topk_ > 0, "topk must be greater than 0"); + CHECK_ARGUMENT(param.parallel_search_thread_count == 1, + "IVF custom query distance does not support parallel search"); + param.enable_reorder = false; + } auto query = request.query_; + if (use_custom_distance) { + CHECK_ARGUMENT(query != nullptr, "query dataset cannot be null"); + CHECK_ARGUMENT(query->GetNumElements() == 1, + "IVF custom search requires exactly one query"); + CHECK_ARGUMENT(query->GetFloat32Vectors() != nullptr, + "query float32 vectors cannot be null"); + CHECK_ARGUMENT(query->GetDim() == this->dim_, "query dimension must match index dimension"); + } if (param.disable_bucket_scan) { CHECK_ARGUMENT(query != nullptr, "query dataset cannot be null"); CHECK_ARGUMENT(query->GetNumElements() >= 1, @@ -1795,6 +1962,19 @@ IVF::SearchWithRequest(const SearchRequest& request) const { param.executors.emplace_back(executor); } } + if (use_custom_distance) { + param.search_mode = KNN_SEARCH; + param.topk = request.topk_; + auto search_result = search_with_custom_distance(query, request, param, ctx); + if (search_result == nullptr || search_result->Empty()) { + auto dataset_results = DatasetImpl::MakeEmptyDataset(); + dataset_results->Statistics(stats.Dump()); + return dataset_results; + } + auto dataset_results = this->pack_knn_result(search_result, ctx.alloc); + dataset_results->Statistics(stats.Dump()); + return dataset_results; + } std::shared_ptr reasoning_ctx; if (not request.expected_labels_.empty()) { reasoning_ctx = std::make_shared(this->allocator_); diff --git a/src/algorithm/ivf/ivf.h b/src/algorithm/ivf/ivf.h index 72db328151..7b31b79d2b 100644 --- a/src/algorithm/ivf/ivf.h +++ b/src/algorithm/ivf/ivf.h @@ -207,6 +207,13 @@ class IVF : public InnerIndexInterface { QueryContext& ctx, ReasoningContext* reasoning_ctx = nullptr) const; + DistHeapPtr + search_with_custom_distance(const DatasetPtr& query, + const SearchRequest& request, + const InnerSearchParam& param, + QueryContext& ctx, + ReasoningContext* reasoning_ctx = nullptr) const; + /** * @brief Re-score the top candidates in @p input with high-precision * codes and return the final result Dataset. diff --git a/src/impl/inner_search_param.h b/src/impl/inner_search_param.h index 8ea012c96e..ecb9ff3b26 100644 --- a/src/impl/inner_search_param.h +++ b/src/impl/inner_search_param.h @@ -22,6 +22,7 @@ #include "utils/filter_search_skip_strategy.h" #include "utils/pointer_define.h" #include "utils/timer.h" +#include "vsag/search_request.h" namespace vsag { @@ -50,6 +51,8 @@ class InnerSearchParam { int range_search_limit_size{-1}; int64_t parallel_search_thread_count{1}; bool enable_rabitq_one_bit_search{false}; + SearchDistanceBatchFunc distance_batch_func{nullptr}; + uint64_t distance_batch_size{1}; // for ivf int scan_bucket_size{1}; diff --git a/src/impl/searcher/basic_searcher.cpp b/src/impl/searcher/basic_searcher.cpp index f2d19d5914..7410d109f2 100644 --- a/src/impl/searcher/basic_searcher.cpp +++ b/src/impl/searcher/basic_searcher.cpp @@ -17,6 +17,7 @@ #include #include +#include #include #include @@ -365,11 +366,15 @@ BasicSearcher::search_impl(const GraphInterfacePtr& graph, auto top_candidates = std::make_shared>(alloc, -1); auto candidate_set = std::make_shared>(alloc, -1); - if (not graph or not flatten) { + const bool use_custom_distance = inner_search_param.distance_batch_func != nullptr; + if (not graph or (not flatten and not use_custom_distance)) { return top_candidates; } - auto computer = preset_computer != nullptr ? preset_computer : flatten->FactoryComputer(query); + ComputerInterfacePtr computer = nullptr; + if (not use_custom_distance) { + computer = preset_computer != nullptr ? preset_computer : flatten->FactoryComputer(query); + } auto is_id_allowed = inner_search_param.is_inner_id_allowed; auto ep = inner_search_param.ep; @@ -385,6 +390,13 @@ BasicSearcher::search_impl(const GraphInterfacePtr& graph, Vector neighbors(graph->MaximumDegree(), alloc); Vector line_dists(graph->MaximumDegree(), alloc); Vector lower_bound_dists(graph->MaximumDegree(), alloc); + const uint64_t custom_batch_capacity = + use_custom_distance + ? std::max(1, + std::min(inner_search_param.distance_batch_size, + graph->MaximumDegree())) + : 0; + Vector custom_labels(custom_batch_capacity, alloc); auto skip_strategy = create_filter_search_skip_strategy( inner_search_param.skip_strategy_type, inner_search_param.is_inner_id_allowed != nullptr @@ -407,7 +419,77 @@ BasicSearcher::search_impl(const GraphInterfacePtr& graph, }; auto* reasoning = ctx == nullptr ? nullptr : ctx->reasoning_ctx; - if (inner_search_param.enable_rabitq_one_bit_search) { + auto score_ids = [&](const InnerIdType* ids, uint64_t count, float* scores) { + if (not use_custom_distance) { + flatten->Query(scores, computer, ids, count, ctx); + return; + } + CHECK_ARGUMENT(label_table != nullptr, "custom distance requires a label table"); + CHECK_ARGUMENT(inner_search_param.distance_batch_size > 0, + "distance_batch_size must be greater than 0"); + for (uint64_t offset = 0; offset < count; + offset += inner_search_param.distance_batch_size) { + const uint64_t batch_count = + std::min(inner_search_param.distance_batch_size, count - offset); + for (uint64_t i = 0; i < batch_count; ++i) { + custom_labels[i] = label_table->GetLabelById(ids[offset + i]); + } + inner_search_param.distance_batch_func( + custom_labels.data(), batch_count, scores + offset); + for (uint64_t i = 0; i < batch_count; ++i) { + CHECK_ARGUMENT(std::isfinite(scores[offset + i]), + "distance callback must return finite scores"); + } + } + }; + auto score_duplicates = [&](const auto& duplicate_ids, uint32_t duplicate_hops) { + if (not use_custom_distance) { + return; + } + + uint64_t duplicate_count = 0; + auto submit_duplicates = [&]() { + if (duplicate_count == 0) { + return; + } + score_ids(neighbors.data(), duplicate_count, lower_bound_dists.data()); + dist_cmp += duplicate_count; + for (uint64_t i = 0; i < duplicate_count; ++i) { + if (reasoning != nullptr) { + reasoning->RecordVisit(neighbors[i], lower_bound_dists[i], duplicate_hops); + } + top_candidates->Push(lower_bound_dists[i], neighbors[i]); + } + duplicate_count = 0; + }; + + for (const auto& item : duplicate_ids) { + if (not check_func(item)) { + continue; + } + neighbors[duplicate_count++] = item; + if (duplicate_count == neighbors.size()) { + submit_duplicates(); + } + } + submit_duplicates(); + + if constexpr (mode == KNN_SEARCH) { + while (top_candidates->Size() > ef) { + if (reasoning != nullptr) { + reasoning->RecordEviction(top_candidates->Top().second, duplicate_hops); + } + top_candidates->Pop(); + } + } + if (not top_candidates->Empty()) { + lower_bound = top_candidates->Top().first; + } + }; + + if (use_custom_distance) { + score_ids(&ep, 1, &dist); + } else if (inner_search_param.enable_rabitq_one_bit_search) { flatten->QueryWithDistanceLowerBound(&dist, nullptr, computer, &ep, 1, ctx); } else { flatten->Query(&dist, computer, &ep, 1, ctx); @@ -422,6 +504,10 @@ BasicSearcher::search_impl(const GraphInterfacePtr& graph, top_candidates->Pop(); } } + if (use_custom_distance and inner_search_param.consider_duplicate) { + const auto duplicate_ids = graph->GetDuplicateIds(ep); + score_duplicates(duplicate_ids, hops); + } candidate_set->Push(-dist, ep); vl->Set(ep); @@ -472,8 +558,10 @@ BasicSearcher::search_impl(const GraphInterfacePtr& graph, neighbors); bool collect_rabitq_lower_bound = false; - if (inner_search_param.enable_rabitq_one_bit_search and top_candidates->Size() == ef and - rabitq_lower_bound_candidates != nullptr) { + if (use_custom_distance) { + score_ids(to_be_visited_id.data(), count_no_visited, line_dists.data()); + } else if (inner_search_param.enable_rabitq_one_bit_search and + top_candidates->Size() == ef and rabitq_lower_bound_candidates != nullptr) { collect_rabitq_lower_bound = true; flatten->QueryWithDistanceLowerBound(line_dists.data(), lower_bound_dists.data(), @@ -500,6 +588,10 @@ BasicSearcher::search_impl(const GraphInterfacePtr& graph, if (reasoning != nullptr) { reasoning->RecordVisit(cur_id, dist, hops); } + if (use_custom_distance and inner_search_param.consider_duplicate) { + const auto duplicate_ids = graph->GetDuplicateIds(cur_id); + score_duplicates(duplicate_ids, hops); + } if constexpr (mode == KNN_SEARCH) { if (collect_rabitq_lower_bound and lower_bound_dists[i] < lower_bound and check_func(cur_id)) { @@ -515,7 +607,7 @@ BasicSearcher::search_impl(const GraphInterfacePtr& graph, } else if (reasoning != nullptr) { reasoning->RecordFilterReject(cur_id); } - if (inner_search_param.consider_duplicate) { + if (inner_search_param.consider_duplicate and not use_custom_distance) { const auto duplicate_ids = graph->GetDuplicateIds(cur_id); for (const auto& item : duplicate_ids) { if (check_func(item)) { @@ -557,7 +649,8 @@ BasicSearcher::search_impl(const GraphInterfacePtr& graph, } // set duplicate id for query vector - if (inner_search_param.find_duplicate and not top_candidates->Empty()) { + if (not use_custom_distance and inner_search_param.find_duplicate and + not top_candidates->Empty()) { const auto* data = top_candidates->GetData(); auto min_distance = data[0].first; auto min_index = data[0].second; diff --git a/tests/test_brute_force.cpp b/tests/test_brute_force.cpp index b940ed9a72..75e2e437b5 100644 --- a/tests/test_brute_force.cpp +++ b/tests/test_brute_force.cpp @@ -13,6 +13,7 @@ // See the License for the specific language governing permissions and // limitations under the License. +#include #include #include #include @@ -1535,3 +1536,95 @@ TEST_CASE("(PR) BruteForce Reasoning No Output When Disabled", "[ft][bruteforce] REQUIRE(result.value()->GetIds()[i] == knn_result.value()->GetIds()[i]); } } + +TEST_CASE("(PR) BruteForce Custom Batch Distance", "[ft][bruteforce][custom_distance][pr]") { + using namespace fixtures; + + auto param = BruteForceTestIndex::GenerateBruteForceBuildParametersString("l2", 16, "fp32"); + auto index = TestIndex::TestFactory(BruteForceTestIndex::name, param, true); + auto dataset = BruteForceTestIndex::pool.GetDatasetAndCreate(16, 256, "l2"); + TestIndex::TestBuildIndex(index, dataset, true); + + std::vector scored_ids; + uint64_t max_batch_size = 0; + vsag::SearchRequest request; + request.topk_ = 3; + request.distance_batch_size_ = 3; + request.enable_filter_ = true; + request.filter_ = std::make_shared(); + request.distance_batch_func_ = [&](const int64_t* ids, uint64_t count, float* distances) { + max_batch_size = std::max(max_batch_size, count); + for (uint64_t i = 0; i < count; ++i) { + scored_ids.push_back(ids[i]); + distances[i] = static_cast(ids[i]); + } + }; + + auto result = index->SearchWithRequest(request); + REQUIRE(result.has_value()); + REQUIRE(max_batch_size == 3); + REQUIRE(result.value()->GetDim() == 3); + for (const auto id : scored_ids) { + REQUIRE(id % 2 == 0); + } + + std::vector expected_ids; + for (int64_t i = 0; i < dataset->base_->GetNumElements(); ++i) { + const auto id = dataset->base_->GetIds()[i]; + if (id % 2 == 0) { + expected_ids.push_back(id); + } + } + std::sort(expected_ids.begin(), expected_ids.end()); + for (int64_t i = 0; i < result.value()->GetDim(); ++i) { + REQUIRE(result.value()->GetIds()[i] == expected_ids[i]); + REQUIRE(result.value()->GetDistances()[i] == static_cast(expected_ids[i])); + } +} + +TEST_CASE("(PR) BruteForce Custom Batch Distance Validation", + "[ft][bruteforce][custom_distance][pr]") { + using namespace fixtures; + + auto param = BruteForceTestIndex::GenerateBruteForceBuildParametersString("l2", 16, "fp32"); + auto index = TestIndex::TestFactory(BruteForceTestIndex::name, param, true); + auto dataset = BruteForceTestIndex::pool.GetDatasetAndCreate(16, 32, "l2"); + TestIndex::TestBuildIndex(index, dataset, true); + + vsag::SearchRequest request; + request.topk_ = 1; + request.distance_batch_func_ = [](const int64_t*, uint64_t, float* distances) { + distances[0] = std::numeric_limits::quiet_NaN(); + }; + + request.distance_batch_size_ = 0; + auto invalid_batch_size = index->SearchWithRequest(request); + REQUIRE_FALSE(invalid_batch_size.has_value()); + REQUIRE(invalid_batch_size.error().type == vsag::ErrorType::INVALID_ARGUMENT); + + request.distance_batch_size_ = 1; + auto non_finite_score = index->SearchWithRequest(request); + REQUIRE_FALSE(non_finite_score.has_value()); + REQUIRE(non_finite_score.error().type == vsag::ErrorType::INVALID_ARGUMENT); + + request.distance_batch_func_ = [](const int64_t*, uint64_t count, float* distances) { + std::fill(distances, distances + count, 0.0F); + }; + request.topk_ = 0; + auto invalid_topk = index->SearchWithRequest(request); + REQUIRE_FALSE(invalid_topk.has_value()); + REQUIRE(invalid_topk.error().type == vsag::ErrorType::INVALID_ARGUMENT); + + request.mode_ = vsag::SearchMode::RANGE_SEARCH; + request.radius_ = -1.0F; + request.limited_size_ = -1; + auto invalid_radius = index->SearchWithRequest(request); + REQUIRE_FALSE(invalid_radius.has_value()); + REQUIRE(invalid_radius.error().type == vsag::ErrorType::INVALID_ARGUMENT); + + request.radius_ = 1.0F; + request.limited_size_ = 0; + auto invalid_limit = index->SearchWithRequest(request); + REQUIRE_FALSE(invalid_limit.has_value()); + REQUIRE(invalid_limit.error().type == vsag::ErrorType::INVALID_ARGUMENT); +} diff --git a/tests/test_hgraph.cpp b/tests/test_hgraph.cpp index bf07cc2fff..e43719c3ec 100644 --- a/tests/test_hgraph.cpp +++ b/tests/test_hgraph.cpp @@ -1292,6 +1292,70 @@ TEST_CASE("(PR) HGraph Reasoning Zero Overhead When Disabled", "[ft][hgraph][rea REQUIRE(ratios[measure_samples / 2] < 1.5); } +TEST_CASE("(PR) HGraph Custom Batch Distance", "[ft][hgraph][custom_distance][pr]") { + using namespace fixtures; + + HGraphTestIndex::HGraphBuildParam build_param("l2", 16, "fp32"); + auto param = HGraphTestIndex::GenerateHGraphBuildParametersString(build_param); + auto index = TestIndex::TestFactory(HGraphTestIndex::name, param, true); + auto dataset = HGraphTestIndex::pool.GetDatasetAndCreate(16, 256, "l2"); + TestIndex::TestBuildIndex(index, dataset, true); + + uint64_t max_batch_size = 0; + vsag::SearchRequest request; + request.topk_ = 5; + request.params_str_ = fmt::format(fixtures::search_param_tmp, 256, false); + request.distance_batch_size_ = 2; + request.distance_batch_func_ = [&](const int64_t* ids, uint64_t count, float* distances) { + max_batch_size = std::max(max_batch_size, count); + for (uint64_t i = 0; i < count; ++i) { + distances[i] = static_cast(ids[i]); + } + }; + + auto result = index->SearchWithRequest(request); + REQUIRE(result.has_value()); + REQUIRE(result.value()->GetDim() > 0); + REQUIRE(max_batch_size <= request.distance_batch_size_); + for (int64_t i = 0; i < result.value()->GetDim(); ++i) { + REQUIRE(result.value()->GetDistances()[i] == + static_cast(result.value()->GetIds()[i])); + } + + request.distance_batch_size_ = 0; + auto invalid_batch_size = index->SearchWithRequest(request); + REQUIRE_FALSE(invalid_batch_size.has_value()); + REQUIRE(invalid_batch_size.error().type == vsag::ErrorType::INVALID_ARGUMENT); + + request.distance_batch_size_ = 1; + request.distance_batch_func_ = [](const int64_t*, uint64_t, float* distances) { + distances[0] = std::numeric_limits::quiet_NaN(); + }; + auto non_finite_score = index->SearchWithRequest(request); + REQUIRE_FALSE(non_finite_score.has_value()); + REQUIRE(non_finite_score.error().type == vsag::ErrorType::INVALID_ARGUMENT); + + request.distance_batch_func_ = [](const int64_t*, uint64_t count, float* distances) { + std::fill(distances, distances + count, 0.0F); + }; + request.mode_ = vsag::SearchMode::RANGE_SEARCH; + auto unsupported_range = index->SearchWithRequest(request); + REQUIRE_FALSE(unsupported_range.has_value()); + REQUIRE(unsupported_range.error().type == vsag::ErrorType::INVALID_ARGUMENT); + + request.mode_ = vsag::SearchMode::KNN_SEARCH; + request.topk_ = 5; + request.params_str_ = R"({"hgraph":{"ef_search":256,"parallelism":2}})"; + auto unsupported_parallel = index->SearchWithRequest(request); + REQUIRE_FALSE(unsupported_parallel.has_value()); + REQUIRE(unsupported_parallel.error().type == vsag::ErrorType::INVALID_ARGUMENT); + + request.params_str_ = R"({"hgraph":{"ef_search":256,"brute_force_threshold":0.1}})"; + auto unsupported_brute_force = index->SearchWithRequest(request); + REQUIRE_FALSE(unsupported_brute_force.has_value()); + REQUIRE(unsupported_brute_force.error().type == vsag::ErrorType::INVALID_ARGUMENT); +} + static void TestHGraphGetRawVector(const fixtures::HGraphTestIndexPtr& test_index, const fixtures::HGraphResourcePtr& resource) { diff --git a/tests/test_ivf.cpp b/tests/test_ivf.cpp index f6bc867aca..d33bc86d5b 100644 --- a/tests/test_ivf.cpp +++ b/tests/test_ivf.cpp @@ -1730,6 +1730,70 @@ TEST_CASE_PERSISTENT_FIXTURE(fixtures::IVFTestIndex, } } +TEST_CASE_PERSISTENT_FIXTURE(fixtures::IVFTestIndex, + "IVF Custom Batch Distance", + "[ft][search][ivf][pr]") { + constexpr int64_t dim = 16; + constexpr int64_t count = 200; + constexpr uint64_t batch_size = 7; + auto dataset = IVFTestIndex::pool.GetDatasetAndCreate(dim, count, "l2"); + auto param = IVFTestIndex::GenerateIVFBuildParametersString("l2", dim, "fp32", 4); + auto index = TestIndex::TestFactory(IVFTestIndex::name, param, true); + TestIndex::TestBuildIndex(index, dataset, true); + + uint64_t largest_batch = 0; + auto query = vsag::Dataset::Make(); + query->NumElements(1) + ->Dim(dim) + ->Float32Vectors(dataset->base_->GetFloat32Vectors()) + ->Owner(false); + vsag::SearchRequest request; + request.query_ = query; + request.topk_ = 3; + request.params_str_ = R"({"ivf":{"scan_buckets_count":4}})"; + request.distance_batch_size_ = batch_size; + request.distance_batch_func_ = [&largest_batch]( + const int64_t* ids, uint64_t size, float* distances) { + largest_batch = std::max(largest_batch, size); + for (uint64_t i = 0; i < size; ++i) { + distances[i] = static_cast(ids[i]); + } + }; + + auto result = index->SearchWithRequest(request); + REQUIRE(result.has_value()); + REQUIRE(result.value()->GetNumElements() == 1); + REQUIRE(result.value()->GetDim() == request.topk_); + REQUIRE(largest_batch > 0); + REQUIRE(largest_batch <= batch_size); + for (int64_t i = 0; i < result.value()->GetDim(); ++i) { + REQUIRE(result.value()->GetDistances()[i] == + static_cast(result.value()->GetIds()[i])); + } + + request.mode_ = vsag::SearchMode::RANGE_SEARCH; + auto range_result = index->SearchWithRequest(request); + REQUIRE_FALSE(range_result.has_value()); + REQUIRE(range_result.error().type == vsag::ErrorType::INVALID_ARGUMENT); + + request.mode_ = vsag::SearchMode::KNN_SEARCH; + request.distance_batch_size_ = 0; + auto invalid_batch_result = index->SearchWithRequest(request); + REQUIRE_FALSE(invalid_batch_result.has_value()); + REQUIRE(invalid_batch_result.error().type == vsag::ErrorType::INVALID_ARGUMENT); + + request.distance_batch_size_ = batch_size; + request.params_str_ = R"({"ivf":{"scan_buckets_count":4,"disable_bucket_scan":true}})"; + auto route_only_result = index->SearchWithRequest(request); + REQUIRE_FALSE(route_only_result.has_value()); + REQUIRE(route_only_result.error().type == vsag::ErrorType::INVALID_ARGUMENT); + + request.params_str_ = R"({"ivf":{"scan_buckets_count":4,"parallelism":2}})"; + auto parallel_result = index->SearchWithRequest(request); + REQUIRE_FALSE(parallel_result.has_value()); + REQUIRE(parallel_result.error().type == vsag::ErrorType::INVALID_ARGUMENT); +} + // RejectAllFilter for testing empty results class RejectAllFilter : public vsag::Filter { public: