[DOC] Document the Terms Enum API (/{index}/_terms_enum) #13462

What do you want to do?

• Request a change to existing documentation
• Add new documentation
• Report a technical problem with the documentation
• Other

Tell us about your request.

OpenSearch 3.10 adds an experimental Terms Enum API, GET and POST /{index}/_terms_enum. It returns up to size distinct indexed terms of one keyword, constant_keyword or _index field that start with a prefix, merged across shards in UTF-8 byte order. Each shard reads its term dictionary instead of running an aggregation, so the cost depends on size rather than on how many documents match. The exception is a shard with deleted documents (with soft deletes, these include earlier versions of updated documents) or documents hidden from the caller, for example by document-level security: there, each candidate term's postings are read until a visible document is found, and that extra work is bounded by timeout. The intended use is autocomplete and filter-value suggestions, which OpenSearch Dashboards builds today with a terms aggregation bounded by terminate_after.

Must appear on the page

• The API is experimental and is not behind a feature flag.
• A response with complete: false can be missing terms anywhere in its range, not only at the end. Do not page with search_after from it.
• index_filter only skips whole shards. On a shard it does not skip, terms come from all visible documents, not only from documents that match the filter.
• The read action group grants the API. The search action group does not.
• The first request to a search-idle shard that has pending writes waits for that shard's refresh, and the wait counts against timeout. At the default 1s, that request can return complete: false, possibly with no terms.
• Cross-cluster requests are not supported, and an index reached only through filtered aliases is rejected.
• search.allow_expensive_queries: false does not block it, even though it blocks a prefix query on a keyword field.

Proposed changes

1. Add _api-reference/search-apis/terms-enum.md (title "Terms enum", parent "Search APIs", nav_order: 86, between Search shards (85) and Validate query (87)). It is then served at https://docs.opensearch.org/latest/api-reference/search-apis/terms-enum/, the URL the core REST spec already links to. Use the "Introduced 3.10" label and the standard experimental {: .warning} that links [Feature Request] Add a _terms_enum API for fast prefix-based term suggestions on a field OpenSearch#23510.
2. In _api-reference/search-apis/index.md, under "Core search APIs", add:

   - **[Terms enum]({{site.url}}{{site.baseurl}}/api-reference/search-apis/terms-enum/)**: List the distinct indexed terms of a field that start with a prefix, for autocomplete and filter suggestions.
   

3. In _install-and-configure/configuring-opensearch/search-settings.md, add:

   - `search.terms_enum.max_size` (Dynamic, integer): The largest `size` that a Terms enum API request can ask for. Minimum is `1`. Default is `1000`.
   

4. In the "Index permissions" table of _security/access-control/permissions.md, add this row between indices:data/read/search/template and indices:data/read/tv, in alphabetical order:

   | `indices:data/read/terms_enum` | Permission to run the [Terms enum API]({{site.url}}{{site.baseurl}}/api-reference/search-apis/terms-enum/). |
   

   _security/access-control/default-action-groups.md needs no change: the read group's indices:data/read* already covers the action.
5. In _search-plugins/searching-data/autocomplete.md, after the list of methods, add a pointer such as: "To suggest values that already exist in a keyword field, such as filter values, use the Terms enum API."

Once the API specification PR is merged, the endpoint, path parameter and query parameter sections can use spec_insert. No page on main mentions this API yet.

Endpoints

GET  /{index}/_terms_enum
POST /{index}/_terms_enum

Path and query parameters

The common parameters also apply, including source with source_content_type for clients that cannot send a body with GET. ignore_throttled is also accepted, with a default of false (search defaults it to true). There is no timeout query parameter. The time limit is set in the body.

Request body fields

Supported fields

• keyword (indexed), constant_keyword and _index, each linked to its page (_mappings/supported-field-types/keyword.md, constant-keyword.md, _mappings/metadata-fields/index-metadata.md). Field aliases and multi-fields enumerate the concrete field. On a data stream, _index returns backing index names.
• Other field types, keyword fields with index: false and derived fields are unsupported (see "Unsupported fields, missing fields and failed shards").
• Only indexed terms are returned, so values longer than ignore_above never appear.
• Terms held only by deleted documents (including earlier versions of updated documents) are not returned.
• Terms are in UTF-8 byte order, so ASCII uppercase sorts before lowercase, and terms that differ only in case are distinct. No document counts or frequencies are returned.

Example

Setup, from the YAML REST tests. The index names can change; if they do, update the _index example (its string prefix and its result) to match. Keep the values so the tested results stay true:

PUT /test
{
  "settings": { "number_of_shards": 1, "number_of_replicas": 0 },
  "mappings": { "properties": { "tag": { "type": "keyword" } } }
}

PUT /test-2
{
  "settings": { "number_of_shards": 1, "number_of_replicas": 0 },
  "mappings": { "properties": { "tag": { "type": "keyword" } } }
}

POST /_bulk?refresh=true
{"index": {"_index": "test", "_id": "1"}}
{"tag": "web-01"}
{"index": {"_index": "test", "_id": "2"}}
{"tag": "web-02"}
{"index": {"_index": "test", "_id": "3"}}
{"tag": "api-01"}
{"index": {"_index": "test", "_id": "4"}}
{"tag": "Web-03"}
{"index": {"_index": "test-2", "_id": "1"}}
{"tag": "web-02"}
{"index": {"_index": "test-2", "_id": "2"}}
{"tag": "web-04"}

POST /test,test-2/_terms_enum
{
  "field": "tag",
  "string": "web-"
}

{
  "_shards": { "total": 2, "successful": 2, "skipped": 0, "failed": 0 },
  "terms": ["web-01", "web-02", "web-04"],
  "complete": true
}

More results from the same tests (on index test unless noted):

• {"field": "tag", "size": 2} returns ["Web-03", "api-01"] (byte order).
• {"field": "tag", "string": "WEB-"} returns []. With "case_insensitive": true it returns ["Web-03", "web-01", "web-02"].
• {"field": "tag", "string": "web-", "search_after": "web-01"} returns ["web-02"]. With "search_after": "a" (before the prefix) it returns ["web-01", "web-02"].
• On test,test-2, {"field": "_index", "string": "test"} returns ["test", "test-2"].
• index_filter, on two other one-shard indexes, each mapped with tag as keyword and @timestamp as date: old (tags old-a and old-b, 2015 @timestamp values) and new (tags new-a and new-b, 2026 values). {"field": "tag", "index_filter": {"range": {"@timestamp": {"gte": "2026-01-01"}}}} on old,new returns ["new-a", "new-b"] with "_shards": {"total": 2, "successful": 2, "skipped": 1, "failed": 0}.

Response body fields

Paging and complete

• complete is false when any target shard timed out, failed or could not be reached. Such a page can be missing terms anywhere in its range, not only at the end. Do not use it for search_after paging.
• complete: true only means every shard answered in full. It does not mean there are no more matching terms after size.
• To page, pass the last term as search_after. A page with complete: true and fewer than size terms is the last one.
• If some target indexes map the field with an unsupported type, every successful response has complete: false (unless index_filter skips those shards). If the field is absent from every other target index (or index_filter skips them), the request fails with a 400 instead. Narrow the target to indexes where the field is supported.

Timeouts and partial results

• timeout covers the whole request. Each shard gets the remaining time minus a small reserve, starting when it receives the request. A shard that reaches its limit returns the terms it has so far and counts as successful, so complete: false can come with failed: 0.
• A shard that has not answered by the deadline is reported in failures with status GATEWAY_TIMEOUT (reason: shard did not respond within [<timeout>]). A late answer is dropped.
• A failed shard request is retried on another copy of the shard while time remains, unless its shard task was cancelled.
• If no shard answers in time, the response is a 200 with no terms and complete: false.
• Search-idle shards: as in search, the first request to a search-idle shard with pending writes waits for its refresh, so recently indexed terms are returned. That wait counts against timeout. The first request after a quiet period can therefore return complete: false (possibly with no terms) within the default 1s. A retry or a longer timeout avoids it.
• The request is cancelled when the client disconnects and can be cancelled with the Tasks API.

A partial response looks like this:

{
  "_shards": {
    "total": 3,
    "successful": 2,
    "skipped": 0,
    "failed": 1,
    "failures": [
      {
        "shard": 0,
        "index": "logs-legacy",
        "status": "BAD_REQUEST",
        "reason": {
          "type": "illegal_argument_exception",
          "reason": "Field [tag] of type [text] does not support terms enumeration"
        }
      }
    ]
  },
  "terms": ["web-01", "web-02"],
  "complete": false
}

Unsupported fields, missing fields and failed shards

The coordinator checks these rules in order, and the first that matches decides the response:

Errors before any shard is queried (400 unless noted)

• A malformed request: an unknown query parameter or body field, or a timeout or index_filter that cannot be parsed.
• Missing field, size below 1 or over search.terms_enum.max_size, string over 1,000 UTF-16 code units, timeout below 1ms.
• More target shards than action.search.shard_count.limit.
• An index reached only through aliases that have a filter (terms would come from the whole index, not only from the documents the filter selects). If the same request also reaches that index by name, through _all or a wildcard that matches the index name, or through an alias without a filter, it is accepted and returns terms from the whole index, as in search.
• An index expression that targets a remote cluster: cross-cluster requests are not supported.
• Every target shard only on nodes older than 3.10.
• The usual index errors, for example a 404 index_not_found_exception for a missing index and a 403 cluster_block_exception for an index with index.blocks.read: true.

Indexes, aliases and routing

• Closed indexes are handled as in search: a closed target fails the request with a 400 index_closed_exception, or is skipped with ignore_unavailable=true. A wildcard skips closed indexes by default; with expand_wildcards=open,closed it fails with index_closed_exception.
• Data streams are supported.
• An alias with search routing targets the same shards as _search.
• Every target shard is queried at once. Unlike search, there is no limit on concurrent shard requests per node.

Security

• Action name: indices:data/read/terms_enum (shard action indices:data/read/terms_enum[s]). The read action group (indices:data/read*) covers it. The search action group (indices:data/read/search*) does not.
• Shards read terms only through the shard's searcher, so reader wrappers such as Lucene-level document-level security apply. A term held only by documents the wrapper hides is not returned.
• A field hidden by the node's field filter (the hook field-level security uses for _field_caps) is treated as absent, with no terms and no error, whatever its type. The same applies to an alias that points to it. Metadata fields such as _index are exempt, as in _field_caps.
• The core PR tests this with test plugins that stand in for the Security plugin. The Security plugin's privilege mapping, its document-level security modes and field masking (a masked keyword field is expected to return no terms, while a masked constant_keyword field still returns its mapped value) should be confirmed with its maintainers before the page goes further than these points.

Workload management and admission control

• Each shard request runs as a search shard task on the search thread pool, so search backpressure applies to the shard tasks. The coordinating task is not tracked by search backpressure. A request with a workloadGroupId header counts toward that workload group, and each shard request can be rejected when the group is over its rejection threshold, as for search. A rejected shard request is retried on another copy while time remains; if no copy accepts it, the shard counts as failed. Search backpressure and workload management can also cancel a running shard task; that shard counts as failed and is not retried.
• In 3.10 the following do not apply, and the page should say so:
  • the workload group check a search goes through on the coordinating node, so a rejection shows up per shard instead of rejecting the whole request;
  • workload group throttling (the throttling limits of a workload group, new in 3.10): a _terms_enum request does not take a throttle permit, so it is not limited or rejected by those limits;
  • workload group settings such as search.default_search_timeout, which apply to search requests only;
  • rule-based auto-tagging, which handles only search and scroll requests (send the workloadGroupId header instead);
  • search admission control (admission_control.search.*);
  • search.allow_expensive_queries: a _terms_enum request runs when this setting is false, even though a prefix query on a keyword field is rejected. Its work is bounded by size and timeout.
• Shard work for system indexes and search-throttled indexes also runs on the search thread pool.

Mixed-version clusters and clients

• Only shard copies on 3.10 or later nodes are queried. A shard with no such copy is reported as failed, so a 200 response has complete: false. If every target shard has copies, but only on nodes older than 3.10, the request fails with a 400. Send requests to a 3.10 node; older nodes do not have the endpoint.
• The deprecated Java high-level REST client has no method for it. Until the language clients add it from the API specification, send it as a generic request.

When to use it instead of a terms aggregation

Use _terms_enum to suggest values as a user types: it returns the first matching terms in byte order, it reads the term dictionary instead of sampling the first documents of each shard (so rare values are not missed), and its cost depends on size rather than on the number of matching documents (on shards with deleted or hidden documents it also reads postings until it finds a visible document, bounded by timeout). Use a terms aggregation when you need document counts, the most frequent values first, or values from only the documents that match a query (index_filter only skips whole shards). Results also differ in order: Dashboards today shows the top terms by count within the first 100,000 documents per shard, while _terms_enum shows the first terms in byte order.

Version

3.10 (main is 3.10.0-SNAPSHOT at the time of writing; adjust if the change ships in a later release).

What other resources are available?

• Core issue: [Feature Request] Add a _terms_enum API for fast prefix-based term suggestions on a field OpenSearch#23510
• Core PR: Add a terms_enum API for fast value suggestions OpenSearch#23539 (open when this was written; please check the final behavior against the merged code, the review may change details)
• The PR description lists the behavior above with the unit, integration and YAML tests that cover it. The YAML tests are in rest-api-spec/src/main/resources/rest-api-spec/test/terms_enum/.
• API specification PR: Add the terms_enum API to the spec opensearch-api-specification#1320 (adds terms_enum; the Search API pages can use spec_insert for the endpoint and parameters once it merges).

Background for the writer (not page content):

• The example setup and results come from 10_basic.yml through 40_index_filter.yml. For the first request, the test asserts terms, complete and _shards.failed; the other _shards values follow from the two one-shard indexes. The partial response under "Timeouts and partial results" is illustrative, not from a test.
• The core PR includes a single-node laptop benchmark (5M documents, 1M distinct values) in which _terms_enum answered in about 0.3 ms at p50 on a force-merged index, against about 20 ms for the terms aggregation Dashboards sends today; multiple nodes and concurrent load were not measured. Most of that gap comes from the execution_hint: map that Dashboards sets: a terms aggregation with the default execution hint and the same terminate_after: 100000 took about 0.6 ms (on the index with 10% of documents deleted: 0.38 ms for _terms_enum, 48 ms for the Dashboards request and 2.3 ms for that aggregation), but it missed suggestions for rare prefixes. Do not present these numbers as a general speedup over the terms aggregation.