What is the bug?
On the Analytics Engine (unified query) path, PPL subsearches run unbounded, while the default (non-AE) pipeline caps them. Unlike the other settings-fidelity gaps in this area, this one diverges at defaults — no operator configuration is required to hit it.
| Setting |
Cluster default |
AE path value |
plugins.ppl.subsearch.maxout |
10000 |
0 (unlimited) |
plugins.ppl.join.subsearch_maxout |
50000 |
0 (unlimited) |
Verified on a default AE context:
SysLimit[querySizeLimit=10000, subsearchLimit=0, joinSubsearchLimit=0]
SysLimit.fromSettings reads both keys, and 0 means "no LogicalSystemLimit injected", so nothing bounds the subsearch.
Root cause — and why this one is a design question, not a plain bug
This is deliberate, and documented in UnifiedQueryContext.Builder:
Settings.Key#PPL_JOIN_SUBSEARCH_MAXOUT defaults to 0 to avoid injecting LogicalSystemLimit into the logical plan, which is an OpenSearch-specific operational concern irrelevant to external consumers of the unified query API. Settings.Key#PPL_SUBSEARCH_MAXOUT is set to 0 for the same reason.
That rationale holds for external consumers of the unified query API as a library (e.g. planning PPL for a non-OpenSearch execution target), where an OpenSearch operational cap is indeed meaningless.
It does not obviously hold for the in-cluster REST path, where the query is being served by an OpenSearch node, consumes that node's memory, and the operator has configured a cap expecting it to apply. Two queries against the same cluster get different memory-safety guarantees purely based on whether the target index happens to use the composite data format.
What is the expected behavior?
Needs a maintainer decision between:
Option A — forward from the REST handler. Add both keys to RestUnifiedQueryAction.FORWARDED_CLUSTER_SETTINGS. This preserves the library default of "unlimited" for external API consumers (who never go through the REST handler) while restoring cluster fidelity in-cluster. Minimal change, and it keeps the documented rationale intact for the case it was written for.
Caveat: this is a behavior change at defaults — AE subsearches would go from unbounded to capped at 10000 / 50000. Queries relying on the current unbounded behavior would start hitting the cap. Needs a release note.
Option B — keep as-is. Accept that AE subsearches are unbounded by design, and document the divergence explicitly so operators know the cap does not apply to composite indices.
Do you have any additional context?
Found while fixing the same class of defect in #5611 (settings the AE path silently ignored). That PR forwards plugins.query.size_limit, plugins.ppl.pattern.*, and plugins.ppl.values.max.limit, and explicitly excludes these two keys pending this decision — the exclusion is pinned by a test (everySeededPlanningSettingIsClassified) so it stays a conscious choice rather than drift.
Related: #5734 (plugins.calcite.all_join_types.allowed guardrail inactive on the AE path).
What is the bug?
On the Analytics Engine (unified query) path, PPL subsearches run unbounded, while the default (non-AE) pipeline caps them. Unlike the other settings-fidelity gaps in this area, this one diverges at defaults — no operator configuration is required to hit it.
plugins.ppl.subsearch.maxout100000(unlimited)plugins.ppl.join.subsearch_maxout500000(unlimited)Verified on a default AE context:
SysLimit.fromSettingsreads both keys, and0means "noLogicalSystemLimitinjected", so nothing bounds the subsearch.Root cause — and why this one is a design question, not a plain bug
This is deliberate, and documented in
UnifiedQueryContext.Builder:That rationale holds for external consumers of the unified query API as a library (e.g. planning PPL for a non-OpenSearch execution target), where an OpenSearch operational cap is indeed meaningless.
It does not obviously hold for the in-cluster REST path, where the query is being served by an OpenSearch node, consumes that node's memory, and the operator has configured a cap expecting it to apply. Two queries against the same cluster get different memory-safety guarantees purely based on whether the target index happens to use the composite data format.
What is the expected behavior?
Needs a maintainer decision between:
Option A — forward from the REST handler. Add both keys to
RestUnifiedQueryAction.FORWARDED_CLUSTER_SETTINGS. This preserves the library default of "unlimited" for external API consumers (who never go through the REST handler) while restoring cluster fidelity in-cluster. Minimal change, and it keeps the documented rationale intact for the case it was written for.Caveat: this is a behavior change at defaults — AE subsearches would go from unbounded to capped at
10000/50000. Queries relying on the current unbounded behavior would start hitting the cap. Needs a release note.Option B — keep as-is. Accept that AE subsearches are unbounded by design, and document the divergence explicitly so operators know the cap does not apply to composite indices.
Do you have any additional context?
Found while fixing the same class of defect in #5611 (settings the AE path silently ignored). That PR forwards
plugins.query.size_limit,plugins.ppl.pattern.*, andplugins.ppl.values.max.limit, and explicitly excludes these two keys pending this decision — the exclusion is pinned by a test (everySeededPlanningSettingIsClassified) so it stays a conscious choice rather than drift.Related: #5734 (
plugins.calcite.all_join_types.allowedguardrail inactive on the AE path).