Repository navigation
Commit 1a401a3
committed
Forward cluster planning settings to the Analytics Engine unified query path
The Analytics Engine (unified query) path silently ignored several cluster
settings, always planning against UnifiedQueryContext.Builder's hardcoded
seed value regardless of what the operator configured. The default
(non-AE) pipeline honored them correctly.
Root cause: RestUnifiedQueryAction.applyClusterOverrides() forwarded only a
hand-picked subset of cluster settings into the UnifiedQueryContext, while
the builder independently seeds a default settings map. Any planning
setting present in the seed map but absent from the forward list regressed
to its default -- a two-lists-drift defect.
Settings fixed (all verified to reach the plan context only with this
change):
plugins.query.size_limit pinned to 10000
plugins.ppl.pattern.method pinned to SIMPLE_PATTERN
plugins.ppl.pattern.mode pinned to LABEL
plugins.ppl.pattern.max.sample.count pinned to 10
plugins.ppl.pattern.buffer.limit pinned to 100000
plugins.ppl.pattern.show.numbered.token pinned to false
plugins.ppl.values.max.limit read back null, so the configured
cap on values() never applied
Each has a cluster-side default identical to the seeded one, so behavior is
unchanged unless an operator explicitly configured the setting -- at which
point the configured value now takes effect.
Replaces the hand-maintained forwardClusterSetting calls with a single
FORWARDED_CLUSTER_SETTINGS allow-list, and adds a drift guard
(everySeededPlanningSettingIsClassified) asserting every key the builder
seeds is either forwarded or explicitly documented as excluded, so this
defect cannot recur silently.
Deliberately not forwarded:
plugins.calcite.enabled -- the unified path is Calcite-based by
definition and must force it on.
plugins.ppl.subsearch.maxout / plugins.ppl.join.subsearch_maxout --
seeded to 0 (unlimited) on purpose to keep LogicalSystemLimit out of
plans built by external consumers of the unified query API. These do
diverge from the cluster defaults (10000 / 50000) even when
unconfigured; whether the in-cluster REST path should override that is
a separate behavioral decision, tracked in opensearch-project#5735.
Testing: the four new unit tests each fail without this change
(expected:<BRAIN> but was:<SIMPLE_PATTERN>, expected:<100> but was:<null>,
expected:<500> but was:<10000>) and pass with it.
Signed-off-by: Jialiang Liang <jiallian@amazon.com>1 parent 802152e commit 1a401a3
2 files changed
Lines changed: 153 additions & 14 deletions
File tree
- plugin/src
- main/java/org/opensearch/sql/plugin/rest
- test/java/org/opensearch/sql/plugin/rest
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
| 13 | + | |
13 | 14 | | |
14 | 15 | | |
15 | 16 | | |
| 17 | + | |
16 | 18 | | |
17 | 19 | | |
18 | 20 | | |
| |||
40 | 42 | | |
41 | 43 | | |
42 | 44 | | |
| 45 | + | |
43 | 46 | | |
44 | 47 | | |
45 | 48 | | |
| |||
346 | 349 | | |
347 | 350 | | |
348 | 351 | | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
349 | 385 | | |
350 | 386 | | |
351 | 387 | | |
352 | | - | |
353 | | - | |
354 | | - | |
355 | | - | |
| 388 | + | |
| 389 | + | |
356 | 390 | | |
357 | | - | |
358 | | - | |
359 | | - | |
360 | | - | |
361 | | - | |
362 | | - | |
363 | | - | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
364 | 394 | | |
365 | 395 | | |
366 | 396 | | |
367 | | - | |
368 | | - | |
| 397 | + | |
369 | 398 | | |
370 | 399 | | |
371 | 400 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| 8 | + | |
8 | 9 | | |
9 | 10 | | |
10 | 11 | | |
11 | 12 | | |
| 13 | + | |
12 | 14 | | |
| 15 | + | |
| 16 | + | |
13 | 17 | | |
14 | 18 | | |
15 | 19 | | |
| |||
22 | 26 | | |
23 | 27 | | |
24 | 28 | | |
| 29 | + | |
| 30 | + | |
25 | 31 | | |
26 | 32 | | |
27 | 33 | | |
| |||
33 | 39 | | |
34 | 40 | | |
35 | 41 | | |
| 42 | + | |
36 | 43 | | |
37 | 44 | | |
38 | 45 | | |
| |||
46 | 53 | | |
47 | 54 | | |
48 | 55 | | |
| 56 | + | |
49 | 57 | | |
50 | 58 | | |
51 | 59 | | |
| |||
54 | 62 | | |
55 | 63 | | |
56 | 64 | | |
57 | | - | |
| 65 | + | |
58 | 66 | | |
59 | 67 | | |
60 | 68 | | |
| |||
237 | 245 | | |
238 | 246 | | |
239 | 247 | | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
240 | 350 | | |
241 | 351 | | |
242 | 352 | | |
| |||
0 commit comments