Stabilize PPL ITs on the analytics-engine route (percentile/float/datetime/json/dedup/union/rename/chart) (opensearch-project#5564)

Brings 14 PPL IT classes to parity on the analytics-engine route
(-Dtests.analytics.parquet_indices=true). Each gated test is skipped
ONLY on the AE route via @RequiresCapability + a matching build.gradle
excludeTestsMatching; the v2/Calcite path runs all of them unchanged.

Engine divergences gated (route-only, behavior is correct elsewhere):
- PERCENTILE_APPROXIMATE: percentile/median is approximate on DataFusion
- FLOAT_ARITHMETIC_PRECISION: float/half_float arithmetic keeps 32-bit
- DATETIME_FORMAT_RENDERING: date_format/strftime token rendering differs
- UNIX_TIMESTAMP_SUBSECOND: unix_timestamp drops sub-second precision
- JSON_DOLLAR_PATH: json_set/json_delete with $-path is a no-op
- DEDUP_NONDETERMINISTIC: dedup surviving-row selection is unstable
- SAME_INDEX_UNION_CONFLATION: same-index union conflates (delegate leak)
- WILDCARD_COLUMN_ORDER: rename * column order differs
- BIN_TIME_FIELD_BUCKETING: span() time bucketing differs
- MULTI_VALUE_FIELD_LOAD: otel_logs/game_of_thrones multi-value field
  can't bulk-load into the parquet store

Two cases were harness-predicate gaps, not real divergences, so they
now PASS on AE instead of being gated: testStatsPercentileWithMin and
testTimestampDiff branch on isCalciteEnabled() for the result type, but
the AE route runs the Calcite path while the cluster setting reads
false; extended the predicate with isAnalyticsParquetIndicesEnabled().

Guarded the otel_logs / game_of_thrones loads in CalciteChartCommandIT
and CalcitePPLJsonBuiltinFunctionIT init() so a multi-value bulk-load
failure no longer aborts init() and mislabels unrelated tests.

CalciteAnalyticsDatetimeWireFormatIT (AE-only via assumeTrue) updated
to assert the date/time UDT types AE now preserves.

Results (this batch, on the AE route):
- CalcitePPLAggregationIT: 100 run, 2 skip
- CalcitePPLBuiltinFunctionIT: 26 run, 3 skip
- CalciteDateTimeFunctionIT: 65 run, 5 skip
- CalcitePPLDedupIT: 15 run, 3 skip
- CalcitePPLJsonBuiltinFunctionIT: 22 run, 2 skip
- CalcitePPLRenameIT: 24 run, 1 skip
- CalciteUnionCommandIT: 15 run, 2 skip
- CalciteChartCommandIT: 15 run, 5 skip
- DateTimeFunctionIT: 59 run, 4 skip
- StatsCommandIT: 59 run, 6 skip
- SystemFunctionIT: 1 run
- CalciteAnalyticsDatetimeWireFormatIT: 11 run
AE route: 0 failures (was 30+ fail across the batch). V2 baseline:
402 run, 0 fail, 3 pre-existing skips, 0 from these gates.

Signed-off-by: Kai Huang <ahkcs@amazon.com>

Author: ahkcs

integ-test/build.gradle

@@ -1239,6 +1239,52 @@ task integTestRemote(type: RestIntegTestTask) {
             // === Excludes: CalcitePPLBasicIT route divergence ===
             // REGEXP filter throws a backend NullPointerException on the AE route.
             excludeTestsMatching '*CalcitePPLBasicIT.testRegexpFilter'
+
+            // === Excludes: percentile is approximate on AE (DataFusion) but exact on v2/Calcite ===
+            excludeTestsMatching '*StatsCommandIT.testStatsPercentileWithNull'
+            excludeTestsMatching '*StatsCommandIT.testStatsPercentileByNullValue'
+            excludeTestsMatching '*StatsCommandIT.testStatsPercentileByNullValueNonNullBucket'
+            excludeTestsMatching '*StatsCommandIT.testStatsPercentileBySpan'
+            excludeTestsMatching '*CalcitePPLAggregationIT.testPercentile'
+            excludeTestsMatching '*CalcitePPLAggregationIT.testPercentileShortcutsFloatingPoint'
+
+            // === Excludes: span() time-field bucketing differs on the AE route ===
+            excludeTestsMatching '*StatsCommandIT.testStatsBySpanTimeWithNullBucket'
+            excludeTestsMatching '*CalciteChartCommandIT.testChartMaxValueByTimestampSpanDayAndWeek'
+
+            // === Excludes: float/half_float arithmetic keeps 32-bit precision on AE ===
+            excludeTestsMatching '*CalcitePPLBuiltinFunctionIT.testDivide'
+            excludeTestsMatching '*CalcitePPLBuiltinFunctionIT.testModFloatAndNegative'
+            excludeTestsMatching '*CalcitePPLBuiltinFunctionIT.testModShouldReturnWiderTypes'
+
+            // === Excludes: date_format/strftime render some tokens differently on AE ===
+            excludeTestsMatching '*DateTimeFunctionIT.testDateFormat'
+            excludeTestsMatching '*CalciteDateTimeFunctionIT.testStrftimeWithDateFields'
+
+            // === Excludes: unix_timestamp drops sub-second precision on AE ===
+            excludeTestsMatching '*DateTimeFunctionIT.testUnixTimestampWithTimestampString'
+
+            // === Excludes: json_set/json_delete with a $-prefixed path is a no-op on AE ===
+            excludeTestsMatching '*CalcitePPLJsonBuiltinFunctionIT.testJsonSetWithDollarPrefixedPath'
+            excludeTestsMatching '*CalcitePPLJsonBuiltinFunctionIT.testJsonDeleteWithDollarPrefixedPath'
+
+            // === Excludes: dedup surviving-row selection is non-deterministic on AE ===
+            excludeTestsMatching '*CalcitePPLDedupIT.testDedupComplex'
+            excludeTestsMatching '*CalcitePPLDedupIT.testDedupExpr'
+            excludeTestsMatching '*CalcitePPLDedupIT.testConsecutiveImplicitFallbackV2'
+
+            // === Excludes: same-index union conflates on AE (delegated predicate leak) ===
+            excludeTestsMatching '*CalciteUnionCommandIT.testUnionThreeSubsearches'
+            excludeTestsMatching '*CalciteUnionCommandIT.testUnionMidPipeline_SingleExplicitDataset'
+
+            // === Excludes: rename * returns columns in a different order on AE ===
+            excludeTestsMatching '*CalcitePPLRenameIT.testRenameFullWildcardExcludesMetadataFields'
+
+            // === Excludes: otel_logs multi-value field can't load into the parquet store ===
+            excludeTestsMatching '*CalciteChartCommandIT.testChartLimit0WithUseOther'
+            excludeTestsMatching '*CalciteChartCommandIT.testChartLimitTopWithUseOther'
+            excludeTestsMatching '*CalciteChartCommandIT.testChartLimitBottomWithUseOther'
+            excludeTestsMatching '*CalciteChartCommandIT.testChartLimitTopWithMinAgg'
         }
     }
 

integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalciteAnalyticsDatetimeWireFormatIT.java

@@ -92,29 +92,24 @@ public void testTimestampRootColumnSpaceFormat() throws IOException {
     verifyDataRows(result, rows("2024-03-15 10:30:00"));
   }
 
-  /**
-   * DATE-mapped col: AE widens to TIMESTAMP at scan time; value must use space separator, not ISO
-   * {@code T}.
-   */
+  /** DATE-mapped col: AE preserves the DATE type (date UDT) and renders {@code yyyy-MM-dd}. */
   @Test
   public void testDateRootColumnYmdFormat() throws IOException {
     String query = "source=" + INDEX + " | where d = '2024-03-15' | fields d";
     assertRoutedToAnalyticsEngine(query);
     JSONObject result = executeQuery(query);
-    verifySchema(result, schema("d", "timestamp"));
-    verifyDataRows(result, rows("2024-03-15 00:00:00"));
+    verifySchema(result, schema("d", "date"));
+    verifyDataRows(result, rows("2024-03-15"));
   }
 
-  /** TIME-mapped col: AE widens to TIMESTAMP; value must use space separator, not ISO {@code T}. */
+  /** TIME-mapped col: AE preserves the TIME type (time UDT) and renders {@code HH:mm:ss}. */
   @Test
   public void testTimeRootColumnHmsFormat() throws IOException {
     String query = "source=" + INDEX + " | sort t | head 1 | fields t";
     assertRoutedToAnalyticsEngine(query);
     JSONObject result = executeQuery(query);
-    verifySchema(result, schema("t", "timestamp"));
-    Assert.assertFalse(
-        "Time-mapped column must not surface as ISO T-separator literal",
-        result.getJSONArray("datarows").getJSONArray(0).getString(0).contains("T"));
+    verifySchema(result, schema("t", "time"));
+    verifyDataRows(result, rows("10:30:00"));
   }
 
   /** Eval-derived TIMESTAMP follows the same wire-format contract as a root column. */

integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalciteChartCommandIT.java

@@ -9,6 +9,8 @@
 import static org.opensearch.sql.legacy.TestsConstants.TEST_INDEX_BANK_WITH_NULL_VALUES;
 import static org.opensearch.sql.legacy.TestsConstants.TEST_INDEX_OTEL_LOGS;
 import static org.opensearch.sql.legacy.TestsConstants.TEST_INDEX_TIME_DATA;
+import static org.opensearch.sql.util.Capability.BIN_TIME_FIELD_BUCKETING;
+import static org.opensearch.sql.util.Capability.MULTI_VALUE_FIELD_LOAD;
 import static org.opensearch.sql.util.MatcherUtils.assertJsonEquals;
 import static org.opensearch.sql.util.MatcherUtils.rows;
 import static org.opensearch.sql.util.MatcherUtils.schema;
@@ -20,6 +22,7 @@
 import org.json.JSONObject;
 import org.junit.jupiter.api.Test;
 import org.opensearch.sql.ppl.PPLIntegTestCase;
+import org.opensearch.sql.util.RequiresCapability;
 
 public class CalciteChartCommandIT extends PPLIntegTestCase {
   @Override
@@ -28,7 +31,13 @@ public void init() throws Exception {
     enableCalcite();
     loadIndex(Index.BANK);
     loadIndex(Index.BANK_WITH_NULL_VALUES);
-    loadIndex(Index.OTELLOGS);
+    // otel_logs has a multi-value array for a scalar-mapped field, which the parquet store rejects
+    // at bulk load (MULTI_VALUE_FIELD_LOAD); skip the load on the AE route so it doesn't abort
+    // init() for the otel-independent tests. The otel tests themselves are
+    // @RequiresCapability-gated.
+    if (!isAnalyticsParquetIndicesEnabled()) {
+      loadIndex(Index.OTELLOGS);
+    }
     loadIndex(Index.TIME_TEST_DATA);
     loadIndex(Index.EVENTS_NULL);
   }
@@ -142,6 +151,11 @@ public void testChartMaxValueOverCategoryByTimestampSpanWeek() throws IOExceptio
   }
 
   @Test
+  @RequiresCapability(
+      value = BIN_TIME_FIELD_BUCKETING,
+      note =
+          "span=2weeks anchors the bucket to a different week origin on the AE route"
+              + " (BIN_TIME_FIELD_BUCKETING).")
   public void testChartMaxValueByTimestampSpanDayAndWeek() throws IOException {
     JSONObject result =
         executeQuery(
@@ -165,6 +179,11 @@ public void testChartMaxValueByTimestampSpanDayAndWeek() throws IOException {
   }
 
   @Test
+  @RequiresCapability(
+      value = MULTI_VALUE_FIELD_LOAD,
+      note =
+          "reads otel_logs whose multi-value field can't load on the AE store"
+              + " (MULTI_VALUE_FIELD_LOAD).")
   public void testChartLimit0WithUseOther() throws IOException {
     JSONObject result =
         executeQuery(
@@ -208,6 +227,11 @@ public void testChartLimit0WithUseOther() throws IOException {
   }
 
   @Test
+  @RequiresCapability(
+      value = MULTI_VALUE_FIELD_LOAD,
+      note =
+          "reads otel_logs whose multi-value field can't load on the AE store"
+              + " (MULTI_VALUE_FIELD_LOAD).")
   public void testChartLimitTopWithUseOther() throws IOException {
     JSONObject result =
         executeQuery(
@@ -230,6 +254,11 @@ public void testChartLimitTopWithUseOther() throws IOException {
   }
 
   @Test
+  @RequiresCapability(
+      value = MULTI_VALUE_FIELD_LOAD,
+      note =
+          "reads otel_logs whose multi-value field can't load on the AE store"
+              + " (MULTI_VALUE_FIELD_LOAD).")
   public void testChartLimitBottomWithUseOther() throws IOException {
     JSONObject result =
         executeQuery(
@@ -246,6 +275,11 @@ public void testChartLimitBottomWithUseOther() throws IOException {
   }
 
   @Test
+  @RequiresCapability(
+      value = MULTI_VALUE_FIELD_LOAD,
+      note =
+          "reads otel_logs whose multi-value field can't load on the AE store"
+              + " (MULTI_VALUE_FIELD_LOAD).")
   public void testChartLimitTopWithMinAgg() throws IOException {
     JSONObject result =
         executeQuery(

integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalciteDateTimeFunctionIT.java

@@ -7,6 +7,7 @@
 
 import static org.opensearch.sql.legacy.TestsConstants.TEST_INDEX_DATE;
 import static org.opensearch.sql.legacy.TestsConstants.TEST_INDEX_DATE_FORMATS;
+import static org.opensearch.sql.util.Capability.DATETIME_FORMAT_RENDERING;
 import static org.opensearch.sql.util.MatcherUtils.rows;
 import static org.opensearch.sql.util.MatcherUtils.verifyDataRows;
 
@@ -17,6 +18,7 @@
 import org.junit.Ignore;
 import org.junit.Test;
 import org.opensearch.sql.ppl.DateTimeFunctionIT;
+import org.opensearch.sql.util.RequiresCapability;
 
 public class CalciteDateTimeFunctionIT extends DateTimeFunctionIT {
   @Override
@@ -73,6 +75,9 @@ public void testStrftimeWithVariousInputTypes() throws IOException {
   }
 
   @Test
+  @RequiresCapability(
+      value = DATETIME_FORMAT_RENDERING,
+      note = "strftime renders sub-second precision differently on the AE route.")
   public void testStrftimeWithDateFields() throws IOException {
     // Test strftime with different date field types from indices
     loadIndex(Index.DATE_FORMATS);

integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalcitePPLAggregationIT.java

@@ -12,6 +12,7 @@
 import static org.opensearch.sql.legacy.TestsConstants.TEST_INDEX_DATE_FORMATS;
 import static org.opensearch.sql.legacy.TestsConstants.TEST_INDEX_LOGS;
 import static org.opensearch.sql.legacy.TestsConstants.TEST_INDEX_TELEMETRY;
+import static org.opensearch.sql.util.Capability.PERCENTILE_APPROXIMATE;
 import static org.opensearch.sql.util.MatcherUtils.assertJsonEquals;
 import static org.opensearch.sql.util.MatcherUtils.rows;
 import static org.opensearch.sql.util.MatcherUtils.schema;
@@ -28,6 +29,7 @@
 import org.opensearch.sql.common.utils.StringUtils;
 import org.opensearch.sql.exception.SemanticCheckException;
 import org.opensearch.sql.ppl.PPLIntegTestCase;
+import org.opensearch.sql.util.RequiresCapability;
 
 public class CalcitePPLAggregationIT extends PPLIntegTestCase {
 
@@ -967,6 +969,9 @@ public void testTake() throws IOException {
   }
 
   @Test
+  @RequiresCapability(
+      value = PERCENTILE_APPROXIMATE,
+      note = "percentile is approximate on the AE route but exact on v2/Calcite.")
   public void testPercentile() throws IOException {
     JSONObject actual =
         executeQuery(
@@ -1184,6 +1189,9 @@ public void testPercentileShortcutsWithDecimals() throws IOException {
   }
 
   @Test
+  @RequiresCapability(
+      value = PERCENTILE_APPROXIMATE,
+      note = "percentile is approximate on the AE route but exact on v2/Calcite.")
   public void testPercentileShortcutsFloatingPoint() throws IOException {
     JSONObject actual =
         executeQuery(

integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalcitePPLBuiltinFunctionIT.java

@@ -9,6 +9,7 @@
 import static org.opensearch.sql.legacy.TestsConstants.TEST_INDEX_DOG;
 import static org.opensearch.sql.legacy.TestsConstants.TEST_INDEX_NULL_MISSING;
 import static org.opensearch.sql.legacy.TestsConstants.TEST_INDEX_STATE_COUNTRY;
+import static org.opensearch.sql.util.Capability.FLOAT_ARITHMETIC_PRECISION;
 import static org.opensearch.sql.util.MatcherUtils.closeTo;
 import static org.opensearch.sql.util.MatcherUtils.rows;
 import static org.opensearch.sql.util.MatcherUtils.schema;
@@ -21,6 +22,7 @@
 import org.json.JSONObject;
 import org.junit.jupiter.api.Test;
 import org.opensearch.sql.ppl.PPLIntegTestCase;
+import org.opensearch.sql.util.RequiresCapability;
 
 public class CalcitePPLBuiltinFunctionIT extends PPLIntegTestCase {
   @Override
@@ -255,6 +257,9 @@ public void testModWithSortAndFields() throws IOException {
   }
 
   @Test
+  @RequiresCapability(
+      value = FLOAT_ARITHMETIC_PRECISION,
+      note = "float modulo keeps 32-bit precision on AE; v2 widens to double.")
   public void testModFloatAndNegative() throws IOException {
     JSONObject actual =
         executeQuery(
@@ -267,6 +272,9 @@ public void testModFloatAndNegative() throws IOException {
   }
 
   @Test
+  @RequiresCapability(
+      value = FLOAT_ARITHMETIC_PRECISION,
+      note = "float modulo keeps 32-bit precision on AE; v2 widens to double.")
   public void testModShouldReturnWiderTypes() throws IOException {
     JSONObject actual =
         executeQuery(
@@ -347,6 +355,9 @@ public void testSignAndRound() throws IOException {
   }
 
   @Test
+  @RequiresCapability(
+      value = FLOAT_ARITHMETIC_PRECISION,
+      note = "float/half_float division keeps 32-bit precision on AE; v2 widens to double.")
   public void testDivide() throws IOException {
     JSONObject actual =
         executeQuery(

integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalcitePPLDedupIT.java

@@ -7,12 +7,14 @@
 
 import static org.opensearch.sql.legacy.TestsConstants.TEST_INDEX_ACCOUNT;
 import static org.opensearch.sql.legacy.TestsConstants.TEST_INDEX_DUPLICATION_NULLABLE;
+import static org.opensearch.sql.util.Capability.DEDUP_NONDETERMINISTIC;
 import static org.opensearch.sql.util.MatcherUtils.*;
 
 import java.io.IOException;
 import org.json.JSONObject;
 import org.junit.jupiter.api.Test;
 import org.opensearch.sql.ppl.PPLIntegTestCase;
+import org.opensearch.sql.util.RequiresCapability;
 
 public class CalcitePPLDedupIT extends PPLIntegTestCase {
 
@@ -95,6 +97,9 @@ public void testDedupMultipleFieldsKeepEmpty() throws IOException {
   }
 
   @Test
+  @RequiresCapability(
+      value = DEDUP_NONDETERMINISTIC,
+      note = "dedup CONSECUTIVE behavior diverges on the AE route.")
   public void testConsecutiveImplicitFallbackV2() throws IOException {
     JSONObject actual =
         executeQuery(
@@ -252,6 +257,10 @@ public void testReorderDedupFieldsShouldNotAffectResult() throws IOException {
   }
 
   @Test
+  @RequiresCapability(
+      value = DEDUP_NONDETERMINISTIC,
+      note =
+          "dedup surviving-duplicate selection diverges on the AE route (no stable merge order).")
   public void testDedupComplex() throws IOException {
     JSONObject actual =
         executeQuery(String.format("source=%s | dedup 1 name", TEST_INDEX_DUPLICATION_NULLABLE));
@@ -364,6 +373,10 @@ public void testSortThenDedupKeepEmpty() throws IOException {
   }
 
   @Test
+  @RequiresCapability(
+      value = DEDUP_NONDETERMINISTIC,
+      note =
+          "dedup surviving-duplicate selection diverges on the AE route (no stable merge order).")
   public void testDedupExpr() throws IOException {
     JSONObject actual =
         executeQuery(

integ-test/src/test/java/org/opensearch/sql/calcite/remote/CalcitePPLJsonBuiltinFunctionIT.java

@@ -7,6 +7,7 @@
 
 import static org.opensearch.sql.expression.function.jsonUDF.JsonUtils.gson;
 import static org.opensearch.sql.legacy.TestsConstants.*;
+import static org.opensearch.sql.util.Capability.JSON_DOLLAR_PATH;
 import static org.opensearch.sql.util.MatcherUtils.*;
 import static org.opensearch.sql.util.MatcherUtils.rows;
 
@@ -15,6 +16,7 @@
 import org.json.JSONObject;
 import org.junit.jupiter.api.Test;
 import org.opensearch.sql.ppl.PPLIntegTestCase;
+import org.opensearch.sql.util.RequiresCapability;
 
 public class CalcitePPLJsonBuiltinFunctionIT extends PPLIntegTestCase {
   @Override
@@ -30,7 +32,12 @@ public void init() throws Exception {
     loadIndex(Index.PEOPLE2);
     loadIndex(Index.BANK);
     loadIndex(Index.JSON_TEST);
-    loadIndex(Index.GAME_OF_THRONES);
+    // game_of_thrones has a multi-value array for the scalar-mapped `titles` field, which the
+    // parquet store rejects at bulk load; skip it on the AE route so it doesn't abort init() for
+    // the rest of the suite. No test in this class queries game_of_thrones.
+    if (!isAnalyticsParquetIndicesEnabled()) {
+      loadIndex(Index.GAME_OF_THRONES);
+    }
   }
 
   @Test
@@ -297,6 +304,9 @@ public void testJsonSetPartialSet() throws IOException {
   }
 
   @Test
+  @RequiresCapability(
+      value = JSON_DOLLAR_PATH,
+      note = "json_set with a $-prefixed path is a no-op on the AE route (JSON_DOLLAR_PATH).")
   public void testJsonSetWithDollarPrefixedPath() throws IOException {
     // Issue #5167: json_set with $.key path should not double-prefix
     JSONObject actual =
@@ -313,6 +323,9 @@ public void testJsonSetWithDollarPrefixedPath() throws IOException {
   }
 
   @Test
+  @RequiresCapability(
+      value = JSON_DOLLAR_PATH,
+      note = "json_delete with a $-prefixed path is a no-op on the AE route (JSON_DOLLAR_PATH).")
   public void testJsonDeleteWithDollarPrefixedPath() throws IOException {
     // Issue #5167: json_delete with $.key path should remove the key
     JSONObject actual =