Skip to content

Run regression tests on JDK 26#1526

Merged
msridhar merged 8 commits intomasterfrom
java-26
Apr 14, 2026
Merged

Run regression tests on JDK 26#1526
msridhar merged 8 commits intomasterfrom
java-26

Conversation

@msridhar
Copy link
Copy Markdown
Collaborator

@msridhar msridhar commented Apr 13, 2026

Partially addresses #1525. We update WALA to get JDK 26 support.

Summary by CodeRabbit

  • New Features

    • Added support for running tests on JDK 26 in addition to JDK 21.
  • Chores

    • Updated WALA library to version 1.7.0.
    • Standardized the Java toolchain to use Azul for test runs.
    • Ensured the test module JAR is built before any JDK-specific test executions.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 13, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

The PR sets a global Java toolchain in the test conventions to languageVersion 26 and vendor Azul, expands the generated per-JDK Test tasks to include JDK 26 (adding testJdk26) and wires check to depend on it, updates gradle/libs.versions.toml to bump wala from 1.6.12 → 1.7.0, and makes all Test tasks depend on :test-java-module:jar so the test module JAR is built before JDK-specific tests run.

Possibly related PRs

  • Update to Error Prone 2.45.0 #1401: Modifies the same buildSrc/src/main/groovy/nullaway.java-test-conventions.gradle to change the JDK-specific test task/toolchain matrix.
  • Test on JDK 25 #1301: Adds support for a newer JDK test target in the same test-conventions script (previously added JDK 25).
  • Simplified set of CI jobs #1271: Alters registration logic for per-JDK testJdk* tasks and toolchain configuration in the test-conventions script.

Suggested reviewers

  • lazaroclapp
  • yuxincs
🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly and concisely summarizes the main objective of the PR: adding JDK 26 regression testing support.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch java-26

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@buildSrc/src/main/groovy/nullaway.java-test-conventions.gradle`:
- Line 81: The shared convention is enabling testJdk26 via the loop that
registers JDK test tasks (e.g., the block iterating [21,26].each { majorVersion
-> ... } that creates testJdk26), which causes modules that explicitly target a
fixed JDK (such as jdk-recent-unit-tests) to inherit it; update those module
build scripts that enforce a single recent JDK target to disable the inherited
testJdk26 by adding onlyIf { false } to the testJdk26 task registration (i.e.,
locate the testJdk26 TaskProvider or task configuration in the module and append
onlyIf { false } so the task is skipped).

In `@gradle/libs.versions.toml`:
- Line 5: The dependency version for the 'wala' entry in
gradle/libs.versions.toml is set to a non-existent "1.7.0"; change the value of
the wala key to a valid release such as "1.6.12" (or "1.6.13-SNAPSHOT" if you
need a snapshot) so the build can resolve the artifact.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: c259de33-1ec9-4637-89bf-8ea5334055fe

📥 Commits

Reviewing files that changed from the base of the PR and between 9c08bc5 and 45e7db4.

📒 Files selected for processing (3)
  • buildSrc/src/main/groovy/nullaway.java-test-conventions.gradle
  • gradle/libs.versions.toml
  • jdk-recent-unit-tests/build.gradle


// Tasks for testing on other JDK versions; see https://jakewharton.com/build-on-latest-java-test-through-lowest-java/
[21].each { majorVersion ->
[21,26].each { majorVersion ->
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Read-only verification: check toolchain resolver + CI JDK 26 setup.
set -euo pipefail

echo "== settings.gradle* files =="
fd -i 'settings.gradle*' -x sed -n '1,220p' {}

echo
echo "== Toolchain-related config in Gradle files/properties =="
rg -n -C3 --iglob '*.gradle' --iglob '*.gradle.kts' --iglob 'gradle.properties' \
  'foojay|toolchainManagement|javaRepositories|javaToolchains|org\.gradle\.java\.installations|auto-download'

echo
echo "== CI workflow Java setup (looking for 26) =="
rg -n -C3 --iglob '*.yml' --iglob '*.yaml' \
  'actions/setup-java|java-version|distribution|temurin|zulu|26'

Repository: uber/NullAway

Length of output: 5555


🏁 Script executed:

# Check if any modules disable testJdk26 using onlyIf
rg -n 'testJdk26' --iglob '*.gradle' --iglob '*.gradle.kts' -A 3

Repository: uber/NullAway

Length of output: 39


🌐 Web query:

Foojay resolver Gradle plugin version 1.0.0 Java 26 support toolchain

💡 Result:

The Foojay resolver Gradle plugin version 1.0.0, released on May 19, 2025, supports Java toolchains via the Foojay Disco API. Java 26 reached general availability on March 17, 2026. The plugin resolves toolchains using the major Java version specified in the toolchain, querying the Disco API for matching distributions. No sources indicate limitations preventing Java 26 support; the plugin dynamically fetches available JDKs from the API, which tracks all OpenJDK builds including recent releases like Java 26. The plugin requires Gradle 7.6+ and Java 17+ to run. Usage example in settings.gradle.kts: plugins { id("org.gradle.toolchains.foojay-resolver-convention") version "1.0.0" } java { toolchain { languageVersion.set(JavaLanguageVersion.of(26)) } }

Citations:


🏁 Script executed:

# Search for onlyIf { false } pattern on test tasks to understand skip pattern
rg -n 'onlyIf\s*\{\s*false\s*\}' --iglob '*.gradle' --iglob '*.gradle.kts' -B 2

Repository: uber/NullAway

Length of output: 1468


🏁 Script executed:

# Check if any modules have JDK version constraints that would exclude JDK 26
rg -n 'sourceCompatibility|targetCompatibility|languageVersion' --iglob '*.gradle' --iglob '*.gradle.kts' | grep -E '(21|22|23|24|25)' | head -20

Repository: uber/NullAway

Length of output: 1973


🏁 Script executed:

# Check jdk-recent-unit-tests full build.gradle to understand its testing strategy
cat -n jdk-recent-unit-tests/build.gradle | head -80

Repository: uber/NullAway

Length of output: 2802


🏁 Script executed:

# Search for any conditional logic or patterns that might auto-skip newer JDK tests
rg -n 'testJdk' buildSrc/src/main/groovy/nullaway.java-test-conventions.gradle -A 5 -B 2

Repository: uber/NullAway

Length of output: 829


Add explicit testJdk26 disable to modules with fixed JDK targets, like jdk-recent-unit-tests.

Foojay resolver 1.0.0 properly provisions Java 26 toolchains via the Disco API; the shared convention's addition of testJdk26 won't cause CI hard-failures. However, jdk-recent-unit-tests explicitly targets JDK 25 and disables testJdk17 and testJdk21 to enforce this (per its comment "we only run on the recent JDK version specified above"). This module now inherits testJdk26 from the shared convention but does not disable it, breaking its module-specific JDK targeting. Add onlyIf { false } to testJdk26 in affected modules using fixed JDK versions.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@buildSrc/src/main/groovy/nullaway.java-test-conventions.gradle` at line 81,
The shared convention is enabling testJdk26 via the loop that registers JDK test
tasks (e.g., the block iterating [21,26].each { majorVersion -> ... } that
creates testJdk26), which causes modules that explicitly target a fixed JDK
(such as jdk-recent-unit-tests) to inherit it; update those module build scripts
that enforce a single recent JDK target to disable the inherited testJdk26 by
adding onlyIf { false } to the testJdk26 task registration (i.e., locate the
testJdk26 TaskProvider or task configuration in the module and append onlyIf {
false } so the task is skipped).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should run jdk-recent-unit-tests on JDK 25 and newer. I'll clarify the comment to indicate this

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@msridhar, thanks for the clarification! That makes sense — jdk-recent-unit-tests is intended to run on JDK 25 and newer, so inheriting testJdk26 from the shared convention is exactly the right behavior. My concern was incorrect. Appreciate you updating the comment to make this intent clear.


✏️ Learnings added
Learnt from: msridhar
Repo: uber/NullAway PR: 1526
File: buildSrc/src/main/groovy/nullaway.java-test-conventions.gradle:0-0
Timestamp: 2026-04-13T19:39:37.891Z
Learning: In the NullAway repo, the `jdk-recent-unit-tests` module is intentionally run on JDK 25 AND all newer JDK versions (e.g., JDK 26). It should NOT disable newer JDK test tasks like `testJdk26` inherited from the shared `nullaway.java-test-conventions.gradle` convention. The module's purpose is to test on the latest/recent JDKs (25+), not exclusively on JDK 25.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.


🧠 Learnings used
Learnt from: msridhar
Repo: uber/NullAway PR: 1245
File: guava-recent-unit-tests/src/test/java/com/uber/nullaway/guava/NullAwayGuavaParametricNullnessTests.java:101-102
Timestamp: 2025-08-14T18:50:06.159Z
Learning: In NullAway JSpecify tests, when JDK version requirements exist due to bytecode annotation reading capabilities, prefer failing tests over skipping them on unsupported versions to ensure CI catches regressions and enforces proper JDK version usage for developers.

Learnt from: msridhar
Repo: uber/NullAway PR: 1243
File: jdk-annotations/astubx-generator/build.gradle:22-22
Timestamp: 2025-10-09T19:59:16.543Z
Learning: When disabling testJdk17 tasks for modules requiring JDK 21, use `onlyIf { false }` to skip the task:
```gradle
tasks.named("testJdk17").configure {
    onlyIf { false }
}
```
Do not use `doFirst { throw new GradleException(...) }` as it will cause CI failures when the task is executed.

Learnt from: CR
Repo: uber/NullAway PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-04-08T19:11:30.754Z
Learning: Run tests for the main NullAway module using `./gradlew :nullaway:test`; use `--tests` flag to run specific test classes or methods

Comment thread gradle/libs.versions.toml
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
buildSrc/src/main/groovy/nullaway.java-test-conventions.gradle (1)

90-93: ⚠️ Potential issue | 🟠 Major

Explicitly set vendor to AZUL in launcherFor to prevent non-Azul JDK resolution.

The launcherFor call with only languageVersion allows Gradle to select any vendor matching the version, bypassing the Azul constraint. According to Gradle toolchain specifications, vendor defaults to "any" if not explicitly set inside the launcherFor block and is not inherited from the project's java.toolchain configuration.

🔧 Proposed fix
-        javaLauncher = javaToolchains.launcherFor {
-            languageVersion = JavaLanguageVersion.of(majorVersion)
-        }
+        javaLauncher = javaToolchains.launcherFor {
+            languageVersion = JavaLanguageVersion.of(majorVersion)
+            vendor = JvmVendorSpec.AZUL
+        }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@buildSrc/src/main/groovy/nullaway.java-test-conventions.gradle` around lines
90 - 93, The toolchain launcher currently only sets languageVersion in the
javaToolchains.launcherFor block (in the test task registered as
"testJdk$majorVersion" that assigns javaLauncher), which allows any vendor;
update the launcherFor block to explicitly set the vendor to Azul by adding
vendor = JvmVendorSpec.AZUL inside javaToolchains.launcherFor so Gradle resolves
an Azul JDK rather than any vendor.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In `@buildSrc/src/main/groovy/nullaway.java-test-conventions.gradle`:
- Around line 90-93: The toolchain launcher currently only sets languageVersion
in the javaToolchains.launcherFor block (in the test task registered as
"testJdk$majorVersion" that assigns javaLauncher), which allows any vendor;
update the launcherFor block to explicitly set the vendor to Azul by adding
vendor = JvmVendorSpec.AZUL inside javaToolchains.launcherFor so Gradle resolves
an Azul JDK rather than any vendor.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 53da8dec-8c7b-4815-8c7f-dd241fb100b7

📥 Commits

Reviewing files that changed from the base of the PR and between 171a0eb and 29f79a6.

📒 Files selected for processing (1)
  • buildSrc/src/main/groovy/nullaway.java-test-conventions.gradle

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.48%. Comparing base (334016d) to head (5c4ce9a).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff            @@
##             master    #1526   +/-   ##
=========================================
  Coverage     88.48%   88.48%           
  Complexity     2843     2843           
=========================================
  Files           103      103           
  Lines          9489     9489           
  Branches       1905     1905           
=========================================
  Hits           8396     8396           
  Misses          531      531           
  Partials        562      562           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@msridhar msridhar enabled auto-merge (squash) April 14, 2026 19:00
@msridhar msridhar merged commit 26a23ce into master Apr 14, 2026
11 checks passed
@msridhar msridhar deleted the java-26 branch April 14, 2026 19:21
This was referenced Apr 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants