README.md

Java conformance scenarios

Everything a JVM scenario shares: the build logic every Java build reuses, the support every scenario needs whatever domain it measures, and the command that builds and runs one.

scenarios/<domain>/java/      a domain's Gradle build root — wrapper, settings, versions
tools/java/gradle-plugins/    the convention plugins, as an included build
tools/java/scenario-support/  what a scenario needs before any telemetry
tools/java/scenario-sdk/      the SDK a library-instrumentation scenario owns
tools/java/agent-control/     the extension controlling an agent-owned SDK
tools/java/src/               `otel-conformance-java`, the launcher
tools/java/tests/             the launcher's tests

A build root per domain

A domain's Java scenarios are one Gradle build, rooted at its own scenarios/<domain>/java, including scenarios/http/java and scenarios/database/java. otel-conformance-java finds it by searching upwards from the scenario directory for settings.gradle.kts, so a scenario file says nothing about how deep it is nested. Within a build, projects are grouped by the library they instrument, so :armeria:scenarios and :armeria:opentelemetry-javaagent sit beside the Armeria scenario packages themselves.

What is not per domain is the build logic. gradle-plugins/ is an included build rather than a buildSrc, so a second domain's build root reuses it instead of restating any of it:

• otel-conformance.java-conventions — toolchain, encoding, Spotless, JUnit, and Error Prone. Style is the formatter's job, so only Error Prone's own errors fail a build here: the patterns that are bugs rather than taste.
• otel-conformance.scenario-launcher — the javaAgent configuration and prepareRuntime, applied only by the projects that are scenario entry points. Which projects those are is therefore visible in the build files.

The projects under tools/java are shared the same way: a build root includes them by directory, so both a scenario-support and a domain's own framework projects sit in one project list.

Java 25 is the Gradle build, compiler, application bytecode, test and default scenario-runtime baseline. Set JAVA_HOME to a JDK 25 installation for prepare. To use a different Java version for a scenario, configure its framework and launcher projects:

java {
    toolchain {
        languageVersion = JavaLanguageVersion.of(27)
    }
}

Both builds use the Foojay toolchain resolver, so Gradle downloads a missing JDK automatically. CI installs only Java 25 for the build JVM and lets Gradle provision additional JDKs required by scenario toolchains. JAVA_HOME stays on Java 25.

Versions are pinned in a build root's gradle/libs.versions.toml. The Java agent and the instrumentation BOM are one release published under two coordinates, so a catalog is what keeps them from drifting apart and having the agent scenarios measure a different instrumentation from the library ones.

What a scenario shares

scenario-support/ carries no OpenTelemetry dependency at all, which is the point: an agent scenario's classpath must hold the agent's instrumentation and nothing else, so what every scenario needs — the runner's environment, and the driver's shutdown protocol — cannot live beside the SDK.

scenario-sdk/ is the other half: SDK autoconfiguration, the OTLP exporter, and shutdown, for scenarios measuring explicit library instrumentation. A framework's launch project supplies only its own decorators. Agent launch projects do not depend on it, so those jars never reach a runtime that is meant to be measuring the agent.

Both SDK paths use the same pre-shutdown order: start trace and log flushes, await both, flush metrics, then let normal SDK and process shutdown continue. The 15-second budget is shared across the flush phases. A timeout or failed flush fails the scenario.

For --agent, prepare packages a Java-agent extension that captures the agent's processors and metric readers through AutoConfigurationCustomizerProvider. The shared scenario launcher invokes that extension through an in-process JMX control after the scenario main returns. Scenario code does not know about the control or call a flush helper.

otel-conformance-java

Builds and runs a JVM conformance scenario, so no conformance.yaml restates how Java is built. A scenario names its Gradle project and main class, and opts into agent attachment when needed; nothing else about Java appears in the file.

setup: otel-conformance-java prepare armeria:opentelemetry-javaagent

scenarios:
  server:
    run: otel-conformance-java run --agent armeria:opentelemetry-javaagent ArmeriaJavaagentServerScenario

The project is a Gradle path, so everything belonging to one instrumented library can be grouped under it and two libraries can each have a javaagent project without colliding.

The main class is unqualified because a scenario entry point sits in the default package. Nothing imports one — it is reached only by name from the command line — so a package would buy it no isolation and only lengthen the line above. Everything reusable stays packaged.

prepare invokes the committed Gradle wrapper's prepareRuntime, which syncs the resolved classpath and the Java agent into the build root's build/scenario-runtime/<project>, with the path flattened to one directory name. That is under the build root rather than under each project, so where a Gradle project sits on disk is the build's business and not something the launcher has to know.

prepareRuntime also writes java-runtime.json beside lib and agent. It records the absolute Java executable path and language version selected by the project's Gradle toolchain. run uses that executable, independent of the current JAVA_HOME or PATH. Missing or invalid metadata, or an unavailable executable, fails with guidance to prepare again. These are local prepared runtimes, not portable distributions: prepare again after moving the checkout or moving or removing a JDK.

run executes the selected JVM directly rather than through Gradle, so the scenario inherits the fresh OTLP endpoint the runner injected instead of whatever a long-lived Gradle daemon started with.

--agent attaches the Java agent from the prepared runtime. It is a JVM launch option, not an application mode, and is not passed to the scenario, so it goes before the main class. Everything after the main class reaches the scenario verbatim, including arguments that begin with -.