Repository navigation
Verify the full stack on the device with protobuf-free telemetry #115
Description
Parent
Part of #102
What to build
End-to-end confirmation on the real car that the protobuf-free firmware produces telemetry indistinguishable from before: traces in Tempo, metrics in Mimir, logs in Loki and profiles in Pyroscope.
Target: dust-mite, branch otlp-json-no-protobuf, updating the component submodule pointers.
The existing device-gated end-to-end tests are the acceptance gate and run unchanged. Their value is that they are encoding-agnostic: they assert on what arrived in the backends, so any JSON mapping error surfaces without new test code. The profiles test doubles as a regression check on the ported exporter.
Acceptance criteria
- Submodule pointers reference the integrated component and fork branches
- The traces, metrics, logs and profiles end-to-end tests all pass against the headless stack with the real device
- Trace context propagation between the car and the rest of the stack is unaffected
- Firmware size is recorded before and after as a sanity check that protobuf is no longer compiled
- No end-to-end test required modification to pass
Activity
ltowarek commentedon Sep 7, 2026
Verified on the real device against dust-mite's
otlp-json-no-protobufbranch (pushed: https://github.com/ltowarek/dust-mite/tree/otlp-json-no-protobuf).Submodule pointers — bumped
car/components/esp-opentelemetry-cppto18c71c7(this branch's tip). Both the outer component and its nestedthird_party/opentelemetry-cppfork submodule are on their respectiveotlp-json-no-protobufbranches.Firmware size (same commit, same
sdkconfig.defaults+sdkconfig.defaults.profiling, only theESP_OPENTELEMETRY_OTLP_HTTP_ENCODINGKconfig choice flipped):check_no_protobuf.shreports 6236 protobuf/Abseil symbols (FAIL)check_no_protobuf.shreports 0 (OK)Bug found and fixed along the way:
car/main/main.cppbuilt a nullProfilesExporterand called the one-argesp_opentelemetry_profiling_setup(nullptr), which unconditionally disabled profiling regardless ofCONFIG_ESP_OPENTELEMETRY_PROFILING_ENABLED. Switched to the component's no-arg overload, which builds its own exporter fromCONFIG_ESP_OPENTELEMETRY_PROFILES_OTLP_BASE_URLwhen enabled and no-ops via its stub otherwise — exactly the "call unconditionally" contract that overload exists for.Follow-up cleanup: also dropped
CONFIG_PTHREAD_TASK_STACK_SIZE_DEFAULT=32768fromcar/sdkconfig.defaults. It existed only because BatchSpanProcessor's pthread worker serialized through protobuf; with the JSON path (cJSON writer, no protobuf/Abseil) that call chain is shallower. Verified on-device at IDF's stock 3072-byte default under sustained real traffic across all three signals with zero stack-overflow panics.E2E —
observability/tests/e2e --dutagainst the headless stack with the real device, no test files modified: 44/48 pass. The remaining 4 (a controller log,ws.command.receiveon the car,streamer.handle_drive_command, anddust_mite_commands_sent) all gate on the Python streamer's autonomous obstacle-avoidance path, which only runs while a browser client is connected to the streamer and the ultrasonic sensor reads <5cm at that same instant — not reliably reproducible unattended in this remote session. Confirmed each piece of that path works in isolation (sensor readings, command delivery, controller code path); they just never lined up together in one test window. Not a regression from this change.dust_mite_largest_free_block_bytes/task_cpu_usage/task_priorityneed thesdkconfig.defaults.alldebug overlay to emit at all (documented as CI/debug-only); they passed once that overlay was built in, but are absent from the stock production config, consistent with existing behavior.Trace context propagation across car → streamer → browser verified intact (
ws.stream.send/ws.telemetry.send/streamer.handle_camera_frame/streamer.handle_telemetry/streamer.process_frameall passed with parent/child relationships preserved).ltowarek commentedon Sep 11, 2026
Closing based on the verification comment above. dust-mite branch
otlp-json-no-protobuf, commits3e0f37cedfe5319f43ca17297a79233a3c129368(submodule bump + profiling-enable fix) and389bb21b0d3139cf4b7d1277aec6f231f601c644(pthread stack-size cleanup).Export traces, metrics and logs over OTLP/JSON without protobuf