evetest: port Eden app and device telemetry smoke tests - #6277
Conversation
|
Claude finished @rene's task in 3m 42s —— View job Code reviewI read all 13 files, cross-checked the framework methods they use ( This is clean, well-documented, test-only code. Framework usage is correct, the doc comments follow the
No correctness, security, or performance issues found. I left two low-priority inline nits on
Both are optional. Note: I couldn't run |
|
|
A minimal container that prints a startup banner followed by numbered heartbeats. Log-related tests need an application that emits output from container creation onwards; evetest-ubuntu-ctr prints nothing, so there is no way to exercise the container-creation log path with it. The heartbeat counter restarts from 1 on every (re)creation of the container, which lets a test tell application restarts apart by counting banner occurrences instead of relying on log timestamps. Signed-off-by: Renê de Souza Pinto <rene@renesp.com.br> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Add tests/apps, a new package for application-lifecycle scenarios that
are not about networking, holding evetest equivalents of three eden
smoke tests (tests/eclient/testdata/{metadata,userdata,app_logs}.txt):
- TestAppInstanceMetadata: an app POSTs a payload to the link-local
metadata server and EVE must report it to the controller as
ZInfoAppInstMetaData. Also covers the appCustomStatus endpoint, to
check that the reported metadata type follows the endpoint used, and
the hostname endpoint.
- TestAppUserData: ~90 KB of plain key=value user-data must not break
deployment and must reach the container as environment variables;
a #cloud-config document must have its write_files section applied,
exactly once per user-data version, so that a file modified by the
application survives an application restart.
- TestAppLogs: application stdout must reach the controller, both the
output produced at container creation and the output produced while
the app runs, and collection must resume after the app is stopped and
started again.
All three are registered in TestApplicationSuite and share the same
device and network requirements so the framework reuses a single VM
across the suite.
Signed-off-by: Renê de Souza Pinto <rene@renesp.com.br>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Add tests/telemetry, a new package covering the three channels through
which EVE reports itself to the controller, holding evetest equivalents
of three eden smoke tests (tests/lim/testdata/{log,info,metric}_test.txt):
- TestDeviceInfo: ZInfoDevice must report the port configuration EVE
actually applied, a plausible hardware inventory, and an HSM state
matching how the device was created.
- TestDeviceMetrics: DeviceMetric must report moving per-port network
counters, plus the memory, CPU and controller-connectivity counters
carried in the same message. Note that NetworkMetric.iName is the
logical label from the controller config, not the interface name;
eden matched it against eth0 only because its device model labels the
port that way, so iName and localName are asserted separately here.
- TestDeviceLogs: a message emitted on the device must reach the
controller, and the microservice log stream must be flowing. Eden
drives SSH connections and waits for sshd's "Disconnected from"
message; that trigger was tried here and does not work -- the sessions
are established but no matching entry is ever uploaded. This test
instead emits a per-run marker to /dev/kmsg, the path EVE's own
ssh-service.sh uses with the comment "this is picked up by newlogd",
which makes the trigger deterministic and independent of third-party
log wording. The kernel log path is gated by debug.kernel.*.loglevel,
separate from the debug.default.* levels the framework sets, so the
test states those explicitly.
Fields that EVE populates conditionally are waited for inside the
Eventually predicate rather than asserted on whichever message arrived
first: DevicePort.Err is never nil (zedagent encodes port test results
into it unconditionally, so "no error" means an empty description), and
DeviceMemory/CpuMetric.TotalNs only appear once domainmgr has published
the host DomainMetric.
All three are registered in TestTelemetrySuite and state identical
device requirements so the framework reuses a single VM across the
suite.
Signed-off-by: Renê de Souza Pinto <rene@renesp.com.br>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
milan-zededa
left a comment
There was a problem hiding this comment.
My comments were addressed
|
Rebased onto master (#6244) and addressed all review comments. All six tests pass on amd64/KVM against EVE
The telemetry three were verified before the rebase (only their comments changed since); the app three were re-run after it. Two caveats:
Note for anyone running evetest against current master: the VERSION bump to 1.1 means |
Description
Ports six Eden smoke tests to the evetest framework, in two new test packages
plus one new test app:
evetest/tests/apps/(TestApplicationSuite) — fromtests/eclient/testdata/:TestAppInstanceMetadata(metadata.txt) — app POSTs to the link-localmetadata server; EVE must report it as
ZInfoAppInstMetaData. Also coversthe
appCustomStatusandhostnameendpoints.TestAppUserData(userdata.txt) — plainkey=valueuser-data must reachthe container as environment variables;
#cloud-configwrite_filesmust beapplied once per user-data version and survive an app restart.
TestAppLogs(app_logs.txt) — app stdout must reach the controller, bothat container creation and continuously, and resume after stop/start.
evetest/tests/telemetry/(TestTelemetrySuite) — fromtests/lim/testdata/:TestDeviceInfo(info_test.txt) —ZInfoDevicereports the applied portconfig, a plausible hardware inventory, and the HSM state.
TestDeviceMetrics(metric_test.txt) —DeviceMetricreports movingper-port network counters plus memory/CPU/controller-connectivity counters.
TestDeviceLogs(log_test.txt) — a message emitted on the device reachesthe controller, and the microservice log stream is flowing.
evetest/testapps/logger-ctr/— minimal container printing a startup bannerand numbered heartbeats. Needed because
evetest-ubuntu-ctrprints nothing, sothe container-creation log path cannot be exercised with it.
Two deliberate deviations from the Eden originals:
metric_testmatchesdm.network.iNameagainsteth0, butiNameis thelogical label from the controller config — it only equals
eth0becauseEden's device model labels the port that way. This asserts
iNameandlocalNameseparately.log_testdrives SSH connections and waits for sshd's"Disconnected from".That trigger was tried and does not work here: sessions are established but no
matching entry is ever uploaded. The test instead emits a per-run marker to
/dev/kmsg— the path EVE's ownssh-service.shuses — making the triggerdeterministic and independent of third-party log wording.
PR dependencies
TestAppLogspullslfedge/evetest-logger-ctr:1.0, which is published bypublish-evetest.ymlon push to master. Until that runs, the test needs theimage pushed to a reachable registry (see below).
How to test and validate this PR
All six pass on amd64/KVM against EVE
17.0.0-lts:Or individually via
make evetest NAME=<TestFunctionName>. Runtimes: 147–426 seach.
Before
evetest-logger-ctris published,TestAppLogsneeds a registry thedevice can reach:
Changelog notes
No user-facing changes.
PR Backports
Checklist
structure required by
evetest/README.md; the new test app is picked upby
testapps/README.md's documented convention with no changes neededarch-specific, and the test app is built for both platforms by CI