Skip to content

Commit 4ef8273

Browse files
authored
Enabled pinned USBX regression and merged coverage on dev (#280)
* Enabled pinned USBX regression and merged coverage on dev Dev pull requests did not run the USBX regression gate, dependencies floated, and coverage represented one filtered configuration. GCC 14 also rejected pointer types and missing declarations in five test sources and the sample. Pinned compatible dependencies and the reviewed ThreadX workflow/bootstrap, enabled all 17 profiles on every dev PR, and merged unfiltered raw coverage with exact source-line union checks. Added descriptor-size regressions, failure-path runner tests, safe manual subsets and Ubuntu 24.04 setup. Pages deployment remains limited to successful full master runs. The check stays auto_tests / run_tests; eclipse-threadx/.eclipsefdn#18 remains disabled pending rollout to dev, so passing CI is not yet an enforced merge requirement. All 17 GCC 14 CMake/Ninja configurations built; 3,043/3,043 CTest cases passed. The exact coverage union measured 66.3244% lines and 55.0718% branches. Eight runner tests, real failure/subset probes, actionlint, ShellCheck and whitespace checks passed. * Excluded reusable ThreadX workflow updates from Dependabot Dependabot identifies reusable workflows by repository and workflow path. A repository-only ignore entry would allow automatic changes to the workflow pin. Matched the full dependency name so workflow and bootstrap revisions retain explicit review together. Verified the name against Dependabot's workflow parser and both workflow calls. Whitespace checks passed. Assisted-by: Codex (GPT-6-Sol) <noreply@openai.com>
1 parent 15ee7cd commit 4ef8273

24 files changed

Lines changed: 876 additions & 189 deletions

‎.github/dependabot.yml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
version: 2
2+
updates:
3+
- package-ecosystem: github-actions
4+
directory: /
5+
target-branch: dev
6+
schedule:
7+
interval: weekly
8+
ignore:
9+
# The cross-repository bootstrap and workflow are reviewed together.
10+
- dependency-name: eclipse-threadx/threadx/.github/workflows/regression_template.yml
Lines changed: 23 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,46 +1,47 @@
11
name: USBX Regression Test
22

3-
# Controls when the action will run. Triggers the workflow on push or pull request
4-
# events but only for the master branch
53
on:
64
workflow_dispatch:
75
inputs:
86
tests_to_run:
9-
description: 'all, single or multiple of default_build_coverage error_check_build_full_coverage tracex_enable_build device_buffer_owner_build device_zero_copy_build nofx_build_coverage optimized_build standalone_device_build_coverage standalone_device_buffer_owner_build standalone_device_zero_copy_build standalone_host_build_coverage standalone_build_coverage generic_build otg_support_build memory_management_build_coverage msrc_rtos_build msrc_standalone_build'
10-
required: false
7+
description: 'all, or space-separated profiles from test/cmake/usbx/CMakeLists.txt'
8+
required: true
119
default: 'all'
10+
type: string
1211
skip_coverage:
1312
required: false
1413
type: boolean
1514
default: false
16-
coverage_name:
17-
required: false
18-
default: 'default_build_coverage'
1915
push:
20-
branches: [ master ]
16+
branches: [dev, master]
2117
pull_request:
22-
branches: [ master ]
18+
branches: [dev, master]
2319

24-
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
25-
jobs:
20+
concurrency:
21+
group: usbx-regression-${{ github.event.pull_request.number || github.ref }}
22+
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
2623

24+
jobs:
2725
manual_tests:
2826
if: github.event_name == 'workflow_dispatch'
2927
permissions:
3028
contents: read
3129
issues: read
3230
checks: write
3331
pull-requests: write
32+
# GitHub validates deployment permissions even when that job is skipped.
33+
# The called test job narrows its own token scope.
3434
pages: write
3535
id-token: write
36-
37-
uses: eclipse-threadx/threadx/.github/workflows/regression_template.yml@master
36+
uses: eclipse-threadx/threadx/.github/workflows/regression_template.yml@b37cd4a81a1cb8c2ebefc438220ab7f009e13362
3837
with:
3938
cmake_path: ./test/cmake/usbx
40-
build_script: ./scripts/build.sh ${{ inputs.tests_to_run }}
41-
test_script: ./scripts/test.sh ${{ inputs.tests_to_run }}
42-
coverage_name: ${{ inputs.coverage_name }}
43-
skip_coverage: ${{ !!inputs.skip_coverage }}
39+
build_script: ./scripts/build.sh --manual
40+
test_script: ./scripts/test.sh --manual
41+
coverage_name: ${{ inputs.tests_to_run == 'all' && 'merged' || 'subset' }}
42+
coverage_thresholds: '0 100'
43+
skip_coverage: ${{ inputs.skip_coverage }}
44+
skip_deploy: ${{ github.ref != 'refs/heads/master' || inputs.tests_to_run != 'all' }}
4445

4546
auto_tests:
4647
if: github.event_name != 'workflow_dispatch'
@@ -49,13 +50,15 @@ jobs:
4950
issues: read
5051
checks: write
5152
pull-requests: write
53+
# The reusable workflow declares a separate deployment job.
5254
pages: write
5355
id-token: write
54-
55-
uses: eclipse-threadx/threadx/.github/workflows/regression_template.yml@master
56+
uses: eclipse-threadx/threadx/.github/workflows/regression_template.yml@b37cd4a81a1cb8c2ebefc438220ab7f009e13362
5657
with:
5758
cmake_path: ./test/cmake/usbx
5859
build_script: ./scripts/build.sh all
5960
test_script: ./scripts/test.sh all
60-
coverage_name: default_build_coverage
61+
coverage_name: merged
62+
coverage_thresholds: '66 100'
6163
skip_coverage: false
64+
skip_deploy: ${{ github.event_name != 'push' || github.ref != 'refs/heads/master' }}

‎.gitignore‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,3 +13,8 @@ compile_commands.json
1313
CTestTestfile.cmake
1414

1515
coverage_report/
16+
17+
/test/externals/
18+
/test/cmake/usbx/.run.sh
19+
/test/cmake/usbx/libs/
20+
__pycache__/

‎samples/demo_usbx.c‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,12 @@
88
/* */
99
/* SPDX-License-Identifier: MIT */
1010
/***************************************************************************/
11+
/* Portions of this file were generated with AI assistance. */
1112

1213
/* This is a small demo of the USBX */
1314

1415
#include "ux_api.h"
16+
#include "ux_device_stack.h"
1517
#include "ux_system.h"
1618
#include "ux_utility.h"
1719
#include "ux_host_class_dpump.h"

‎scripts/build.sh‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,4 +10,10 @@
1010
# SPDX-License-Identifier: MIT
1111
##############################################################################
1212

13-
$(dirname `realpath $0`)/../test/cmake/usbx/run.sh build $@
13+
set -euo pipefail
14+
export CC=${CC:-gcc-14}
15+
export GCOV=${GCOV:-gcov-14}
16+
export CMAKE_BUILD_PARALLEL_LEVEL=${CMAKE_BUILD_PARALLEL_LEVEL:-4}
17+
"$(dirname "$(realpath "$0")")/../test/cmake/usbx/run.sh" build "$@"
18+
echo "Checking runner and coverage failure handling in isolated fixtures."
19+
python3 "$(dirname "$(realpath "$0")")/../test/cmake/usbx/test_runner.py"

‎scripts/install.sh‎

Lines changed: 47 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -10,27 +10,50 @@
1010
# SPDX-License-Identifier: MIT
1111
##############################################################################
1212

13-
#
14-
# Install necessary softwares for Ubuntu.
15-
16-
sudo apt-get update
17-
sudo apt-get install -y \
18-
gcc-multilib \
19-
git \
20-
g++ \
21-
python3-pip \
22-
ninja-build \
23-
unifdef \
24-
p7zip-full \
25-
tofrodos \
26-
gawk \
27-
software-properties-common
28-
29-
wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | sudo apt-key add -
30-
CODENAME=$(lsb_release -c | cut -f2 -d':' | sed 's/\t//')
31-
sudo apt-add-repository -y "deb https://apt.kitware.com/ubuntu/ $CODENAME main"
32-
sudo apt-get -y install cmake
33-
34-
python3 -m pip install --upgrade pip
35-
pip3 install gcovr==4.1
36-
pip3 install --upgrade cmake
13+
set -euo pipefail
14+
15+
retry() {
16+
local attempt
17+
for attempt in 1 2 3; do
18+
if "$@"; then
19+
return 0
20+
fi
21+
if [ "$attempt" -lt 3 ]; then
22+
sleep $((attempt * 5))
23+
fi
24+
done
25+
return 1
26+
}
27+
28+
apt_options=(-o Acquire::Retries=3 -o DPkg::Lock::Timeout=60)
29+
if ! retry sudo timeout 150 apt-get "${apt_options[@]}" update; then
30+
echo "Package index update failed; package installation will verify availability." >&2
31+
fi
32+
retry sudo timeout 150 apt-get "${apt_options[@]}" install -y \
33+
cmake gcc-14 gcc-14-multilib git ninja-build python3-venv \
34+
unifdef p7zip-full tofrodos gawk
35+
36+
venv_dir="${RUNNER_TEMP:-${TMPDIR:-/tmp}}/usbx-ci-venv"
37+
python3 -m venv "$venv_dir"
38+
retry timeout 120 "$venv_dir/bin/python" -m pip install \
39+
--retries 3 --timeout 30 gcovr==8.6
40+
41+
cc="${CC:-gcc-14}"
42+
gcov="${GCOV:-gcov-14}"
43+
cc_version=$("$cc" -dumpfullversion)
44+
gcov_version=$("$gcov" --version | sed -n '1{s/.* \([0-9][0-9]*\.[0-9][0-9]*\(\.[0-9][0-9]*\)\?\).*/\1/p;}')
45+
if [ -z "$gcov_version" ] || [ "$cc_version" != "$gcov_version" ]; then
46+
echo "Compiler $cc and coverage tool $gcov have different versions." >&2
47+
exit 1
48+
fi
49+
50+
if [ -n "${GITHUB_ENV:-}" ]; then
51+
printf 'CC=%s\nGCOV=%s\n' "$cc" "$gcov" >> "$GITHUB_ENV"
52+
printf '%s\n' "$venv_dir/bin" >> "$GITHUB_PATH"
53+
fi
54+
55+
"$venv_dir/bin/gcovr" --version | head -1
56+
"$cc" --version | head -1
57+
"$gcov" --version | head -1
58+
59+
printf 'Coverage environment: %s\n' "$venv_dir/bin/activate"

‎scripts/test.sh‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,4 +10,10 @@
1010
# SPDX-License-Identifier: MIT
1111
##############################################################################
1212

13-
CTEST_PARALLEL_LEVEL=4 $(dirname `realpath $0`)/../test/cmake/usbx/run.sh test $@
13+
set -euo pipefail
14+
export CTEST_PARALLEL_LEVEL=${CTEST_PARALLEL_LEVEL:-4}
15+
export CTEST_REPEAT_FAIL=1
16+
export CC=${CC:-gcc-14}
17+
export GCOV=${GCOV:-gcov-14}
18+
export CMAKE_BUILD_PARALLEL_LEVEL=${CMAKE_BUILD_PARALLEL_LEVEL:-4}
19+
exec "$(dirname "$(realpath "$0")")/../test/cmake/usbx/run.sh" test "$@"

‎test/cmake/usbx/BASELINE.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# GCC 14 baseline
2+
3+
Measured at USBX commit `ffc0f79d951c716a03addcf7255020328e0d5b47`, with the
4+
dependency revisions in `dependencies.txt`, GCC/gcov 14.3.0 and gcovr 8.6.
5+
All 17 configurations were configured and built from clean trees with CMake and
6+
Ninja. CTest ran every configuration once, including configurations with build
7+
failures: 3,019/3,043 cases passed in 666.17 seconds of summed CTest wall time.
8+
9+
Twenty-three cases could not run because five regression sources had incompatible
10+
pointer arguments or missing declarations. The sample also lacked device-stack
11+
declarations. One standalone printer transfer assertion failed. These results
12+
are retained as failures; a subsequent isolated printer pass does not erase them.
13+
14+
Coverage below uses the existing profile-specific exclusions, so percentages are
15+
not comparable across profiles and cannot be averaged or merged. A dash means
16+
the profile was not instrumented. Standalone-device collection initially failed
17+
gcovr's 2^32 suspicious-hit heuristic; the same raw counters were then collected
18+
with the documented 2^40 threshold and unchanged exclusions. No tests were rerun
19+
to produce that coverage. Build times are Ninja execution spans and exclude
20+
configuration and dependency setup.
21+
22+
| Profile | Passed / tests | Build s | Test s | Lines hit / valid | Branches hit / valid |
23+
| --- | ---: | ---: | ---: | ---: | ---: |
24+
| default_build_coverage | 427/430 | 10.838 | 13.89 | 7701/8014 | 3420/3738 |
25+
| error_check_build_full_coverage | 425/430 | 11.190 | 13.53 | 14767/24084 | 6238/11250 |
26+
| tracex_enable_build | 427/430 | 11.390 | 13.48 | — | — |
27+
| device_buffer_owner_build | 427/430 | 11.371 | 11.18 | — | — |
28+
| device_zero_copy_build | 427/430 | 11.271 | 16.53 | — | — |
29+
| nofx_build_coverage | 46/46 | 6.778 | 5.57 | 2751/7955 | 993/3738 |
30+
| optimized_build | 129/129 | 6.893 | 3.46 | — | — |
31+
| standalone_device_build_coverage | 55/57 | 6.669 | 185.88 | 2625/3098 | 880/1279 |
32+
| standalone_device_buffer_owner_build | 56/57 | 6.782 | 185.39 | — | — |
33+
| standalone_device_zero_copy_build | 56/57 | 6.896 | 177.48 | — | — |
34+
| standalone_host_build_coverage | 44/44 | 6.633 | 20.78 | 4005/5255 | 1464/2289 |
35+
| standalone_build_coverage | 20/20 | 5.412 | 0.25 | 2011/8277 | 720/3567 |
36+
| generic_build | 1/1 | 6.047 | 0.01 | — | — |
37+
| otg_support_build | 427/430 | 10.614 | 11.31 | — | — |
38+
| memory_management_build_coverage | 11/11 | 6.619 | 4.63 | 4748/8069 | 1715/3780 |
39+
| msrc_rtos_build | 33/33 | 6.482 | 0.76 | — | — |
40+
| msrc_standalone_build | 8/8 | 5.069 | 2.04 | — | — |

‎test/cmake/usbx/CMakeLists.txt‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,8 @@ cmake_policy(SET CMP0077 NEW)
55

66
project(usbx_test LANGUAGES C)
77

8+
option(USBX_CI_COVERAGE "Instrument every selected configuration" "$ENV{TX_COVERAGE}")
9+
810
# Use customized ux_user.h
911
set(UX_USER_FILE ${CMAKE_CURRENT_SOURCE_DIR}/ux_user.h)
1012

@@ -215,7 +217,7 @@ else()
215217
message(STATUS "Building usbx BUILD_SHARED_LIBS: ${BUILD_SHARED_LIBS}")
216218
endif()
217219
if(NOT BUILD_SHARED_LIBS)
218-
if(CMAKE_BUILD_TYPE MATCHES ".*_coverage")
220+
if(USBX_CI_COVERAGE OR CMAKE_BUILD_TYPE MATCHES ".*_coverage")
219221
add_link_options(-fprofile-arcs)
220222
add_link_options(-lgcov)
221223
endif()
@@ -255,13 +257,17 @@ if(NOT (CMAKE_BUILD_TYPE MATCHES "standalone.*"))
255257
endif()
256258

257259
# Coverage
258-
if(CMAKE_BUILD_TYPE MATCHES ".*_coverage")
260+
if(USBX_CI_COVERAGE OR CMAKE_BUILD_TYPE MATCHES ".*_coverage")
259261
target_compile_options(usbx PRIVATE -fprofile-arcs -ftest-coverage)
260262
target_link_options(usbx PRIVATE -fprofile-arcs -ftest-coverage)
261263
endif()
262264

263265
# Build ThreadX library once
264-
execute_process(COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/run.sh build_libs)
266+
execute_process(COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/run.sh build_libs
267+
RESULT_VARIABLE dependency_status)
268+
if(NOT dependency_status EQUAL 0)
269+
message(FATAL_ERROR "Dependency build failed: ${dependency_status}")
270+
endif()
265271
add_custom_target(build_libs ALL COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/run.sh
266272
build_libs)
267273

‎test/cmake/usbx/README.md‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Linux regression and coverage
2+
3+
The regression workflow builds and tests all 17 profiles on every pull request to
4+
`dev` or `master`. Its check name is `auto_tests / run_tests`. Workflow
5+
success alone does not enforce merging: repository rules must separately require
6+
that check. The staged Otterdog ruleset must remain disabled until this workflow
7+
is available on `dev`.
8+
9+
Run `scripts/install.sh` on Ubuntu 24.04, then use GCC 14 and its matching gcov with
10+
CMake and Ninja:
11+
12+
```sh
13+
TX_COVERAGE=ON ./scripts/build.sh all
14+
TX_COVERAGE=ON ./scripts/test.sh all
15+
python3 test/cmake/usbx/test_runner.py
16+
```
17+
18+
The installer installs gcovr 8.6 in a virtual environment and exports its location
19+
through GitHub Actions environment files. Outside Actions, activate the reported
20+
virtual environment before running coverage. `CC` and `GCOV` can select matching
21+
compiler and coverage executables. The default build and test concurrency is four.
22+
CTest runs each case once; failing cases are not retried into a passing result.
23+
24+
`dependencies.txt` records immutable ThreadX, FileX and NetX Duo revisions. FileX
25+
PR #106 is open at the pinned revision; this setup does not assume it has merged.
26+
Existing dependency checkouts must be clean and match their pins. Remove an old
27+
checkout explicitly when updating a pin. The ThreadX bootstrap supplies JUnit
28+
output without runtime edits to dependency files.
29+
30+
Pass profile names instead of `all` to either script, or select them in the manual
31+
workflow. Build and test output names the exact selection. JUnit files, test logs
32+
and `build/results.txt` report actual counts and elapsed test times. The generic
33+
profile's single CTest case is a build-only placeholder, not a functional USB test.
34+
35+
Every selected profile is instrumented when `TX_COVERAGE=ON`, including the generic
36+
build. Raw JSON, Cobertura XML and HTML use one inclusion policy: all compiled C
37+
sources matching `common/*/src/*.c`. Host controllers, simulators, optional classes
38+
and untested source lines remain in the denominator. Dependency and test code are
39+
outside that policy. No profile-specific diagnostic exclusion affects the union.
40+
41+
Each input must have nonempty JSON, XML and HTML, measured source lines, and
42+
repository-relative paths. The merger checks that JSON and XML agree and that its
43+
measured and covered source-line sets equal the exact unions of its inputs.
44+
Reports and execution counters are cleared before each test selection. A subset
45+
cannot consume an earlier full run's reports or publish complete coverage.
46+
Available diagnostics and coverage survive failing tests; missing coverage itself
47+
fails the runner. Full-suite line and branch floors are in `coverage-floors.json`.
48+
Standalone polling produces more than six billion positive hits in a single
49+
profile. Direct GCC 14 output confirms balanced loop counts. Collection uses a
50+
2^40 suspicious-hit threshold instead of gcovr's default 2^32 heuristic; negative
51+
counts and parse errors remain fatal, and no lines are ignored to obtain a report.
52+
53+
Only successful complete runs on `master` can deploy Pages. Callers grant the
54+
permissions required for GitHub to validate the pinned reusable workflow's
55+
separate deployment job; its test job retains narrower permissions. The template
56+
bounds installation, build and test steps and uploads diagnostics on failure.
57+
The manual selector is parsed from the event file, never interpolated as shell
58+
code. All pull requests retain the full matrix; there is no conditional routing.
59+
60+
Dependabot targets `dev` once its configuration is present on the default branch.
61+
The cross-repository workflow/bootstrap revision requires a separate review and
62+
is excluded from routine action updates.

0 commit comments

Comments
 (0)