Skip to content

[TC-GC-2.8] Validate GroupcastTesting event in Python test#72737

Merged
mergify[bot] merged 5 commits into
project-chip:masterfrom
jmartinez-silabs:tc-gc-2.8-groupcast-event
Jun 26, 2026
Merged

[TC-GC-2.8] Validate GroupcastTesting event in Python test#72737
mergify[bot] merged 5 commits into
project-chip:masterfrom
jmartinez-silabs:tc-gc-2.8-groupcast-event

Conversation

@jmartinez-silabs

Copy link
Copy Markdown
Member

Summary

Problem
  • TC_GC_2_8.py only validates the FabricUnderTest attribute side effect of the GroupcastTesting command. The GroupcastTesting event itself is never validated, so a mandatory spec behavior is untested.
Solution
  • Expand TC_GC_2_8.py to validate the GroupcastTesting event through the Listener flow, interleaved with the existing FabricUnderTest attribute checks.
  • The test joins a group, enables Listener testing, sends group commands, and asserts the event fields for four outcomes:
    • Success / AccessAllowed=true: key present and an ACL grants the group Operate access.
    • NoAvailableKey (unknown group): command sent to a different GroupID the DUT never joined / has no key for.
    • Success / AccessAllowed=false: ACL grant revoked, so the message is processed but Access Control denies invocation.
    • NoAvailableKey (cleared key): GroupKeyMap cleared so no key is available to decrypt.
  • Reuses helpers in TC_GC_common.py (get_feature_map, get_operate_only_commands, get_iana_multicast_address, generate_fabric_under_test_matcher).
  • Related testplan PR: https://github.com/CHIP-Specifications/chip-test-plans/pull/6223
Caveats
  • For SD-only DUTs the event steps are skipped: a DUT-originated (Sender) groupcast cannot be generically triggered by the TH. Only the FabricUnderTest attribute behavior is validated in that case.

Related issues

  • CHIP-Specifications/chip-test-plans#5974.

Testing

  • Test TC_GC_2_8.py, run against all-devices-app --device on-off-light:1 --groupcast.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request expands the TC_GC_2_8.py test script to include comprehensive testing steps and logic for Groupcast Listener nodes, covering group membership, access control, and decryption failure scenarios. The review feedback highlights two issues: an undefined logger variable on line 171 that will cause a NameError (suggesting logging.info instead), and an incorrect step range in mark_step_range_skipped(5, 17) on line 307, which should be (5, 16) since step 17 is meant to be executed.

Comment thread src/python_testing/TC_GC_2_8.py
Comment thread src/python_testing/TC_GC_2_8.py Outdated
@github-actions

github-actions Bot commented Jun 25, 2026

Copy link
Copy Markdown

PR #72737: Size comparison from e2077a4 to 675223b

Full report (30 builds for cc13x4_26x4, cc32xx, efr32, esp32, nrfconnect, psoc6, qpg, realtek, stm32, telink)
platform target config section e2077a4 675223b change % change
cc13x4_26x4 lighting-app LP_EM_CC1354P10_6 FLASH 777368 777368 0 0.0
RAM 103404 103404 0 0.0
lock-ftd LP_EM_CC1354P10_6 FLASH 790120 790120 0 0.0
RAM 108684 108684 0 0.0
pump-app LP_EM_CC1354P10_6 FLASH 739376 739376 0 0.0
RAM 97612 97612 0 0.0
pump-controller-app LP_EM_CC1354P10_6 FLASH 719548 719548 0 0.0
RAM 97644 97644 0 0.0
cc32xx air-purifier CC3235SF_LAUNCHXL FLASH 569654 569654 0 0.0
RAM 205112 205112 0 0.0
lock CC3235SF_LAUNCHXL FLASH 597214 597214 0 0.0
RAM 205272 205272 0 0.0
efr32 lighting-app BRD4187C FLASH 1094924 1094924 0 0.0
RAM 135256 135256 0 0.0
lock-app BRD4187C FLASH 995184 995184 0 0.0
RAM 131292 131292 0 0.0
BRD4338a FLASH 799809 799809 0 0.0
RAM 243432 243432 0 0.0
esp32 all-clusters-app c3devkit DRAM 99556 99556 0 0.0
FLASH 1626146 1626146 0 0.0
IRAM 94776 94776 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 FLASH 844772 844772 0 0.0
RAM 157771 157771 0 0.0
psoc6 all-clusters cy8ckit_062s2_43012 FLASH 1750756 1750756 0 0.0
RAM 215492 215492 0 0.0
all-clusters-minimal cy8ckit_062s2_43012 FLASH 1626548 1626548 0 0.0
RAM 211604 211604 0 0.0
light cy8ckit_062s2_43012 FLASH 1470860 1470860 0 0.0
RAM 197436 197436 0 0.0
lock cy8ckit_062s2_43012 FLASH 1504308 1504308 0 0.0
RAM 225268 225268 0 0.0
qpg lighting-app qpg6200+debug FLASH 843156 843156 0 0.0
RAM 127908 127908 0 0.0
lock-app qpg6200+debug FLASH 782976 782976 0 0.0
RAM 118840 118840 0 0.0
realtek light-switch-app rtl8777g FLASH 689368 689368 0 0.0
RAM 101780 101780 0 0.0
lighting-app rtl8777g FLASH 730304 730304 0 0.0
RAM 102052 102052 0 0.0
stm32 light STM32WB5MM-DK FLASH 478976 478976 0 0.0
RAM 141492 141492 0 0.0
telink all-devices-app tl7218x FLASH 881716 881716 0 0.0
RAM 99716 99716 0 0.0
tlsr9118bdk40d FLASH 673322 673322 0 0.0
RAM 120848 120848 0 0.0
bridge-app tl7218x FLASH 734156 734156 0 0.0
RAM 97700 97700 0 0.0
light-app-ota-compress-lzma-factory-data tl3218x FLASH 800682 800682 0 0.0
RAM 42380 42380 0 0.0
light-app-ota-compress-lzma-shell-factory-data tl7218x FLASH 845822 845822 0 0.0
RAM 101492 101492 0 0.0
light-switch-app-ota-compress-lzma-factory-data tl7218x_retention FLASH 734714 734714 0 0.0
RAM 57824 57824 0 0.0
light-switch-app-ota-compress-lzma-shell-factory-data tlsr9528a FLASH 795802 795802 0 0.0
RAM 75176 75176 0 0.0
light-switch-app-ota-factory-data tl3218x_retention FLASH 734630 734630 0 0.0
RAM 34480 34480 0 0.0
lighting-app-ota-factory-data tlsr9118bdk40d FLASH 615214 615214 0 0.0
RAM 118508 118508 0 0.0
lighting-app-ota-rpc-factory-data-4mb tlsr9518adk80d FLASH 842038 842042 4 0.0
RAM 97376 97376 0 0.0

@codecov

codecov Bot commented Jun 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 56.61%. Comparing base (44fb474) to head (7e2cde3).
⚠️ Report is 35 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #72737      +/-   ##
==========================================
- Coverage   56.76%   56.61%   -0.15%     
==========================================
  Files        1634     1642       +8     
  Lines      112660   113137     +477     
  Branches    13144    13243      +99     
==========================================
+ Hits        63946    64048     +102     
- Misses      48714    49089     +375     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions Bot added the tests label Jun 25, 2026
Comment thread src/python_testing/TC_GC_2_8.py Outdated
Comment thread src/python_testing/TC_GC_2_8.py
@github-actions

github-actions Bot commented Jun 25, 2026

Copy link
Copy Markdown

PR #72737: Size comparison from e2077a4 to 1286580

Full report (33 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, efr32, esp32, nrfconnect, psoc6, qpg, realtek, stm32, telink)
platform target config section e2077a4 1286580 change % change
bl602 lighting-app bl602+mfd+littlefs+rpc FLASH 1099176 1099176 0 0.0
RAM 133418 133418 0 0.0
bl702 lighting-app bl702+eth FLASH 1085726 1085726 0 0.0
RAM 109029 109029 0 0.0
bl702l contact-sensor-app bl702l+mfd+littlefs FLASH 882218 882218 0 0.0
RAM 108596 108596 0 0.0
cc13x4_26x4 lighting-app LP_EM_CC1354P10_6 FLASH 777368 777368 0 0.0
RAM 103404 103404 0 0.0
lock-ftd LP_EM_CC1354P10_6 FLASH 790120 790120 0 0.0
RAM 108684 108684 0 0.0
pump-app LP_EM_CC1354P10_6 FLASH 739376 739376 0 0.0
RAM 97612 97612 0 0.0
pump-controller-app LP_EM_CC1354P10_6 FLASH 719548 719548 0 0.0
RAM 97644 97644 0 0.0
cc32xx air-purifier CC3235SF_LAUNCHXL FLASH 569654 569654 0 0.0
RAM 205112 205112 0 0.0
lock CC3235SF_LAUNCHXL FLASH 597214 597214 0 0.0
RAM 205272 205272 0 0.0
efr32 lighting-app BRD4187C FLASH 1094924 1094924 0 0.0
RAM 135256 135256 0 0.0
lock-app BRD4187C FLASH 995184 995184 0 0.0
RAM 131292 131292 0 0.0
BRD4338a FLASH 799809 799809 0 0.0
RAM 243432 243432 0 0.0
esp32 all-clusters-app c3devkit DRAM 99556 99556 0 0.0
FLASH 1626146 1626146 0 0.0
IRAM 94776 94776 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 FLASH 844772 844772 0 0.0
RAM 157771 157771 0 0.0
psoc6 all-clusters cy8ckit_062s2_43012 FLASH 1750756 1750756 0 0.0
RAM 215492 215492 0 0.0
all-clusters-minimal cy8ckit_062s2_43012 FLASH 1626548 1626548 0 0.0
RAM 211604 211604 0 0.0
light cy8ckit_062s2_43012 FLASH 1470860 1470860 0 0.0
RAM 197436 197436 0 0.0
lock cy8ckit_062s2_43012 FLASH 1504308 1504308 0 0.0
RAM 225268 225268 0 0.0
qpg lighting-app qpg6200+debug FLASH 843156 843156 0 0.0
RAM 127908 127908 0 0.0
lock-app qpg6200+debug FLASH 782976 782976 0 0.0
RAM 118840 118840 0 0.0
realtek light-switch-app rtl8777g FLASH 689368 689368 0 0.0
RAM 101780 101780 0 0.0
lighting-app rtl8777g FLASH 730304 730304 0 0.0
RAM 102052 102052 0 0.0
stm32 light STM32WB5MM-DK FLASH 478976 478976 0 0.0
RAM 141492 141492 0 0.0
telink all-devices-app tl7218x FLASH 881716 881716 0 0.0
RAM 99716 99716 0 0.0
tlsr9118bdk40d FLASH 673322 673322 0 0.0
RAM 120848 120848 0 0.0
bridge-app tl7218x FLASH 734156 734156 0 0.0
RAM 97700 97700 0 0.0
light-app-ota-compress-lzma-factory-data tl3218x FLASH 800682 800682 0 0.0
RAM 42380 42380 0 0.0
light-app-ota-compress-lzma-shell-factory-data tl7218x FLASH 845822 845822 0 0.0
RAM 101492 101492 0 0.0
light-switch-app-ota-compress-lzma-factory-data tl7218x_retention FLASH 734714 734714 0 0.0
RAM 57824 57824 0 0.0
light-switch-app-ota-compress-lzma-shell-factory-data tlsr9528a FLASH 795802 795802 0 0.0
RAM 75176 75176 0 0.0
light-switch-app-ota-factory-data tl3218x_retention FLASH 734630 734630 0 0.0
RAM 34480 34480 0 0.0
lighting-app-ota-factory-data tlsr9118bdk40d FLASH 615214 615214 0 0.0
RAM 118508 118508 0 0.0
lighting-app-ota-rpc-factory-data-4mb tlsr9518adk80d FLASH 842038 842042 4 0.0
RAM 97376 97376 0 0.0

jmartinez-silabs and others added 2 commits June 26, 2026 13:59
…th the provided epochkey. Use those helper in GC_2-8 step 13 to exercise the FailAuth path
@github-actions

github-actions Bot commented Jun 26, 2026

Copy link
Copy Markdown

PR #72737: Size comparison from e2077a4 to 59af9c9

Full report (33 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, efr32, esp32, nrfconnect, psoc6, qpg, realtek, stm32, telink)
platform target config section e2077a4 59af9c9 change % change
bl602 lighting-app bl602+mfd+littlefs+rpc FLASH 1099176 1099194 18 0.0
RAM 133418 133418 0 0.0
bl702 lighting-app bl702+eth FLASH 1085726 1085744 18 0.0
RAM 109029 109029 0 0.0
bl702l contact-sensor-app bl702l+mfd+littlefs FLASH 882218 882236 18 0.0
RAM 108596 108596 0 0.0
cc13x4_26x4 lighting-app LP_EM_CC1354P10_6 FLASH 777368 777384 16 0.0
RAM 103404 103404 0 0.0
lock-ftd LP_EM_CC1354P10_6 FLASH 790120 790136 16 0.0
RAM 108684 108684 0 0.0
pump-app LP_EM_CC1354P10_6 FLASH 739376 739392 16 0.0
RAM 97612 97612 0 0.0
pump-controller-app LP_EM_CC1354P10_6 FLASH 719548 719564 16 0.0
RAM 97644 97644 0 0.0
cc32xx air-purifier CC3235SF_LAUNCHXL FLASH 569654 569670 16 0.0
RAM 205112 205112 0 0.0
lock CC3235SF_LAUNCHXL FLASH 597214 597230 16 0.0
RAM 205272 205272 0 0.0
efr32 lighting-app BRD4187C FLASH 1094924 1094956 32 0.0
RAM 135256 135256 0 0.0
lock-app BRD4187C FLASH 995184 995184 0 0.0
RAM 131292 131292 0 0.0
BRD4338a FLASH 799809 799825 16 0.0
RAM 243432 243432 0 0.0
esp32 all-clusters-app c3devkit DRAM 99556 99556 0 0.0
FLASH 1626146 1626160 14 0.0
IRAM 94776 94776 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 FLASH 844772 844784 12 0.0
RAM 157771 157771 0 0.0
psoc6 all-clusters cy8ckit_062s2_43012 FLASH 1750756 1750772 16 0.0
RAM 215492 215492 0 0.0
all-clusters-minimal cy8ckit_062s2_43012 FLASH 1626548 1626564 16 0.0
RAM 211604 211604 0 0.0
light cy8ckit_062s2_43012 FLASH 1470860 1470876 16 0.0
RAM 197436 197436 0 0.0
lock cy8ckit_062s2_43012 FLASH 1504308 1504324 16 0.0
RAM 225268 225268 0 0.0
qpg lighting-app qpg6200+debug FLASH 843156 843172 16 0.0
RAM 127908 127908 0 0.0
lock-app qpg6200+debug FLASH 782976 783008 32 0.0
RAM 118840 118840 0 0.0
realtek light-switch-app rtl8777g FLASH 689368 689392 24 0.0
RAM 101780 101780 0 0.0
lighting-app rtl8777g FLASH 730304 730320 16 0.0
RAM 102052 102052 0 0.0
stm32 light STM32WB5MM-DK FLASH 478976 478996 20 0.0
RAM 141492 141492 0 0.0
telink all-devices-app tl7218x FLASH 881716 881732 16 0.0
RAM 99716 99716 0 0.0
tlsr9118bdk40d FLASH 673322 673338 16 0.0
RAM 120848 120848 0 0.0
bridge-app tl7218x FLASH 734156 734172 16 0.0
RAM 97700 97700 0 0.0
light-app-ota-compress-lzma-factory-data tl3218x FLASH 800682 800698 16 0.0
RAM 42380 42380 0 0.0
light-app-ota-compress-lzma-shell-factory-data tl7218x FLASH 845822 845838 16 0.0
RAM 101492 101492 0 0.0
light-switch-app-ota-compress-lzma-factory-data tl7218x_retention FLASH 734714 734730 16 0.0
RAM 57824 57824 0 0.0
light-switch-app-ota-compress-lzma-shell-factory-data tlsr9528a FLASH 795802 795818 16 0.0
RAM 75176 75176 0 0.0
light-switch-app-ota-factory-data tl3218x_retention FLASH 734630 734646 16 0.0
RAM 34480 34480 0 0.0
lighting-app-ota-factory-data tlsr9118bdk40d FLASH 615214 615230 16 0.0
RAM 118508 118508 0 0.0
lighting-app-ota-rpc-factory-data-4mb tlsr9518adk80d FLASH 842038 842058 20 0.0
RAM 97376 97376 0 0.0

Comment thread src/python_testing/TC_GC_2_8.py
Comment thread src/python_testing/TC_GC_2_8.py Outdated
Comment thread src/python_testing/TC_GC_common.py Outdated
Comment thread src/python_testing/TC_GC_common.py Outdated
Comment thread src/python_testing/TC_GC_common.py Outdated
@github-actions

github-actions Bot commented Jun 26, 2026

Copy link
Copy Markdown

PR #72737: Size comparison from e2077a4 to 7e2cde3

Full report (33 builds for bl602, bl702, bl702l, cc13x4_26x4, cc32xx, efr32, esp32, nrfconnect, psoc6, qpg, realtek, stm32, telink)
platform target config section e2077a4 7e2cde3 change % change
bl602 lighting-app bl602+mfd+littlefs+rpc FLASH 1099176 1099194 18 0.0
RAM 133418 133418 0 0.0
bl702 lighting-app bl702+eth FLASH 1085726 1085744 18 0.0
RAM 109029 109029 0 0.0
bl702l contact-sensor-app bl702l+mfd+littlefs FLASH 882218 882236 18 0.0
RAM 108596 108596 0 0.0
cc13x4_26x4 lighting-app LP_EM_CC1354P10_6 FLASH 777368 777384 16 0.0
RAM 103404 103404 0 0.0
lock-ftd LP_EM_CC1354P10_6 FLASH 790120 790136 16 0.0
RAM 108684 108684 0 0.0
pump-app LP_EM_CC1354P10_6 FLASH 739376 739392 16 0.0
RAM 97612 97612 0 0.0
pump-controller-app LP_EM_CC1354P10_6 FLASH 719548 719564 16 0.0
RAM 97644 97644 0 0.0
cc32xx air-purifier CC3235SF_LAUNCHXL FLASH 569654 569670 16 0.0
RAM 205112 205112 0 0.0
lock CC3235SF_LAUNCHXL FLASH 597214 597230 16 0.0
RAM 205272 205272 0 0.0
efr32 lighting-app BRD4187C FLASH 1094924 1094956 32 0.0
RAM 135256 135256 0 0.0
lock-app BRD4187C FLASH 995184 995184 0 0.0
RAM 131292 131292 0 0.0
BRD4338a FLASH 799809 799825 16 0.0
RAM 243432 243432 0 0.0
esp32 all-clusters-app c3devkit DRAM 99556 99556 0 0.0
FLASH 1626146 1626306 160 0.0
IRAM 94776 94776 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 FLASH 844772 845892 1120 0.1
RAM 157771 157923 152 0.1
psoc6 all-clusters cy8ckit_062s2_43012 FLASH 1750756 1752716 1960 0.1
RAM 215492 215644 152 0.1
all-clusters-minimal cy8ckit_062s2_43012 FLASH 1626548 1626564 16 0.0
RAM 211604 211604 0 0.0
light cy8ckit_062s2_43012 FLASH 1470860 1470876 16 0.0
RAM 197436 197436 0 0.0
lock cy8ckit_062s2_43012 FLASH 1504308 1504324 16 0.0
RAM 225268 225268 0 0.0
qpg lighting-app qpg6200+debug FLASH 843156 843172 16 0.0
RAM 127908 127908 0 0.0
lock-app qpg6200+debug FLASH 782976 783008 32 0.0
RAM 118840 118840 0 0.0
realtek light-switch-app rtl8777g FLASH 689368 689392 24 0.0
RAM 101780 101780 0 0.0
lighting-app rtl8777g FLASH 730304 730320 16 0.0
RAM 102052 102052 0 0.0
stm32 light STM32WB5MM-DK FLASH 478976 478996 20 0.0
RAM 141492 141492 0 0.0
telink all-devices-app tl7218x FLASH 881716 881732 16 0.0
RAM 99716 99716 0 0.0
tlsr9118bdk40d FLASH 673322 673338 16 0.0
RAM 120848 120848 0 0.0
bridge-app tl7218x FLASH 734156 734172 16 0.0
RAM 97700 97700 0 0.0
light-app-ota-compress-lzma-factory-data tl3218x FLASH 800682 800698 16 0.0
RAM 42380 42380 0 0.0
light-app-ota-compress-lzma-shell-factory-data tl7218x FLASH 845822 845838 16 0.0
RAM 101492 101492 0 0.0
light-switch-app-ota-compress-lzma-factory-data tl7218x_retention FLASH 734714 734730 16 0.0
RAM 57824 57824 0 0.0
light-switch-app-ota-compress-lzma-shell-factory-data tlsr9528a FLASH 795802 795818 16 0.0
RAM 75176 75176 0 0.0
light-switch-app-ota-factory-data tl3218x_retention FLASH 734630 734646 16 0.0
RAM 34480 34480 0 0.0
lighting-app-ota-factory-data tlsr9118bdk40d FLASH 615214 615230 16 0.0
RAM 118508 118508 0 0.0
lighting-app-ota-rpc-factory-data-4mb tlsr9518adk80d FLASH 842038 842058 20 0.0
RAM 97376 97376 0 0.0

@mergify mergify Bot merged commit 50e2653 into project-chip:master Jun 26, 2026
79 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants