Skip to content

Fix tox diffcover#569

Merged
webknjaz merged 2 commits into
aio-libs:masterfrom
loqs:tox-diffcover
May 7, 2026
Merged

Fix tox diffcover#569
webknjaz merged 2 commits into
aio-libs:masterfrom
loqs:tox-diffcover

Conversation

@loqs
Copy link
Copy Markdown
Contributor

@loqs loqs commented Apr 10, 2026

What do these changes do?

Fix tox diffcover failure by:

  • set cov-report output location to the location expected by diffcover.
  • use --compare-branch master for diff-cover to match the project's branch.

Are there changes in behavior for the user?

No.

Related issue number

Related #544.

Checklist

  • I think the code is well written
  • Unit tests for the changes exist
  • tox testenvs have been executed in the following environments:
    • Linux (Ubuntu 18.04, Ubuntu 20.04, Arch): {py36,py37,py38,py39}-diffcov
    • Windows (7, 10): {py36,py37,py38,py39}-{nocov,cov,diffcov}
    • WSL 1.0 (Ubuntu 18.04): {py36,py37,py38,py39}-{nocov,cov,diffcov}, pypy3-{nocov,cov}, qa, docs
    • FreeBSD (12.2, 12.1, 11.4): {py36,pypy3}-{nocov,cov,diffcov}, qa
    • Cygwin: py36-{nocov,cov,diffcov}, qa, docs
  • Documentation reflects the changes
  • Add a news fragment into the NEWS.rst file

Comment thread tox.ini Outdated
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 24, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.84%. Comparing base (c8c4cd1) to head (0b7f43e).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #569      +/-   ##
==========================================
- Coverage   97.89%   97.84%   -0.06%     
==========================================
  Files          23       23              
  Lines        5701     5701              
  Branches      766      766              
==========================================
- Hits         5581     5578       -3     
- Misses         75       79       +4     
+ Partials       45       44       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@webknjaz webknjaz added tech-debt Things that needs to be tidied up to avoid being bitten in the future... testing Issues/PRs related to the test suite code-quality labels May 6, 2026
loqs added 2 commits May 6, 2026 20:12
cov-report uses current working directory as the default output
location.  Set two cov-report output locations; the default location
used by Codecov and _dump/coverage-{env:INTERP}.xml used by diffcover.
diff-cover changed its default comparison branch from `origin/master` to
`origin/main` in release 5.0.0.
@loqs loqs force-pushed the tox-diffcover branch from 0b92195 to 0b7f43e Compare May 6, 2026 19:21
@webknjaz
Copy link
Copy Markdown
Member

webknjaz commented May 7, 2026

There's 5 failed tests in the Windows job: https://github.com/aio-libs/aiosmtpd/actions/runs/25456153086/job/74686050166#step:5:589

This seems unrelated so I've restarted the CI. Though, it's probably worth investing and digging through the logs. If the tests are flaky, they should be stabilized.

@webknjaz
Copy link
Copy Markdown
Member

webknjaz commented May 7, 2026

Note: I realized just now that the CI is not calling tox but invokes pytest and coverage directly, bypassing some shared configuration. I think it'd be good to start integrating reusable-tox.yml in a follow-up.

@webknjaz webknjaz merged commit c579966 into aio-libs:master May 7, 2026
26 of 38 checks passed
@loqs
Copy link
Copy Markdown
Contributor Author

loqs commented May 7, 2026

Though, it's probably worth investing and digging through the logs. If the tests are flaky, they should be stabilized.

I looked through the log. The failing tests are appear to Windows no allowing the same port to be reused in quick succession while this is not triggered on the Linux. I do not have a Windows system to debug the issue. I did notice some tests are already skipped on Windows.

@loqs
Copy link
Copy Markdown
Contributor Author

loqs commented May 7, 2026

Note: I realized just now that the CI is not calling tox but invokes pytest and coverage directly, bypassing some shared configuration. I think it'd be good to start integrating reusable-tox.yml in a follow-up.

I was planning on getting all current tox environments passing first. I estimate about five more pull requests four singles and a multiple commit one covering flake8 and hopefully they should all be passing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

code-quality tech-debt Things that needs to be tidied up to avoid being bitten in the future... testing Issues/PRs related to the test suite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants