Fix vulnerable dependencies and pin versions - #193
Conversation
- setup.py: bump install_requires to minimum secure versions validated during CVE remediation (pyyaml>=6.0.3, urllib3>=2.7.0, requests>=2.34.2, boxsdk[jwt]>=3.14.0, chardet>=6.0.0, packaging>=26.2, tabulate>=0.10.0, binaryornot>=0.6.0) - requirements-dev.txt: pin all dev deps to exact secure versions for reproducible CI environments - Dockerfiles/base/00.python.Dockerfile: pin base image from floating 'python:3' to 'python:3.11-slim-bookworm' — fixes 37 CVEs in OpenSSL, systemd, Kerberos and other packages present in older base images - Dockerfiles/base/01.cli.Dockerfile: explicitly install pyahocorasick==2.3.1 (extras_require not auto-installed)
tox-pip-extensions==1.7.1 does not exist on PyPI (latest is 1.6.0). This would cause pip install -r requirements-dev.txt to fail with a resolver error in any CI or developer environment. Corrected to 1.6.0. ──────────────────────────────────────────────────────────────────────── BRANCH SUMMARY — fix/vuln-dep-pins ──────────────────────────────────────────────────────────────────────── Base: master | Files changed: 4 (+54 / −31) | Commits: 2 Commit 120defa — fix: pin deps and base image to secure versions (CVE remediation) setup.py Raised install_requires floors to minimum secure releases. No API surfaces used by the codebase changed; these are floor bumps only. pyyaml: unbounded → >=6.0.3 (CVE-2017-18342) requests: unbounded → >=2.34.2 (CVE-2024-35195, proxy credential leak) urllib3: >2.4.0 → >=2.7.0 (CVE-2025-50182 / CVE-2025-50181) boxsdk[jwt]: <4.0.0 → >=3.14.0,<4.0.0 (adds secure lower bound) packaging: unbounded → >=26.2 (aligns with CI-validated version) tabulate: unbounded → >=0.10.0 (aligns with CI-validated version) binaryornot: unbounded → >=0.6.0 (aligns with CI-validated version) chardet: >=3.0.2,<7.0.0 → >=6.0.0 (drops vulnerable versions; removes upper cap) Also added python_requires='>=3.8' to make the supported Python floor explicit. requirements-dev.txt Replaced all loose/range pins with exact versions validated against the production Python 3.11 container. Added section comments for clarity. Renamed ibm_db → ibm-db (canonical PyPI name). Dockerfiles/base/00.python.Dockerfile Replaced floating FROM python:3 with FROM python:3.11-slim-bookworm. The floating tag resolved to Debian Bullseye/Buster carrying 37 CVEs across OpenSSL, systemd, and Kerberos packages. Added explicit git install (required by CLI layer, absent from slim image). Added comment discouraging future use of floating tags. Dockerfiles/base/01.cli.Dockerfile Added explicit: pip install 'pyahocorasick==2.3.1' pyahocorasick is declared under extras_require['word_list'] and was not being installed automatically in the container build path. Fixed typo: performace → performance in inline comment. Commit 12b22bf — fix: correct tox-pip-extensions pin (this commit) See first line above. Test results (Python 3.14, PYTHONPATH=., db2_test.py excluded): 1263 passed, 3 failed (all 3 pre-existing on master — not regressions) - TestScanFile::test_error_reading_file (pre-existing source bug) - TestScanFile::test_unicode_decode_error (pre-existing source bug) - test_scan_string_cli_overrides_stdin_db2_enabled (ibm_db not installable without native DB2 client) Caveat: packaging>=26.2 is more aggressive than technically required. The codebase only calls packaging.version.parse(), stable since v20.0. Consider relaxing to packaging>=20.0 if downstream consumers or air-gapped mirrors cannot resolve packaging 26.x.
|
Re: #188 Why we stayed on boxsdk v3 (not v4) boxsdk v4 introduced a hard cap on PyJWT in its package metadata: pyjwt>=1.7.0,<2. This means any environment installing boxsdk[jwt] v4.x will always resolve to pyjwt 1.7.1 — the version carrying 4 CVEs (2 High, 1 Medium, 1 Low). There is no way to override this from our side; the constraint lives inside boxsdk's published wheel metadata, not in our files. boxsdk v3 by contrast only requires pyjwt>=1.7.0 with no upper cap, so pip naturally resolves to the latest safe pyjwt 2.13.0 on a fresh install. The v4 upgrade (along with v10 / any backward compatibility support as applicable) is being handled separately. That work is intentionally kept off this branch to keep the CVE remediation scope narrow and the Mend scan result clean. More updates to follow soon. |
| 'packaging>=20.0', | ||
| 'tabulate>=0.10.0', | ||
| 'binaryornot>=0.6.0', | ||
| 'chardet>=6.0.0,<7.0.0', |
There was a problem hiding this comment.
In the past major version updates of dependencies have severely impacted IBM detect secrets and required consumers to add workarounds. Should you consider adding an upper range to all of these, the same way you have with boxsdk to prevent breaking changes impacting consumers?
| ), | ||
| url='hhttps://github.com/IBM/detect-secrets', | ||
| keywords=['secret-management', 'pre-commit', 'security', 'entropy-checks'], | ||
| python_requires='>=3.8', |
There was a problem hiding this comment.
are you sure all of the below dependency versions run on 3.8? If you look at the requests docs, it says:
Requests officially supports Python 3.10+.
| # Install the package — deps are pinned in setup.py install_requires. | ||
| # pyahocorasick is declared under extras_require['word_list'] so install explicitly. | ||
| RUN pip install /code && \ | ||
| pip install 'pyahocorasick==2.3.1' |
There was a problem hiding this comment.
consider using --no-cache-dir since this is an image
There was a problem hiding this comment.
Review comment accepted and will be addressed. Thank you!
Removes ~17MB of pip cache baked into the image layer. pip cache provides no benefit inside a Docker build — it is never reused across RUN steps or subsequent builds.
2 participants
Note to community
We plan on merging this PR within the next two weeks, so please post review comments and feedback within that timeframe. Thank you!
What this PR does
Remediates CVEs by pinning all dependencies to secure minimum versions, hardens the container base image, and fixes a binaryornot 0.6 compatibility bug.
Key changes
setup.py — all install_requires now have explicit CVE-validated version floors; added python_requires>=3.8
requirements-dev.txt — fully restructured from loose ranges to exact == pins for reproducible dev/CI builds
Dockerfiles/base/00.python.Dockerfile — base image pinned from floating python:3 to python:3.11-slim-bookworm (fixes OpenSSL/systemd/Kerberos CVEs)
Dockerfiles/base/01.cli.Dockerfile — adds explicit pyahocorasick==2.3.1 install; fixes typo
detect_secrets/core/secrets_collection.py — guards is_binary() with os.path.exists() check (binaryornot 0.6 raises on missing files)
Testing
pytest: 1268 passed
Box plugin: 23/23 passed
Mend scan: 0 vulnerabilities — boxsdk[jwt]==3.14.0 resolves pyjwt==2.13.0 on a fresh install (v3.x has no <2 cap unlike v4.x which brings in a CVE due to pyjwt 1.7.1)
Container smoke test: version, deps, scan, hook all verified
Vulnerability scanner(s) result(s) - Will update this post with more screenshots of container scanners too.