Skip to content

Fix 32-bit zstdmt max job size allocation failure#4675

Open
danyalahmed1995 wants to merge 1 commit into
facebook:devfrom
danyalahmed1995:fix/4628-zstdmt-32bit-job-size
Open

Fix 32-bit zstdmt max job size allocation failure#4675
danyalahmed1995 wants to merge 1 commit into
facebook:devfrom
danyalahmed1995:fix/4628-zstdmt-32bit-job-size

Conversation

@danyalahmed1995
Copy link
Copy Markdown

Fixes #4628.

Summary

Reduces the 32-bit zstdmt maximum job size from 512 MB / log 29 to 256 MB / log 28.

On 32-bit builds, the previous bound could allow a threaded compression configuration at max compression level to attempt an oversized allocation, matching the failure described in #4628.

This follows the issue reporter's diagnosis and proposed cap reduction.

Changes

  • Lower ZSTDMT_JOBLOG_MAX on 32-bit builds from 29 to 28.
  • Lower ZSTDMT_JOBSIZE_MAX on 32-bit builds from 512 MB to 256 MB.
  • Add a bounds check in the fuzzer unit tests so ZSTD_c_jobSize reports the expected architecture-specific upper bound.

Validation

  • git diff --check
  • Verified with a temporary GitHub Actions workflow on linux/386 using Debian testing:
    • make clean
    • make -j$(nproc)
    • make check

The temporary validation workflow was kept out of this PR branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Compression fails on linux/i386 with ZSTD_maxCLevel + ZSTD_c_nbWorkers

1 participant