feat(pdf): adopt performance-oriented core-pdfminer.six fork

[KRRT7]

Summary

This migrates Unstructured’s PDF extraction path to core-pdfminer.six, a performance-oriented fork of pdfminer.six.

core-pdfminer.six preserves the parsing and extraction semantics of upstream pdfminer.six under the core_pdfminer_six namespace. It passes the same upstream test cases, with only the import namespace adjusted.

This PR:

• Adds core-pdfminer-six as a dependency.
• Migrates Unstructured’s PDF imports to core_pdfminer_six.
• Updates dependency checks, runtime patch targets, tests, and logging configuration.
• Removes the legacy PSBaseParser monkey patch because the corresponding fixes are already included in the fork.
• Leaves the existing pdfminer.six image-extra dependency unchanged for now.

Performance

Benchmarks against the upstream baseline across Python 3.10, 3.11, and 3.12 showed:

• 1.23× overall geometric-mean speedup
• 1.37× speedup on representative workloads
• 1.51× faster converters
• 1.42× faster parsing
• 1.31× faster layout processing
• Approximately 12–14% lower peak memory usage
• Improved scalability for deep cross-reference chains that exceed upstream recursion limits

Compatibility

The fork maintains upstream parsing and extraction behavior. The package namespace is intentionally distinct, allowing core-pdfminer.six and pdfminer.six to coexist in the same environment.

Testing

• The complete upstream pdfminer.six test suite passes against the fork.
• Ruff checks passed.
• 177 focused Unstructured PDF integration tests passed.

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	pypi/​core-pdfminer-six@​0.0.0

View full report

[KRRT7]

core-pdfminer.six retains the original project’s license, and I have no intention of changing it. I also plan to actively maintain the fork and continue improving its performance.

[KRRT7]

@badGarnet @cragwolfe for your review.

[cubic-dev-ai]

0 issues found across 2 files (changes from recent commits).

<sub>Shadow auto-approve: would auto-approve. Migrates to a performance-oriented fork of pdfminer.six while preserving semantics, removing a now-included monkey-patch. The change is bounded and clearly beneficial with no new exposure or tradeoffs.

Re-trigger cubic</sub>

[KRRT7]

Unstructured performance comparison

I benchmarked upstream main (d309caf8) against this branch (d10999b2) using Unstructured’s quick_partition_bench.py harness on Python 3.12.13.

PDF	main	PR branch	Speedup	Elements
a1977-backus-p21.pdf	59.73 ms	40.02 ms	1.49×	41
copy-protected.pdf	56.73 ms	45.38 ms	1.25×	32
reliance.pdf	149.05 ms	120.94 ms	1.23×	75
pdf-with-ocr-text.pdf	17.61 ms	10.51 ms	1.68×	20
Aggregate	70.78 ms	54.21 ms	1.31×	—

This is a 23.4% reduction in warm partition latency. Element counts matched exactly for every document.

Methodology:

• 30 measured warm runs per PDF after 3 warmups
• 120 samples per revision
• Same virtual environment and locked dependencies
• strategy="fast"

Cold process performance was effectively unchanged (7.027 seconds on main versus 7.013 seconds on this branch) because Unstructured process startup dominates that measurement.

The broader CI partition benchmark also passes.

[cubic-dev-ai]

All reported issues were addressed

<sub>Shadow auto-approve: would not auto-approve because issues were found.

Re-trigger cubic</sub>

[cubic-dev-ai]

0 issues found across 2 files (changes from recent commits).

<sub>Shadow auto-approve: would auto-approve. Bounded, mechanical migration to a compatible performance-oriented fork, preserving all semantics; only import paths and a now-included monkey-patch removal change.

Re-trigger cubic</sub>

[KRRT7]

@agarctfi