Skip to content

About

PDF layers for PDFium — the missing optional-content API, with tests, a wasm build, and a working layers panel in the browser, built in a fork of EmbedPDF..

Resources

Stars

1 star

Watchers

0 watching

Forks

Latest commit

 

History

27 Commits

Folders and files

Repository files navigation

PDF layers for PDFium

PDF has had layers since 2001 — optional content groups, in the specification's words. CAD exports, multilingual artwork, maps with switchable overlays and watermarked drafts all use them. Almost no viewer outside Acrobat lets you touch them.

PDFium is the reason why, and it is also the fix. It renders PDFs in Chrome, Edge, and most of the embedded viewers on the web, and it has understood optional content internally for years — it just never exposed it. There is no fpdf_ocg.h. An embedder cannot ask a document what layers it has, let alone turn one off.

This repository adds that API and demonstrates it end to end: a eight-patch series against upstream PDFium, a port onto EmbedPDF's WebAssembly build, and a working Layers panel you can click through in a browser.

Toggling layers in the panel: Buildings vanishes and returns, Grid switches off, and ticking Labels (French) switches Labels (English) off automatically — the document's own radio pair

Buildings vanishes and returns, Grid switches off, and ticking Labels (French) switches English off by itself — the document declares the two a /RBGroups radio pair, and the panel honours it. The headings — SITE, SERVICES, ANNOTATION — are the document's own /Order tree, not something the viewer invented, and nothing here writes to the file: visibility is view state.

Annotations belong to layers too. Draw with a layer current and the mark lands on it; select one and the layers button on its toolbar drops down the list to move it to:

The layer dropdown on an annotation's toolbar


Try it in 30 seconds

Live demo: https://jmckalex.github.io/pdfium-ocg/ — no install, no clone; the wasm build runs in your browser.

Or run it locally: demo/ holds a prebuilt static site: the EmbedPDF viewer with a full layers panel, compiled against a patched PDFium, plus a ten-layer test drawing. No toolchain, no npm install.

git clone https://github.com/<you>/pdfium-ocg
cd pdfium-ocg/demo
python3 -m http.server 8000     # any static server; it must be http, not file://

Then open http://localhost:8000 and click the layers icon in the top-right of the toolbar.

Things worth trying:

  • Uncheck a layer. The drawing repaints without it. Nothing is written to the file — visibility is view state that PDFium keeps beside the document, so this works on a read-only PDF.
  • Tick Labels (French). Labels (English) switches itself off. The document declares the two as a /RBGroups radio pair, and the panel honours it — turning one on turns its siblings off.
  • Click a row. It becomes the current layer, shown by highlighting rather than a second control. That is what Delete acts on, and where new annotations go.
  • Draw something, then hide the layer you drew it on. Pick a layer, switch to Shapes, draw a rectangle: it lands on that layer and hides with it. This is the part that matters most — almost nobody hand-authors optional content, so an annotation is usually the only thing that will ever be on a layer.
  • Select an annotation and press the layers button on its toolbar for a dropdown of every layer — pick one to move it there, or No layer to take it off layers entirely.
  • Delete a layer. It asks what should happen to what is drawn on it — keep it, delete it too, or move it to another layer. There is no safe default: content still naming a removed layer renders visible, so silently keeping it would reveal the very thing you meant to delete.
  • Drag a row. Reordering rewrites /Order. Drag the SERVICES heading and the whole group moves with its children.
  • Look at Watermark. It carries a lock, because the document lists it under /Locked — but the checkbox still works. That is deliberate, and it is the one open design question in this series: the API reports the lock and lets the embedder decide, rather than refusing the call.

The drawing is generated by demo/make_layers_depot_plan.py, which is commented and worth a look if you want a minimal layered PDF to test against.

What the patches actually do

Eight patches, each a self-contained change with its own tests, in patches/pdfium/. They are written to be reviewed in order.

# Patch What it adds
1 Honour /OC on annotations A spec-compliance fix with no API change (see below)
2 Enumerate and query FPDFDoc_GetOCGCount, FPDFDoc_GetOCG, FPDFOCG_GetName, FPDFOCG_GetVisibility
3 Runtime toggle FPDFOCG_SetVisibility, FPDFOCG_ResetVisibility — the headline
4 The /Order tree FPDFDoc_GetOCGOrder, FPDFOCGNode_* — what a Layers panel needs to draw itself
5 /Locked and /RBGroups FPDFOCG_IsLocked, FPDFDoc_GetOCGRadioGroup*
6 Authoring FPDFDoc_AddOCG, FPDFAnnot_GetOCG / FPDFAnnot_SetOCG
7 Editing FPDFDoc_RemoveOCG, FPDFOCGNode_MoveChild, FPDFPageObj_GetOCG / FPDFPageObj_SetOCG
8 Tree shaping FPDFOCGNode_AddGroup, FPDFOCGNode_MoveChildTo — headings and re-parenting

Patch 1 is the one to read first if you only read one. It is not part of the API work at all: CPDF_AnnotList::DisplayPass filters annotations on the Hidden, Print and NoView flags and nothing else, so an annotation on a hidden layer draws anyway. ISO 32000-1 §12.5.2 says /OC shall be evaluated before drawing. That is a plain bug, it predates this work, and it is fixed on its own with its own test — which is what PDFium's CONTRIBUTING.md asks for.

The design note behind patch 3 — where a runtime override can live without inverting PDFium's parser/ → page/ layering — is in PLAN.md §1, along with the alternatives that were rejected and why.

The state of it

At the tip of the series: 934 embedder tests and 1097 unit tests pass, testing/tools/api_check.py is clean, and git cl format is clean.

Nothing has been submitted upstream. A Google CLA has to be signed first, and there is one open design question worth settling on the mailing list before review rather than during it — whether FPDFOCG_SetVisibility should refuse to change a /Locked group, or report the lock and let the embedder decide. This series takes the second position; patches/pdfium-alternatives/ holds a working implementation of the first, so the discussion can be about code. PLAN.md §6 and §7 have the full list.

Prior art: pdfium bug 1136, "Public API for optional contents groups", filed in 2018 and still open. A maintainer called it feasible and asked for a sample PDF; the reporter narrowed the request to get count / name / state and set state. That is patches 2 and 3.

Building from source

scripts/bootstrap.sh clones the three upstream trees at their pinned base commits and replays this repository's patches onto each. It fetches and patches; it deliberately does not build.

./scripts/bootstrap.sh            # everything
./scripts/bootstrap.sh pdfium     # just the upstream series
./scripts/bootstrap.sh v2         # just the v2 viewer (the complete one)

Everything lands in checkout/, each tree on a branch named ocg.

PDFium

The API and its tests. This is the part that matters upstream.

cd checkout/pdfium
# One-time: fetch the build toolchain. Several GB, and it takes a while.
gclient config --unmanaged --name pdfium https://pdfium.googlesource.com/pdfium
gclient sync

gn gen out/Debug --args='pdf_is_standalone=true is_debug=true pdf_use_skia=false'
ninja -C out/Debug pdfium_embeddertests
./out/Debug/pdfium_embeddertests --gtest_filter='*OCG*:*OptionalContent*'

You need depot_tools on PATH for gn, ninja, and gclient.

Which EmbedPDF? Both.

The patches cover two EmbedPDF lines, because they are different products with the same name.

v2 (patches/embedpdf-v2/) is what people ship today — the @embedpdf/snippet package, with a complete viewer. This is where the layers work is finished: the panel, annotations on layers, moving an annotation between layers, deleting a layer with a choice about its content, radio groups, reordering. demo/ is a build of it.

v3 (patches/embedpdf-runtime/ and patches/embedpdf-viewer/) is the ground-up rewrite, still catching up with v2 feature by feature — its own code says so, and its comment sidebar, for instance, is not yet ported. The layers work there is equivalent but sits on a different architecture.

Doing both was not the plan; it is what the work turned up. If you only care about one, care about v2.

The WebAssembly build and the viewer

checkout/embedpdf-runtime compiles PDFium to wasm; the patches add the OCG exports and the TypeScript bindings for them. checkout/embedpdf-viewer is the viewer, where the patches add an engine service, a plugin, and the Layers panel.

Building the wasm needs emsdk 3.1.72, which the runtime tree vendors under third_party/emsdk. Once the runtime is built, point the viewer at it:

cd checkout/embedpdf-viewer
pnpm install
pnpm build                              # the packages, before any example
cd examples/snippet-react && pnpm dev

One trap worth stating outright, because it cost an hour here: the examples resolve @embedpdf/viewer-chrome from that package's dist/, not its source. Editing the panel and rebuilding only the example gets you a stale panel with no error. Run pnpm build at the root first.

Building v2 (the complete one)

v2 compiles PDFium itself, in Docker, from EmbedPDF's own fork. The series patches that fork and the build picks the new API up on its own — its export list and TypeScript bindings are generated by globbing public/*.h, so adding a header is the whole integration.

cd checkout/embedpdf-v2
pnpm install
( cd packages/pdfium && docker compose build pdfium-build )   # one-time image
( cd packages/pdfium && docker compose run --rm pdfium-build ) # the long one
pnpm build
( cd viewers/snippet && pnpm build )

The PDFium compile is the only slow step, and only the first time — ninja objects live in a named Docker volume, so later builds recompile just what the patch touches. On Apple silicon the image is linux/amd64 and runs emulated, which makes every file slower but changes nothing else. Give Docker plenty of memory: PDFium's link step wants it.

Two things that will waste your afternoon otherwise, both fixed in the patched build.sh but worth knowing:

  • gn must run from inside the checkout. Run it from the workspace root and gn.py cannot find a gclient root, falls back to hunting gn on PATH, finds depot_tools' own wrapper, and re-execs until the kernel kills it. The symptom is Killed, which reads exactly like an out-of-memory failure and is not one.
  • The gn-args block must be guarded on the wasm marker, not on args.gn merely existing. A half-finished earlier run leaves an args.gn with no target_os in the cache volume; the build then compiles all of PDFium for the host and fails minutes later at the emscripten link with hundreds of undefined symbols — from an archive that contains every one of them.

When you point a viewer at a local build, set wasmUrl. The engine otherwise fetches the published pdfium from jsDelivr and fails at runtime with exported native function FPDFDoc_GetOCGCount not found, which accuses the wasm and says nothing about where it came from. Compute it absolutely (new URL('./pdfium.wasm', location.href).href): the worker is created from a blob and cannot resolve a relative path.

If you are on a machine like the one this was written on

scripts/build-env.sh handles two local problems that are not PDFium's fault but will each cost you an afternoon: a TLS-intercepting antivirus root that tools carrying their own CA bundle don't trust, and an xcode-select pointing at the Command Line Tools when Chromium's build needs full Xcode. Source it before building. If neither applies to you, ignore it.

Layout

Path
patches/pdfium/ The upstream series — the point of the project
patches/pdfium-alternatives/ The rejected answer to the open design question, kept as code
patches/embedpdf-v2/ The complete port — v2's wasm build, engine, plugin and viewer
patches/embedpdf-runtime/ The v3 port onto EmbedPDF's PDFium fork, plus wasm bindings
patches/embedpdf-viewer/ v3 engine service, plugin, and Layers panel
demo/ Prebuilt static demo (v2) — clone and serve
demo-v3/ The same, built from the v3 viewer
scripts/bootstrap.sh Fetch and patch the three trees
PLAN.md The plan of record: design decisions, test strategy, open questions
CLAUDE.md Working notes and the gap map the series was built from
HANDOVER.md Session log, opening with the post-mortem of a bug that failed for every human click and no automated one

Known gaps

Stated plainly, because a README that only lists what works is not much use.

  • Text extraction ignores optional content. FPDFText_* returns text from hidden layers. Arguably right for search and accessibility, arguably wrong for copy-paste. Upstream will have an opinion, so this series does not guess — it is flagged, not fixed.
  • The form-fill render path does not check annotation /OC. Patch 1 fixes CPDF_AnnotList; FPDF_FFLDraw takes a different route.

Licence

The patches are against PDFium (BSD 3-clause) and the packages/ tree of EmbedPDF (Apache-2.0), and are offered under the licence of whichever project they apply to. demo/ is a build of those two and carries theirs. Everything else in this repository — the scripts, the documentation, the demo generator — is under the BSD 3-clause licence in LICENSE.

About

PDF layers for PDFium — the missing optional-content API, with tests, a wasm build, and a working layers panel in the browser, built in a fork of EmbedPDF..

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages