Capture a rendered page, then open it offline in Chromium. The address bar still shows the original URL. The bytes come from disk.
Unlike wget -mpk or HTTrack, this does not rewrite links into a local folder. Unlike monolith, it is not a single HTML file. It records real browser traffic and replays it with Playwright route interception, so cookies, CORS, redirects, and absolute URLs keep working. Unmatched requests are aborted. There is no live-network fallback.
uv sync --all-groups
uv run playwright install chromium
uv run website-clone capture https://example.com/ --output captures/example
uv run website-clone open captures/exampleopen loads the archive in headed Chromium and waits until you close the window. That is the command you want for "just show me the page."
Python 3.11+ and Chromium are required.
Install with pip instead of uv
python -m pip install .
python -m playwright install chromium| Command | What it does |
|---|---|
capture <url> --output <dir> |
Record a page and its resources into an archive |
open <archive> |
View the archive in Chromium (no verification pass) |
replay <archive> |
Verify the archive can be replayed offline |
inspect <archive> |
Print a JSON summary, no browser |
export <archive> --output <dir> |
Write readable request/response evidence |
analyze <archive> --output <file> |
Scan retained bodies for traceable candidates |
uv run website-clone capture https://example.com/ --output captures/example--profile sets how hard capture looks for resources:
| Profile | Behavior |
|---|---|
rendered |
Natural browser traffic, final DOM, screenshot, Blob and data-URL evidence. No supplemental fetches. |
offline |
Adds top-document DOM and transitive CSS-closure discovery. |
exhaustive |
Default. Scrolls, walks frames and open shadow roots, inspects lazy and <noscript> content, repeats until two settled passes converge. |
uv run website-clone capture `
"https://asurascans.com/comics/surviving-as-a-genius-on-borrowed-time-b57aa235/chapter/74" `
--profile exhaustive `
--output ".\asurascans\chapter-74"uv run website-clone open captures/exampleChromium navigates to the captured start URL. You will see https://example.com/ (or whatever you captured) in the address bar. Playwright intercepts every request and fulfills it from the archive. A miss is aborted, not fetched live.
open is always headed. It does not replay recipes, scroll exhaustive surfaces, or take a screenshot. --timeout only bounds startup, navigation, and the initial idle wait, not the time you spend looking at the page.
uv run website-clone replay captures/example
uv run website-clone replay captures/example --keep-open
uv run website-clone replay captures/example --headedreplay is the CI-style check: goto, idle, recipe (if any), exhaustive scroll (if the capture used that profile), screenshot. --keep-open still runs that pass first, then leaves the window up. Use open when you just want to look.
uv run website-clone inspect captures/example
uv run website-clone export captures/example --output exports/example
uv run website-clone analyze captures/example --output reports/example.json- Capture drives Chromium, records requests/responses, the final DOM, and a screenshot into a schema-versioned archive.
- Open / replay launch Chromium with
page.goto(start_url)andcontext.route("**/*", ...). - Each request is matched on method + origin + URL + body. A hit is
route.fulfill'd from a content-addressed object. A miss isroute.abort'd. - The page therefore looks like the live origin in the URL bar, but Chromium is not talking to that host.
A static python -m http.server folder is not the replay surface. It cannot preserve methods, origins, redirects, cookies, or request order.
Every command accepts --show-secrets (prints redacted values exactly; does not change archive files).
capture: --profile, --headed, --overwrite, --timeout, --idle, --max-total-bytes, --max-resource-bytes, --max-references, --storage-state, --headers, --recipe, --max-actions, --action-timeout, --max-scroll-steps, --max-resources
replay: --headed, --keep-open, --timeout, --idle, --max-scroll-steps, --no-screenshot
open: --timeout, --idle
export: --overwrite analyze: --output, --overwrite, --max-scan-bytes, --max-leads
Capture favors breadth: 300s deadline, 2s quiet period, 10 GiB total retained payload, 2 GiB per payload, 10,000 supplemental references (offline / exhaustive only). Action defaults: 1,000 expanded operations, 30s per action, 10,000 combined recipe and exhaustive-scroll steps, 100,000 browser resources. 0 disables a byte/reference/resource ceiling. The overall deadline always applies.
Exit codes: 0 complete, 1 persisted partial / misses, 2 invalid input, invalid archive, or refused overwrite.
Archives hold a schema-versioned manifest, final HTML, a capture screenshot, content-addressed response bodies, and replay evidence. New captures use schema v3 with a provenance sidecar. Readers accept v1, v2, and v3.
Treat archives as sensitive. Console output redacts URL credentials, query values, fragments, and credential-shaped headers unless you pass --show-secrets. Archives, exports, and analyze reports contain exact evidence and must be protected.
Capture session inputs are JSON files:
--storage-state— Playwright storage state (cookies,origins)--headers— string-to-string object--recipe— schema v1wait/navigate/scroll/click/input(no implicit submit, no arbitrary script)
Capture follows resource-bearing references in the rendered DOM, inline CSS, and stylesheets. It does not crawl navigation links or automate login. Cross-origin, loopback, private, and link-local HTTP(S) traffic is not filtered by profile; a body can still be dropped if collection fails or a limit is hit.
A live request that fails in the source browser is kept as evidence and does not by itself make capture partial. Collector loss, limits, and cancellation persist a partial archive with structured issues. WebSockets and service workers are inventoried as unsupported. blob: and data: payloads are recorded as evidence, not network requests.
Replay and open serve only exact archived matches. Misses go to replay/last-run.json. WebSockets and service workers are blocked. Non-GET/HEAD archived redirects are aborted rather than risking changed body semantics.
Rerunning capture into an existing directory requires --overwrite, and only if that directory is already a valid archive.
Schema, capture reports, and analysis details
For schema v2+, request bodies are stored byte-for-byte when retained. Identity is method + URL + post_data_sha256. post_data_object_digest is a separate optional object reference. Missing or truncated bodies are recorded as collector evidence, not silently replaced.
capture_report (or null) includes profile, discovery loop/scroll/convergence counts, and image stats (fetched, embedded, failed URLs). missing_numbered only reports internal gaps when at least three observed numeric basenames share origin, directory, extension, and zero-padding. Recognized image signatures: PNG, JPEG, GIF, WebP, AVIF, SVG, BMP, ICO.
analyze items are candidates with confidence and provenance, not validated vulnerabilities. It scans valid UTF-8 even when MIME is misleading, and does not treat recognized binary media as invalid text. Staging lives next to the named output, never in the system temp directory.
Recipe replay (on replay, not open) starts after the initial quiet period. A different pre-action DOM is recorded as a mismatch and returns partial.
| website-clone | wget / HTTrack | monolith / SingleFile | |
|---|---|---|---|
| JS-rendered pages | Yes (Chromium) | No | Partial / browser extension |
| Offline viewing | Chromium + archive routes | Local folder, rewritten links | One HTML file |
| Address bar | Original URL | file:// or localhost |
file:// |
| Live-network fallback | Never | N/A | N/A |
| Best for | Faithful interactive replay | Static site mirrors | Portable single-file snapshots |
uv lock --check
uv sync --all-groups
uv run ruff check .
uv run mypy src/website_clone
uv run pytest -q
uv run website-clone --helpTests use local fixture servers and Chromium. They cover capture, offline replay, open, request identity, archive containment, and visual equivalence.