Skip to content

feat: add JavBus Python scraper (supports censored + uncensored search) - #2820

Open
xy952666680 wants to merge 3 commits into
stashapp:masterfrom
xy952666680:xy952666680-patch-4
Open

feat: add JavBus Python scraper (supports censored + uncensored search)#2820
xy952666680 wants to merge 3 commits into
stashapp:masterfrom
xy952666680:xy952666680-patch-4

Conversation

@xy952666680

@xy952666680 xy952666680 commented Jul 31, 2026

Copy link
Copy Markdown

Summary

New JavBus Python scraper supporting both censored and uncensored search.

  • sceneByFragment / sceneByQueryFragment / sceneByName / sceneByURL
  • Searches javbus.com/search/<code> first, falls back to /uncensored/search/<code> when no censored results
  • Hardcoded Cookie: over18=18; existmag=mag to bypass the site's age gate
  • Stdlib-only (urllib + regex), no third-party dependencies

Scraper types

  • sceneByFragment
  • sceneByName
  • sceneByQueryFragment
  • sceneByURL

Examples to test

Notes

  • Code extraction strips junk tokens (CD1, DISC2, JAV24, ...), supports alphanumeric prefixes (s2mbd-055S2MBD-055), normalizes dashless codes (DDT246DDT-246), and ignores dates before the code
  • Fragment paths return a single scene object; sceneByName returns a list
  • Duration returned in seconds (Stash convention)

@xy952666680 xy952666680 changed the title Xy952666680 patch 4 feat: add JavBus Python scraper (supports censored + uncensored search) Jul 31, 2026
- dispatch: read code/title/urls fields (Stash fragment has no 'name');
  fragment paths return a single scene object, not a list; prefer
  exact-code match over first search hit
- duration: convert minutes to seconds (x60), omit key when absent
- ec(): strip CD/DVD/DISC/PART/JAV junk tokens; support alphanumeric
  prefixes (s2mbd-055 -> S2MBD-055); keep dashless normalization
  (DDT246 -> DDT-246) and date-immune matching
- tags: anchor regex to <label> so performers no longer leak into tags
- ssl: use create_default_context() instead of unverified context
- yml: drop dead 'validSearch' arg (never read by the script)
@xy952666680

Copy link
Copy Markdown
Author

Review fixes applied (verified against live javbus.com)

This branch includes fixes from a full review — Codex CLI review plus end-to-end testing against live javbus.com pages (censored + uncensored), both locally and on a staging Stash instance.

  1. Fragment dispatch: sceneByFragment / sceneByQueryFragment now read code/title/urls (Stash scene fragments have no name field), return a single scene object (Stash unmarshals fragment paths into one scene — see script.go), and prefer an exact code match over the first fuzzy search hit.
  2. Duration: converted minutes → seconds (×60); key omitted when the page has no runtime.
  3. Code extraction ec(): strips CD|DVD|DISC|PART|JAV junk tokens, supports alphanumeric prefixes (s2mbd-055S2MBD-055), normalizes dashless codes (DDT246DDT-246), and is immune to dates appearing before the code.
  4. Tags: regex anchored to class="genre"><label> so performer names no longer leak into the tag list.
  5. SSL: ssl.create_default_context() instead of an unverified context.
  6. YML: removed the dead validSearch argument (the script never read it; Stash doesn't pass query strings to script scrapers).

Tested entry points (all against live pages):

  • Censored SSIS-123 and uncensored HEYZO-3016 × fragment (code-only) / name / url
  • Edge cases: junk-token codes (JAV24 CD1 ABC-123ABC-123), dashless codes, empty fragment → {}
  • Fields verified: title, code, date, duration (seconds), director, studio, tags (no performers), performers, image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant