Skip to content

Fix support for streaming Web Publications with EPUB profile #731

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
mickael-menu wants to merge 1 commit into
base: develop
Choose a base branch
from
Draft

Fix support for streaming Web Publications with EPUB profile #731

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 20 additions & 7 deletions Sources/Streamer/Parser/Readium/ReadiumWebPubParser.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ public class ReadiumWebPubParser: PublicationParser, Loggable {
case let .resource(asset):
return await parse(resource: asset.resource, format: asset.format.specifications, warnings: warnings)
case let .container(asset):
return await parse(container: asset.container, format: asset.format.specifications, warnings: warnings)
return await parse(container: asset.container, isPackage: true, format: asset.format.specifications, warnings: warnings)
}
}

Expand Down Expand Up @@ -79,6 +79,7 @@ public class ReadiumWebPubParser: PublicationParser, Loggable {
.asyncFlatMap { container in
await parse(
container: container,
isPackage: false,
format: FormatSpecifications(.rpf),
warnings: warnings
)
Expand All @@ -87,6 +88,7 @@ public class ReadiumWebPubParser: PublicationParser, Loggable {

private func parse(
container: Container,
isPackage: Bool,
format: FormatSpecifications,
warnings: WarningLogger?
) async -> Result<Publication.Builder, PublicationParseError> {
Expand All @@ -103,17 +105,25 @@ public class ReadiumWebPubParser: PublicationParser, Loggable {
.map { manifest in
var manifest = manifest

// Remove any self link as it is a packaged publication. It
// might be packaged from a streamed manifest which would cause
// issues when serving the relative reading order resources.
manifest.links = manifest.links.filter { !$0.rels.contains(.self) }
if isPackage {
// Remove any self link as it is a packaged publication. It
// might be packaged from a streamed manifest which would
// cause issues when serving the relative reading order
// resources.
manifest.links = manifest.links.filter { !$0.rels.contains(.self) }
}

return Publication.Builder(
manifest: manifest,
container: container,
servicesBuilder: PublicationServicesBuilder(setup: {
if manifest.conforms(to: .epub) {
$0.setPositionsServiceFactory(EPUBPositionsService.makeFactory(reflowableStrategy: epubReflowablePositionsStrategy))
// We don't use the EPUBPositionsService with a
// streamed manifest, because it will fetch all
// the resources to compute the positions.
if isPackage {
$0.setPositionsServiceFactory(EPUBPositionsService.makeFactory(reflowableStrategy: epubReflowablePositionsStrategy))
}

} else if manifest.conforms(to: .divina) {
$0.setPositionsServiceFactory(PerResourcePositionsService.makeFactory(fallbackMediaType: MediaType("image/*")!))
Expand All @@ -128,7 +138,10 @@ public class ReadiumWebPubParser: PublicationParser, Loggable {

// FIXME: WebPositionsService from Kotlin?

if manifest.readingOrder.allAreHTML {
// We don't use the search and content services with a
// streamed manifest because it will fetch all the
// resources on the server.
if manifest.readingOrder.allAreHTML, isPackage {
$0.setSearchServiceFactory(StringSearchService.makeFactory())

@ddfreiling ddfreiling Mar 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it would be desirable to have a SearchService for streamed WebPubs, but that it only fetches content, when/if used to actually search.

@mickael-menu mickael-menu Mar 25, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My comment was ambiguous, the SearchService and ContentService are already fetching the resources only when requested (when moving forward in the search iterator, for example). But this allows users to crawl multiple times all the resources of the server, if it does not provide a proper Web Search service. So I think this should be discouraged in the toolkit by default.

As a user of the toolkit, you can always add the StringSearchService yourself after parsing the publication, if you have some control over the server and are okay with potential crawling.

This could be alleviated by adding a CachingContainer when opening the publication, to cache the resources on the disk or in memory.

$0.setContentServiceFactory(DefaultContentService.makeFactory(
resourceContentIteratorFactories: [
Expand Down