From c3a97dea59fd516458a3a20e7b39673b9138b3e6 Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 22:42:30 +0400 Subject: [PATCH 01/35] Add MkDocs Material configuration --- mkdocs.yml | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) create mode 100644 mkdocs.yml diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 00000000..42d0d030 --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,39 @@ +site_name: PyLD +site_description: Python implementation of the JSON-LD API +site_url: https://digitalbazaar.github.io/pyld/ +repo_url: https://github.com/digitalbazaar/pyld +repo_name: digitalbazaar/pyld + +theme: + name: material + features: + - content.code.copy + - navigation.footer + - navigation.sections + - navigation.top + - search.highlight + - search.suggest + +nav: + - Home: index.md + - Installation: installation.md + - Quick Examples: quick-examples.md + - Document Loaders: + - Overview: document-loaders.md + - RequestsDocumentLoader: document-loaders/requests.md + - AioHttpDocumentLoader: document-loaders/aiohttp.md + - FrozenDocumentLoader: document-loaders/frozen.md + - API Reference: api-reference.md + +markdown_extensions: + - admonition + - attr_list + - pymdownx.details + - pymdownx.highlight: + anchor_linenums: true + - pymdownx.superfences + - toc: + permalink: true + +plugins: + - search From 559455963984d6fa57a9f33370a41aeda33a40fa Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 22:42:34 +0400 Subject: [PATCH 02/35] Use MkDocs Material for documentation dependencies --- docs/requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 231419d2..4c8f017d 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -1 +1 @@ -sphinx-autobuild +mkdocs-material From 8908ed46472939e6fb761b7899a6d78ed8471190 Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 22:42:39 +0400 Subject: [PATCH 03/35] Add documentation homepage --- docs/index.md | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 docs/index.md diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 00000000..a083ad7c --- /dev/null +++ b/docs/index.md @@ -0,0 +1,40 @@ +# PyLD + +PyLD is a Python implementation of the [JSON-LD][] processor API. + +JSON-LD is a lightweight syntax for expressing Linked Data in JSON. It lets +applications add meaning to existing JSON documents with in-band or out-of-band +contexts, while keeping the document shape practical for web APIs, JavaScript, +and JSON document stores. + +## Conformance + +PyLD aims to conform with: + +- [JSON-LD 1.1][json-ld-11] +- [JSON-LD 1.1 Processing Algorithms and API][json-ld-11-api] +- [JSON-LD 1.1 Framing][json-ld-11-framing] +- The JSON-LD Working Group [test suite][wg-test-suite] + +The test runner is updated over time to note or skip newer tests that are not +yet supported. + +## Requirements + +- Python 3.10 or later +- `requests` for the default synchronous document loader, when installed with + the `requests` extra +- `aiohttp` for the asynchronous document loader, when installed with the + `aiohttp` extra + +## Project Links + +- [Source code](https://github.com/digitalbazaar/pyld) +- [Package on PyPI](https://pypi.org/project/PyLD/) +- [JSON-LD](https://json-ld.org/) + +[JSON-LD]: https://json-ld.org/ +[json-ld-11]: https://www.w3.org/TR/json-ld11/ +[json-ld-11-api]: https://www.w3.org/TR/json-ld11-api/ +[json-ld-11-framing]: https://www.w3.org/TR/json-ld11-framing/ +[wg-test-suite]: https://github.com/w3c/json-ld-api/tree/master/tests From 1b4491fe42490906acc270b15ac2082690d495e4 Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 22:42:45 +0400 Subject: [PATCH 04/35] Add installation documentation --- docs/installation.md | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100644 docs/installation.md diff --git a/docs/installation.md b/docs/installation.md new file mode 100644 index 00000000..57b3cce2 --- /dev/null +++ b/docs/installation.md @@ -0,0 +1,41 @@ +# Installation + +Install PyLD from PyPI: + +```bash +pip install PyLD +``` + +PyLD's core package does not install `requests` or `aiohttp` automatically. If +your application needs one of the built-in remote document loaders, install the +matching extra: + +```bash +pip install "PyLD[requests]" +pip install "PyLD[aiohttp]" +``` + +You can also depend on `requests` or `aiohttp` directly if your project already +manages those dependencies. + +## Development Install + +From a local checkout: + +```bash +pip install -e . +``` + +Run the project tests with: + +```bash +pytest +``` + +The JSON-LD specification test suites are stored under `specifications/` and are +usually initialized as git submodules: + +```bash +git submodule init +git submodule update +``` From c3e84e6c432457206d985bcd22adafb97b181eb2 Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 22:42:49 +0400 Subject: [PATCH 05/35] Add JSON-LD quick examples documentation --- docs/quick-examples.md | 73 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) create mode 100644 docs/quick-examples.md diff --git a/docs/quick-examples.md b/docs/quick-examples.md new file mode 100644 index 00000000..0a904804 --- /dev/null +++ b/docs/quick-examples.md @@ -0,0 +1,73 @@ +# Quick Examples + +```python +from pyld import jsonld +import json + +doc = { + "http://schema.org/name": "Manu Sporny", + "http://schema.org/url": {"@id": "http://manu.sporny.org/"}, + "http://schema.org/image": {"@id": "http://manu.sporny.org/images/manu.png"}, +} + +context = { + "name": "http://schema.org/name", + "homepage": {"@id": "http://schema.org/url", "@type": "@id"}, + "image": {"@id": "http://schema.org/image", "@type": "@id"}, +} + +compacted = jsonld.compact(doc, context) +print(json.dumps(compacted, indent=2)) +``` + +The compacted output uses terms from the supplied context: + +```json +{ + "@context": { + "name": "http://schema.org/name", + "homepage": { + "@id": "http://schema.org/url", + "@type": "@id" + }, + "image": { + "@id": "http://schema.org/image", + "@type": "@id" + } + }, + "image": "http://manu.sporny.org/images/manu.png", + "homepage": "http://manu.sporny.org/", + "name": "Manu Sporny" +} +``` + +Expand a compacted document: + +```python +expanded = jsonld.expand(compacted) +print(json.dumps(expanded, indent=2)) +``` + +Flatten a document: + +```python +flattened = jsonld.flatten(doc) +``` + +Frame a document: + +```python +framed = jsonld.frame(doc, frame) +``` + +Normalize a document using RDF Dataset Canonicalization: + +```python +normalized = jsonld.normalize( + doc, + {"algorithm": "URDNA2015", "format": "application/n-quads"}, +) +``` + +The normalized value is a canonical N-Quads string that can be used for hashing, +comparison, or signing workflows. From 01c3f5fb745af1413627f0f51931156c8d5e634e Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 22:42:54 +0400 Subject: [PATCH 06/35] Add document loader overview documentation --- docs/document-loaders.md | 57 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) create mode 100644 docs/document-loaders.md diff --git a/docs/document-loaders.md b/docs/document-loaders.md new file mode 100644 index 00000000..70c01b90 --- /dev/null +++ b/docs/document-loaders.md @@ -0,0 +1,57 @@ +# Document Loaders + +Document loaders retrieve remote JSON-LD documents and contexts. PyLD accepts +any callable with the loader signature, and also ships class-based loaders for +common cases. + +## Built-In Loader Classes + +- [RequestsDocumentLoader](document-loaders/requests.md) uses `requests` for + synchronous remote document loading. +- [AioHttpDocumentLoader](document-loaders/aiohttp.md) uses `aiohttp` for + asynchronous fetching while keeping JSON-LD processing synchronous. +- [FrozenDocumentLoader](document-loaders/frozen.md) serves only documents from + an allowlist. + +The default document loader is selected at import time. PyLD uses +`RequestsDocumentLoader` if `requests` is available, falls back to +`AioHttpDocumentLoader` if `aiohttp` is available, and otherwise installs a +dummy loader that raises when invoked. + +## Custom Loaders + +A custom loader returns a remote document mapping with `contextUrl`, +`documentUrl`, and `document` keys: + +```python +from pyld import jsonld + +document_cache = { + "https://example.com/context": { + "contextUrl": None, + "documentUrl": "https://example.com/context", + "document": {"@context": {"name": "https://schema.org/name"}}, + } +} + + +def load_document(url, options=None): + return document_cache[url] + + +jsonld.set_document_loader(load_document) +``` + +For advanced context caching, pass a `ContextResolver` in operation options: + +```python +from cachetools import LRUCache +from pyld import ContextResolver, jsonld + +resolver = ContextResolver(LRUCache(maxsize=1000), load_document) + +expanded = jsonld.expand( + doc, + options={"contextResolver": resolver}, +) +``` From 813b4825212857e667bec6683a9d4b59ea492f3e Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 22:43:00 +0400 Subject: [PATCH 07/35] Document RequestsDocumentLoader usage --- docs/document-loaders/requests.md | 48 +++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) create mode 100644 docs/document-loaders/requests.md diff --git a/docs/document-loaders/requests.md b/docs/document-loaders/requests.md new file mode 100644 index 00000000..0ab4118f --- /dev/null +++ b/docs/document-loaders/requests.md @@ -0,0 +1,48 @@ +# RequestsDocumentLoader + +`RequestsDocumentLoader` retrieves JSON-LD documents with `requests`. + +The default remote document loader uses `requests` when it is available. +Production applications should usually set at least a timeout: + +```python +from pyld import jsonld + +jsonld.set_document_loader(jsonld.requests_document_loader(timeout=10)) +``` + +The concrete loader class is exported from `pyld`: + +```python +from pyld import RequestsDocumentLoader, jsonld + +jsonld.set_document_loader(RequestsDocumentLoader(timeout=10)) +``` + +Use `secure=True` to require HTTPS URLs: + +```python +jsonld.set_document_loader( + jsonld.requests_document_loader(secure=True, timeout=10) +) +``` + +Extra keyword arguments are forwarded to `requests.get()`: + +```python +from pyld import RequestsDocumentLoader, jsonld + +loader = RequestsDocumentLoader( + timeout=10, + verify=True, + cert=("client.crt", "client.key"), +) + +jsonld.set_document_loader(loader) +``` + +Install the optional dependency with: + +```bash +pip install "PyLD[requests]" +``` From 3f3348a602c50268d48ebac4fdeb6adfcededf83 Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 22:43:06 +0400 Subject: [PATCH 08/35] Document AioHttpDocumentLoader usage --- docs/document-loaders/aiohttp.md | 44 ++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) create mode 100644 docs/document-loaders/aiohttp.md diff --git a/docs/document-loaders/aiohttp.md b/docs/document-loaders/aiohttp.md new file mode 100644 index 00000000..186d818f --- /dev/null +++ b/docs/document-loaders/aiohttp.md @@ -0,0 +1,44 @@ +# AioHttpDocumentLoader + +`AioHttpDocumentLoader` retrieves JSON-LD documents with `aiohttp`. + +```python +from pyld import jsonld + +jsonld.set_document_loader(jsonld.aiohttp_document_loader(timeout=10)) +``` + +This loader uses asynchronous fetching internally, but JSON-LD processing itself +remains synchronous. + +The concrete loader class is exported from `pyld`: + +```python +from pyld import AioHttpDocumentLoader, jsonld + +jsonld.set_document_loader(AioHttpDocumentLoader(timeout=10)) +``` + +Use `secure=True` to require HTTPS URLs: + +```python +jsonld.set_document_loader( + jsonld.aiohttp_document_loader(secure=True, timeout=10) +) +``` + +Extra keyword arguments are forwarded to `aiohttp` request calls: + +```python +from pyld import AioHttpDocumentLoader, jsonld + +loader = AioHttpDocumentLoader(timeout=10) + +jsonld.set_document_loader(loader) +``` + +Install the optional dependency with: + +```bash +pip install "PyLD[aiohttp]" +``` From 570840b0029f8ebed32fe2bce79f927d91351811 Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 22:43:13 +0400 Subject: [PATCH 09/35] Document FrozenDocumentLoader usage --- docs/document-loaders/frozen.md | 37 +++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100644 docs/document-loaders/frozen.md diff --git a/docs/document-loaders/frozen.md b/docs/document-loaders/frozen.md new file mode 100644 index 00000000..f0ebcdae --- /dev/null +++ b/docs/document-loaders/frozen.md @@ -0,0 +1,37 @@ +# FrozenDocumentLoader + +`FrozenDocumentLoader` serves only URLs in an allowlist and refuses all other +document loads. It is intended for air-gapped runs, reproducible builds, and +deployments that must avoid remote context fetching. + +With no arguments, the loader serves the curated `BUNDLED_CONTEXTS` mapping: + +```python +from pyld import FrozenDocumentLoader, jsonld + +jsonld.set_document_loader(FrozenDocumentLoader()) +``` + +Extend the bundled mapping with additional vetted contexts: + +```python +from pathlib import Path + +from pyld import BUNDLED_CONTEXTS, FrozenDocumentLoader, jsonld + +loader = FrozenDocumentLoader( + documents=dict( + BUNDLED_CONTEXTS, + **{"https://example.com/context": Path("contexts/example.jsonld")}, + ) +) + +jsonld.expand(doc, options={"documentLoader": loader}) +``` + +The `documents` mapping may contain parsed JSON-LD dictionaries or +`pathlib.Path` instances pointing to JSON files. Path entries are read lazily +and cached after the first request. + +Any URL outside the allowlist raises `JsonLdError` with code +`loading document failed`. From d27ad2293789079c3274d35f3f2f4e43845a6c22 Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 22:43:18 +0400 Subject: [PATCH 10/35] Add manual public API reference --- docs/api-reference.md | 135 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 135 insertions(+) create mode 100644 docs/api-reference.md diff --git a/docs/api-reference.md b/docs/api-reference.md new file mode 100644 index 00000000..6b802e99 --- /dev/null +++ b/docs/api-reference.md @@ -0,0 +1,135 @@ +# API Reference + +This page lists the public APIs that are intended for direct user code. PyLD +also contains internal processor classes and helper functions that are not +documented here. + +## Module `pyld.jsonld` + +### `compact(input_, ctx, options=None)` + +Compact a JSON-LD document using the provided context. + +### `expand(input_, options=None, on_property_dropped=noop)` + +Expand a JSON-LD document, removing context aliases and producing expanded +JSON-LD form. `on_property_dropped` can be used to observe or reject properties +that do not expand to absolute IRIs. + +### `flatten(input_, ctx=None, options=None)` + +Flatten a JSON-LD document. If `ctx` is supplied, compact the flattened output +with that context. + +### `frame(input_, frame, options=None)` + +Frame a JSON-LD document according to the supplied frame. + +### `link(input_, ctx, options=None)` + +Experimentally link a JSON-LD document's nodes in memory. This is equivalent to +framing with `@embed: @link`. + +### `normalize(input_, options=None)` + +Normalize a JSON-LD document. Common options include `algorithm` and `format`. +Use `{"algorithm": "URDNA2015", "format": "application/n-quads"}` to produce +canonical N-Quads. + +### `from_rdf(input_, options=None)` + +Convert RDF input to JSON-LD. + +### `to_rdf(input_, options=None)` + +Convert JSON-LD input to RDF dataset form. + +### `set_document_loader(load_document_)` + +Set the global document loader callable. + +### `get_document_loader()` + +Return the current global document loader callable. + +### `load_document(url, options, base=None, profile=None, request_profile=None)` + +Load a remote document using the configured or supplied document loader. + +### `requests_document_loader(**kwargs)` + +Create a `requests`-based document loader. Pass `secure=True` to require HTTPS. +Other keyword arguments are forwarded to `requests.get()`. + +### `aiohttp_document_loader(**kwargs)` + +Create an `aiohttp`-based document loader. Pass `secure=True` to require HTTPS. +Other keyword arguments are forwarded to `aiohttp` request calls. + +### `register_rdf_parser(content_type, parser)` + +Register an RDF parser for a content type. + +### `unregister_rdf_parser(content_type)` + +Remove a registered RDF parser for a content type. + +### `parse_link_header(header)` + +Parse an HTTP `Link` header. + +### `JsonLdProcessor` + +Processor class behind the module-level convenience functions. Most callers use +the module-level functions directly. + +### `JsonLdError` + +Exception type raised for JSON-LD processing and loading errors. + +### `ContextResolver` + +Context resolver that can be supplied in operation options for custom context +loading and caching behavior. + +### `freeze(value)` + +Return an immutable mapping for dictionary values. This is used by PyLD's +context caches. + +## Top-Level Exports + +### `jsonld` + +The main JSON-LD processing module. + +### `DocumentLoader` + +Abstract base class for class-based document loaders. PyLD still accepts any +callable with the loader signature. + +### `RemoteDocument` + +Typed mapping shape returned by document loaders. + +### `RequestsDocumentLoader` + +Class-based remote document loader implemented with `requests`. + +### `AioHttpDocumentLoader` + +Class-based remote document loader implemented with `aiohttp`. + +### `FrozenDocumentLoader` + +Allowlist-only document loader for deployments that must not fetch arbitrary +remote contexts. + +### `BUNDLED_CONTEXTS` + +Mapping of selected common JSON-LD context URLs to bundled local context files. + +### `ContextResolver` + +Context resolver that can be supplied in operation options for custom context +loading and caching behavior. From 01e2c445ee729ca707b5834dd095346bad9185ee Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 22:43:23 +0400 Subject: [PATCH 11/35] Add documentation deployment workflow --- .github/workflows/docs.yaml | 57 +++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) create mode 100644 .github/workflows/docs.yaml diff --git a/.github/workflows/docs.yaml b/.github/workflows/docs.yaml new file mode 100644 index 00000000..f44a2b23 --- /dev/null +++ b/.github/workflows/docs.yaml @@ -0,0 +1,57 @@ +name: Documentation + +on: + push: + branches: + - 'master' + pull_request: + branches: + - '**' + +permissions: {} + +concurrency: + group: pages + cancel-in-progress: false + +jobs: + build: + runs-on: ubuntu-latest + timeout-minutes: 10 + permissions: + contents: read + steps: + - uses: actions/checkout@v5 + with: + persist-credentials: false + - uses: actions/setup-python@v6 + with: + python-version: '3.14' + cache: 'pip' + - name: Install documentation dependencies + run: | + python -m pip install --upgrade pip + pip install -r docs/requirements.txt + - name: Build documentation + run: mkdocs build --strict + - name: Upload Pages artifact + if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }} + uses: actions/upload-pages-artifact@v3 + with: + path: site + + deploy: + if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }} + needs: build + runs-on: ubuntu-latest + timeout-minutes: 10 + permissions: + pages: write + id-token: write + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + steps: + - name: Deploy to GitHub Pages + id: deployment + uses: actions/deploy-pages@v4 From a966bfa96fb44ade6e849646c10549be296c5dc9 Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 22:43:30 +0400 Subject: [PATCH 12/35] Add MkDocs serve target --- Makefile | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 28da5f38..e16887e0 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,4 @@ -.PHONY: install test upgrade-submodules download-bundled-contexts +.PHONY: install test serve upgrade-submodules download-bundled-contexts install: pip install -e . @@ -6,6 +6,9 @@ install: test: pytest --cov=pyld +serve: + mkdocs serve --dev-addr 127.0.0.1:8008 + upgrade-submodules: git submodule update --remote --init --recursive From 0747f9c7c9ce8c3de9cd5629149ab59e6a67101f Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 22:43:34 +0400 Subject: [PATCH 13/35] Document MkDocs contributor commands --- CONTRIBUTING.rst | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index 5053e549..9668a42a 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -16,6 +16,27 @@ Code * Use version X.Y.Z-dev in dev mode. * Use version X.Y.Z for releases. +Documentation +------------- + +The public documentation site is built with MkDocs Material. + +* Install documentation dependencies: + + * ``pip install -r docs/requirements.txt`` + +* Preview documentation locally: + + * ``mkdocs serve`` + +* Check documentation before submitting changes: + + * ``mkdocs build --strict`` + +* Refresh bundled JSON-LD context files: + + * ``make download-bundled-contexts`` + Versioning ---------- From daa6776aae7367df25af70142248b4163af56791 Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 22:43:38 +0400 Subject: [PATCH 14/35] Ignore MkDocs site output --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 51c08296..820ee873 100644 --- a/.gitignore +++ b/.gitignore @@ -11,6 +11,7 @@ build cover dist docs/_build +site/ lib/PyLD.egg-info profiler tests/test_caching.py From 51f60bdb21612d91758f839c8676b07403014c98 Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 22:43:42 +0400 Subject: [PATCH 15/35] Remove Sphinx docs Makefile --- docs/Makefile | 136 -------------------------------------------------- 1 file changed, 136 deletions(-) delete mode 100644 docs/Makefile diff --git a/docs/Makefile b/docs/Makefile deleted file mode 100644 index 88452c53..00000000 --- a/docs/Makefile +++ /dev/null @@ -1,136 +0,0 @@ -# Makefile for Sphinx documentation -# - -# You can set these variables from the command line. -SPHINXOPTS = -SPHINXBUILD = sphinx-build -PAPER = -BUILDDIR = _build -HOST = 127.0.0.1 -PORT = 8000 - -# Internal variables. -PAPEROPT_a4 = -D latex_paper_size=a4 -PAPEROPT_letter = -D latex_paper_size=letter -ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . - -.PHONY: help clean html dirhtml singlehtml serve pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest - -help: - @echo "Please use \`make ' where is one of" - @echo " html to make standalone HTML files" - @echo " dirhtml to make HTML files named index.html in directories" - @echo " singlehtml to make a single large HTML file" - @echo " serve to serve HTML files with auto rebuild" - @echo " pickle to make pickle files" - @echo " json to make JSON files" - @echo " htmlhelp to make HTML files and a HTML help project" - @echo " qthelp to make HTML files and a qthelp project" - @echo " devhelp to make HTML files and a Devhelp project" - @echo " epub to make an epub" - @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" - @echo " latexpdf to make LaTeX files and run them through pdflatex" - @echo " text to make text files" - @echo " man to make manual pages" - @echo " changes to make an overview of all changed/added/deprecated items" - @echo " linkcheck to check all external links for integrity" - @echo " doctest to run all doctests embedded in the documentation (if enabled)" - -clean: - -rm -rf $(BUILDDIR)/* - -html: - $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html - @echo - @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." - -dirhtml: - $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml - @echo - @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." - -singlehtml: - $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml - @echo - @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." - -serve: - sphinx-autobuild -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html --host $(HOST) --port $(PORT) - -pickle: - $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle - @echo - @echo "Build finished; now you can process the pickle files." - -json: - $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json - @echo - @echo "Build finished; now you can process the JSON files." - -htmlhelp: - $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp - @echo - @echo "Build finished; now you can run HTML Help Workshop with the" \ - ".hhp project file in $(BUILDDIR)/htmlhelp." - -qthelp: - $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp - @echo - @echo "Build finished; now you can run "qcollectiongenerator" with the" \ - ".qhcp project file in $(BUILDDIR)/qthelp, like this:" - @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/PyLD.qhcp" - @echo "To view the help file:" - @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/PyLD.qhc" - -devhelp: - $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp - @echo - @echo "Build finished." - @echo "To view the help file:" - @echo "# mkdir -p $$HOME/.local/share/devhelp/PyLD" - @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/PyLD" - @echo "# devhelp" - -epub: - $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub - @echo - @echo "Build finished. The epub file is in $(BUILDDIR)/epub." - -latex: - $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex - @echo - @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." - @echo "Run \`make' in that directory to run these through (pdf)latex" \ - "(use \`make latexpdf' here to do that automatically)." - -latexpdf: - $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex - @echo "Running LaTeX files through pdflatex..." - make -C $(BUILDDIR)/latex all-pdf - @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." - -text: - $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text - @echo - @echo "Build finished. The text files are in $(BUILDDIR)/text." - -man: - $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man - @echo - @echo "Build finished. The manual pages are in $(BUILDDIR)/man." - -changes: - $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes - @echo - @echo "The overview file is in $(BUILDDIR)/changes." - -linkcheck: - $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck - @echo - @echo "Link check complete; look for any errors in the above output " \ - "or in $(BUILDDIR)/linkcheck/output.txt." - -doctest: - $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest - @echo "Testing of doctests in the sources finished, look at the " \ - "results in $(BUILDDIR)/doctest/output.txt." From d417b98030c43b48f4ec09a06475ee8f9c22ceae Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 22:43:48 +0400 Subject: [PATCH 16/35] Remove Sphinx docs configuration --- docs/conf.py | 223 --------------------------------------------------- 1 file changed, 223 deletions(-) delete mode 100644 docs/conf.py diff --git a/docs/conf.py b/docs/conf.py deleted file mode 100644 index c4bf2b85..00000000 --- a/docs/conf.py +++ /dev/null @@ -1,223 +0,0 @@ -# -*- coding: utf-8 -*- -# -# PyLD documentation build configuration file, created by -# sphinx-quickstart on Mon Aug 29 15:25:28 2011. -# -# This file is execfile()d with the current directory set to its containing dir. -# -# Note that not all possible configuration values are present in this -# autogenerated file. -# -# All configuration values have a default; values that are commented out -# serve to show the default. - -import sys, os - -# If extensions (or modules to document with autodoc) are in another directory, -# add these directories to sys.path here. If the directory is relative to the -# documentation root, use os.path.abspath to make it absolute, like shown here. -#sys.path.insert(0, os.path.abspath('.')) - -current_path = os.path.abspath(os.path.dirname(__file__)) -path = os.path.join(current_path, '..') - -sys.path[0:0] = [ - os.path.join(path, 'lib'), -] - -# -- General configuration ----------------------------------------------------- - -# If your documentation needs a minimal Sphinx version, state it here. -#needs_sphinx = '1.0' - -# Add any Sphinx extension module names here, as strings. They can be extensions -# coming with Sphinx (named 'sphinx.ext.*') or your custom ones. -extensions = ['sphinx.ext.autodoc', 'sphinx.ext.todo', 'sphinx.ext.coverage', 'sphinx.ext.ifconfig', 'sphinx.ext.viewcode'] - -# Add any paths that contain templates here, relative to this directory. -templates_path = ['_templates'] - -# The suffix of source filenames. -source_suffix = '.rst' - -# The encoding of source files. -#source_encoding = 'utf-8-sig' - -# The master toctree document. -master_doc = 'index' - -# General information about the project. -project = u'PyLD' -copyright = u'2011, Digital Bazaar' - -# The version info for the project you're documenting, acts as replacement for -# |version| and |release|, also used in various other places throughout the -# built documents. -# -# The short X.Y version. -version = '0.0' -# The full version, including alpha/beta/rc tags. -release = '0.0.1' - -# The language for content autogenerated by Sphinx. Refer to documentation -# for a list of supported languages. -#language = None - -# There are two options for replacing |today|: either, you set today to some -# non-false value, then it is used: -#today = '' -# Else, today_fmt is used as the format for a strftime call. -#today_fmt = '%B %d, %Y' - -# List of patterns, relative to source directory, that match files and -# directories to ignore when looking for source files. -exclude_patterns = ['_build'] - -# The reST default role (used for this markup: `text`) to use for all documents. -#default_role = None - -# If true, '()' will be appended to :func: etc. cross-reference text. -#add_function_parentheses = True - -# If true, the current module name will be prepended to all description -# unit titles (such as .. function::). -#add_module_names = True - -# If true, sectionauthor and moduleauthor directives will be shown in the -# output. They are ignored by default. -#show_authors = False - -# The name of the Pygments (syntax highlighting) style to use. -pygments_style = 'sphinx' - -# A list of ignored prefixes for module index sorting. -#modindex_common_prefix = [] - - -# -- Options for HTML output --------------------------------------------------- - -# The theme to use for HTML and HTML Help pages. See the documentation for -# a list of builtin themes. -html_theme = 'default' - -# Theme options are theme-specific and customize the look and feel of a theme -# further. For a list of options available for each theme, see the -# documentation. -#html_theme_options = {} - -# Add any paths that contain custom themes here, relative to this directory. -#html_theme_path = [] - -# The name for this set of Sphinx documents. If None, it defaults to -# " v documentation". -#html_title = None - -# A shorter title for the navigation bar. Default is the same as html_title. -#html_short_title = None - -# The name of an image file (relative to this directory) to place at the top -# of the sidebar. -#html_logo = None - -# The name of an image file (within the static path) to use as favicon of the -# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 -# pixels large. -#html_favicon = None - -# Add any paths that contain custom static files (such as style sheets) here, -# relative to this directory. They are copied after the builtin static files, -# so a file named "default.css" will overwrite the builtin "default.css". -html_static_path = ['_static'] - -# If not '', a 'Last updated on:' timestamp is inserted at every page bottom, -# using the given strftime format. -#html_last_updated_fmt = '%b %d, %Y' - -# If true, SmartyPants will be used to convert quotes and dashes to -# typographically correct entities. -#html_use_smartypants = True - -# Custom sidebar templates, maps document names to template names. -#html_sidebars = {} - -# Additional templates that should be rendered to pages, maps page names to -# template names. -#html_additional_pages = {} - -# If false, no module index is generated. -#html_domain_indices = True - -# If false, no index is generated. -#html_use_index = True - -# If true, the index is split into individual pages for each letter. -#html_split_index = False - -# If true, links to the reST sources are added to the pages. -#html_show_sourcelink = True - -# If true, "Created using Sphinx" is shown in the HTML footer. Default is True. -#html_show_sphinx = True - -# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. -#html_show_copyright = True - -# If true, an OpenSearch description file will be output, and all pages will -# contain a tag referring to it. The value of this option must be the -# base URL from which the finished HTML is served. -#html_use_opensearch = '' - -# This is the file name suffix for HTML files (e.g. ".xhtml"). -#html_file_suffix = None - -# Output file base name for HTML help builder. -htmlhelp_basename = 'PyLDdoc' - - -# -- Options for LaTeX output -------------------------------------------------- - -# The paper size ('letter' or 'a4'). -#latex_paper_size = 'letter' - -# The font size ('10pt', '11pt' or '12pt'). -#latex_font_size = '10pt' - -# Grouping the document tree into LaTeX files. List of tuples -# (source start file, target name, title, author, documentclass [howto/manual]). -latex_documents = [ - ('index', 'PyLD.tex', u'PyLD Documentation', - u'Digital Bazaar', 'manual'), -] - -# The name of an image file (relative to this directory) to place at the top of -# the title page. -#latex_logo = None - -# For "manual" documents, if this is true, then toplevel headings are parts, -# not chapters. -#latex_use_parts = False - -# If true, show page references after internal links. -#latex_show_pagerefs = False - -# If true, show URL addresses after external links. -#latex_show_urls = False - -# Additional stuff for the LaTeX preamble. -#latex_preamble = '' - -# Documents to append as an appendix to all manuals. -#latex_appendices = [] - -# If false, no module index is generated. -#latex_domain_indices = True - - -# -- Options for manual page output -------------------------------------------- - -# One entry per manual page. List of tuples -# (source start file, name, description, authors, manual section). -man_pages = [ - ('index', 'pyld', u'PyLD Documentation', - [u'Digital Bazaar'], 1) -] From f7e5e15f8cf550a557ab88f42c68b0d7d80c6a6d Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 22:43:53 +0400 Subject: [PATCH 17/35] Remove Sphinx docs index --- docs/index.rst | 68 -------------------------------------------------- 1 file changed, 68 deletions(-) delete mode 100644 docs/index.rst diff --git a/docs/index.rst b/docs/index.rst deleted file mode 100644 index 89b11570..00000000 --- a/docs/index.rst +++ /dev/null @@ -1,68 +0,0 @@ -.. PyLD documentation master file, created by - sphinx-quickstart on Mon Aug 29 15:25:28 2011. - You can adapt this file completely to your liking, but it should at least - contain the root `toctree` directive. - -Welcome to PyLD! -================ - -PyLD is a `Python`_ implementation of a `JSON-LD`_ processor. - - -API Reference -------------- - -.. toctree:: - :maxdepth: 2 - -.. module:: pyld.jsonld -.. autofunction:: compact -.. autofunction:: expand -.. autofunction:: flatten -.. autofunction:: frame -.. autofunction:: normalize - -.. module:: pyld -.. autoclass:: DocumentLoader - :members: -.. autoclass:: RequestsDocumentLoader - :members: -.. autoclass:: AioHttpDocumentLoader - :members: -.. autoclass:: FrozenDocumentLoader - :members: -.. autodata:: BUNDLED_CONTEXTS - -Indices and tables ------------------- - -* :ref:`genindex` -* :ref:`modindex` -* :ref:`search` - -Requirements ------------- - -PyLD is compatible with `Python`_ 2.5 and newer. - -Credits -------- - -Thanks to `Digital Bazaar`_, the JavaScript JSON-LD parser, and the `JSON-LD`_ community. - -Contribute ----------- - -Source code is available: - - https://github.com/digitalbazaar/pyld - -License -------- - -PyLD is licensed under a `BSD 3-Clause license`_. - -.. _JSON-LD: https://json-ld.org/ -.. _Digital Bazaar: https://digitalbazaar.com/ -.. _Python: https://www.python.org/ -.. _BSD 3-Clause License: https://opensource.org/licenses/BSD-3-Clause From a5b00e8ef08dd557370074395b50c8ff811039ff Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 22:43:59 +0400 Subject: [PATCH 18/35] Remove Sphinx Windows build script --- docs/make.bat | 170 -------------------------------------------------- 1 file changed, 170 deletions(-) delete mode 100644 docs/make.bat diff --git a/docs/make.bat b/docs/make.bat deleted file mode 100644 index a24b446a..00000000 --- a/docs/make.bat +++ /dev/null @@ -1,170 +0,0 @@ -@ECHO OFF - -REM Command file for Sphinx documentation - -if "%SPHINXBUILD%" == "" ( - set SPHINXBUILD=sphinx-build -) -set BUILDDIR=_build -set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% . -if NOT "%PAPER%" == "" ( - set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS% -) - -if "%1" == "" goto help - -if "%1" == "help" ( - :help - echo.Please use `make ^` where ^ is one of - echo. html to make standalone HTML files - echo. dirhtml to make HTML files named index.html in directories - echo. singlehtml to make a single large HTML file - echo. pickle to make pickle files - echo. json to make JSON files - echo. htmlhelp to make HTML files and a HTML help project - echo. qthelp to make HTML files and a qthelp project - echo. devhelp to make HTML files and a Devhelp project - echo. epub to make an epub - echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter - echo. text to make text files - echo. man to make manual pages - echo. changes to make an overview over all changed/added/deprecated items - echo. linkcheck to check all external links for integrity - echo. doctest to run all doctests embedded in the documentation if enabled - goto end -) - -if "%1" == "clean" ( - for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i - del /q /s %BUILDDIR%\* - goto end -) - -if "%1" == "html" ( - %SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html - if errorlevel 1 exit /b 1 - echo. - echo.Build finished. The HTML pages are in %BUILDDIR%/html. - goto end -) - -if "%1" == "dirhtml" ( - %SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml - if errorlevel 1 exit /b 1 - echo. - echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml. - goto end -) - -if "%1" == "singlehtml" ( - %SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml - if errorlevel 1 exit /b 1 - echo. - echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml. - goto end -) - -if "%1" == "pickle" ( - %SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle - if errorlevel 1 exit /b 1 - echo. - echo.Build finished; now you can process the pickle files. - goto end -) - -if "%1" == "json" ( - %SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json - if errorlevel 1 exit /b 1 - echo. - echo.Build finished; now you can process the JSON files. - goto end -) - -if "%1" == "htmlhelp" ( - %SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp - if errorlevel 1 exit /b 1 - echo. - echo.Build finished; now you can run HTML Help Workshop with the ^ -.hhp project file in %BUILDDIR%/htmlhelp. - goto end -) - -if "%1" == "qthelp" ( - %SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp - if errorlevel 1 exit /b 1 - echo. - echo.Build finished; now you can run "qcollectiongenerator" with the ^ -.qhcp project file in %BUILDDIR%/qthelp, like this: - echo.^> qcollectiongenerator %BUILDDIR%\qthelp\PyLD.qhcp - echo.To view the help file: - echo.^> assistant -collectionFile %BUILDDIR%\qthelp\PyLD.ghc - goto end -) - -if "%1" == "devhelp" ( - %SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp - if errorlevel 1 exit /b 1 - echo. - echo.Build finished. - goto end -) - -if "%1" == "epub" ( - %SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub - if errorlevel 1 exit /b 1 - echo. - echo.Build finished. The epub file is in %BUILDDIR%/epub. - goto end -) - -if "%1" == "latex" ( - %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex - if errorlevel 1 exit /b 1 - echo. - echo.Build finished; the LaTeX files are in %BUILDDIR%/latex. - goto end -) - -if "%1" == "text" ( - %SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text - if errorlevel 1 exit /b 1 - echo. - echo.Build finished. The text files are in %BUILDDIR%/text. - goto end -) - -if "%1" == "man" ( - %SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man - if errorlevel 1 exit /b 1 - echo. - echo.Build finished. The manual pages are in %BUILDDIR%/man. - goto end -) - -if "%1" == "changes" ( - %SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes - if errorlevel 1 exit /b 1 - echo. - echo.The overview file is in %BUILDDIR%/changes. - goto end -) - -if "%1" == "linkcheck" ( - %SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck - if errorlevel 1 exit /b 1 - echo. - echo.Link check complete; look for any errors in the above output ^ -or in %BUILDDIR%/linkcheck/output.txt. - goto end -) - -if "%1" == "doctest" ( - %SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest - if errorlevel 1 exit /b 1 - echo. - echo.Testing of doctests in the sources finished, look at the ^ -results in %BUILDDIR%/doctest/output.txt. - goto end -) - -:end From a75f9a8abb6030fac17120d2f9eef1150504e072 Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 22:45:27 +0400 Subject: [PATCH 19/35] Build documentation on every branch push --- .github/workflows/docs.yaml | 2 -- 1 file changed, 2 deletions(-) diff --git a/.github/workflows/docs.yaml b/.github/workflows/docs.yaml index f44a2b23..f15f7e6d 100644 --- a/.github/workflows/docs.yaml +++ b/.github/workflows/docs.yaml @@ -2,8 +2,6 @@ name: Documentation on: push: - branches: - - 'master' pull_request: branches: - '**' From af6c2e94d8519829ad6708a82f4b4ae5610d274d Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 22:54:27 +0400 Subject: [PATCH 20/35] Pin documentation workflow actions --- .github/workflows/docs.yaml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/docs.yaml b/.github/workflows/docs.yaml index f15f7e6d..e049e9ed 100644 --- a/.github/workflows/docs.yaml +++ b/.github/workflows/docs.yaml @@ -19,10 +19,10 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@v5 + - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 with: persist-credentials: false - - uses: actions/setup-python@v6 + - uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0 with: python-version: '3.14' cache: 'pip' @@ -34,7 +34,7 @@ jobs: run: mkdocs build --strict - name: Upload Pages artifact if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }} - uses: actions/upload-pages-artifact@v3 + uses: actions/upload-pages-artifact@56afc609e74202658d3ffba0e8f6dda462b719fa # v3.0.1 with: path: site @@ -52,4 +52,4 @@ jobs: steps: - name: Deploy to GitHub Pages id: deployment - uses: actions/deploy-pages@v4 + uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4.0.5 From 178bb760920826e76d582e27228f5de0d0933631 Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 23:06:32 +0400 Subject: [PATCH 21/35] Add documentation macros module --- docs_macros.py | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) create mode 100644 docs_macros.py diff --git a/docs_macros.py b/docs_macros.py new file mode 100644 index 00000000..28f1aa7e --- /dev/null +++ b/docs_macros.py @@ -0,0 +1,60 @@ +import ast +from pathlib import Path + + +ROOT_DIR = Path(__file__).resolve().parent +BUNDLED_CONTEXTS_SOURCE = ( + ROOT_DIR / 'lib' / 'pyld' / 'documentloader' / 'frozen' / '__init__.py' +) +BUNDLED_CONTEXTS_DIR = ( + ROOT_DIR / 'lib' / 'pyld' / 'documentloader' / 'frozen' / 'bundled' +) + + +def _bundled_contexts(): + module = ast.parse(BUNDLED_CONTEXTS_SOURCE.read_text(encoding='utf-8')) + for statement in module.body: + if not isinstance(statement, ast.AnnAssign): + continue + if not isinstance(statement.target, ast.Name): + continue + if statement.target.id != 'BUNDLED_CONTEXTS': + continue + if not isinstance(statement.value, ast.Dict): + break + contexts = {} + for key_node, value_node in zip(statement.value.keys, statement.value.values): + if ( + not isinstance(key_node, ast.Constant) + or not isinstance(key_node.value, str) + ): + continue + contexts[key_node.value] = _bundled_context_path(value_node) + return contexts + raise RuntimeError(f'Could not find BUNDLED_CONTEXTS in {BUNDLED_CONTEXTS_SOURCE}') + + +def _bundled_context_path(value_node): + if ( + isinstance(value_node, ast.BinOp) + and isinstance(value_node.op, ast.Div) + and isinstance(value_node.left, ast.Name) + and value_node.left.id == '_BUNDLED_DIR' + and isinstance(value_node.right, ast.Constant) + and isinstance(value_node.right.value, str) + ): + return BUNDLED_CONTEXTS_DIR / value_node.right.value + raise RuntimeError('Unsupported BUNDLED_CONTEXTS value shape') + + +def define_env(env): + @env.macro + def bundled_contexts_table(): + rows = [ + '| Context URL | Bundled file |', + '| --- | --- |', + ] + for url, path in sorted(_bundled_contexts().items()): + relative_path = Path(path).relative_to(ROOT_DIR) + rows.append(f'| `{url}` | `{relative_path}` |') + return '\n'.join(rows) From 747e7d2224f5ebb301768057244f56b0381be51b Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 23:06:37 +0400 Subject: [PATCH 22/35] Add MkDocs macros dependency --- docs/requirements.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/requirements.txt b/docs/requirements.txt index 4c8f017d..bba465d8 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -1 +1,2 @@ mkdocs-material +mkdocs-macros-plugin From fc68f8ca33555af6d71182128fb1ed12227c8cf2 Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 23:06:42 +0400 Subject: [PATCH 23/35] Enable MkDocs macros plugin --- mkdocs.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mkdocs.yml b/mkdocs.yml index 42d0d030..835940a7 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -36,4 +36,6 @@ markdown_extensions: permalink: true plugins: + - macros: + module_name: docs_macros - search From d49ca6d2b1ac99c331d2e353e579d69a18848ae1 Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 23:06:49 +0400 Subject: [PATCH 24/35] Render bundled contexts in frozen loader docs --- docs/document-loaders/frozen.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/document-loaders/frozen.md b/docs/document-loaders/frozen.md index f0ebcdae..7b307c19 100644 --- a/docs/document-loaders/frozen.md +++ b/docs/document-loaders/frozen.md @@ -12,6 +12,10 @@ from pyld import FrozenDocumentLoader, jsonld jsonld.set_document_loader(FrozenDocumentLoader()) ``` +## Bundled Contexts + +{{ bundled_contexts_table() }} + Extend the bundled mapping with additional vetted contexts: ```python From 8d5417ffe775536a0c5254d4365cdb3639c96c46 Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 23:09:27 +0400 Subject: [PATCH 25/35] Import bundled contexts in docs macro --- docs_macros.py | 49 +++++-------------------------------------------- 1 file changed, 5 insertions(+), 44 deletions(-) diff --git a/docs_macros.py b/docs_macros.py index 28f1aa7e..4b8f0fe2 100644 --- a/docs_macros.py +++ b/docs_macros.py @@ -1,60 +1,21 @@ -import ast from pathlib import Path +import sys ROOT_DIR = Path(__file__).resolve().parent -BUNDLED_CONTEXTS_SOURCE = ( - ROOT_DIR / 'lib' / 'pyld' / 'documentloader' / 'frozen' / '__init__.py' -) -BUNDLED_CONTEXTS_DIR = ( - ROOT_DIR / 'lib' / 'pyld' / 'documentloader' / 'frozen' / 'bundled' -) - - -def _bundled_contexts(): - module = ast.parse(BUNDLED_CONTEXTS_SOURCE.read_text(encoding='utf-8')) - for statement in module.body: - if not isinstance(statement, ast.AnnAssign): - continue - if not isinstance(statement.target, ast.Name): - continue - if statement.target.id != 'BUNDLED_CONTEXTS': - continue - if not isinstance(statement.value, ast.Dict): - break - contexts = {} - for key_node, value_node in zip(statement.value.keys, statement.value.values): - if ( - not isinstance(key_node, ast.Constant) - or not isinstance(key_node.value, str) - ): - continue - contexts[key_node.value] = _bundled_context_path(value_node) - return contexts - raise RuntimeError(f'Could not find BUNDLED_CONTEXTS in {BUNDLED_CONTEXTS_SOURCE}') - - -def _bundled_context_path(value_node): - if ( - isinstance(value_node, ast.BinOp) - and isinstance(value_node.op, ast.Div) - and isinstance(value_node.left, ast.Name) - and value_node.left.id == '_BUNDLED_DIR' - and isinstance(value_node.right, ast.Constant) - and isinstance(value_node.right.value, str) - ): - return BUNDLED_CONTEXTS_DIR / value_node.right.value - raise RuntimeError('Unsupported BUNDLED_CONTEXTS value shape') +sys.path.insert(0, str(ROOT_DIR / 'lib')) def define_env(env): @env.macro def bundled_contexts_table(): + from pyld import BUNDLED_CONTEXTS + rows = [ '| Context URL | Bundled file |', '| --- | --- |', ] - for url, path in sorted(_bundled_contexts().items()): + for url, path in sorted(BUNDLED_CONTEXTS.items()): relative_path = Path(path).relative_to(ROOT_DIR) rows.append(f'| `{url}` | `{relative_path}` |') return '\n'.join(rows) From d9d60353a1fd02505f1296f3dd92db61ecf7acb2 Mon Sep 17 00:00:00 2001 From: Anatoly Scherbakov Date: Mon, 1 Jun 2026 23:09:34 +0400 Subject: [PATCH 26/35] Install runtime dependencies for docs macros --- docs/requirements.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/requirements.txt b/docs/requirements.txt index bba465d8..845f4434 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -1,2 +1,3 @@ +-r ../requirements.txt mkdocs-material mkdocs-macros-plugin From 7de8a94b1f665d5db0324a10a95354f3e1ae9118 Mon Sep 17 00:00:00 2001 From: Miel Vander Sande Date: Tue, 9 Jun 2026 14:00:35 +0200 Subject: [PATCH 27/35] Convert README from rst to md and update the contents. --- CHANGELOG.md | 1 + CONTRIBUTING.rst | 2 +- MANIFEST.in | 2 +- README.rst | 446 ----------------------------------------------- setup.py | 4 +- 5 files changed, 5 insertions(+), 450 deletions(-) delete mode 100644 README.rst diff --git a/CHANGELOG.md b/CHANGELOG.md index 80a614bb..9838cd63 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ class-based `DocumentLoader` instances while preserving the existing callable factory API. The concrete `RequestsDocumentLoader` and `AioHttpDocumentLoader` classes are also importable from `pyld`. +- Convert `./README.rst` (reStructuredText) to `./README.md` (markdown) and update its contents. ### Added - `pyld.DocumentLoader` abstract base class for class-based document loaders, diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index 9668a42a..bdb90ecb 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -78,7 +78,7 @@ As of early 2020, the process to generate an EARL report for the official `JSON-LD Processor Conformance`_ page is: * Run the tests on the ``json-ld-api`` and ``json-ld-framing`` test repos to - generate a ``.jsonld`` test report as explained in [README.rst](./README.rst#tests) + generate a ``.jsonld`` test report as explained in [README.md](./README.md#tests) * Use the rdf_ tool to generate a ``.ttl``: * ``rdf serialize pyld-earl.jsonld --output-format turtle -o pyld-earl.ttl`` diff --git a/MANIFEST.in b/MANIFEST.in index 8cd6fdad..96efd64f 100644 --- a/MANIFEST.in +++ b/MANIFEST.in @@ -1,2 +1,2 @@ -include README.rst README.txt LICENSE CHANGELOG.md +include README.md README.txt LICENSE CHANGELOG.md recursive-include lib/pyld/documentloader/frozen/bundled *.jsonld diff --git a/README.rst b/README.rst deleted file mode 100644 index 1fe27f40..00000000 --- a/README.rst +++ /dev/null @@ -1,446 +0,0 @@ -PyLD -==== - -.. image:: https://travis-ci.org/digitalbazaar/pyld.png?branch=master - :target: https://travis-ci.org/digitalbazaar/pyld - :alt: Build Status - -Introduction ------------- - -This library is an implementation of the JSON-LD_ specification in Python_. - -JSON, as specified in RFC7159_, is a simple language for representing -objects on the Web. Linked Data is a way of describing content across -different documents or Web sites. Web resources are described using -IRIs, and typically are dereferencable entities that may be used to find -more information, creating a "Web of Knowledge". JSON-LD_ is intended -to be a simple publishing method for expressing not only Linked Data in -JSON, but for adding semantics to existing JSON. - -JSON-LD is designed as a light-weight syntax that can be used to express -Linked Data. It is primarily intended to be a way to express Linked Data -in JavaScript and other Web-based programming environments. It is also -useful when building interoperable Web Services and when storing Linked -Data in JSON-based document storage engines. It is practical and -designed to be as simple as possible, utilizing the large number of JSON -parsers and existing code that is in use today. It is designed to be -able to express key-value pairs, RDF data, RDFa_ data, -Microformats_ data, and Microdata_. That is, it supports every -major Web-based structured data model in use today. - -The syntax does not require many applications to change their JSON, but -easily add meaning by adding context in a way that is either in-band or -out-of-band. The syntax is designed to not disturb already deployed -systems running on JSON, but provide a smooth migration path from JSON -to JSON with added semantics. Finally, the format is intended to be fast -to parse, fast to generate, stream-based and document-based processing -compatible, and require a very small memory footprint in order to operate. - -Conformance ------------ - -This library aims to conform with the following: - -- `JSON-LD 1.1 `_, - W3C Candidate Recommendation, - 2019-12-12 or `newer `_ -- `JSON-LD 1.1 Processing Algorithms and API `_, - W3C Candidate Recommendation, - 2019-12-12 or `newer `_ -- `JSON-LD 1.1 Framing `_, - W3C Candidate Recommendation, - 2019-12-12 or `newer `_ -- Working Group `test suite `_ - -The `test runner`_ is often updated to note or skip newer tests that are not -yet supported. - -Requirements ------------- - -- Python_ (3.10 or later) -- Requests_ (optional) -- aiohttp_ (optional, Python 3.5 or later) - -Installation ------------- - -PyLD can be installed with a pip_ `package `_ - -.. code-block:: bash - - pip install PyLD - -Defining a dependency on pyld will not pull in Requests_ or aiohttp_. If you -need one of these for a `Document Loader`_ then either depend on the desired -external library directly or define the requirement as ``PyLD[requests]`` or -``PyLD[aiohttp]``. - -Quick Examples --------------- - -.. code-block:: Python - - from pyld import jsonld - import json - - doc = { - "http://schema.org/name": "Manu Sporny", - "http://schema.org/url": {"@id": "http://manu.sporny.org/"}, - "http://schema.org/image": {"@id": "http://manu.sporny.org/images/manu.png"} - } - - context = { - "name": "http://schema.org/name", - "homepage": {"@id": "http://schema.org/url", "@type": "@id"}, - "image": {"@id": "http://schema.org/image", "@type": "@id"} - } - - # compact a document according to a particular context - # see: https://json-ld.org/spec/latest/json-ld/#compacted-document-form - compacted = jsonld.compact(doc, context) - - print(json.dumps(compacted, indent=2)) - # Output: - # { - # "@context": {...}, - # "image": "http://manu.sporny.org/images/manu.png", - # "homepage": "http://manu.sporny.org/", - # "name": "Manu Sporny" - # } - - # compact using URLs - jsonld.compact('http://example.org/doc', 'http://example.org/context') - - # expand a document, removing its context - # see: https://json-ld.org/spec/latest/json-ld/#expanded-document-form - expanded = jsonld.expand(compacted) - - print(json.dumps(expanded, indent=2)) - # Output: - # [{ - # "http://schema.org/image": [{"@id": "http://manu.sporny.org/images/manu.png"}], - # "http://schema.org/name": [{"@value": "Manu Sporny"}], - # "http://schema.org/url": [{"@id": "http://manu.sporny.org/"}] - # }] - - # expand using URLs - jsonld.expand('http://example.org/doc') - - # flatten a document - # see: https://json-ld.org/spec/latest/json-ld/#flattened-document-form - flattened = jsonld.flatten(doc) - # all deep-level trees flattened to the top-level - - # frame a document - # see: https://json-ld.org/spec/latest/json-ld-framing/#introduction - framed = jsonld.frame(doc, frame) - # document transformed into a particular tree structure per the given frame - - # normalize a document using the RDF Dataset Normalization Algorithm - # (URDNA2015), see: https://www.w3.org/TR/rdf-canon/ - normalized = jsonld.normalize( - doc, {'algorithm': 'URDNA2015', 'format': 'application/n-quads'}) - # normalized is a string that is a canonical representation of the document - # that can be used for hashing, comparison, etc. - -Document Loader ---------------- - -The default document loader for PyLD uses Requests_. In a production -environment you may want to setup a custom loader that, at a minimum, sets a -timeout value. You can also force requests to use https, set client certs, -disable verification, or set other Requests_ parameters. - -.. code-block:: Python - - jsonld.set_document_loader(jsonld.requests_document_loader(timeout=...)) - -The factory remains the compatibility API, and the concrete class is also -available when class-based construction is preferred: - -.. code-block:: Python - - from pyld import RequestsDocumentLoader - - jsonld.set_document_loader(RequestsDocumentLoader(timeout=...)) - -An asynchronous document loader using aiohttp_ is also available. Please note -that this document loader limits asynchronicity to fetching documents only. -The processing loops remain synchronous. - -.. code-block:: Python - - jsonld.set_document_loader(jsonld.aiohttp_document_loader(timeout=...)) - -The concrete aiohttp loader class is available from ``pyld`` as well: - -.. code-block:: Python - - from pyld import AioHttpDocumentLoader - - jsonld.set_document_loader(AioHttpDocumentLoader(timeout=...)) - -When no document loader is specified, the default loader is set to Requests_. -If Requests_ is not available, the loader is set to aiohttp_. The fallback -document loader is a dummy document loader that raises an exception on every -invocation. - -Frozen Document Loader -###################### - -For air-gapped runs, reproducible builds, and security-hardened deployments -that must not perform any remote context fetches at all, PyLD ships -``FrozenDocumentLoader``: a class-based loader that serves only the URLs in -its ``documents`` allowlist and refuses everything else with -``JsonLdError(code='loading document failed')``. - -Instantiating with no arguments serves the curated ``BUNDLED_CONTEXTS`` set -(ActivityStreams, DID v1, Verifiable Credentials v1 and v2, Linked Data -Security v1/v2, Ed25519-2020, and JWS-2020). To extend the bundle with -additional pre-vetted contexts, pass a merged mapping: - -.. code-block:: Python - - from pyld import jsonld, FrozenDocumentLoader, BUNDLED_CONTEXTS - - loader = FrozenDocumentLoader(documents=dict( - BUNDLED_CONTEXTS, - **{'https://example.com/my-ctx': Path('contexts/my-ctx.jsonld')}, - )) - jsonld.expand(doc, options={'documentLoader': loader}) - -This honors the W3C *JSON-LD Best Practices* recommendation that clients -SHOULD attempt to use a locally cached version of contexts (see -`§ Cache JSON-LD Contexts `_). -Refresh the bundled copies with ``make download-bundled-contexts``. - -Customizing the ContextLoader ------------------------------ - -You can customize the way contexts are loaded and cached by passing an instance -of ``ContextResolver``. The following example implements a loader with a -prefilled custom document cache and uses a custom LRU cache for resolved -contexts: - -.. code-block:: Python - - from pyld.jsonld import compact, expand, set_document_loader, ContextResolver - import json - from cachetools import LRUCache - - # Load the Linked Art context from file-system - fh = open('linked-art.json') - js = json.load(fh) - fh.close() - - # Add to document cache - docCache = { - "https://linked.art/ns/v1/linked-art.json": { - "contextUrl": None, - "documentUrl": "https://linked.art/ns/v1/linked-art.json", - "document": js - } - } - - # Custom loader that uses the document cache - def load_document_and_cache(url, options={}): - if url in docCache: - return docCache[url] - doc = {"contextUrl": None, "documentUrl": url, "document": ""} - resp = requests.get(url) - doc["document"] = resp.json() - docCache[url] = doc - return doc - - # Set the custom loader as global document loader - set_document_loader(load_document_and_cache) - # Create custom context resolver with custom LRU cache and custom loader - resolved_context_cache = LRUCache(maxsize=1000) - resolver = ContextResolver(resolved_context_cache, load_document_and_cache) - - # Expand JSON-LD document using custom context resolver - input = {"@context":"https://linked.art/ns/v1/linked-art.json", "id": "tag:foo", "type": "Person"} - output = expand(input, options={'contextResolver': resolver}) - - -It is also possible to change the maximum number of times that the loader recusively fetches contexts, -by passing the ``max_context_urls`` parameter: - -.. code-block:: Python - - resolver = ContextResolver(resolved_context_cache, load_document_and_cache, max_context_urls=20) - # Or you can do... - # resolver = ContextResolver(resolved_context_cache, load_document_and_cache) - # resolver.max_context_urls = 20 - output = expand(input, options={'contextResolver': resolver}) - - -Handling ignored properties during JSON-LD expansion ----------------------------------------------------- - -If a property in a JSON-LD document does not map to an absolute IRI then it is -ignored. You can customize this behaviour by passing a customizable handler to -`on_property_dropped` parameter of `jsonld.expand()`. - -For example, you can introduce a strict mode by raising a ValueError on every -dropped property: - -.. code-block:: Python - - def raise_this(value): - raise ValueError(value) - - jsonld.expand(doc, None, on_property_dropped=raise_this) - -Commercial Support ------------------- - -Commercial support for this library is available upon request from -`Digital Bazaar`_: support@digitalbazaar.com. - -Source ------- - -The source code for the Python implementation of the JSON-LD API -is available at: - -https://github.com/digitalbazaar/pyld - -Tests ------ - -This library includes a sample testing utility which may be used to verify -that changes to the processor maintain the correct output. - -To run the sample tests you will need to get the test suite files, which -by default, are stored in the `specifications/` folder. -The test suites can be obtained by either using git submodules or by cloning -them manually. - -Using git submodules -#################### - -The test suites are included as git submodules to ensure versions are in sync. -When cloning the repository, use the ``--recurse-submodules`` flag to -automatically clone the submodules. -If you have cloned the repository without the submodules, you can initialize -them with the following commands: - -.. code-block:: bash - - git submodule init - git submodule update - -Cloning manually -#################### - -You can also avoid using git submodules by manually cloning -the ``json-ld-api``, ``json-ld-framing``, and ``normalization`` repositories -hosted on GitHub using the following commands: - -.. code-block:: bash - - git clone https://github.com/w3c/json-ld-api ./specifications/json-ld-api - git clone https://github.com/w3c/json-ld-framing ./specifications/json-ld-framing - git clone https://github.com/json-ld/normalization ./specifications/normalization - -Note that you can clone these repositories into any location you wish; however, -if you do not clone them into the default ``specifications/`` folder, you will -need to provide the paths to the test runner as arguments when running the -tests, as explained below - -Running the sample test suites and unittests using pytest -######################################################### - -If the suites repositories are available in the `specifications/` folder of the -PyLD source directory, then all unittests, including the sample test suites, -can be run with `pytest`: - -.. code-block:: bash - - pytest - -If you wish to store the test suites in a different location than the default -``specifications/`` folder, or you want to test individual manifest ``.jsonld`` -files or directories containing a ``manifest.jsonld``, then you can supply -these files or directories as arguments: - -.. code-block:: bash - - # use: pytest --tests=TEST_PATH [--tests=TEST_PATH...] - pytest --tests=./specifications/json-ld-api/tests - -The test runner supports different document loaders by setting -``--loader requests`` or ``--loader aiohttp``. The default document loader is -set to Requests_. - -.. code-block:: bash - - pytest --loader=requests --tests=./specifications/json-ld-api/tests - -An EARL report can be generated using the ``--earl`` option. - -.. code-block:: bash - - pytest --earl=./earl-report.json - -Running the sample test suites using the original test runner -############################################################# - -You can also run the JSON-LD test suites using the original test runner script -provided: - -.. code-block:: bash - - python tests/runtests.py - -If you wish to store the test suites in a different location than the default -``specifications/`` folder, or you want to test individual manifest ``.jsonld`` -files or directories containing a ``manifest.jsonld``, then you can supply -these files or directories as arguments: - -.. code-block:: bash - - python tests/runtests.py TEST_PATH [TEST_PATH...] - -The test runner supports different document loaders by setting ``-l requests`` -or ``-l aiohttp``. The default document loader is set to Requests_. - -.. code-block:: bash - - python tests/runtests.py -l requests ./specifications/json-ld-api/tests - -An EARL report can be generated using the ``-e`` or ``--earl`` option. - -.. code-block:: bash - - python tests/runtests.py -e ./earl-report.json - - -.. _Digital Bazaar: https://digitalbazaar.com/ - -.. _JSON-LD WG 1.1 API: https://www.w3.org/TR/json-ld11-api/ -.. _JSON-LD WG 1.1 Framing: https://www.w3.org/TR/json-ld11-framing/ -.. _JSON-LD WG 1.1: https://www.w3.org/TR/json-ld11/ - -.. _JSON-LD WG API latest: https://w3c.github.io/json-ld-api/ -.. _JSON-LD WG Framing latest: https://w3c.github.io/json-ld-framing/ -.. _JSON-LD WG latest: https://w3c.github.io/json-ld-syntax/ - -.. _JSON-LD Benchmarks: https://json-ld.org/benchmarks/ -.. _JSON-LD WG: https://www.w3.org/2018/json-ld-wg/ -.. _JSON-LD: https://json-ld.org/ -.. _Microdata: http://www.w3.org/TR/microdata/ -.. _Microformats: http://microformats.org/ -.. _Python: https://www.python.org/ -.. _Requests: http://docs.python-requests.org/ -.. _aiohttp: https://aiohttp.readthedocs.io/ -.. _RDFa: http://www.w3.org/TR/rdfa-core/ -.. _RFC7159: http://tools.ietf.org/html/rfc7159 -.. _WG test suite: https://github.com/w3c/json-ld-api/tree/master/tests -.. _errata: http://www.w3.org/2014/json-ld-errata -.. _pip: http://www.pip-installer.org/ -.. _test runner: https://github.com/digitalbazaar/pyld/blob/master/tests/runtests.py -.. _test suite: https://github.com/json-ld/json-ld.org/tree/master/test-suite diff --git a/setup.py b/setup.py index 41fe18e9..62f533b1 100644 --- a/setup.py +++ b/setup.py @@ -18,7 +18,7 @@ os.path.dirname(__file__), 'lib', 'pyld', '__about__.py')) as fp: exec(fp.read(), about) -with open('README.rst') as fp: +with open('README.md') as fp: long_description = fp.read() setup( @@ -26,7 +26,7 @@ version=about['__version__'], description='Python implementation of the JSON-LD API', long_description=long_description, - long_description_content_type="text/x-rst", + long_description_content_type="text/markdown", author='Digital Bazaar', author_email='support@digitalbazaar.com', url='https://github.com/digitalbazaar/pyld', From df2cd9a61e037abc8e79881e3514621ddb5c27e0 Mon Sep 17 00:00:00 2001 From: Miel Vander Sande Date: Tue, 9 Jun 2026 14:03:01 +0200 Subject: [PATCH 28/35] Add README in markdown --- README.md | 319 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 319 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 00000000..ecbf151c --- /dev/null +++ b/README.md @@ -0,0 +1,319 @@ +# PyLD + +## Introduction + +This library is an implementation of the JSON-LD specification in [Python](https://www.python.org/). + +JSON, as specified in [RFC7159](http://tools.ietf.org/html/rfc7159), is a simple language for representing objects on the Web. Linked Data is a way of describing content across different documents or Web sites. Web resources are described using IRIs, and typically are dereferencable entities that may be used to find more information, creating a "Web of Knowledge". [JSON-LD](https://json-ld.org/) is intended to be a simple publishing method for expressing not only Linked Data in JSON, but for adding semantics to existing JSON. + +JSON-LD is designed as a light-weight syntax that can be used to express Linked Data. It is primarily intended to be a way to express Linked Data in JavaScript and other Web-based programming environments. It is also useful when building interoperable Web Services and when storing Linked Data in JSON-based document storage engines. It is practical and designed to be as simple as possible, utilizing the large number of JSON parsers and existing code that is in use today. It is designed to be able to express key-value pairs, RDF data, [RDFa](http://www.w3.org/TR/rdfa-core/) data, [Microformats](http://microformats.org/) data, and [Microdata](http://www.w3.org/TR/microdata/). That is, it supports every major Web-based structured data model in use today. + +The syntax does not require many applications to change their JSON, but easily add meaning by adding context in a way that is either in-band or out-of-band. The syntax is designed to not disturb already deployed systems running on JSON, but provide a smooth migration path from JSON to JSON with added semantics. Finally, the format is intended to be fast to parse, fast to generate, stream-based and document-based processing compatible, and require a very small memory footprint in order to operate. + +## Conformance + +This library aims to conform with the following W3C Recommendations: + +| Standard | Status | +| :--- | :--- | +| [JSON-LD 1.1](https://www.w3.org/TR/json-ld11/) | W3C Recommendation | +| [JSON-LD 1.1 Processing Algorithms and API](https://www.w3.org/TR/json-ld11-api/) | W3C Recommendation | +| [JSON-LD 1.1 Framing](https://www.w3.org/TR/json-ld11-framing/) | W3C Recommendation | +| [RDF Dataset Canonicalization](https://www.w3.org/TR/rdf-canon/) | W3C Recommendation | + + +The [`test runner`](https://github.com/digitalbazaar/pyld/blob/master/tests/runtests.py) is often updated to note or skip newer tests that are not yet supported. + +## Requirements + +* Python (3.10 or later) +* [Requests](http://docs.python-requests.org/) (optional) +* [aiohttp](https://aiohttp.readthedocs.io/) (optional) + +## Installation + +PyLD can be installed with a [pip](http://www.pip-installer.org/) [package](https://pypi.org/project/PyLD/): + +```bash +pip install PyLD +``` + +Defining a dependency on pyld will not pull in [Requests](http://docs.python-requests.org/) or [aiohttp](https://aiohttp.readthedocs.io/). If you need one of these for a [Document Loader]() then either depend on the desired external library directly or define the requirement as `PyLD[requests]` or `PyLD[aiohttp]`. + +## Quick Examples + +```python +from pyld import jsonld +import json + +doc = { + "http://schema.org/name": "Manu Sporny", + "http://schema.org/url": {"@id": "http://manu.sporny.org/"}, + "http://schema.org/image": {"@id": "http://manu.sporny.org/images/manu.png"} +} + +context = { + "name": "http://schema.org/name", + "homepage": {"@id": "http://schema.org/url", "@type": "@id"}, + "image": {"@id": "http://schema.org/image", "@type": "@id"} +} + +# compact a document according to a particular context +# see: https://json-ld.org/spec/latest/json-ld/#compacted-document-form +compacted = jsonld.compact(doc, context) + +print(json.dumps(compacted, indent=2)) +# Output: +# { +# "@context": {...}, +# "image": "http://manu.sporny.org/images/manu.png", +# "homepage": "http://manu.sporny.org/", +# "name": "Manu Sporny" +# } + +# compact using URLs +jsonld.compact('http://example.org/doc', 'http://example.org/context') + +# expand a document, removing its context +# see: https://json-ld.org/spec/latest/json-ld/#expanded-document-form +expanded = jsonld.expand(compacted) + +print(json.dumps(expanded, indent=2)) +# Output: +# [{ +# "http://schema.org/image": [{"@id": "http://manu.sporny.org/images/manu.png"}], +# "http://schema.org/name": [{"@value": "Manu Sporny"}], +# "http://schema.org/url": [{"@id": "http://manu.sporny.org/"}] +# }] + +# expand using URLs +jsonld.expand('http://example.org/doc') + +# flatten a document +# see: https://json-ld.org/spec/latest/json-ld/#flattened-document-form +flattened = jsonld.flatten(doc) +# all deep-level trees flattened to the top-level + +# frame a document +# see: https://json-ld.org/spec/latest/json-ld-framing/#introduction +framed = jsonld.frame(doc, frame) +# document transformed into a particular tree structure per the given frame + +# normalize a document using the RDF Dataset Normalization Algorithm +# (URDNA2015), see: https://www.w3.org/TR/rdf-canon/ +normalized = jsonld.normalize( + doc, {'algorithm': 'URDNA2015', 'format': 'application/n-quads'}) +# normalized is a string that is a canonical representation of the document +# that can be used for hashing, comparison, etc. +``` + +## Document Loader + +The default document loader for PyLD uses [Requests](http://docs.python-requests.org/). In a production environment you may want to setup a custom loader that, at a minimum, sets a timeout value. You can also force requests to use https, set client certs, disable verification, or set other Requests parameters. + +```python +jsonld.set_document_loader(jsonld.requests_document_loader(timeout=...)) +``` + +The factory remains the compatibility API, and the concrete class is also available when class-based construction is preferred: + +```python +from pyld import RequestsDocumentLoader + +jsonld.set_document_loader(RequestsDocumentLoader(timeout=...)) +``` + +An asynchronous document loader using aiohttp is also available. Please note that this document loader limits asynchronicity to fetching documents only. The processing loops remain synchronous. + +```python +jsonld.set_document_loader(jsonld.aiohttp_document_loader(timeout=...)) +``` + +The concrete aiohttp loader class is available from `pyld` as well: + +```python +from pyld import AioHttpDocumentLoader + +jsonld.set_document_loader(AioHttpDocumentLoader(timeout=...)) +``` + +When no document loader is specified, the default loader is set to [Requests](http://docs.python-requests.org/). If Requests is not available, the loader is set to aiohttp. The fallback document loader is a dummy document loader that raises an exception on every invocation. + +## Frozen Document Loader + +For air-gapped runs, reproducible builds, and security-hardened deployments that must not perform any remote context fetches at all, PyLD ships `FrozenDocumentLoader`: a class-based loader that serves only the URLs in its `documents` allowlist and refuses everything else with `JsonLdError(code='loading document failed')`. + +Instantiating with no arguments serves the curated `BUNDLED_CONTEXTS` set (ActivityStreams, DID v1, Verifiable Credentials v1 and v2, Linked Data Security v1/v2, Ed25519-2020, and JWS-2020). To extend the bundle with additional pre-vetted contexts, pass a merged mapping: + +```python +from pyld import jsonld, FrozenDocumentLoader, BUNDLED_CONTEXTS + +loader = FrozenDocumentLoader(documents=dict( + BUNDLED_CONTEXTS, + **{'https://example.com/my-ctx': Path('contexts/my-ctx.jsonld')}, +)) +jsonld.expand(doc, options={'documentLoader': loader}) +``` + +This honors the W3C *JSON-LD Best Practices* recommendation that clients SHOULD attempt to use a locally cached version of contexts (see `§ Cache JSON-LD Contexts `_). +Refresh the bundled copies with `make download-bundled-contexts`. + +## Customizing the ContextLoader + +You can customize the way contexts are loaded and cached by passing an instance of `ContextResolver`. The following example implements a loader with a prefilled custom document cache and uses a custom LRU cache for resolved contexts: + +```python +from pyld.jsonld import compact, expand, set_document_loader, ContextResolver +import json +from cachetools import LRUCache + +# Load the Linked Art context from file-system +fh = open('linked-art.json') +js = json.load(fh) +fh.close() + +# Add to document cache +docCache = { + "https://linked.art/ns/v1/linked-art.json": { + "contextUrl": None, + "documentUrl": "https://linked.art/ns/v1/linked-art.json", + "document": js + } +} + +# Custom loader that uses the document cache +def load_document_and_cache(url, options={}): + if url in docCache: + return docCache[url] + doc = {"contextUrl": None, "documentUrl": url, "document": ""} + resp = requests.get(url) + doc["document"] = resp.json() + docCache[url] = doc + return doc + +# Set the custom loader as global document loader +set_document_loader(load_document_and_cache) +# Create custom context resolver with custom LRU cache and custom loader +resolved_context_cache = LRUCache(maxsize=1000) +resolver = ContextResolver(resolved_context_cache, load_document_and_cache) + +# Expand JSON-LD document using custom context resolver +input = {"@context":"https://linked.art/ns/v1/linked-art.json", "id": "tag:foo", "type": "Person"} +output = expand(input, options={'contextResolver': resolver}) +``` + +It is also possible to change the maximum number of times that the loader recursively fetches contexts, by passing the `max_context_urls` parameter: + +```python +resolver = ContextResolver(resolved_context_cache, load_document_and_cache, max_context_urls=20) +# Or you can do... +# resolver = ContextResolver(resolved_context_cache, load_document_and_cache) +# resolver.max_context_urls = 20 +output = expand(input, options={'contextResolver': resolver}) +``` + +## Handling ignored properties during JSON-LD expansion + +If a property in a JSON-LD document does not map to an absolute IRI then it is ignored. You can customize this behaviour by passing a customizable handler to `on_property_dropped` parameter of `jsonld.expand()`. + +For example, you can introduce a strict mode by raising a ValueError on every dropped property: + +```python +def raise_this(value): + raise ValueError(value) + +jsonld.expand(doc, None, on_property_dropped=raise_this) +``` + +## Commercial Support + +Commercial support for this library is available upon request from [`Digital Bazaar`](mailto:support@digitalbazaar.com). + +## Source + +The source code for the Python implementation of the JSON-LD API is available at: + +[https://github.com/digitalbazaar/pyld](https://github.com/digitalbazaar/pyld) + +## Tests + +This library includes a sample testing utility which may be used to verify that changes to the processor maintain the correct output. + +To run the sample tests you will need to get the test suite files, which by default, are stored in the `specifications/` folder. +The test suites can be obtained by either using git submodules or by cloning them manually. + +### Using git submodules + +The test suites are included as git submodules to ensure versions are in sync. +When cloning the repository, use the `--recurse-submodules` flag to automatically clone the submodules. +If you have cloned the repository without the submodules, you can initialize them with the following commands: + +```bash +git submodule init +git submodule update +``` + +### Cloning manually + +You can also avoid using git submodules by manually cloning the `json-ld-api`, `json-ld-framing`, and `normalization` repositories hosted on GitHub using the following commands: + +```bash +git clone https://github.com/w3c/json-ld-api ./specifications/json-ld-api +git clone https://github.com/w3c/json-ld-framing ./specifications/json-ld-framing +git clone https://github.com/json-ld/normalization ./specifications/normalization +``` + +Note that you can clone these repositories into any location you wish; however, if you do not clone them into the default `specifications/` folder, you will need to provide the paths to the test runner as arguments when running the tests, as explained below. + +### Running the sample test suites and unit tests using pytest + +If the suites repositories are available in the `specifications/` folder of the PyLD source directory, then all unittests, including the sample test suites, can be run with `pytest`: + +```bash +pytest +``` + +If you wish to store the test suites in a different location than the default `specifications/` folder, or you want to test individual manifest `.jsonld` files or directories containing a `manifest.jsonld`, then you can supply these files or directories as arguments: + +```bash +# use: pytest --tests=TEST_PATH [--tests=TEST_PATH...] +pytest --tests=./specifications/json-ld-api/tests +``` + +The test runner supports different document loaders by setting `--loader requests` or `--loader aiohttp`. The default document loader is set to [Requests](http://docs.python-requests.org/). + +```bash +pytest --loader=requests --tests=./specifications/json-ld-api/tests +``` + +An EARL report can be generated using the `--earl` option. + +```bash +pytest --earl=./earl-report.json +``` + +### Running the sample test suites using the original test runner + +You can also run the JSON-LD test suites using the original test runner script provided: + +```bash +python tests/runtests.py +``` + +If you wish to store the test suites in a different location than the default `specifications/` folder, or you want to test individual manifest `.jsonld` files or directories containing a `manifest.jsonld`, then you can supply these files or directories as arguments: + +```bash +python tests/runtests.py TEST_PATH [TEST_PATH...] +``` + +The test runner supports different document loaders by setting `-l requests` or `-l aiohttp`. The default document loader is set to [Requests](http://docs.python-requests.org/). + +```bash +python tests/runtests.py -l requests ./specifications/json-ld-api/tests +``` + +An EARL report can be generated using the `-e` or `--earl` option. + +```bash +python tests/runtests.py -e ./earl-report.json +``` \ No newline at end of file From ee797ac33852c22f3f13a0e09cf5caa47c609eba Mon Sep 17 00:00:00 2001 From: Miel Vander Sande Date: Tue, 9 Jun 2026 15:06:17 +0200 Subject: [PATCH 29/35] Change symbolic link README.txt --- README.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.txt b/README.txt index 92cacd28..42061c01 120000 --- a/README.txt +++ b/README.txt @@ -1 +1 @@ -README.rst \ No newline at end of file +README.md \ No newline at end of file From 54105bb623cc454a6be6f8f52c0a24abd4b309c9 Mon Sep 17 00:00:00 2001 From: Miel Vander Sande Date: Tue, 9 Jun 2026 15:10:57 +0200 Subject: [PATCH 30/35] Wrap markdown at 80 ch --- README.md | 152 +++++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 115 insertions(+), 37 deletions(-) diff --git a/README.md b/README.md index ecbf151c..d58c674d 100644 --- a/README.md +++ b/README.md @@ -2,13 +2,36 @@ ## Introduction -This library is an implementation of the JSON-LD specification in [Python](https://www.python.org/). - -JSON, as specified in [RFC7159](http://tools.ietf.org/html/rfc7159), is a simple language for representing objects on the Web. Linked Data is a way of describing content across different documents or Web sites. Web resources are described using IRIs, and typically are dereferencable entities that may be used to find more information, creating a "Web of Knowledge". [JSON-LD](https://json-ld.org/) is intended to be a simple publishing method for expressing not only Linked Data in JSON, but for adding semantics to existing JSON. - -JSON-LD is designed as a light-weight syntax that can be used to express Linked Data. It is primarily intended to be a way to express Linked Data in JavaScript and other Web-based programming environments. It is also useful when building interoperable Web Services and when storing Linked Data in JSON-based document storage engines. It is practical and designed to be as simple as possible, utilizing the large number of JSON parsers and existing code that is in use today. It is designed to be able to express key-value pairs, RDF data, [RDFa](http://www.w3.org/TR/rdfa-core/) data, [Microformats](http://microformats.org/) data, and [Microdata](http://www.w3.org/TR/microdata/). That is, it supports every major Web-based structured data model in use today. - -The syntax does not require many applications to change their JSON, but easily add meaning by adding context in a way that is either in-band or out-of-band. The syntax is designed to not disturb already deployed systems running on JSON, but provide a smooth migration path from JSON to JSON with added semantics. Finally, the format is intended to be fast to parse, fast to generate, stream-based and document-based processing compatible, and require a very small memory footprint in order to operate. +This library is an implementation of the JSON-LD specification in +[Python](https://www.python.org/). + +JSON, as specified in [RFC7159](http://tools.ietf.org/html/rfc7159), is a simple +language for representing objects on the Web. Linked Data is a way of describing +content across different documents or Web sites. Web resources are described +using IRIs, and typically are dereferencable entities that may be used to find +more information, creating a "Web of Knowledge". [JSON-LD](https://json-ld.org/) +is intended to be a simple publishing method for expressing not only Linked Data +in JSON, but for adding semantics to existing JSON. + +JSON-LD is designed as a light-weight syntax that can be used to express Linked +Data. It is primarily intended to be a way to express Linked Data in JavaScript +and other Web-based programming environments. It is also useful when building +interoperable Web Services and when storing Linked Data in JSON-based document +storage engines. It is practical and designed to be as simple as possible, +utilizing the large number of JSON parsers and existing code that is in use +today. It is designed to be able to express key-value pairs, RDF data, +[RDFa](http://www.w3.org/TR/rdfa-core/) data, +[Microformats](http://microformats.org/) data, and +[Microdata](http://www.w3.org/TR/microdata/). That is, it supports every major +Web-based structured data model in use today. + +The syntax does not require many applications to change their JSON, but easily +add meaning by adding context in a way that is either in-band or out-of-band. +The syntax is designed to not disturb already deployed systems running on JSON, +but provide a smooth migration path from JSON to JSON with added semantics. +Finally, the format is intended to be fast to parse, fast to generate, +stream-based and document-based processing compatible, and require a very small +memory footprint in order to operate. ## Conformance @@ -22,7 +45,9 @@ This library aims to conform with the following W3C Recommendations: | [RDF Dataset Canonicalization](https://www.w3.org/TR/rdf-canon/) | W3C Recommendation | -The [`test runner`](https://github.com/digitalbazaar/pyld/blob/master/tests/runtests.py) is often updated to note or skip newer tests that are not yet supported. +The [`test +runner`](https://github.com/digitalbazaar/pyld/blob/master/tests/runtests.py) is +often updated to note or skip newer tests that are not yet supported. ## Requirements @@ -32,13 +57,18 @@ The [`test runner`](https://github.com/digitalbazaar/pyld/blob/master/tests/runt ## Installation -PyLD can be installed with a [pip](http://www.pip-installer.org/) [package](https://pypi.org/project/PyLD/): +PyLD can be installed with a [pip](http://www.pip-installer.org/) +[package](https://pypi.org/project/PyLD/): ```bash pip install PyLD ``` -Defining a dependency on pyld will not pull in [Requests](http://docs.python-requests.org/) or [aiohttp](https://aiohttp.readthedocs.io/). If you need one of these for a [Document Loader]() then either depend on the desired external library directly or define the requirement as `PyLD[requests]` or `PyLD[aiohttp]`. +Defining a dependency on pyld will not pull in +[Requests](http://docs.python-requests.org/) or +[aiohttp](https://aiohttp.readthedocs.io/). If you need one of these for a +[Document Loader]() then either depend on the desired external library directly +or define the requirement as `PyLD[requests]` or `PyLD[aiohttp]`. ## Quick Examples @@ -109,13 +139,18 @@ normalized = jsonld.normalize( ## Document Loader -The default document loader for PyLD uses [Requests](http://docs.python-requests.org/). In a production environment you may want to setup a custom loader that, at a minimum, sets a timeout value. You can also force requests to use https, set client certs, disable verification, or set other Requests parameters. +The default document loader for PyLD uses +[Requests](http://docs.python-requests.org/). In a production environment you +may want to setup a custom loader that, at a minimum, sets a timeout value. You +can also force requests to use https, set client certs, disable verification, or +set other Requests parameters. ```python jsonld.set_document_loader(jsonld.requests_document_loader(timeout=...)) ``` -The factory remains the compatibility API, and the concrete class is also available when class-based construction is preferred: +The factory remains the compatibility API, and the concrete class is also +available when class-based construction is preferred: ```python from pyld import RequestsDocumentLoader @@ -123,7 +158,9 @@ from pyld import RequestsDocumentLoader jsonld.set_document_loader(RequestsDocumentLoader(timeout=...)) ``` -An asynchronous document loader using aiohttp is also available. Please note that this document loader limits asynchronicity to fetching documents only. The processing loops remain synchronous. +An asynchronous document loader using aiohttp is also available. Please note +that this document loader limits asynchronicity to fetching documents only. The +processing loops remain synchronous. ```python jsonld.set_document_loader(jsonld.aiohttp_document_loader(timeout=...)) @@ -137,13 +174,23 @@ from pyld import AioHttpDocumentLoader jsonld.set_document_loader(AioHttpDocumentLoader(timeout=...)) ``` -When no document loader is specified, the default loader is set to [Requests](http://docs.python-requests.org/). If Requests is not available, the loader is set to aiohttp. The fallback document loader is a dummy document loader that raises an exception on every invocation. +When no document loader is specified, the default loader is set to +[Requests](http://docs.python-requests.org/). If Requests is not available, the +loader is set to aiohttp. The fallback document loader is a dummy document +loader that raises an exception on every invocation. ## Frozen Document Loader -For air-gapped runs, reproducible builds, and security-hardened deployments that must not perform any remote context fetches at all, PyLD ships `FrozenDocumentLoader`: a class-based loader that serves only the URLs in its `documents` allowlist and refuses everything else with `JsonLdError(code='loading document failed')`. +For air-gapped runs, reproducible builds, and security-hardened deployments that +must not perform any remote context fetches at all, PyLD ships +`FrozenDocumentLoader`: a class-based loader that serves only the URLs in its +`documents` allowlist and refuses everything else with +`JsonLdError(code='loading document failed')`. -Instantiating with no arguments serves the curated `BUNDLED_CONTEXTS` set (ActivityStreams, DID v1, Verifiable Credentials v1 and v2, Linked Data Security v1/v2, Ed25519-2020, and JWS-2020). To extend the bundle with additional pre-vetted contexts, pass a merged mapping: +Instantiating with no arguments serves the curated `BUNDLED_CONTEXTS` set +(ActivityStreams, DID v1, Verifiable Credentials v1 and v2, Linked Data Security +v1/v2, Ed25519-2020, and JWS-2020). To extend the bundle with additional +pre-vetted contexts, pass a merged mapping: ```python from pyld import jsonld, FrozenDocumentLoader, BUNDLED_CONTEXTS @@ -155,12 +202,16 @@ loader = FrozenDocumentLoader(documents=dict( jsonld.expand(doc, options={'documentLoader': loader}) ``` -This honors the W3C *JSON-LD Best Practices* recommendation that clients SHOULD attempt to use a locally cached version of contexts (see `§ Cache JSON-LD Contexts `_). -Refresh the bundled copies with `make download-bundled-contexts`. +This honors the W3C *JSON-LD Best Practices* recommendation that clients SHOULD +attempt to use a locally cached version of contexts (see `§ Cache JSON-LD +Contexts `_). Refresh +the bundled copies with `make download-bundled-contexts`. ## Customizing the ContextLoader -You can customize the way contexts are loaded and cached by passing an instance of `ContextResolver`. The following example implements a loader with a prefilled custom document cache and uses a custom LRU cache for resolved contexts: +You can customize the way contexts are loaded and cached by passing an instance +of `ContextResolver`. The following example implements a loader with a prefilled +custom document cache and uses a custom LRU cache for resolved contexts: ```python from pyld.jsonld import compact, expand, set_document_loader, ContextResolver @@ -202,7 +253,8 @@ input = {"@context":"https://linked.art/ns/v1/linked-art.json", "id": "tag:foo", output = expand(input, options={'contextResolver': resolver}) ``` -It is also possible to change the maximum number of times that the loader recursively fetches contexts, by passing the `max_context_urls` parameter: +It is also possible to change the maximum number of times that the loader +recursively fetches contexts, by passing the `max_context_urls` parameter: ```python resolver = ContextResolver(resolved_context_cache, load_document_and_cache, max_context_urls=20) @@ -214,9 +266,12 @@ output = expand(input, options={'contextResolver': resolver}) ## Handling ignored properties during JSON-LD expansion -If a property in a JSON-LD document does not map to an absolute IRI then it is ignored. You can customize this behaviour by passing a customizable handler to `on_property_dropped` parameter of `jsonld.expand()`. +If a property in a JSON-LD document does not map to an absolute IRI then it is +ignored. You can customize this behaviour by passing a customizable handler to +`on_property_dropped` parameter of `jsonld.expand()`. -For example, you can introduce a strict mode by raising a ValueError on every dropped property: +For example, you can introduce a strict mode by raising a ValueError on every +dropped property: ```python def raise_this(value): @@ -227,26 +282,31 @@ jsonld.expand(doc, None, on_property_dropped=raise_this) ## Commercial Support -Commercial support for this library is available upon request from [`Digital Bazaar`](mailto:support@digitalbazaar.com). +Commercial support for this library is available upon request from [`Digital +Bazaar`](mailto:support@digitalbazaar.com). ## Source -The source code for the Python implementation of the JSON-LD API is available at: +The source code for the Python implementation of the JSON-LD API is available +at: [https://github.com/digitalbazaar/pyld](https://github.com/digitalbazaar/pyld) ## Tests -This library includes a sample testing utility which may be used to verify that changes to the processor maintain the correct output. +This library includes a sample testing utility which may be used to verify that +changes to the processor maintain the correct output. -To run the sample tests you will need to get the test suite files, which by default, are stored in the `specifications/` folder. -The test suites can be obtained by either using git submodules or by cloning them manually. +To run the sample tests you will need to get the test suite files, which by +default, are stored in the `specifications/` folder. The test suites can be +obtained by either using git submodules or by cloning them manually. ### Using git submodules The test suites are included as git submodules to ensure versions are in sync. -When cloning the repository, use the `--recurse-submodules` flag to automatically clone the submodules. -If you have cloned the repository without the submodules, you can initialize them with the following commands: +When cloning the repository, use the `--recurse-submodules` flag to +automatically clone the submodules. If you have cloned the repository without +the submodules, you can initialize them with the following commands: ```bash git submodule init @@ -255,7 +315,9 @@ git submodule update ### Cloning manually -You can also avoid using git submodules by manually cloning the `json-ld-api`, `json-ld-framing`, and `normalization` repositories hosted on GitHub using the following commands: +You can also avoid using git submodules by manually cloning the `json-ld-api`, +`json-ld-framing`, and `normalization` repositories hosted on GitHub using the +following commands: ```bash git clone https://github.com/w3c/json-ld-api ./specifications/json-ld-api @@ -263,24 +325,34 @@ git clone https://github.com/w3c/json-ld-framing ./specifications/json-ld-framin git clone https://github.com/json-ld/normalization ./specifications/normalization ``` -Note that you can clone these repositories into any location you wish; however, if you do not clone them into the default `specifications/` folder, you will need to provide the paths to the test runner as arguments when running the tests, as explained below. +Note that you can clone these repositories into any location you wish; however, +if you do not clone them into the default `specifications/` folder, you will +need to provide the paths to the test runner as arguments when running the +tests, as explained below. ### Running the sample test suites and unit tests using pytest -If the suites repositories are available in the `specifications/` folder of the PyLD source directory, then all unittests, including the sample test suites, can be run with `pytest`: +If the suites repositories are available in the `specifications/` folder of the +PyLD source directory, then all unittests, including the sample test suites, can +be run with `pytest`: ```bash pytest ``` -If you wish to store the test suites in a different location than the default `specifications/` folder, or you want to test individual manifest `.jsonld` files or directories containing a `manifest.jsonld`, then you can supply these files or directories as arguments: +If you wish to store the test suites in a different location than the default +`specifications/` folder, or you want to test individual manifest `.jsonld` +files or directories containing a `manifest.jsonld`, then you can supply these +files or directories as arguments: ```bash # use: pytest --tests=TEST_PATH [--tests=TEST_PATH...] pytest --tests=./specifications/json-ld-api/tests ``` -The test runner supports different document loaders by setting `--loader requests` or `--loader aiohttp`. The default document loader is set to [Requests](http://docs.python-requests.org/). +The test runner supports different document loaders by setting `--loader +requests` or `--loader aiohttp`. The default document loader is set to +[Requests](http://docs.python-requests.org/). ```bash pytest --loader=requests --tests=./specifications/json-ld-api/tests @@ -294,19 +366,25 @@ pytest --earl=./earl-report.json ### Running the sample test suites using the original test runner -You can also run the JSON-LD test suites using the original test runner script provided: +You can also run the JSON-LD test suites using the original test runner script +provided: ```bash python tests/runtests.py ``` -If you wish to store the test suites in a different location than the default `specifications/` folder, or you want to test individual manifest `.jsonld` files or directories containing a `manifest.jsonld`, then you can supply these files or directories as arguments: +If you wish to store the test suites in a different location than the default +`specifications/` folder, or you want to test individual manifest `.jsonld` +files or directories containing a `manifest.jsonld`, then you can supply these +files or directories as arguments: ```bash python tests/runtests.py TEST_PATH [TEST_PATH...] ``` -The test runner supports different document loaders by setting `-l requests` or `-l aiohttp`. The default document loader is set to [Requests](http://docs.python-requests.org/). +The test runner supports different document loaders by setting `-l requests` or +`-l aiohttp`. The default document loader is set to +[Requests](http://docs.python-requests.org/). ```bash python tests/runtests.py -l requests ./specifications/json-ld-api/tests From c612d48247ec4634f443bb47a149d3e74b9c849d Mon Sep 17 00:00:00 2001 From: Miel Vander Sande Date: Wed, 10 Jun 2026 13:41:02 +0200 Subject: [PATCH 31/35] Convert CONTRIBUTING to markdown --- AGENTS.md | 2 +- CONTRIBUTING.md | 89 ++++++++++++++++++++++++++++++++++++++++++++ CONTRIBUTING.rst | 97 ------------------------------------------------ 3 files changed, 90 insertions(+), 98 deletions(-) create mode 100644 CONTRIBUTING.md delete mode 100644 CONTRIBUTING.rst diff --git a/AGENTS.md b/AGENTS.md index df89c8d6..e4187d8f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,6 +1,6 @@ # Agent guidelines -Read [CONTRIBUTING.rst](CONTRIBUTING.rst) for code style, linting (e.g. `make lint`, `make fmt`), and release process. +Read [CONTRIBUTING.md](CONTRIBUTING.md) for code style, linting (e.g. `make lint`, `make fmt`), and release process. ## Testing diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..a12040b9 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,89 @@ +# Contributing to PyLD + +Want to contribute to PyLD? Great! Here are a few notes: + +## Code + +* In general, follow the common [PEP 8 Style Guide](https://www.python.org/dev/peps/pep-0008/). +* Try to make the code pass [ruff](https://docs.astral.sh/ruff/) checks. + + * `make lint` or `ruff check lib/pyld/*` + * You can also apply automatic fixing and formatting + using `make fmt` + +* Use version `X.Y.Z-dev` in dev mode. +* Use version `X.Y.Z` for releases. + +## Documentation + +The public documentation site is built with MkDocs Material. + +* Install documentation dependencies: + + * `pip install -r docs/requirements.txt` + +* Preview documentation locally: + + * `mkdocs serve` + +* Check documentation before submitting changes: + + * `mkdocs build --strict` + +* Refresh bundled JSON-LD context files: + + * `make download-bundled-contexts` + +## Versioning + +* Follow the [Semantic Versioning](https://semver.org/) guidelines. + +## Release Process + +* `$EDITOR CHANGELOG.md`: update CHANGELOG with new notes, version, and date. +* commit changes +* `$EDITOR lib/pyld/__about__.py`: update to release version and remove `-dev` suffix. +* `git commit CHANGELOG.md lib/pyld/__about__.py -m "Release {version}."` +* `git tag {version}` +* `$EDITOR lib/pyld/__about__.py`: update to next version and add `-dev` suffix. +* `git commit lib/pyld/__about__.py -m "Start {next-version}."` +* `git push --tags` + +To ensure a clean [package](https://pypi.org/project/PyLD/) upload to [PyPI](https://pypi.org/), +use a clean checkout, and run the following: + +* For more info, look at the packaging + [guide](https://packaging.python.org/en/latest/guides/distributing-packages-using-setuptools/). +* Setup an [API token](https://pypi.org/help/#apitoken). Recommend using a + specific "PyLD" token and set it up as a "repository" in your + [`~/.pypirc`](https://packaging.python.org/en/latest/specifications/pypirc/) + for use in the upload command. +* The below builds and uploads a sdist and wheel. Adjust as needed depending + on how you manage and clean "dist/" dir files. +* `git checkout {version}` +* `python3 -m build` +* `twine check dist/*` +* `twine upload -r PyLD dist/*` + +## Implementation Report Process + +As of early 2020, the process to generate an EARL report for the official +[JSON-LD Processor Conformance](https://w3c.github.io/json-ld-api/reports/) page is: + +* Run the tests on the `json-ld-api` and `json-ld-framing` test repos to + generate a `.jsonld` test report as explained in [README.md](./README.md#tests) +* Use the [rdf](https://rubygems.org/gems/rdf) tool to generate a `.ttl`: + + * `rdf serialize pyld-earl.jsonld --output-format turtle -o pyld-earl.ttl` + +* Optionally follow the [report instructions](https://github.com/w3c/json-ld-api/tree/master/reports) to generate the HTML report for inspection. +* Submit a PR to the [json-ld-api repository](https://github.com/w3c/json-ld-api/pulls) with at least the `.ttl`. + +.. _JSON-LD Processor Conformance: https://w3c.github.io/json-ld-api/reports/ +.. _PEP 8 Style Guide: https://www.python.org/dev/peps/pep-0008/ +.. _Semantic Versioning: https://semver.org/ +.. _ruff: https://docs.astral.sh/ruff/ +.. _json-ld-api repository: https://github.com/w3c/json-ld-api/pulls +.. _rdf: https://rubygems.org/gems/rdf +.. _report instructions: https://github.com/w3c/json-ld-api/tree/master/reports +.. _PyPI: https://pypi.org/ \ No newline at end of file diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst deleted file mode 100644 index bdb90ecb..00000000 --- a/CONTRIBUTING.rst +++ /dev/null @@ -1,97 +0,0 @@ -Contributing to PyLD -==================== - -Want to contribute to PyLD? Great! Here are a few notes: - -Code ----- - -* In general, follow the common `PEP 8 Style Guide`_. -* Try to make the code pass ruff_ checks. - - * ``make lint`` or ``ruff check lib/pyld/*`` - * you can also apply automatic fixing and formatting - using ``make fmt`` - -* Use version X.Y.Z-dev in dev mode. -* Use version X.Y.Z for releases. - -Documentation -------------- - -The public documentation site is built with MkDocs Material. - -* Install documentation dependencies: - - * ``pip install -r docs/requirements.txt`` - -* Preview documentation locally: - - * ``mkdocs serve`` - -* Check documentation before submitting changes: - - * ``mkdocs build --strict`` - -* Refresh bundled JSON-LD context files: - - * ``make download-bundled-contexts`` - -Versioning ----------- - -* Follow the `Semantic Versioning`_ guidelines. - -Release Process ---------------- - -* ``$EDITOR CHANGELOG.md``: update CHANGELOG with new notes, version, and date. -* commit changes -* ``$EDITOR lib/pyld/__about__.py``: update to release version and remove ``-dev`` - suffix. -* ``git commit CHANGELOG.md lib/pyld/__about__.py -m "Release {version}."`` -* ``git tag {version}`` -* ``$EDITOR lib/pyld/__about__.py``: update to next version and add ``-dev`` suffix. -* ``git commit lib/pyld/__about__.py -m "Start {next-version}."`` -* ``git push --tags`` - -To ensure a clean `package `_ upload to PyPI_, -use a clean checkout, and run the following: - -* For more info, look at the packaging - `guide `_. -* Setup an `API token `_. Recommend using a - specific "PyLD" token and set it up as a "repository" in your - `~/.pypirc `_ - for use in the upload command. -* The below builds and uploads a sdist and wheel. Adjust as needed depending - on how you manage and clean "dist/" dir files. -* ``git checkout {version}`` -* ``python3 -m build`` -* ``twine check dist/*`` -* ``twine upload -r PyLD dist/*`` - -Implementation Report Process ------------------------------ - -As of early 2020, the process to generate an EARL report for the official -`JSON-LD Processor Conformance`_ page is: - -* Run the tests on the ``json-ld-api`` and ``json-ld-framing`` test repos to - generate a ``.jsonld`` test report as explained in [README.md](./README.md#tests) -* Use the rdf_ tool to generate a ``.ttl``: - - * ``rdf serialize pyld-earl.jsonld --output-format turtle -o pyld-earl.ttl`` - -* Optionally follow the `report instructions`_ to generate the HTML report for - inspection. -* Submit a PR to the `json-ld-api repository`_ with at least the ``.ttl``: - -.. _JSON-LD Processor Conformance: https://w3c.github.io/json-ld-api/reports/ -.. _PEP 8 Style Guide: https://www.python.org/dev/peps/pep-0008/ -.. _Semantic Versioning: https://semver.org/ -.. _ruff: https://docs.astral.sh/ruff/ -.. _json-ld-api repository: https://github.com/w3c/json-ld-api/pulls -.. _rdf: https://rubygems.org/gems/rdf -.. _report instructions: https://github.com/w3c/json-ld-api/tree/master/reports -.. _PyPI: https://pypi.org/ From 09e509ca47453b3e48c4fd2611f05162c4c787ed Mon Sep 17 00:00:00 2001 From: Miel Vander Sande Date: Wed, 10 Jun 2026 13:43:04 +0200 Subject: [PATCH 32/35] Update README.md Co-authored-by: Anatoly Scherbakov --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index d58c674d..0dcac4e7 100644 --- a/README.md +++ b/README.md @@ -67,7 +67,7 @@ pip install PyLD Defining a dependency on pyld will not pull in [Requests](http://docs.python-requests.org/) or [aiohttp](https://aiohttp.readthedocs.io/). If you need one of these for a -[Document Loader]() then either depend on the desired external library directly +[Document Loader](#document-loader) then either depend on the desired external library directly or define the requirement as `PyLD[requests]` or `PyLD[aiohttp]`. ## Quick Examples From 4cca15306618272bed28dc3b13d2ec9e6d764d31 Mon Sep 17 00:00:00 2001 From: Miel Vander Sande Date: Wed, 10 Jun 2026 13:43:36 +0200 Subject: [PATCH 33/35] Update README.md Co-authored-by: Anatoly Scherbakov --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 0dcac4e7..a4ead101 100644 --- a/README.md +++ b/README.md @@ -203,9 +203,9 @@ jsonld.expand(doc, options={'documentLoader': loader}) ``` This honors the W3C *JSON-LD Best Practices* recommendation that clients SHOULD -attempt to use a locally cached version of contexts (see `§ Cache JSON-LD -Contexts `_). Refresh -the bundled copies with `make download-bundled-contexts`. +attempt to use a locally cached version of contexts (see +[§ Cache JSON-LD Contexts](https://w3c.github.io/json-ld-bp/#cache-json-ld-contexts)). +Refresh the bundled copies with `make download-bundled-contexts`. ## Customizing the ContextLoader From 2a867850754800ac663e04e09e03bc56067078e2 Mon Sep 17 00:00:00 2001 From: Miel Vander Sande Date: Wed, 10 Jun 2026 13:46:09 +0200 Subject: [PATCH 34/35] Remove trailing links in contributing.md --- CONTRIBUTING.md | 9 --------- 1 file changed, 9 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a12040b9..5f79a983 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -78,12 +78,3 @@ As of early 2020, the process to generate an EARL report for the official * Optionally follow the [report instructions](https://github.com/w3c/json-ld-api/tree/master/reports) to generate the HTML report for inspection. * Submit a PR to the [json-ld-api repository](https://github.com/w3c/json-ld-api/pulls) with at least the `.ttl`. - -.. _JSON-LD Processor Conformance: https://w3c.github.io/json-ld-api/reports/ -.. _PEP 8 Style Guide: https://www.python.org/dev/peps/pep-0008/ -.. _Semantic Versioning: https://semver.org/ -.. _ruff: https://docs.astral.sh/ruff/ -.. _json-ld-api repository: https://github.com/w3c/json-ld-api/pulls -.. _rdf: https://rubygems.org/gems/rdf -.. _report instructions: https://github.com/w3c/json-ld-api/tree/master/reports -.. _PyPI: https://pypi.org/ \ No newline at end of file From 1d7079c5d25dab119da489b033c2e72705b2aed9 Mon Sep 17 00:00:00 2001 From: Miel Vander Sande Date: Wed, 10 Jun 2026 13:50:59 +0200 Subject: [PATCH 35/35] Adjust changelog --- CHANGELOG.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9838cd63..12770a14 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,7 +16,9 @@ class-based `DocumentLoader` instances while preserving the existing callable factory API. The concrete `RequestsDocumentLoader` and `AioHttpDocumentLoader` classes are also importable from `pyld`. -- Convert `./README.rst` (reStructuredText) to `./README.md` (markdown) and update its contents. +- Convert `./README.rst` and `./CONTRIBUTING.rst` (reStructuredText) to + `./README.md` and `./CONTRIBUTING.md` (markdown). Also update their + contents to reflect the current state of the repo. ### Added - `pyld.DocumentLoader` abstract base class for class-based document loaders, @@ -259,7 +261,7 @@ - **1.0.0**! - [Semantic Versioning](https://semver.org/) is now past the "initial development" 0.x.y stage (after 6+ years!). -- [Conformance](README.rst#conformance): +- [Conformance](README.md#conformance): - JSON-LD 1.0 + JSON-LD 1.0 errata - JSON-LD 1.1 drafts - Thanks to the JSON-LD and related communities and the many many people over