NRL only content for GitHub pages

[kheiss-uwzoo]

Summary

This PR adds a NeMo Retriever Library (NRL)–scoped documentation build and publishing path, reorganizes the table of contents to match the proposed NeMo Retriever doc structure, and introduces short hub pages so major topics have a clear landing page before deeper guides. It aligns naming with NeMo Retriever Library where we mean the library product, and applies NVIDIA style guidance (plain language, “and” instead of “&”, descriptive links, list lead-ins).

Use this for a rendered preview of the current draft content:

https://sw-docs-dgx-station.nvidia.com/nemo/retriever/latest/extraction/overview/

Discussion, scope, and merge status live here in this PR.

Feedback welcome on either link.

Note that this is draft / staging quality, not a final production publication.

Scope

Documentation and CI for the NRL-focused site; it does not replace the full multi-package Sphinx/API doc pipeline for the whole repo.

What’s included

Build and automation

• .github/workflows/nrl-docs-github-pages.yml — Builds and deploys the NRL docs to GitHub Pages (staging/nightly: push to main under docs/**, schedule, workflow_dispatch).

• docs/mkdocs.nrl-github-pages.yml — MkDocs config for the NRL-only site (Material theme, staging overrides, exclude_docs for pages outside this build).

• docs/overrides-nrl-staging/main.html — Staging theme override.

• docs/scripts/print_nrl_mkdocs_nav.py — Prints the nav tree from the NRL MkDocs config for review.

• docs/scripts/scan_non_nrl_doc_references.py — Optional scan for legacy naming (informational).

• Information architecture and content

• Nav (13 sections) — Introduction → Get started → Choose deployment → Core workflows → Multimodal extraction → Embedding/indexing → Retrieval → Deployment → Customize → Integrations → Evaluation → Reference → Support; plus additional resources.

• New hub / topic pages (examples): how to use this documentation, hosted vs self-hosted NIMs, workflows (query/rerank, agentic, video OCR), semantic/hybrid retrieval, reranking, production checklist, multimodal extraction stubs, embedding NIMs, vector DB partners, published metrics, and related links.

• Supporting updates to existing pages (choose-your-path, getting-started-about, key-features, concepts, integrations, resources, small fixes across FAQ, data-store, audio, etc.).

[greptile-apps]

Greptile Summary

This PR introduces a NRL-scoped MkDocs documentation build and GitHub Pages publishing pipeline, reorganizes the nav into 13 sections with new hub/topic pages, renames releasenotes-nv-ingest.md to releasenotes.md, and adds two helper scripts for pre-deploy nav inspection and legacy-naming scanning.

• Workflow permissions: is scoped too broadly — pages: write and id-token: write are declared at the workflow level and inherited by the build job, which does not need them; these should be moved to the deploy job only.
• Staging banner will not render — theme.announcement is not a recognized Material for MkDocs YAML key; the warning will be silently dropped, and the {% block announce %} override is not implemented in main.html.

Confidence Score: 3/5

Not safe to merge until the broken build job structure is fixed and the over-broad token permissions are addressed.

The build job in the workflow is structurally invalid (missing steps: key — flagged in prior thread) and will fail immediately on push. The new security finding of workflow-level pages: write / id-token: write gives the build job unnecessary token access. The staging banner misconfiguration (theme.announcement) means the site will silently deploy without any staging indicator. Together these are two new P1s plus one unresolved P1 from a prior thread.

.github/workflows/nrl-docs-github-pages.yml (broken job structure + over-broad permissions) and docs/mkdocs.nrl-github-pages.yml + docs/overrides-nrl-staging/main.html (missing announcement block).

Security Review

• Over-broad token grants (.github/workflows/nrl-docs-github-pages.yml): pages: write and id-token: write are granted at the workflow level, giving the build job OIDC and Pages-write permissions it does not need. A compromised third-party action in the build job (all six are pinned to mutable version tags, not full SHAs) could abuse these tokens. Scope both permissions to the deploy job only.

Important Files Changed

Filename	Overview
.github/workflows/nrl-docs-github-pages.yml	New workflow for NRL docs build/deploy; has structural issues (missing steps: in build job, duplicate checkout actions, mutable tag pins) and over-broad permissions at workflow level instead of job level.
docs/mkdocs.nrl-github-pages.yml	New MkDocs config for NRL-only GitHub Pages site; theme.announcement is not a recognized Material key so the staging banner will be silently dropped from the rendered site.
docs/scripts/print_nrl_mkdocs_nav.py	New utility script to print nav tree for pre-deploy review; missing required SPDX license header (flagged in prior thread).
docs/scripts/scan_non_nrl_doc_references.py	New scan script for legacy naming; missing SPDX header (prior thread) and has a dead NV-Ingest regex pattern shadowed by the preceding case-insensitive superset entry.
docs/overrides-nrl-staging/main.html	Minimal staging theme override; does not implement {% block announce %}, so the staging banner in mkdocs.nrl-github-pages.yml will not render.
docs/docs/extraction/releasenotes.md	Renamed from releasenotes-nv-ingest.md; nav reference in mkdocs config now correctly points to this file, resolving the prior broken-nav finding.
docs/docs/extraction/overview.md	Updated NRL product overview page; "NIVIDIA" typo was flagged in a prior thread and needs correction.

Sequence Diagram

sequenceDiagram
    participant Push as Push / Schedule
    participant Build as build job
    participant MkDocs as MkDocs --strict
    participant Artifact as Pages artifact
    participant Deploy as deploy job
    participant Pages as GitHub Pages

    Push->>Build: trigger (docs/**, schedule, dispatch)
    Build->>Build: actions/checkout
    Build->>Build: pip install mkdocs + nemo_retriever
    Build->>Build: print_nrl_mkdocs_nav.py
    Build->>Build: scan_non_nrl_doc_references.py
    Build->>MkDocs: mkdocs build -f mkdocs.nrl-github-pages.yml --strict
    MkDocs-->>Artifact: docs/site/
    Build->>Artifact: upload-pages-artifact
    Artifact-->>Deploy: needs: build
    Deploy->>Pages: actions/deploy-pages
    Pages-->>Deploy: page_url

Loading

Prompt To Fix All With AI

This is a comment left during a code review.
Path: .github/workflows/nrl-docs-github-pages.yml
Line: 18-21

Comment:
**Over-broad workflow-level permissions**

The `permissions:` block is declared at the workflow level, which grants `pages: write` and `id-token: write` to every job — including `build`, which needs neither. Under the `github-actions-security` rule, permissions should be scoped to the minimum required at the job level. Move the elevated permissions to the `deploy` job only and restrict `build` to `contents: read`:

```yaml
permissions:
  contents: read   # workflow-level default

jobs:
  build:
    permissions:
      contents: read
    ...
  deploy:
    permissions:
      pages: write
      id-token: write
```

This limits the blast radius if the `build` job is ever compromised by a supply-chain attack.

**Rule Used:** GitHub Actions workflows must: pin third-party act... ([source](https://app.greptile.com/review/custom-context?memory=github-actions-security))

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: docs/mkdocs.nrl-github-pages.yml
Line: 33-34

Comment:
**`theme.announcement` is not a valid Material for MkDocs config key**

Material for MkDocs does not read an `announcement:` key from the `theme:` block — MkDocs will silently ignore it, and the staging warning banner will never appear on the site. The correct approach is to override the `{% block announce %}` in the theme's custom template.

Add the block to `docs/overrides-nrl-staging/main.html`:

```html
{% block announce %}
  <strong>Staging (nightly):</strong> NeMo Retriever documentation only.
  This site is not a production or release publication.
{% endblock %}
```

Without this, the site looks identical to a production build and could mislead readers into treating draft content as authoritative.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: docs/scripts/scan_non_nrl_doc_references.py
Line: 19-21

Comment:
**Dead `NV-Ingest` pattern — always shadowed by the case-insensitive superset above it**

`PATTERNS` is evaluated in order with a `break` after the first match. The first entry uses `re.IGNORECASE`, which matches `NV-Ingest` (and any other casing), so the second entry `re.compile(r"NV-Ingest")` is unreachable — it will never fire. Both lines that match `NV-Ingest` will be reported under the `nv-ingest (substring)` label instead.

If you need to distinguish the mixed-case product name from the lowercase CLI artifact in the output, move the case-sensitive patterns before the case-insensitive superset, or remove the redundant entry.

How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (6): Last reviewed commit: "Update docs/docs/extraction/overview.md" | Re-trigger Greptile}

[randerzander]

Approving as an initial re-factor for the doc content

@jdye64 to review the scripts and gh pages changes

[jdye64]

What is the point of this new workflow? There is already a workflow that does the same thing

[kheiss-uwzoo]

Existing Pages workflows run the full Docker/Sphinx docs build; this one is a lightweight NRL-only MkDocs path for staging/nightly without that pipeline.