domdinicola · Oct 26, 2026 · Oct 14, 2026 · Oct 16, 2026 · Oct 16, 2026 · Oct 20, 2026
diff --git a/docs/src/about.md b/docs/src/about.md
@@ -1,7 +1,11 @@
 # About
 
-This is the documentation for the HOPE Documents project.
+HOPE Documents is a Django-based application for scanning and processing identity documents to extract structured data using OCR.
 
-HOPE Documents is a Django-based application designed for scanning and processing documents to extract data using OCR capabilities. It provides a command-line interface to extract text from images and PDFs.
+It provides:
 
-This documentation provides information on how to use, contribute to, and understand the architecture of the project.
+- **REST API** — submit a document image for analysis and retrieve extracted field values as JSON. Supports synchronous (inline) and asynchronous (Celery-backed) processing with an optional callback URL.
+- **Admin catalog UI** — configure which document types, countries, attributes, and extraction regions the system recognises. A built-in canvas tool lets operators define per-attribute bounding boxes directly on a reference image.
+- **CLI** — a `doc` command for development and debugging, with sub-commands for single-image extraction and batch processing.
+
+This documentation covers how to use the REST API, configure the document catalog, and understand the system architecture.
diff --git a/docs/src/api.md b/docs/src/api.md
@@ -1,20 +1,110 @@
-# API
+# REST API
 
+The REST API accepts document images, runs OCR extraction, and returns structured field values.
 
-### Search pattenrn in an image
+Swagger UI: `/api/rest/swagger/`
+ReDoc: `/api/rest/redoc/`
 
-``` python
-    from hope_documents.ocr.engine import CV2Config, Processor, SearchInfo, TSConfig
+---
 
-    ts_config = TSConfig(psm=11, oem=3)
-    cv2_config = CV2Config()
+## Authentication
 
-    processor = Processor(ts_config, cv2_config)
-    si: SearchInfo
-    for si in processor.find_text("my_image.png", "pattern"):
-        if si.match:
-            print("Text found in image. Distance is:", si.match.distance)
-        else:
-            print("Text not found")
+All endpoints require authentication. Three methods are supported:
 
+| Method | Header |
+|---|---|
+| Basic | `Authorization: Basic <base64(username:password)>` |
+| Token | `Authorization: Token <token>` |
+| Session | Django session cookie (browser / admin) |
+
+Unauthenticated requests receive `401 Unauthorized`.
+
+---
+
+## POST /api/analyze/
+
+Submit a document image for analysis.
+
+**Query parameters**
+
+| Parameter | Values | Default |
+|---|---|---|
+| `sync` | `true` / `1` | — |
+
+When `?sync=true` the task runs in-process and the response contains the full result (HTTP 200). Without it the task is queued via Celery and the response contains only the `job_id` (HTTP 202).
+
+**Request body** — `multipart/form-data` or JSON
+
+| Field | Type | Required | Notes |
+|---|---|---|---|
+| `image_file` | File upload | One of three required | Uploaded image file |
+| `image_url` | URL string | One of three required | Image fetched by the server |
+| `image_base64` | String | One of three required | Base64-encoded image, optional `data:…;base64,` prefix |
+| `document_type` | String | Yes | `DocumentType.code`, e.g. `PAS` |
+| `country` | String | Yes | `Country.code2`, e.g. `IT` |
+| `callback_url` | URL string | No | Called via POST with the result payload when the job completes |
+| `validation_mode` | String | No | See [Validation modes](#validation-modes). Default: `fall_back_to_best_match` |
+
+**Synchronous response — HTTP 200**
+
+```json
+{
+  "job_id": "5ef25336-94ab-4819-9c25-510cb88914ad",
+  "status": "completed",
+  "document_type": "PAS",
+  "country": "IT",
+  "validation_mode": "fall_back_to_best_match",
+  "version": "2024",
+  "version_matched": true,
+  "attributes": [
+    {"attribute_name": "first_name", "extracted_value": "MARIO"},
+    {"attribute_name": "last_name",  "extracted_value": "ROSSI"},
+    {"attribute_name": "document_number", "extracted_value": "AA1234567"}
+  ],
+  "error": null
+}
 ```
+
+**Asynchronous response — HTTP 202**
+
+```json
+{"job_id": "5ef25336-94ab-4819-9c25-510cb88914ad"}
+```
+
+**Error responses**
+
+| Status | Cause |
+|---|---|
+| 400 | Missing or invalid fields (see error body for detail) |
+| 401 | Not authenticated |
+
+---
+
+## GET /api/analyze/{job_id}/
+
+Poll the status of a submitted job.
+
+**Response**
+
+Same shape as the synchronous `POST` response. `version`, `version_matched`, and `attributes` are `null` until the job reaches `completed`. `error` is populated (and `status` is `failed`) if processing fails.
+
+**Status values:** `pending` → `running` → `completed` / `failed`
+
+---
+
+## Validation modes
+
+The `validation_mode` field controls how the extraction pipeline handles attribute values that do not match a `DocumentAttribute.validation_expression` regex:
+
+| Value | Behaviour |
+|---|---|
+| `fall_back_to_best_match` | Non-conforming values are penalised in version scoring; the algorithm prefers a version where values match. Default. |
+| `keep_value_mark_invalid` | Return the raw extracted value and mark it as invalid in the result. |
+| `treat_as_not_found` | Discard the value; treat the attribute as not found for this job. |
+| `fail_analysis` | Abort the job with `status: failed` if any attribute fails validation. |
+
+---
+
+## Callback
+
+When `callback_url` is provided, the server sends a `POST` request to that URL on job completion. The request body is identical to the `GET /api/analyze/{job_id}/` response. No retries are attempted on failure.
diff --git a/docs/src/architecture.md b/docs/src/architecture.md
@@ -1,45 +1,124 @@
 # Architecture
 
-This section describes the architecture of the HOPE Documents project.
-
 ## Overview
 
-HOPE Documents is a Django application designed for document processing and OCR. It is structured into several key components:
+HOPE Documents is split into two Django apps:
+
+- **`hope_ocr`** — OCR engine, document catalog (archive), and admin UI. Self-contained; can be used independently of the REST API.
+- **`hope_documents`** — REST API, async analysis pipeline, and the `AnalysisJob` model. Depends on `hope_ocr`.
 
--   **`hope_documents.archive`**: A Django app responsible for storing and managing document configuration. It includes Django models, admin configurations, and migrations.
+### Key components
 
--   **`hope_documents.ocr`**: This is the core component for OCR processing. It contains:
-    -   `engine.py`: The main OCR engine, likely using Tesseract and OpenCV.
-    -   `__cli__.py`: Implements the command-line interface for batch processing.
-    -   `loaders.py`: Likely responsible for loading documents from different sources.
-    -   `reader.py`: Reads the content of the documents.
+| Component | Location | Purpose |
+|---|---|---|
+| OCR engine | `hope_ocr/ocr/engine.py` | Tesseract + OpenCV image processing |
+| Document catalog | `hope_ocr/archive/models.py` | Config layer: countries, document types, attributes, regions |
+| Extraction protocol | `hope_documents/extractors.py` | `AttributeExtractor` Protocol + `OCRAttributeExtractor` |
+| Analysis task | `hope_documents/tasks.py` | Celery task that runs extraction and persists results |
+| REST API | `hope_documents/api/` | DRF views + serializers for `POST /api/analyze/` |
+| Admin | `hope_ocr/archive/admin.py` | Catalog management + Define Regions canvas UI |
 
--   **`hope_documents.utils`**: A collection of utility modules for common tasks such as image manipulation, language detection, logging, and performance timing.
+---
 
 ## Domain model
 
-The `archive` app contains two independent layers:
+### Configuration layer (`hope_ocr.archive`)
 
-**Configuration layer** — defines *which fields exist* for a given document type, country and version:
+Defines *which fields exist* for each document type, country, and version:
 
 ```
 Country ──────────────────┐
                           ▼
 DocumentType ────────► Document ──► DocumentAttribute ──► Attribute
-                       (version)        (region)
+                       (version)     (region, validation_expression)
+```
+
+- `Country` and `DocumentType` are reference objects shared across documents.
+- `Document` groups a `DocumentType`, `Country`, and free-text `version`. A version labels one complete layout — e.g. all attributes valid for Italian passports issued from 2024 onwards.
+- `Attribute` is a catalog of extractable field names. Three system attributes (`first_name`, `last_name`, `document_number`) are seeded automatically and cannot be renamed.
+- `DocumentAttribute` links one `Attribute` to a `Document` and stores:
+  - `region` — a `BoundingBox` (pixel coordinates) on the document image used to crop before OCR.
+  - `validation_expression` — an optional regex the extracted value must satisfy.
+
+### Processing layer (`hope_documents`)
+
+`AnalysisJob` records every extraction request:
+
+| Field | Notes |
+|---|---|
+| `id` | UUID primary key |
+| `status` | `pending` → `running` → `completed` / `failed` |
+| `document_type` / `country` | FK to catalog |
+| `image` | Uploaded document image |
+| `validation_mode` | Controls how validation_expression failures are handled |
+| `callback_url` | Optional POST target on completion |
+| `matched_version` / `version_matched` | Which Document version was selected |
+| `result` | JSON array of `{attribute_name, extracted_value}` |
+
+`DocumentRule` (existing) stores per-country regex patterns used by the legacy OCR engine. It is independent of the configuration layer above.
+
+---
+
+## Extraction pipeline
+
+```
+POST /api/analyze/
+       │
+       ▼
+AnalyzeView
+  ├─ validate input (document_type, country, image)
+  ├─ create AnalysisJob (status=pending)
+  └─ save image to media/analysis_jobs/
+       │
+       ├─ ?sync=true ──► process_analysis_job() in-process
+       └─ default ──────► process_analysis_job.delay() via Celery
+                                   │
+                                   ▼
+                         resolve Document version
+                         (score candidate versions against extracted values)
+                                   │
+                                   ▼
+                         OCRAttributeExtractor.extract()
+                           for each DocumentAttribute:
+                             ├─ region defined → crop image → OCR crop
+                             └─ no region → OCR full image
+                                   │
+                                   ▼
+                         validate against validation_expression
+                         apply validation_mode logic
+                                   │
+                                   ▼
+                         persist result on AnalysisJob
+                         fire callback_url if set
 ```
 
-- `Country` and `DocumentType` are the base reference objects.
-- `Document` groups a `DocumentType`, `Country` and free-text `version` together. The version labels a set of extractable attributes (e.g. all attributes valid for Italian driving licences issued after 2019).
-- `Attribute` is a catalog of extractable field names (`first_name`, `last_name`, `number` are built-in system attributes).
-- `DocumentAttribute` links one `Attribute` to a `Document` and stores the bounding-box `region` on the document image where the value should be extracted.
+---
+
+## Extraction abstraction
+
+`AttributeExtractor` is a `Protocol` in `hope_documents/extractors.py`:
+
+```python
+class AttributeExtractor(Protocol):
+    def extract_attribute(
+        self,
+        image: Image.Image,
+        attribute_name: str,
+        region: BoundingBox | None,
+    ) -> str: ...
+```
 
-**Processing layer** (existing) — validates extracted values at runtime:
+`OCRAttributeExtractor` is the current implementation — it uses `hope_ocr` directly. A future `LLMAttributeExtractor` is planned for low-confidence fallback (see todo list).
 
-- `DocumentRule` links a `Country` and `DocumentType` and holds regex patterns used by the OCR engine to validate number formats. It is independent of the configuration layer above.
+---
 
 ## Technologies
 
--   **Backend Framework**: Django
--   **OCR Engine**: Tesseract, with image processing capabilities provided by OpenCV.
--   **Command-Line Interface**: Click
+| Layer | Technology |
+|---|---|
+| Web framework | Django + Django REST Framework |
+| OCR | Tesseract (via pytesseract), OpenCV |
+| Async tasks | Celery |
+| Admin UI | Unfold + django-admin-extra-buttons |
+| API schema | drf-spectacular (Swagger / ReDoc) |
+| CLI | Click (via mkdocs-click autogen) |