02_workflow.md

Workflow

The ingestion pipeline has four composable steps:

1. Source trace extraction.
2. Successful-route normalization.
3. CGR/SB-CGR enrichment and route provenance.
4. Artifact and HTML writing.

The CLI handlers are thin orchestrators around these steps.

Step 1: Source Trace Extraction

Source trace extraction reads all created search nodes from the raw planner output. It does not require successful routes.

Outputs:

• SearchTrace.nodes
• SearchTrace.edges
• source, algorithm, target metadata
• source-specific warnings and trace precision metadata

Trace extraction entrypoints:

• extract_source_trace("synplanner", tree_path=...)
• extract_source_trace("aizynthfinder", tree_path=...)
• extract_source_trace("syntheseus", graph_path=...)

Step 2: Route Normalization

Route normalization produces successful routes only. This is intentionally separate from trace extraction because RetroCast route adapters are route-level, not full-search-tree adapters.

For SynPlanner, routes are exported from the direct Tree using SynPlanner route helpers, then adapted into the neutral route layer.

For AiZynthFinder and Syntheseus, route JSON artifacts are loaded directly if they already look like grouped RetroCast routes. Otherwise, the raw route payload is adapted through the local RetroCast/Procrustes bridge.

Output:

• routes.json.gz

Step 3: CGR Enrichment And Provenance

CGR enrichment maps route reactions into a SynPlanner-compatible reaction dict, then calls:

• compose_all_route_cgrs
• compose_all_sb_cgrs
• cluster_routes

For AiZynthFinder and Syntheseus, reactions can be mapped on the fly if mapped reaction SMILES are missing. Existing partial atom-map numbers are preserved when possible.

Route provenance attaches successful routes back onto raw search trace nodes:

• SynPlanner uses direct tree route provenance where available.
• AiZynthFinder matches normalized route states to MCTS state nodes.
• Syntheseus matches route molecules/reactions to trace graph nodes.

Output:

• strategic_bonds.json.gz
• SearchTrace.routes
• TraceNode.route_ids

Step 4: Artifact And HTML Writing

The final step writes canonical files and optionally renders HTML:

• search_trace.json.gz
• routes.json.gz
• strategic_bonds.json.gz
• candidates.json.gz, when failed route candidates are preserved
• run_manifest.json
• index.html

HTML rendering builds one payload schema for all engines, then applies source-specific visual expansions only in the payload layer.

Typical Commands

Render from existing canonical artifacts:

retro-tree-visual render \
  --trace search_trace.json.gz \
  --routes routes.json.gz \
  --strategic-bonds strategic_bonds.json.gz \
  --target-id apatinib \
  --output index.html

Run full AiZynthFinder ingestion:

retro-tree-visual ingest aizynthfinder \
  --tree full_search_tree.json.gz \
  --routes routes.json \
  --target-id apatinib \
  --target-smiles 'N#CC1(c2ccc(NC(=O)c3cccnc3NCc3ccncc3)cc2)CCCC1' \
  --out-dir runs/apatinib_mcts_500/aizynthfinder \
  --html runs/apatinib_mcts_500/aizynthfinder/index.html