docs(training): align cookbook docs and skill references with main#422
Merged
Conversation
- recipes.md: rename IGPO row from "Importance-weighted GRPO" to "Information Gain-based Policy Optimization (IGPO)" -- matches the recipe's actual algorithm (Wang et al., ICLR 2026). - examples.md: fix wrong paths (sft_getting_started, deepmath_rl, frozen_lake), add single_turn_token_in / multi_turn_message_in / manual entries, drop the misleading "Generic RL wiring -> examples/rl/" parent-dir row, relabel Multi-hop QA RL -> Multi-hop QA IGPO. - README.md: add Async RL and IGPO recipe rows; fix the Directory layout block (snippets -> tools, list every existing example subdir). - checkpoints.md: include async_rl_loop in the warm_start support list (the recipe already wires validate_warm_start_config). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The multi-hop QA example previously bundled its own training loop in `train_multihop_qa_igpo.py` (~1k lines) with a custom client/sampler wiring around `igpo_loop`'s lower-level helpers. Replace it with a thin `train.py` that targets `recipes/async_rl_loop.main()` and a `rollout.py` factory that emits `RolloutSample` per call. The multi-turn search/submit_answer tool loop and per-turn IG scoring are preserved by reusing `MultiHopQARolloutProcessor` (the rollout factory wraps a single-row eval-protocol invocation and threads `scorer.on_turn_complete` through `turn_callback`). Per-turn IG rewards are folded into the trajectory's scalar reward as `outcome + ig_weight * sum(ig_per_turn)` so the async loop's GRPO advantage z-normalises across the prompt group. Trade-off vs. the legacy recipe: per-token credit assignment is collapsed to a per-trajectory scalar (the IG signal still propagates via group z-score, just without per-token weighting). This is what the `RolloutSample` flat contract supports; per-turn formulations should fork `rollout.py`. Updates `run.sh`, the example README, the cookbook README directory listing, and the skill index entry. Adds a `.gitignore` for the materialised dataset. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
xiaoyifan
approved these changes
May 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Doc-only PR fixing training docs / dev skill references that drifted from the current tree, plus the SDK pin bump.
training/pyproject.toml— bumpfireworks-ai[training]to the latest available prerelease,>=1.2.0a66,<2.skills/dev/references/recipes.md— IGPO row label corrected: "Importance-weighted GRPO" -> "Information Gain-based Policy Optimization (IGPO)"; async RL row documents the existing async recipe.skills/dev/references/examples.md— fix wrong example paths (sft_getting_started/,deepmath_rl, top-levelfrozen_lake/don't exist on main); add current async/manual example paths; relabel multi-hop as IGPO.training/README.md— add Async RL and IGPO recipe rows; fix the Directory layout block (snippets/->tools/; list current example subdirs).skills/dev/references/checkpoints.md— includeasync_rl_loopin the warm-start support list.skills/dev/SKILL.md,skills/dev/references/shapes.md,skills/dev/references/recipes.md— align shape guidance with auto-selection and refresh the SDK pin example tofireworks-ai[training]>=1.2.0a66,<2.training/examples/tools/README.md— relabel snippets as tools and document the current utility scripts / control-plane checkpoint source.No recipe, async loop, IGPO implementation, or multi-hop example code changes remain in this PR.
Test plan
python -m pip index versions fireworks-ai --preconfirms latest prerelease is1.2.0a66git diff --check origin/main...HEADFIREWORKS_API_KEYappears in docs