Add GSM8K and identity conversational SFT recipes

docs/guides/training/lora-and-qlora.md

@@ -21,8 +21,10 @@ Shipped SFT configs:
 |---|---|---|
 | `configs/qwen3_8b_full.json` | `Qwen/Qwen3-8B` | full-parameter (default) |
 | `configs/qwen3_8b_lora.json` | `Qwen/Qwen3-8B` | LoRA, rank 32 |
+| `configs/qwen35_9b_full.json` | `Qwen/Qwen3.5-9B` | full-parameter |
+| `configs/qwen35_9b_lora.json` | `Qwen/Qwen3.5-9B` | LoRA, rank 32, lr 5e-6 |
 | `configs/qwen36_35b_a3b_full.json` | `Qwen/Qwen3.6-35B-A3B` | full-parameter |
-| `configs/qwen36_35b_a3b_lora.json` | `Qwen/Qwen3.6-35B-A3B` | LoRA |
+| `configs/qwen36_35b_a3b_lora.json` | `Qwen/Qwen3.6-35B-A3B` | LoRA, lr 5e-6 |
 
 Copy one and edit `peft_config.r` and `lora_alpha` to change the adapter. For
 dense Qwen models, you can also adjust `target_modules`. Keep the shipped

recipes/README.md

@@ -8,9 +8,9 @@ under its `configs/` directory.
 
 | Recipe | Method | Dataset | Status |
 |---|---|---|---|
-| [Conversational SFT](sft/conversational/README.md) | LoRA or full-parameter SFT | Hugging Face chat datasets | Runnable |
+| [Conversational SFT](sft/conversational/README.md) | LoRA or full-parameter SFT | Hugging Face chat datasets, GSM8K, identity | Runnable |
 | [Math GRPO](rl/math_grpo/README.md) | Reinforcement learning | Hendrycks MATH and MATH-500 | Runnable |
-| [Inference endpoint](inference/README.md) | Serve, generate, eval | Open weights, checkpoints, MATH-500 | Runnable |
+| [Inference endpoint](inference/README.md) | Serve, generate, eval | Open weights, checkpoints, MATH-500, GSM8K, identity | Runnable |
 
 ## Prerequisites
 

recipes/inference/README.md

@@ -91,12 +91,20 @@ python -m recipes.inference.serve \
   config=/path/to/config.json \
   job_config=configs/qwen3_8b_lora.json
 
-# Qwen3.6-35B-A3B LoRA
+# Qwen3.5-9B LoRA / full
+python -m recipes.inference.serve \
+  config=/path/to/config.json \
+  job_config=configs/qwen35_9b_lora.json
+
+python -m recipes.inference.serve \
+  config=/path/to/config.json \
+  job_config=configs/qwen35_9b_full.json
+
+# Qwen3.6-35B-A3B LoRA / full
 python -m recipes.inference.serve \
   config=/path/to/config.json \
   job_config=configs/qwen36_35b_a3b_lora.json
 
-# Qwen3.6-35B-A3B full-parameter
 python -m recipes.inference.serve \
   config=/path/to/config.json \
   job_config=configs/qwen36_35b_a3b_full.json
@@ -147,3 +155,48 @@ python -m recipes.inference.evaluate \
   source_job_id=TRAINING_JOB_ID \
   checkpoint_id=CHECKPOINT_ID
 ```
+
+### Evaluate (GSM8K test)
+
+Exact-match on the `####` final answer. Use `temperature=0` for a deterministic compare.
+
+```bash
+python -m recipes.inference.evaluate \
+  config=/path/to/config.json \
+  job_config=configs/qwen3_8b_full.json \
+  task=gsm8k \
+  temperature=0 \
+  max_tokens=1024
+
+python -m recipes.inference.evaluate \
+  config=/path/to/config.json \
+  job_config=configs/qwen3_8b_lora.json \
+  source_job_id=TRAINING_JOB_ID \
+  checkpoint_id=CHECKPOINT_ID \
+  task=gsm8k \
+  temperature=0 \
+  max_tokens=1024
+```
+
+### Evaluate (identity)
+
+Score is the percent of completions that contain `Snowflake AI Research`.
+Default prompts are `recipes/sft/conversational/data/identity_eval.jsonl`.
+
+```bash
+python -m recipes.inference.evaluate \
+  config=/path/to/config.json \
+  job_config=configs/qwen3_8b_lora.json \
+  source_job_id=TRAINING_JOB_ID \
+  checkpoint_id=CHECKPOINT_ID \
+  task=identity \
+  temperature=0 \
+  max_tokens=128
+```
+
+Score an existing `generate` JSONL (needs a `completion` field):
+
+```bash
+python -m recipes.inference.identity \
+  completions_file=/tmp/identity_eval.jsonl
+```

recipes/inference/configs/qwen35_9b_full.json

@@ -0,0 +1,18 @@
+{
+  "sub_job_configs": [
+    {
+      "job_type": "sampling",
+      "model_name": "Qwen/Qwen3.5-9B",
+      "dtype": "bfloat16",
+      "seed": 42,
+      "inference_config": {
+        "max_seq_len": 32768,
+        "n_gpus": 2,
+        "vllm_config": {
+          "tensor_parallel_size": 1,
+          "gpu_memory_utilization": 0.8
+        }
+      }
+    }
+  ]
+}

recipes/inference/configs/qwen35_9b_lora.json

@@ -0,0 +1,34 @@
+{
+  "sub_job_configs": [
+    {
+      "job_type": "sampling",
+      "model_name": "Qwen/Qwen3.5-9B",
+      "dtype": "bfloat16",
+      "seed": 42,
+      "inference_config": {
+        "max_seq_len": 32768,
+        "n_gpus": 2,
+        "vllm_config": {
+          "tensor_parallel_size": 1,
+          "gpu_memory_utilization": 0.8
+        },
+        "peft_config": {
+          "peft_type": "Lora",
+          "r": 32,
+          "lora_alpha": 32,
+          "lora_dropout": 0.0,
+          "bias": "none",
+          "target_modules": [
+            "q_proj",
+            "k_proj",
+            "v_proj",
+            "o_proj",
+            "gate_proj",
+            "up_proj",
+            "down_proj"
+          ]
+        }
+      }
+    }
+  ]
+}