Add GSM8K and identity conversational SFT recipes

[sfc-gh-wenli]

Summary

• Add GSM8K conversational SFT: map openai/gsm8k into the chat messages schema, add exact-match eval (#### then last number), and ship 9B/35B full and LoRA train plus inference recipes (LoRA lr 1e-4, full-parameter lr 5e-6).
• Add the builtin identity JSONL (Who trained you? paraphrases → Snowflake AI Research), register it in the dataset catalog, and add substring-match eval plus batched generate from a prompts file.

Test plan

Experimentation doc: https://docs.google.com/document/d/1YxjZOoOtmJKBwAuIoHjav5OXqw2Xwt0gHPlbnLAEjvs/edit?tab=t.irb9zy5h4lqt

Unit:

• uv run pytest tests/test_conversational_gsm8k.py tests/test_gsm8k_eval.py tests/test_identity_eval.py tests/test_model_catalog.py

GSM8K on qa6 H200 (task=gsm8k, temperature=0, test 1,319, last-number fallback). 9B is one epoch (934 steps, batch 8). 35B is 100 steps on the shipped SP8/EP4 configs (100 rows, not a full epoch).

The 9B full 5e-6 number was trained from a 64k copy (/tmp/qwen35_9b_full_64k_lr5e6.json). That matches shipped qwen35_9b_full.json except max_seq_len (32k vs 64k); GSM8K rows are ~300 tokens, so the cap does not change the batch.

Model	Setup	LR	GSM8K	Format	vs base
Qwen3.5-9B	Base	—	82.0%	0%	—
Qwen3.5-9B	LoRA (qwen35_9b_lora.json)	1e-4	87.7%	99.8%	+5.7
Qwen3.5-9B	Full (qwen35_9b_full.json)	5e-6	86.5%	100%	+4.5
Qwen3.5-9B	Full (not shipped)	2e-5	75.3%	99.8%	−6.7
Qwen3.5-9B	Full (not shipped)	1e-4	41.8%	96.7%	−40.2
Qwen3.6-35B-A3B	Base	—	83.7%	0.1%	—
Qwen3.6-35B-A3B	LoRA (qwen36_35b_a3b_lora.json)	1e-4	90.8%	99.7%	+7.1
Qwen3.6-35B-A3B	Full (qwen36_35b_a3b_full.json)	5e-6	88.1%	99.6%	+4.4
Qwen3.6-35B-A3B	Full (not shipped)	2e-5	46.9%	98.4%	−36.8
Qwen3.6-35B-A3B	Full (not shipped)	1e-6	91.0%	99.6%	+7.3

• 9B LoRA 1e-4 and full 5e-6 beat base; full 2e-5 and 1e-4 forget
• 35B LoRA 1e-4 and full 5e-6 beat base on a 100-row slice; shipped-old full 2e-5 forgets (46.9%)
• 35B full GSM8K for a full epoch (7,473 steps) — only 100 rows were trained

Identity (50 eval prompts, substring Snowflake AI Research, temperature=0, 100 steps):

Model	Setup	LR	Identity
Qwen3.5-9B	LoRA (qwen35_9b_lora.json)	1e-4	100% (50/50)
Qwen3.5-9B	Full (qwen35_9b_full.json)	5e-6	100% (50/50)
Qwen3.6-35B-A3B	LoRA (qwen36_35b_a3b_lora.json)	1e-4	96% (48/50)
Qwen3.6-35B-A3B	Full (qwen36_35b_a3b_full.json)	2e-5	100% (50/50)

• 9B identity LoRA 1e-4 and full 5e-6
• 35B identity LoRA 1e-4 and full (ran at the previous shipped 2e-5)
• 35B identity full at the new shipped 5e-6

[snowflake-security-bot]

Snowflake Security Review

Security grade: A — Passed ✅

This PR was classified as LOW risk by the automated pre-screen.

[snowflake-security-bot]

Snowflake Security Review

Security grade: A — Passed ✅

This PR was classified as LOW risk by the automated pre-screen.

[snowflake-security-bot]

Snowflake Security Review

Security grade: A — Passed ✅

This PR was classified as LOW risk by the automated pre-screen.

[snowflake-security-bot]

Snowflake Security Review

Security grade: A — Passed ✅

This PR was classified as LOW risk by the automated pre-screen.

[snowflake-security-bot]

Snowflake Security Review

Security grade: A — Passed ✅

This PR was classified as LOW risk by the automated pre-screen.

[sfc-gh-wenli]

Closing in favor of the same-repo PR: #62

[sfc-gh-wenli]

Reopened — we'll use this original PR instead of the same-repo duplicate.

[sfc-gh-halu]

if this is used for throughput calculation, it likely inflates the throughput number given it's counting padded tokens.

[sfc-gh-wenli]

Agree. Originally I thought the padded tokens would still hit the gpu and cause compute.
If DSS strips pad and packs valid tokens into, those pad positions never hit the GPU, so they shouldn’t count as compute tok/s.

[snowflake-security-bot]

Snowflake Security Review

Security grade: A — Passed ✅

This PR was classified as LOW risk by the automated pre-screen.