feat(rlix): lower whole-GPU residual gate default to 7.0 GiB; preload dual smoke

docs/m11-implementation-guide.md

@@ -535,6 +535,7 @@ Four small fixes from the M11 review report.
 | MED | MED1 | concurrent-resize stress test under `max_concurrency=4` | Open (Codex carryforward) |
 | NOTE | F8 / F10 | `orchestrator.cleanup_stale_pipelines()` RPC | M11.3 production hardening |
 | LOW | LOW1 | verify `generate_rollout_fully_async` accepts `rlix_hooks` kw | ~30 min check |
+| MED | F2 v2 | whole-GPU used-memory gate landed (`rlops/rlix#17` merged 2026-07-05, default 13.0); follow-up lowers default to 7.0 + preload dual smoke (this PR) — basis: torch-hook residual 11.07 GiB vs preload 5.02 GiB (4x5090 cu130); rollback `MILES_TMS_HOOK_MODE=torch` + `MILES_MAX_RESIDUAL_GPU_MEM_GB=13` _(added 2026-07-05)_ | #17 merged; threshold-7 follow-up in flight |
 
 ## §3.6 Tianye's PR #14 — finalize-always-continue + release-on-sync-failure ✅ MERGED 2026-05-18 _(added 2026-05-24)_
 
@@ -718,6 +719,7 @@ New deferred-work entries surfaced during M11.2 real-overlap work (not in the hi
 | **F8 / F10 (NOTE)** | Detached coordinator persists after driver crash | M11.3 cleanup RPC scope | Stale actors on operator-killed runs | Still open; needs `orchestrator.cleanup_stale_pipelines()` RPC. | _(added 2026-05-24)_ |
 | **F9 (NOTE)** | `_split_pools_for_dual` silently ignored extra GPUs on non-2N machines | Static-analysis finding | Silent GPU leak on 5/6/7-GPU machines | **✅ RESOLVED** by miles `1487c3f` — explicit `ValueError` when `num_gpus_per_node != 2*infer_pool_size`; error message points at MILES_DUAL_P* workaround. | _(added 2026-05-24)_ |
 | **LOW1 (Codex carryforward)** | Verify `generate_rollout_fully_async` actually accepts `rlix_hooks` kw | `inspect.signature` forward silently no-ops if kw is missing | Demand signal path dead-codes silently | Open; ~30 min grep + startup assert. | _(added 2026-05-24)_ |
+| **F2 v2 (MED)** | Whole-GPU used-memory gate landed (`rlops/rlix#17` merged 2026-07-05, default 13.0); follow-up threshold 7.0 + preload dual smoke (this PR) | Measured basis: torch-hook residual 11.07 GiB (Megatron optimizer state never offloads) vs preload 5.02 GiB stable on 4x5090 cu130 dual smoke | Tighter gate catches offload regressions; torch-hook deployments trip 7.0 by design | Landed; rollback knobs `MILES_TMS_HOOK_MODE=torch` + `MILES_MAX_RESIDUAL_GPU_MEM_GB=13`. | _(added 2026-07-05)_ |
 
 ---
 
@@ -737,6 +739,7 @@ A vast.ai (or equivalent) GPU instance with:
 | Model checkpoint | Qwen2.5-0.5B at `/root/Qwen2.5-0.5B`, torch_dist at `/root/Qwen2.5-0.5B_torch_dist`, miles bundle at `/root/Qwen2.5-0.5B_miles/` | Any HF Megatron-compatible Qwen2.5-0.5B mirror works. |
 | Datasets | `/root/dapo-math-17k/dapo-math-17k.jsonl`, `/root/aime-2024/aime-2024.jsonl` | Eval is gated and skipped in smoke; the file just needs to exist. |
 | Branches | rlix on `zhenyu/miles-mvp-e2e` (or main once merged), miles on `zhenyu/m11-mvp-test` | See Appendix C for HEADs. |
+| CUDA — update | 13.0 (cu130 wheels, 4×RTX 5090) | `run_smoke_dual.sh` now defaults to `MILES_TMS_HOOK_MODE=preload` (verified 2026-07-05: preload segfault gone on cu13 wheels; residual gate default 7.0 sized for preload). Pre-cu13 Blackwell stacks: fall back to `MILES_TMS_HOOK_MODE=torch` + `MILES_MAX_RESIDUAL_GPU_MEM_GB=13` (`run_smoke_e2e.sh`/`run_smoke_inject_fault.sh` keep that combination). _(added 2026-07-05)_ |
 
 ### 7.2 Smoke scripts
 

rlix/pipeline/miles_coordinator.py

@@ -439,12 +439,14 @@ def _shrink_workers(self, engine_indices: Set[int]) -> None:
         # Whole-GPU residual threshold (GiB). Miles shrink_engines logs
         # SGLang per-process/server_info attribution diagnostics with this
         # value; the hard whole-GPU gate runs in MilesPipeline after the
-        # engines report offloaded. Default 13.0 is a temporary smoke-safe
-        # value based on observed whole-GPU residuals with the known Megatron
-        # train-offload gap; lower it after that follow-up is fixed and
-        # re-measured.
+        # engines report offloaded. Default 7.0 = measured preload-mode
+        # whole-GPU floor 5.02 GiB (2026-07-05 audit, 4x RTX 5090 cu130 dual
+        # smoke) + margin; requires MILES_TMS_HOOK_MODE=preload (miles guard
+        # allows it on cu13+ Blackwell). torch-hook deployments must set
+        # MILES_MAX_RESIDUAL_GPU_MEM_GB=13. Next step: ~3.0 after the
+        # suspected tms.disable() weight-sync-window tail (~1.6 GiB) is fixed.
         residual_threshold_gb = parse_env_positive_float(
-            "MILES_MAX_RESIDUAL_GPU_MEM_GB", 13.0
+            "MILES_MAX_RESIDUAL_GPU_MEM_GB", 7.0
         )
         shrunk = ray.get(
             rollout_manager.shrink_engines.remote(

rlix/pipeline/miles_pipeline.py

@@ -577,7 +577,7 @@ def _wait_for_overlap_engines_offloaded(self, allocated_train_gpus, *, timeout_s
         # intentionally broader than the SGLang process tree: it catches
         # non-child/orphan/co-tenant VRAM (Megatron/Miles/vLLM/etc.) on the
         # GPUs that must be clear before actor_train wakes up.
-        threshold_gb = parse_env_positive_float("MILES_MAX_RESIDUAL_GPU_MEM_GB", 13.0)
+        threshold_gb = parse_env_positive_float("MILES_MAX_RESIDUAL_GPU_MEM_GB", 7.0)
         max_used_gb = self._probe_max_used_gpu_mem_gb(target_gpu_ids)
         if max_used_gb is None:
             logger.warning(
@@ -600,7 +600,10 @@ def _wait_for_overlap_engines_offloaded(self, allocated_train_gpus, *, timeout_s
                 "check may be caused by non-SGLang co-tenants such as "
                 "Megatron/Miles/vLLM/orphan processes. SGLang per-engine "
                 "process-resident and server_info residual diagnostics are logged "
-                "engine-side for attribution."
+                "engine-side for attribution. Note: with MILES_TMS_HOOK_MODE=torch, "
+                "~11 GiB residual is expected (Megatron optimizer state is not "
+                "tms-pausable) — either run the preload hook mode (cu13+ stacks) "
+                "or set MILES_MAX_RESIDUAL_GPU_MEM_GB=13."
             )
 
     @staticmethod

scripts/run_smoke_dual.sh

@@ -49,13 +49,20 @@ else
 fi
 export NVTE_ALLOW_NONDETERMINISTIC_ALGO=0
 export CUBLAS_WORKSPACE_CONFIG=:4096:8
-export MILES_TMS_HOOK_MODE=torch
-# B-14 mitigation on 16 GB GPUs: enable torch_memory_saver.pause() so
-# Megatron weights actually move off-GPU between train cycles. Original
-# `MILES_SKIP_TMS_PAUSE=1` was a workaround for tms.pause segfault on
-# CUDA 12.9 + Blackwell + tms 0.0.9. On this RTX 4060 Ti + torch 2.11
-# stack, attempt to use real tms.pause to free GPU memory between
-# rollouts.
+# preload = full-catchment offload (optimizer state pausable), verified on
+# cu130 Blackwell 2026-07-05; matches the 7.0 residual gate default.
+# Fallback for pre-cu13 Blackwell stacks:
+#   export MILES_TMS_HOOK_MODE=torch AND MILES_MAX_RESIDUAL_GPU_MEM_GB=13.
+export MILES_TMS_HOOK_MODE=preload
+# miles' arch guard blocks preload on Blackwell (a tms 0.0.9 + CUDA 12.9
+# segfault, since fixed on cu13 images — verified 2026-07-05 on 4x RTX 5090
+# cu130, mock + dual E2E). Authorize preload via the guard's own escape
+# hatch; on pre-cu13 Blackwell stacks use the torch+13 fallback above instead.
+export MILES_TMS_ALLOW_PRELOAD_ON_BLACKWELL=1
+# Real tms.pause stays enabled (MILES_SKIP_TMS_PAUSE unset). The historical
+# skip was a workaround for the tms 0.0.9 pause segfault on pre-cu13
+# Blackwell stacks; if you must revive it, also pin
+# MILES_MAX_RESIDUAL_GPU_MEM_GB=13 (nothing offloads with the skip).
 # export MILES_SKIP_TMS_PAUSE=1
 export MILES_SKIP_NODE_PG_PIN=1
 # NOTE: PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True is incompatible with

scripts/run_smoke_e2e.sh

@@ -26,6 +26,10 @@ export CUBLAS_WORKSPACE_CONFIG=:4096:8
 # aggressive offload).
 export MILES_TMS_HOOK_MODE=torch
 export MILES_SKIP_TMS_PAUSE=1
+# torch-mode/skip-pause leaves ~11+ GiB residual on the train GPUs; pin the
+# residual gate to the torch-mode rollback value (code default is now 7.0,
+# sized for preload mode — see run_smoke_dual.sh).
+export MILES_MAX_RESIDUAL_GPU_MEM_GB=13
 # ROLL ResourceManagerProxy creates a node-PG holding all GPUs which would
 # pin the MilesPipeline actor and block Phase B's _create_placement_group.
 # Skip the pin so the coordinator can remove the node-PG mid-run.

scripts/run_smoke_inject_fault.sh

@@ -32,6 +32,9 @@ export CUBLAS_WORKSPACE_CONFIG=:4096:8
 export MILES_TMS_HOOK_MODE=torch
 export MILES_SKIP_TMS_PAUSE=1
 export MILES_SKIP_NODE_PG_PIN=1
+# torch-mode/skip-pause rollback value for the residual gate (code default 7.0
+# is sized for preload mode — see run_smoke_dual.sh).
+export MILES_MAX_RESIDUAL_GPU_MEM_GB=13
 
 # Phase 1 fault injection — raises inside train_group.train at first rollout.
 export MILES_INJECT_TRAIN_FAULT=1

tests/test_miles_residual_threshold_wiring.py

@@ -33,9 +33,9 @@ def test_miles_uses_whole_gpu_residual_threshold() -> None:
         and isinstance(node.args[0], ast.Constant)
         and node.args[0].value == "MILES_MAX_RESIDUAL_GPU_MEM_GB"
         and isinstance(node.args[1], ast.Constant)
-        and node.args[1].value == 13.0
+        and node.args[1].value == 7.0
         for node in ast.walk(shrink_fn)
-    ), "_shrink_workers must parse the residual threshold env var with 13GB default"
+    ), "_shrink_workers must parse the residual threshold env var with 7GB default"
 
     assert any(
         isinstance(node, ast.Call)
@@ -90,9 +90,9 @@ def test_miles_pipeline_enforces_whole_gpu_residual_gate() -> None:
         and isinstance(node.args[0], ast.Constant)
         and node.args[0].value == "MILES_MAX_RESIDUAL_GPU_MEM_GB"
         and isinstance(node.args[1], ast.Constant)
-        and node.args[1].value == 13.0
+        and node.args[1].value == 7.0
         for node in ast.walk(wait_fn)
-    ), "MilesPipeline hard gate must parse MILES_MAX_RESIDUAL_GPU_MEM_GB with 13GB default"
+    ), "MilesPipeline hard gate must parse MILES_MAX_RESIDUAL_GPU_MEM_GB with 7GB default"
     assert any(isinstance(node, ast.Raise) for node in ast.walk(wait_fn)), (
         "MilesPipeline must raise when whole-GPU residual exceeds the threshold"
     )