Skip to content

fix: submit monitors the GPU that serves the model (v1.1.6) - #14

Merged
Prajwal7-2002 merged 1 commit into
mainfrom
fix/submit-serving-gpu
Nov 6, 2026
Merged

Prajwal7-2002 merged 1 commit into
mainfrom
fix/submit-serving-gpu

Conversation

@Prajwal7-2002

Copy link
Copy Markdown
Owner

Summary

On multi-GPU machines servebench submit monitored GPU 0 even when the engine loaded the model on another GPU. In the first Kaggle T4 x2 run, Ollama put 3 of 5 models on the second T4, so those results reported an idle GPU (0% busy, 585 MHz, 29 W). The speeds were right; the conditions described the wrong card.

Changes

  • submit sends one 1-token request to load the model (load_model), picks the GPU with the most memory in use (pick_serving_gpu, pure), and monitors that GPU for telemetry and limits.
  • It records hardware.serving_gpu only when there is more than one GPU, so single-GPU results are unchanged and still validate on older RunsOnMine.
  • Fixes the battery warning's flag name (RunsOnMine flags battery).
  • Version 1.1.6.

Pairs with Prajwal7-2002/runsonmine#2, which adds serving_gpu to the format. Merge that PR first.

Test verification

  • New tests: pick_serving_gpu (second GPU holds the model, single GPU, empty and garbage input); serving_gpu is recorded only with several GPUs.
  • Full suite: 169 passed; ruff clean.
  • Live, laptop (single RTX 3050, on battery): no serving_gpu field; conditions recorded (power limit 38.9 W, clock 1073 of 2100 MHz); RunsOnMine flags it battery (146 tok/s vs 202 on AC).

On multi-GPU machines submit monitored GPU 0 even when the engine put
the model on another GPU (Kaggle T4 x2: 3 of 5 runs showed an idle
GPU). Load the model with a 1-token request, pick the GPU with the most
memory in use, monitor it, and record hardware.serving_gpu when there
are several GPUs. Fix the battery warning's flag name. Bump to 1.1.6.
@Prajwal7-2002
Prajwal7-2002 merged commit 43bdbe2 into main Nov 6, 2026
7 checks passed
@Prajwal7-2002
Prajwal7-2002 deleted the fix/submit-serving-gpu branch November 6, 2026 06:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant