Skip to content

About

Which model? How much effort? Lightning talk on choosing Claude models and effort levels at HGSC.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

3 Commits

Folders and files

Repository files navigation

Choosing models

Which model, and how much effort. Two defaults and a few rules for when to move off them.

From a lightning talk at the HGSC AI meeting, 30 July 2026. Questions: hale@bcm.edu


The short version

Two independent decisions, and people freeze because they collapse them into one:

  • Where you run — Claude's own endpoints, AWS Bedrock, or our local GPU node. This is a data question, decided by data-classification policy, not by preference. That policy is actively being revised; the working group that owns it reports at the HGSC AI meeting. Go by them, not by this page.
  • Which model, and how much effort — a task question. That's what's below.

Claude Code

Two dials, not one:

Dial What it controls
/model how capable the model is
/effort how long it's allowed to think

Turn up effort before you turn up the model. It's the cheaper move and it fixes more than people expect.

Start here

/model     → Default (Sonnet)
/effort    → medium

Use that for weeks without thinking about it.

Turn up effort — high → xhigh → max

  • it already failed once (don't re-prompt the same setup harder — change something)
  • you don't know where the problem is
  • you're designing, not editing
  • you can't cheaply check whether the answer is right ← the one that matters most for us

If more thinking doesn't do it, then /model opus.

Turn both down — haiku, low

  • renames, boilerplate, mechanical edits
  • you know the answer, you just don't want to type it

Notes

  • /effort auto resets effort to the default.
  • Use the aliases (opus, sonnet, haiku), not version numbers — versions roll.
  • Available models differ by machine and app version. The HPC install currently offers Default (Sonnet), Opus, and Haiku; laptop/desktop also offer Sonnet explicitly and Fable, and the desktop app can select the previous version of each.
  • xhigh and max are not supported on every model; unsupported levels fall back.

The mental model

Sonnet at medium is your own hands — what you use all day.

Opus, or xhigh effort, is hiring a consultant — sometimes for a genuinely two-minute gig, and that's fine.

The metaphor cuts both ways: you don't bill a consultant to rename a variable, and you also don't stay stuck for an hour to avoid calling one.


Inference

Running a model over your own data. Two places:

Bedrock Frontier quality, no GPU queue. Your data leaves the node.
analysis-app1 (local GPU node) Nothing leaves the building. You own the quality problem.

On the local node

Honestly: just use gemma. Two reasons to move off it — it's code, or it isn't good enough.

You're doing Use
Text — summarize, extract, classify gemma4:latest
Same, but quality matters more gemma4:31b
Writing or reviewing code qwen3-coder:latest
Thousands of trivial calls, need speed llama3.2:latest
Small/fast coder option qwen2.5-coder:latest

Bigger = better and slower. That's the whole sizing rule.

Run ollama list on analysis-app1 to see what's actually loaded rather than trusting this table.


Claude Code against local models

If you don't have a Claude account, or nothing may leave the node, Claude Code will run against local Ollama. It works. It is meaningfully worse — tool use and plan mode get shaky.

Point it at qwen3-coder:latest, not gemma. Driving Claude Code needs tool-use support, so the model recommended above for inference is the wrong one here. Two different jobs, two different answers.

export ANTHROPIC_BASE_URL=http://localhost:11434
export ANTHROPIC_AUTH_TOKEN=ollama
export ANTHROPIC_API_KEY=

Needs a tool-use-capable model and ≥32K context.

Claude Code against Bedrock

export CLAUDE_CODE_USE_BEDROCK=1
export AWS_REGION=<region>
export ANTHROPIC_MODEL=<model ID or inference-profile ARN>

Per-model overrides also exist: ANTHROPIC_DEFAULT_OPUS_MODEL, ANTHROPIC_DEFAULT_SONNET_MODEL, ANTHROPIC_DEFAULT_HAIKU_MODEL.


The recipe

  1. Where am I allowed to run? Policy, not preference.
  2. Start at the default. Claude Code → Default (Sonnet), /effort medium. Local inference → gemma4:latest.
  3. Escalate on evidence, not on nerves. Failed once · don't know where to look · designing · can't check the answer → effort first, then model.
  4. Downshift for mechanical work. Don't bill the consultant for a rename.

Deliberately out of scope

Benchmarks and leaderboards, token pricing, parameter counts and quantization, context-window sizes, how reasoning works internally, fine-tuning and RAG, agent frameworks, cross-vendor comparisons, and the data-classification policy itself.

Also skipped: MAX_THINKING_TOKENS and the ultrathink prompt keyword. Both real, both a distraction, and their interaction isn't worth explaining before you've used /effort for a while.

Reference

About

Which model? How much effort? Lightning talk on choosing Claude models and effort levels at HGSC.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors