perf(engine): keep the mapped file after a repacked load is killed or stalls

docs/evidence/2026-10-05-android-conversations/README.md

@@ -0,0 +1,90 @@
+# Android chat before/after, Pixel 6a (4–5 Oct 2026)
+
+Raw results behind PR #72 (`perf/android-chat-conversations`). Every number here comes from the phone: the app's
+`execution_telemetry` table for conversations, and `npm run eval:device` JSONL for the question sets.
+
+## Setup
+
+- **Phone:** Google Pixel 6a, Tensor G1, 6 GB RAM (5.46 GiB usable), Android 16, dev build (`expo-dev-client`, arm64) served by
+  Metro over `adb reverse`, Wi-Fi adb.
+- **Models:** Qwen3-4B-Instruct-2507 Q4_K_M (catalog default) and Qwen2.5-1.5B-Instruct Q4_K_M (default on 6–8 GB phones).
+- **Library:** bundled corpus + Standard set. No knowledge packs.
+- **Conditions:** on battery; runs started from a fresh app launch (force-stop, relaunch, model loaded) and waited until the
+  battery was at or below 33.0 °C with thermal status 0, for at most 30 minutes. Exceptions are marked in the tables.
+- **Baseline:** `main` at a6633f8d (the branch's base before the rebase), served from a clean worktree to the same app.
+
+## Conversation protocol (`conversation/`)
+
+The harness runs single questions, so it can't see history growth, follow-ups or answers cut by the step timeout. These runs
+drive the real chat screen over adb (`driver.py`: `uiautomator dump` to find the controls, `input text`, Send tapped once):
+one new chat, six questions in order, a 30 s pause after each answer, personality **Thorough & Detailed**:
+
+1. How do vaccines work?  2. What are the side effects?  3. Who proposed the theory of evolution by natural selection?
+4. What did he publish?  5. Compare the French Revolution and the Industrial Revolution.  6. What is the capital of Australia?
+
+Each `*.json` (the driver's `bench/LABEL.results.json`, renamed) holds the telemetry rows (`ttft_ms`, `tok_per_sec`, `tokens_generated`, `total_latency_ms`, …) and the
+chat messages of the run. "Finished" means the answer ends with sentence punctuation instead of mid-sentence.
+
+| Run | Model | What changed | Code | First word, median | Slowest decode | Longest answer | Tokens | Finished |
+|---|---|---|---|---|---|---|---|---|
+| `base-4b` | Qwen3-4B | baseline | main a6633f8d | 45.4 s | 0.1 tok/s | 129 s | 1134 | 0 of 6 |
+| `fix-4b` | Qwen3-4B | history budget, warm-up fix, follow-up search, length target | branch at 98fd3ed2 (first 5 changes) | 15.3 s | 2.5 tok/s | 122 s | 1644 | 4 of 6 |
+| `fix2-4b` | Qwen3-4B | + length from the current speed, history race fixed | 98fd3ed2 + e334679c + 2c92437e (working copy) | 13.4 s | 0.1 tok/s | 127 s | 1187 | 5 of 6 |
+| `norepack-4b` | Qwen3-4B | + weights kept as the mapped file, by hand | fix2 + 8-bit KV + no_extra_bufts forced on (working copy) | 22.9 s | 3.6 tok/s | 122 s | 1359 | 5 of 6 |
+| `auto-4b` | Qwen3-4B | strict repack rule, 8-bit KV, length from each prompt | b19e7037 (working copy) | 26.3 s | 3.7 tok/s | 98 s | 1306 | 6 of 6 |
+| `narrow-4b` | Qwen3-4B | final: narrowed repack rule, 8-bit KV only when streaming | PR head 4622454c; app started by hand, background apps stopped | 9.1 s | 3.7 tok/s | 85 s | 1580 | 6 of 6 |
+| `base-15b` | Qwen2.5-1.5B | baseline | main a6633f8d | 20.0 s | 6.4 tok/s | 112 s | 2849 | 3 of 6 |
+| `fix-15b` | Qwen2.5-1.5B | branch | b19e7037 (the 1.5B repacks under every rule) | 4.7 s | 8.0 tok/s | 48 s | 2478 | 4 of 6 |
+
+## Question sets (`harness/`)
+
+`npm run eval:device -- --models qwen3-4b` (standard set v1, 17 questions) and `--queries vitalik` (6), Succinct, no
+history, questions back to back. Each folder has the JSONL (authoritative), the report from `npm run eval:summary`, and the
+answers for grading where the run produced them.
+
+| Run | Question set | Code | OK | First word, median | Decode, median | Total, median |
+|---|---|---|---|---|---|---|
+| `base-4b-std` | standard (17) | main a6633f8d | 17/17 | 9.0 s | 5.5 tok/s | 21.8 s |
+| `fix-4b-std` | standard (17) | 98fd3ed2; ran right after a low-memory kill and cold restart | 17/17 | 9.2 s | 4.5 tok/s | 28.6 s |
+| `fix-4b-std-rerun` | standard (17) | 98fd3ed2, fresh start | 17/17 | 9.1 s | 5.0 tok/s | 21.4 s |
+| `final-4b-std` | standard (17) | 429917b5: strict repack rule | 17/17 | 23.0 s | 5.0 tok/s | 34.3 s |
+| `narrow-4b-std` | standard (17) | 568ad1c7: narrowed rule, 8-bit KV always | 17/17 | 11.6 s | 5.9 tok/s | 27.6 s |
+| `narrow-f16kv-4b-std` | standard (17) | narrowed rule, 16-bit KV (what 4622454c runs for the 4B on this phone) | 17/17 | 9.5 s | 5.3 tok/s | 19.9 s |
+| `base-4b-vitalik` | vitalik (6) | main a6633f8d | 6/6 | 1.0 s | 6.1 tok/s | 18.6 s |
+| `fix-4b-vitalik` | vitalik (6) | 98fd3ed2 | 6/6 | 0.9 s | 6.3 tok/s | 13.9 s |
+| `final-4b-vitalik` | vitalik (6) | 429917b5: strict repack rule | 6/6 | 1.7 s | 6.2 tok/s | 15.1 s |
+
+## Reading the results
+
+- **Final code, single questions (the harness):** same as `main`. Standard set first word 9.5 s vs 9.0 s median, total 25.9 s vs
+  25.3 s average; per question the first word is slower than `main` on 9 of 17 (median ratio 1.00×). Not rerun for the Vitalik set
+  on the final code; on the earlier branch it was 13.9 s vs 18.6 s total.
+- **Final code, conversations:** the 4B finished 6 of 6 answers (0 of 6 on `main`) with a 9.1 s median first word (45 s on `main`)
+  and no stalls; the 1.5B's first word went from a 20 s median to 4.7 s.
+- **How the repack rule got here.** A strict rule (repack only with 0.75 GB to spare) kept the 4B as a mapped file on this phone:
+  stable, but prompt reading ~2× slower (`final-4b-std`, 23 s first word). The final rule keeps the mapped file only when the model
+  file itself doesn't fit the memory budget (a 35B mixture of experts on 12 GB), so the 4B repacks here (`narrow-*`). The 8-bit KV
+  cache cost ~19% of prompt reading (`narrow-4b-std` vs `narrow-f16kv-4b-std`), so it now applies only to streamed models.
+
+## Known risk on 6 GB phones (not fixed by this PR, same as `main`)
+
+With repacked weights the 4B's load peaks at ~2.8 GB of anonymous memory. When the phone has little free memory, Android's
+low-memory killer stops the app during the load: three times in a row on 5 Oct (18:38, 18:49, 18:50; `am_low_memory` followed by
+`am_proc_died team.sopa.aoair`, ~0.8 GB free before the load), after which the app's crash guard falls back to the 1.5B. Earlier
+repacked runs also had occasional 0.1 tok/s stalls (`base-4b` answer 2, `fix2-4b` answer 3) while weights came back from swap.
+A follow-up can decide at load time from the memory actually free, and switch to the mapped file after a stall.
+
+## Caveats
+
+- One phone, one room; n = 6 per conversation and 17 + 6 per question set.
+- Runs marked "working copy" had dev-only console logging on top of the listed commits (timing lines, and a tokenizer check
+  before each answer that adds a few tens of milliseconds). No behavior change otherwise.
+- `narrow-4b` started from the app as the user opened it (after the load-time kills above) with cached background apps stopped
+  (`am kill-all`), not from a scripted restart, since another restart risked another kill. It began with 2.2 GB of the app in swap.
+- `narrow-f16kv-4b-std` ran the narrowed rule with the 8-bit cache disabled by hand, which is what the final code does for a
+  repacked model.
+- `fix-4b-std` ran right after Android killed the app for low memory and is kept for completeness; `fix-4b-std-rerun` is the
+  comparable run.
+- Answer quality was not scored. The answers are here for the team's judge.
+- Qwen3-4B on a 6 GB phone is not the setup wizard's recommendation (it recommends the 4B from ~10 GB); it is used here because
+  it's the catalog default and shows the memory behavior most clearly.

docs/evidence/2026-10-05-android-conversations/conversation/auto-4b.json

@@ -0,0 +1,134 @@
+{
+  "label": "auto-4b",
+  "since_ms": 1791228483265,
+  "telemetry": [
+    {
+      "created_at": 1791228554016,
+      "model_id": "qwen3-4b-instruct-2507-q4km",
+      "task_type": "research",
+      "ttft_ms": 12383.707525968552,
+      "tok_per_sec": 5.4827027993605455,
+      "tokens_generated": 277,
+      "generation_latency_ms": 50522.526961028576,
+      "total_latency_ms": 63417.44566899538,
+      "outcome": "success"
+    },
+    {
+      "created_at": 1791228719249,
+      "model_id": "qwen3-4b-instruct-2507-q4km",
+      "task_type": "lookup",
+      "ttft_ms": 23189.493501007557,
+      "tok_per_sec": 4.843178910252969,
+      "tokens_generated": 314,
+      "generation_latency_ms": 64833.45047098398,
+      "total_latency_ms": 88836.05558598042,
+      "outcome": "success"
+    },
+    {
+      "created_at": 1791228882876,
+      "model_id": "qwen3-4b-instruct-2507-q4km",
+      "task_type": "lookup",
+      "ttft_ms": 29502.60438501835,
+      "tok_per_sec": 4.513697984150574,
+      "tokens_generated": 255,
+      "generation_latency_ms": 56494.69700795412,
+      "total_latency_ms": 86732.36727100611,
+      "outcome": "success"
+    },
+    {
+      "created_at": 1791228986293,
+      "model_id": "qwen3-4b-instruct-2507-q4km",
+      "task_type": "lookup",
+      "ttft_ms": 14253.910447955132,
+      "tok_per_sec": 4.640694837290158,
+      "tokens_generated": 184,
+      "generation_latency_ms": 39649.2349640131,
+      "total_latency_ms": 55022.6159529686,
+      "outcome": "success"
+    },
+    {
+      "created_at": 1791229163581,
+      "model_id": "qwen3-4b-instruct-2507-q4km",
+      "task_type": "compare",
+      "ttft_ms": 57122.61554700136,
+      "tok_per_sec": 3.7118641886819623,
+      "tokens_generated": 149,
+      "generation_latency_ms": 40141.55487000942,
+      "total_latency_ms": 98131.43177002668,
+      "outcome": "success"
+    },
+    {
+      "created_at": 1791229292803,
+      "model_id": "qwen3-4b-instruct-2507-q4km",
+      "task_type": "lookup",
+      "ttft_ms": 45009.53802597523,
+      "tok_per_sec": 3.873281368336207,
+      "tokens_generated": 127,
+      "generation_latency_ms": 32788.7359380126,
+      "total_latency_ms": 78621.58723700047,
+      "outcome": "success"
+    }
+  ],
+  "messages": [
+    {
+      "created_at": 1791228490449,
+      "role": "user",
+      "text": "How do vaccines work?"
+    },
+    {
+      "created_at": 1791228554021,
+      "role": "assistant",
+      "text": "Vaccines work by training the immune system to recognize and fight specific pathogens\u2014like viruses or bacteria\u2014without causing the full disease. They introduce a harmless version of the pathogen, such as a weakened or inactivated form, or a piece of it (like a surface protein), so the body can learn to identify it [1]. This exposure prompts the immune system to produce antibodies and memory cells, which remain active long after vaccination.\n\nUnlike natural infection, vaccines do not cause illness. For example, the measles vaccine uses a weakened virus that stimulates immunity without leading to the actual disease. This makes vaccination safer and more reliable. In contrast, natural infection carries risks like severe illness or complications, especially in vulnerable populations.\n\nThe immune response created by vaccines is highly specific and durable. Memory B and T cells \"remember\" the pathogen, allowing a faster, stronger response upon future exposure. This is like training a soldier to recognize an enemy\u2014once trained, they respond quickly when the threat reappears.\n\nComparatively, vaccines offer consistent, controlled immunity, while natural exposure varies in severity and outcome. Vaccines also allow for mass immunization, protecting entire communities through herd immunity. This prevents outbreaks and reduces the spread of disease, making public health interventions far more effective than relying on individual recoveries.\n\nIn essence, vaccines are a scientifically proven method to build immunity safely and efficiently."
+    },
+    {
+      "created_at": 1791228630380,
+      "role": "user",
+      "text": "What are the side effects?"
+    },
+    {
+      "created_at": 1791228719260,
+      "role": "assistant",
+      "text": "Vaccine side effects are typically mild and short-lived, often resembling common illnesses like fever or soreness at the injection site. These reactions occur because the immune system is actively responding to the vaccine, as it would to a real infection. For example, soreness at the injection site or fatigue may happen within hours to days after vaccination, and usually resolve within 24\u201348 hours [1].\n\nMore serious side effects are extremely rare. The U.S. Centers for Disease Control and Prevention (CDC) reports that adverse events from vaccines are overwhelmingly mild and often not different from those seen with natural infections. For instance, fever after a flu shot is more common and less severe than the fever from actual influenza.\n\nSome people may experience allergic reactions\u2014such as hives or anaphylaxis\u2014but these are rare and typically occur within minutes of vaccination. Such reactions are closely monitored, and immediate medical care is available. In contrast, natural infection carries a much higher risk of severe complications, including hospitalization or death.\n\nVaccines are rigorously tested and continuously monitored for safety. The benefits of vaccination\u2014such as preventing millions of cases of disease annually\u2014far outweigh the risks. For example, the measles vaccine prevents serious complications like pneumonia or encephalitis, which can occur in 1 in 1,000 cases of natural infection.\n\nIn summary, vaccine side effects are generally mild, transient, and well-managed, while the risks of contracting the actual disease are significantly greater. This makes vaccination one of the most effective public health tools available."
+    },
+    {
+      "created_at": 1791228796130,
+      "role": "user",
+      "text": "Who proposed the theory of evolution by natural selection?"
+    },
+    {
+      "created_at": 1791228882883,
+      "role": "assistant",
+      "text": "The theory of evolution by natural selection was primarily proposed by Charles Darwin, published in his 1859 work *On the Origin of Species*. Darwin synthesized observations from natural history, including those of Alfred Russel Wallace, who independently developed a similar idea. Darwin\u2019s key insight was that individuals with traits better suited to their environment are more likely to survive and reproduce, passing those advantageous traits to offspring over generations [1].\n\nUnlike Lamarckism, which suggested traits were passed on due to use or disuse, Darwin emphasized variation and differential survival. Natural selection acts on random genetic variation, favoring traits that enhance reproductive success\u2014such as camouflage in birds or drought-resistant plants in arid regions.\n\nDarwin\u2019s theory explained biodiversity through gradual, observable changes over time, supported by fossil records and biogeography. For example, the adaptive differences in finch beaks on the Gal\u00e1pagos Islands illustrate how environmental pressures shape evolution.\n\nWhile Wallace contributed significantly, Darwin was the first to present a comprehensive, testable mechanism. His theory laid the foundation for modern evolutionary biology, integrating genetics in the 20th century through the modern synthesis.\n\nThus, Darwin is credited as the originator of the theory, though it was refined through later scientific advances. [1]"
+    },
+    {
+      "created_at": 1791228931238,
+      "role": "user",
+      "text": "What did he publish?"
+    },
+    {
+      "created_at": 1791228986295,
+      "role": "assistant",
+      "text": "Charles Darwin published *On the Origin of Species* in 1859, which introduced the theory of evolution by natural selection as a comprehensive explanation for biodiversity. In this work, he demonstrated how small, inherited variations in organisms lead to differential survival and reproduction in changing environments. For instance, in the case of finches on the Gal\u00e1pagos Islands, those with beaks better suited to available food sources survived and passed on those traits [1].\n\nHe contrasted natural selection with Lamarckism, which claimed traits were passed based on use or disuse. Darwin\u2019s mechanism offered a testable, observable process\u2014unlike speculative theories. His examples, like the peppered moth during industrialization, showed how environmental pressure shifts allele frequencies over time.\n\nThe book was revolutionary because it unified biology, showing species are not fixed but descended from common ancestors. This framework laid the foundation for modern genetics and evolutionary biology."
+    },
+    {
+      "created_at": 1791229065436,
+      "role": "user",
+      "text": "Compare the French Revolution and the Industrial Revolution."
+    },
+    {
+      "created_at": 1791229163582,
+      "role": "assistant",
+      "text": "The French Revolution (1789\u20131799) was a political and social upheaval that dismantled monarchy and aristocracy, promoting ideals of liberty, equality, and fraternity [1]. In contrast, the Industrial Revolution (starting c. 1760) was an economic transformation marked by mechanization, urbanization, and rising productivity [2]. While the French Revolution changed societal structures and governance, the Industrial Revolution reshaped economies and labor systems. The former disrupted power hierarchies; the latter redefined production and class relations. Both were catalysts of modernity, but one focused on political change, the other on technological and economic advancement. Neither was a single event but a complex process with lasting global impacts."
+    },
+    {
+      "created_at": 1791229214162,
+      "role": "user",
+      "text": "What is the capital of Australia?"
+    },
+    {
+      "created_at": 1791229292804,
+      "role": "assistant",
+      "text": "The capital of Australia is Canberra. Unlike most countries, Australia does not have a capital city at a single geographic location, as it was deliberately chosen to be a compromise between Sydney and Melbourne. Canberra was established in 1913 as a neutral site, reflecting the nation\u2019s commitment to federal balance. It is located in the Australian Capital Territory, a non-contiguous territory designed to serve as a national center. This contrasts with capitals like London or Washington, D.C., which are central to historical or political developments. Canberra\u2019s design, with its planned layout and government institutions, symbolizes Australia\u2019s modern federalism and democratic governance."
+    }
+  ]
+}