ITU compliance Stage 5 (core): the preset as library API, the C ABI chain, the claims

HANDOFF.md

@@ -375,6 +375,19 @@ make_chain_config) must be kept in lockstep until then. Note: Stage 4's
 notebook ended up NOT needing the C ABI (its dump harness includes the
 test machinery directly), so the "C ABI extension" item is now about
 FFI consumers generally — scope it when the MuTap half lands.*
+*CORE HALF DONE — `mutap::aec_chain_preset<Sample>(block, partitions,
+fs)` lives in postfilter.h (the follow-up above, resolved: the preset
+generalizes the per-rate floor-bias calibration by interpolating
+between the two measured points, clamped to the measured neighborhood);
+tests/support/itu_chain.h chain_config() is now a one-line call into it,
+so any preset change lands as a failing gate, never silent drift — the
+full suite re-verified green after the refactor. C ABI grew the
+`mutap_aec_*` family (create-from-preset / process / echo_explained /
+converged / clone; probed via a C driver). README gained the
+compliance-claims table with margins; the book's echo chapter gained
+"The last 30 dB: @postfilter" plus the two new knobs. REMAINING in
+MuTap-Max, after this merges: re-pin the submodule and switch the
+external's make_chain_config to call the library preset.*
 
 ---
 

README.md

@@ -247,6 +247,41 @@ work (CMSIS-DSP/Helium mapping, instruction-count ratchets) and the
 Hexagon data-layout work on real hardware (VTCM residency, FastRPC
 offload).
 
+## ITU-T compliance
+
+The echo-cancellation chain — `mutap::aec_chain` configured by
+`mutap::aec_chain_preset` (and exposed as `mutap.aec~ @postfilter 1` in
+[MuTap-Max](https://github.com/tap/MuTap-Max)) — **meets every
+requirement of the in-force ITU-T automotive/hands-free recommendations
+at both required rates, 48 kHz and 16 kHz**, on one pinned
+configuration. Measured margins over the requirements (worst rate,
+cabin path):
+
+| Claim | Requirement | Measured (worst rate) | Margin |
+|---|---|---|---|
+| Terminal coupling loss (P.1110/P.1120 §11.11.1) | ≥ 46 dB | 68.4 dB | +22 dB |
+| Single-talk echo level (§11.11.2) | < −58 dBm0(A) | −76.4 | +18 dB |
+| Attenuation spectrum vs the WB mask (§11.11.3) | mask | mask +13.3 dB | +13 dB |
+| Convergence from cold start (§11.11.4) | ≥ 40 dB by 1.2 s | 45.4 dB | +5.4 dB |
+| Convergence in driving noise (§11.11.5) | at ref by 1.5 s | at ref by 0.75 s | 2× |
+| Double-talk send attenuation (P.340 Cat. 1) | ≤ 3 dB | 0.9 dB integrated | +2.1 dB |
+| Double-talk echo loss, every band (P.340 Cat. 1) | ≥ 27 dB | 37.5 dB | +10 dB |
+| Comfort-noise level match (§11.13) | +2/−5 dB | −2.15 dB | inside |
+| Closed-loop stability (P.1110 Annex E) | stable | stable at 0 dB far-end ERL | sweep floor |
+| Algorithmic delay | ≤ 70 ms budget | 10.7 / 32 ms | ≥ 2× |
+
+Scope, honestly stated: G.168's battery is reported as **G.168-adapted**
+(the rec disclaims acoustic scope); G.167 is withdrawn and **run and
+reported**, not claimed; compressed real speech is method-equivalent
+pending ITU test-vector procurement (synthetic P.501 signals generated
+from the recommendations' algorithmic descriptions); and self-imposed
+margin *targets* missed at 16 kHz are documented regression gates, never
+silent. The clause-by-clause matrix is
+[`docs/itu-compliance.md`](docs/itu-compliance.md); every row is
+asserted by the test suite on every CI run, and
+[`notebooks/itu_compliance.ipynb`](notebooks/itu_compliance.ipynb)
+re-measures the whole battery live and renders the trajectories.
+
 ## Quick start
 
 ```cmake

book/src/echo-cancellation.md

@@ -153,7 +153,79 @@ directions of this trade so neither silently erodes.
 
 The latency contract is identical to `mutap.afc~`: the cleaned output is
 delayed by exactly `@block` samples (256 by default, 5.3 ms at 48 kHz),
-and that is the only latency the object adds.
+and that is the only latency the object adds — until `@postfilter`, which
+adds one more block (next section).
+
+## The last 30 dB: `@postfilter`
+
+A linear canceller subtracts its best estimate of the echo, and its best
+estimate is never the whole story: the estimate converges asymptotically,
+the room drifts, and small nonlinearities in the speaker never fit a
+linear filter at all. What survives subtraction is **residual echo** —
+far quieter than the original, still audibly *there* on a quiet line.
+Telephony standards do not grade "much quieter"; they grade *inaudible*,
+with numbers attached.
+
+`@postfilter 1` engages MuTap's answer, and it is a different kind of
+machine than everything before it. The canceller *subtracts*; the
+post-filter *decides*. Per frequency band, it asks: does what is left in
+the microphone still **cohere** with what the canceller believes the echo
+is? Where the answer is yes, the leftover is residual echo, and the band
+is turned down — by a learned amount, deep only where the evidence is.
+Where the answer is no, the leftover is *you*, and the band passes
+untouched. Double-talk transparency is not a detector bolted on the side;
+it is the shape of the rule, because your voice destroys exactly the
+coherence that would justify suppression.
+
+Two companions make the decision maker livable:
+
+- **Comfort noise** (`@comfort`, default on). Turning bands down also
+  removes the room tone under them, and a noise floor that breathes —
+  present while you talk, gone while the far end talks — is more
+  annoying than the echo was. So the post-filter tracks the *real*
+  noise floor during pauses (by watching minima, which speech cannot
+  fake) and fills what it suppresses back to exactly that level. Fill
+  only, never subtraction: turn it off and suppressed bands go silent
+  instead.
+- **The receive guard.** For the first fraction of a second of a call
+  the canceller has learned nothing, its echo estimate is zero, and a
+  coherence rule that references it is structurally blind — raw echo
+  would pass. The guard applies a modest switched loss (< 14 dB, inside
+  the standards' own switching allowance) only while the far end is
+  active and convergence is uncertified, then latches off permanently.
+  Measured latch: under half a second of far-end speech.
+
+The combination is not tuned to taste; it is the configuration MuTap's
+**ITU-T compliance battery** certifies — every requirement of the
+in-force automotive/hands-free recommendations (P.1110/P.1120 clause 11,
+P.340 full-duplex Category 1, the G.168-adapted battery) met at both
+required rates, 48 and 16 kHz. Headlines from the measured tables:
+single-talk residual below **−76 dBm0(A)** where the clause wants −58,
+double-talk cost to your voice about **1 dB** integrated, comfort noise
+matched within ~1–2 dB of the true floor, full-duplex echo loss
+**≥ 37 dB in every band** while both sides talk. The full
+requirement/measured/margin story — with the trajectories, and the one
+documented deviation (deep re-convergence after an *abrupt* path change
+is slow; the mask element still holds) — lives in MuTap's
+`docs/itu-compliance.md` and the executed proof notebook
+`notebooks/itu_compliance.ipynb`.
+
+The fine print. `@postfilter` selects its own engine — the raw Kalman
+canceller, because the battery measured the PEM engines dozens of dB
+worse on open-loop echo (there is no closed-loop bias for prewhitening
+to fix out here, so its predictor refit is pure gradient noise) — which
+means `@mu`, `@warp` and `@kalman` are ignored while it is on, and
+`@gate` selects the receive guard. The right outlet switches from IPC to
+the suppressor's **echo-explained** fraction (0..1 — watch it climb as
+the canceller converges; the guard releases at 0.9). And the constrained
+gain filter that keeps the suppression click-free costs one extra block
+of latency, 10.7 ms total at the defaults.
+
+When to use it: any conversation — calls, conferencing, streams — where
+the far end must not hear themselves; this is the product mode. When to
+leave it off: measurement and monitoring patches where you want the
+linear path untouched, or any time you need the bare canceller's output
+to study what it learned.
 
 ## The knobs, revisited
 
@@ -202,6 +274,13 @@ what changes is the advice.
   engine for the same reason as chapter 1: seniority, and one
   default-engine decision for the package, made once, after real-room
   listening.)
+- **`@postfilter`.** The residual-echo suppressor + comfort noise +
+  receive guard of the previous section — the ITU-certified chain. On
+  for conversation, off for measurement. Ignores `@mu`/`@warp`/`@kalman`
+  while on; `@gate` becomes the receive guard; +1 block of latency.
+- **`@comfort`.** With `@postfilter` on: fill suppressed bands to the
+  room's tracked noise floor (default on). Off = suppressed bands go
+  silent — useful when metering how much the suppressor is doing.
 - **`reset`.** Same as chapter 1: after physically moving speaker or
   mic, a fresh start beats un-learning.
 

docs/itu-compliance.md

@@ -565,4 +565,27 @@ note: CSS generation through the NOTE 2 resampler dominates the dump's
 runtime, so it memoizes `make_css_at` — verified byte-identical output
 against the uncached run.)
 
-Remaining: Stage 5 (externals/docs).
+## Stage 5 delivered: the preset, the external, the claims
+
+- **The pinned configuration is now a library API**:
+  `mutap::aec_chain_preset<Sample>(block, partitions, fs)` in
+  `mutap/postfilter.h`. `tests/support/itu_chain.h` builds the
+  compliance chain through it, so the suite gates the preset itself;
+  the per-rate comfort-noise floor-bias calibration is generalized by
+  interpolating the two measured points (ratio 1 → 4.0, ratio 3 → 5.6)
+  with a clamp to the measured neighborhood — geometries far outside
+  block 256 at 16..96 kHz are uncalibrated and say so in the header.
+- **`mutap.aec~ @postfilter 1`** (MuTap-Max) engages the certified
+  chain; `@comfort` controls the fill, `@gate` maps to the receive
+  guard, the right outlet reports echo-explained. The external
+  currently replicates the preset's scaling rule and will switch to
+  calling the library preset at the next submodule re-pin.
+- **C ABI**: the `mutap_aec_*` family exposes the chain to FFI
+  consumers, configured by the preset.
+- **Claims**: README carries the requirement/measured/margin table
+  (worst-rate values) with the scope caveats (G.168-adapted framing,
+  G.167 run-and-reported, synthetic-signal method equivalence);
+  the book's echo chapter explains the post-filter for non-DSP readers.
+
+Remaining: ITU real-speech test-vector procurement (tracked above) and
+the fd_kalman uncertainty re-inflation core follow-up (HANDOFF).

include/mutap/postfilter.h

@@ -761,4 +761,57 @@ namespace mutap {
         Sample                      m_guard      = Sample(1);
     };
 
+    /// THE COMPLIANCE PRESET: the aec_chain configuration MuTap's ITU-T
+    /// battery certifies (docs/itu-compliance.md — every requirement of
+    /// the automotive/hands-free recommendations met at both required
+    /// rates), generalized to any (block_size, sample_rate) geometry.
+    ///
+    /// The measurements behind every constant were taken at the reference
+    /// geometry, block 256 at 48 kHz; portability to other geometries is
+    /// one rule applied uniformly: every per-block constant is rescaled so
+    /// the PHYSICAL time constants hold — smoothing factors a' =
+    /// a^(block_s / ref_block_s), windows counted in blocks divide by the
+    /// same ratio, and the low-band suppression cap keeps covering
+    /// 0..300 Hz whatever the bin width. Two constants deliberately do
+    /// NOT follow the rule:
+    ///   - the canceller transition stays 0.9998 per block (rescaling it
+    ///     traded the time-variant-path row to the wire for hangover);
+    ///   - the comfort-noise floor bias is calibrated per geometry, not
+    ///     rescaled: longer blocks put fewer meter samples in each
+    ///     minimum-statistics window and the minima bias deeper. Two
+    ///     points are measured (ratio 1: 4.0; ratio 3, the 16 kHz rate:
+    ///     5.6); between and near them the preset interpolates linearly
+    ///     in the ratio and clamps to the measured neighborhood — treat
+    ///     readings far outside block 256 at 16..96 kHz as uncalibrated.
+    ///
+    /// The Tier A / Tier B batteries pin their chain through this preset
+    /// (tests/support/itu_chain.h), so the configuration it returns for
+    /// block 256 at 48 and 16 kHz is exactly the measured one; the
+    /// mutap.aec~ external's @postfilter mode builds from it as well.
+    template <typename Sample>
+    typename aec_chain<Sample>::config aec_chain_preset(size_t block_size, size_t partitions, double sample_rate) {
+        typename aec_chain<Sample>::config cfg;
+        cfg.canceller.block_size          = block_size;
+        cfg.canceller.partitions          = partitions;
+        cfg.canceller.transition          = Sample(0.9998); // the measured AEC sweet spot; NOT rescaled (see above)
+        cfg.canceller.initial_uncertainty = Sample(10);
+        const double ratio                = (static_cast<double>(block_size) / sample_rate) / (256.0 / 48000.0);
+        cfg.canceller.noise_smoothing     = Sample(std::pow(0.9, ratio));
+        auto& pf                          = cfg.postfilter;
+        pf.leakage_smoothing              = Sample(std::pow(static_cast<double>(pf.leakage_smoothing), ratio));
+        pf.gain_attack                    = Sample(std::pow(static_cast<double>(pf.gain_attack), ratio));
+        pf.gain_release                   = Sample(std::pow(static_cast<double>(pf.gain_release), ratio));
+        pf.floor_smoothing                = Sample(std::pow(static_cast<double>(pf.floor_smoothing), ratio));
+        pf.floor_window = std::max<size_t>(8, static_cast<size_t>(static_cast<double>(pf.floor_window) / ratio));
+        // Low-band suppression cap from 300 Hz (protect voice fundamentals
+        // no analysis resolution can separate from echo; the canceller
+        // owns low-frequency echo) with sustained certification held at
+        // 0.3 s of real time.
+        const double n_analysis    = static_cast<double>(pf.analysis_blocks * block_size);
+        pf.low_band_bins           = static_cast<size_t>(300.0 * n_analysis / sample_rate) + 1;
+        pf.low_band_certify_blocks = std::max<size_t>(8, static_cast<size_t>(56.0 / ratio));
+        pf.floor_bias              = Sample(std::clamp(4.0 + 0.8 * (ratio - 1.0), 3.0, 5.6));
+        return cfg;
+    }
+
 } // namespace mutap

tests/support/itu_chain.h

@@ -55,47 +55,16 @@ namespace mutap_test::itu {
         return {setup_48k(), setup_16k()};
     }
 
+    /// The pinned compliance configuration IS the library preset
+    /// (mutap::aec_chain_preset — Stage 5 moved the scaling rule there;
+    /// its header documents every constant and the two deliberate
+    /// exceptions). The suite measures what the preset returns for
+    /// block 256 at 48 and 16 kHz, so any preset change lands here as a
+    /// failing gate, never as silent drift. The historical per-rate
+    /// rationale (why transition is NOT rescaled, why floor_bias is
+    /// calibrated per geometry) lives with the preset.
     inline compliance_chain::config chain_config(const rate_setup& rs) {
-        compliance_chain::config cfg;
-        cfg.canceller.block_size          = rs.block;
-        cfg.canceller.partitions          = rs.taps / rs.block;
-        cfg.canceller.initial_uncertainty = 10;
-        // ALL per-block constants — canceller and suppressor — are
-        // rescaled to keep the same PHYSICAL time constants; blocks at
-        // 16 kHz last 3x longer (16 ms vs 5.33), so rescale every
-        // constant to keep the same PHYSICAL time constants the Stage 2
-        // measurements calibrated (a' = a^(block_s / ref_block_s)),
-        // and the floor window to the same seconds.
-        const double ratio = (static_cast<double>(rs.block) / rs.fs) / (256.0 / 48000.0);
-        // Canceller: transition (state decay per block) and the noise-PSD
-        // smoothing follow block duration — unscaled, the 16 kHz Kalman's
-        // observation-noise tracker reacted 3x slower in wall time and
-        // double talk dragged its filter (hangover recovery measured
-        // 16.4 dB bare where 48 kHz reads 42.6).
-        cfg.canceller.transition =
-            0.9998; // NOT rescaled: rescaling traded TimeVariantPath to the wire (-52.3 vs req -52) for hangover
-        cfg.canceller.noise_smoothing = std::pow(0.9, ratio);
-        auto& pf                      = cfg.postfilter;
-        pf.leakage_smoothing          = std::pow(pf.leakage_smoothing, ratio);
-        pf.gain_attack                = std::pow(pf.gain_attack, ratio);
-        pf.gain_release               = std::pow(pf.gain_release, ratio);
-        pf.floor_smoothing            = std::pow(pf.floor_smoothing, ratio);
-        pf.floor_window = std::max<size_t>(8, static_cast<size_t>(static_cast<double>(pf.floor_window) / ratio));
-        // Low-band suppression cap at 300 Hz (see postfilter.h): protect
-        // voice fundamentals where no analysis resolution can separate
-        // them from echo; the canceller owns low-frequency echo.
-        const size_t n_analysis    = pf.analysis_blocks * rs.block;
-        pf.low_band_bins           = static_cast<size_t>(300.0 * static_cast<double>(n_analysis) / rs.fs) + 1;
-        pf.low_band_certify_blocks = std::max<size_t>(8, static_cast<size_t>(56.0 / ratio));
-        // Comfort-noise floor bias, calibrated per rate: at 16 ms blocks
-        // the minimum-statistics window holds 3x fewer meter samples and
-        // the minima bias deeper (measured -2.8 dB comfort-noise step
-        // tracking at bias 4, against G.168's +-2 requirement; 5.6
-        // restores the 48 kHz calibration).
-        if (rs.fs != 48000.0) {
-            pf.floor_bias = 5.6;
-        }
-        return cfg;
+        return mutap::aec_chain_preset<double>(rs.block, rs.taps / rs.block, rs.fs);
     }
 
     enum class room { cabin, studio };

tools/capi/mutap_capi.cpp

@@ -42,6 +42,10 @@ struct MutapAfc {
     std::variant<speech_afc, warped_afc, kalman_speech_afc, kalman_warped_afc> impl;
 };
 
+struct MutapAec {
+    mutap::aec_chain<double> impl;
+};
+
 unsigned mutap_version(void) {
     return MUTAP_VERSION_MAJOR * 10000U + MUTAP_VERSION_MINOR * 100U + MUTAP_VERSION_PATCH;
 }
@@ -236,4 +240,65 @@ MutapAfc* mutap_afc_clone(const MutapAfc* h) {
     }
 }
 
+MutapAec* mutap_aec_create(size_t block_size, size_t partitions, double sample_rate, int comfort_noise,
+                           int receive_guard) {
+    try {
+        if (sample_rate <= 0.0) {
+            return nullptr;
+        }
+        auto cfg                     = mutap::aec_chain_preset<double>(block_size, partitions, sample_rate);
+        cfg.postfilter.comfort_noise = comfort_noise != 0;
+        if (receive_guard == 0) {
+            cfg.guard_attenuation_db = 0.0;
+        }
+        return new MutapAec{mutap::aec_chain<double>(cfg)};
+    }
+    catch (...) {
+        return nullptr;
+    }
+}
+
+void mutap_aec_destroy(MutapAec* h) {
+    delete h;
+}
+
+void mutap_aec_process(MutapAec* h, const double* x, const double* y, double* e) {
+    if (h != nullptr) {
+        h->impl.process_block(x, y, e);
+    }
+}
+
+size_t mutap_aec_block_size(const MutapAec* h) {
+    return h != nullptr ? h->impl.block_size() : 0;
+}
+
+double mutap_aec_echo_explained(const MutapAec* h) {
+    return h != nullptr ? h->impl.postfilter().echo_explained() : 0.0;
+}
+
+int mutap_aec_converged(const MutapAec* h) {
+    return (h != nullptr && h->impl.converged()) ? 1 : 0;
+}
+
+void mutap_aec_set_adaptation(MutapAec* h, int enabled) {
+    if (h != nullptr) {
+        h->impl.set_adaptation(enabled != 0);
+    }
+}
+
+void mutap_aec_reset(MutapAec* h) {
+    if (h != nullptr) {
+        h->impl.reset();
+    }
+}
+
+MutapAec* mutap_aec_clone(const MutapAec* h) {
+    try {
+        return h != nullptr ? new MutapAec{*h} : nullptr;
+    }
+    catch (...) {
+        return nullptr;
+    }
+}
+
 } // extern "C"