feat(preprocessing,compose): dataframe-agnostic StandardScaler & compose mini-batching via narwhals

[MaxHalford]

Motivation

Part of #1919 (migrate all _many methods to narwhals). This makes preprocessing.StandardScaler and the compose composition primitives dataframe-agnostic, so a whole pipeline can be mini-batched on any narwhals-supported eager backend (pandas, polars, pyarrow, nullable/arrow-backed pandas, ...). The numpy compute cores are untouched and the input backend (including the pandas index) is rebuilt on output.

Changes

preprocessing/scale.py — StandardScaler

• learn_many wraps the input via into_frame and drops to a float64 numpy matrix with to_numpy; the windowed branch iterates rows via iter_rows.
• transform_many keeps a verbatim classic-pandas fast path (in-place np.divide(out=), no-copy frame, float dtype preservation) and adds an agnostic float64 path for every other backend. Pandas output is byte-for-byte unchanged.

compose

• Pipeline: type hints only — the orchestration was already backend-agnostic.
• TransformerUnion: pd.concat(axis=1) → nw.concat(how="horizontal").
• Select: X.loc[...].copy() → narwhals select (still pure).
• TransformerProduct: keeps the pandas Sparse[uint8] fast path, adds an agnostic elementwise-product path for other backends.
• FuncTransformer: unchanged (already passes the native frame through).

Tests

New cross-backend tests via the frame_backend fixture, with pandas as the oracle:

• value/stat parity for learn_many & transform_many (incl. with_std and window_size variants)
• mixed-dtype real dataset (TrumpApproval), chunked learning
• emerging / disappearing / reordered features between mini-batches
• native-backend round-trip, float32 preservation on the pandas fast path

All scale + compose tests pass, 481 estimator-checks pass, mypy clean.

Performance

Pandas (dominant case) has zero regression: transform_many's pandas path is the old code verbatim, and learn_many matches the old baseline at realistic sizes (10.96 vs 11.12 ms at 50k×100). The only added cost is the constant ~60µs narwhals.from_native boundary, identical to the other narwhalified methods.

🤖 Generated with Claude Code

[MaxHalford]

@FBruzzesi does it look ok to you?

[FBruzzesi]

Hey Max, I was about to push the changes for StandardScaler with some general improvements to create a DataFrame from a 2d numpy array without having to slice it. preprocessing module seems great!

If you are not in a rush I can make a more in depth review later

[MaxHalford]

Sure, take your time! And feel free to push on top of this PR 👍

I'm done for today hehe

[FBruzzesi]

Went through it a bit more thoroughly: nothing to add for the compose module, most ops are straightforward. For the other comment on the StandardScaler, I am proposing two changes in #1935.