updates.Manager: steady-state cost (per-channel idle getChannelDifference + goroutine-per-channel) scales with channel count — ideas welcome

[vzd3v]

Version

v0.157.1 (referencing code by function name in telegram/updates, in case line numbers drift).

Context

I run gotd for accounts that are members of thousands of channels (one account here sits in ~6.1k). Everything works correctly — this isn't a bug report. It's about steady-state cost that scales linearly with the number of tracked channels, which gets heavy at this scale. Related to, but different from, the startup deadlock in #1658 — that was init; this is ongoing.

I wanted to lay out the observations and a few possible directions and hear what you think before writing code — I may well be missing a reason it works the way it does.

Observation 1 — unconditional periodic getChannelDifference per channel

Each tracked channel runs channelState.Run, whose idle path fires getDifference → UpdatesGetChannelDifference on a fixed idleTimeout (15 min) whenever the channel is quiet. With N tracked channels that's a steady N / 15 min baseline of mostly-empty channel-difference RPCs, independent of whether anything could have changed.

This matches what I measure: an account with ~6.1k channels emits ~6.8 getChannelDifference/s of empty diffs (6138 / 900s), continuously. On a healthy connection these polls are largely redundant with the pushes the server already delivers, and they compete with the app's own RPCs for the rate-limit budget (at higher N they can start drawing FLOOD_WAIT).

The server-provided Timeout on ChannelDifference* is stored into diffTimeout, but in practice the 15-min idle timer is the effective cadence (my measured rate ≈ N / 15 min), so for empty diffs it doesn't seem to push the next poll out much.

Observation 2 — goroutine + timer per tracked channel

internalState starts one goroutine (channelState.Run) per channel, each with its own idle timer and buffered channels, and ForEachChannels spawns one for every stored channel at boot. At thousands of channels per account this is a non-trivial fixed memory/scheduler cost (tens of MB/account in my case), multiplied across accounts.

Possible directions (genuinely unsure — curious what you think)

Just options, not a settled proposal:

1. Honor the server Timeout as the primary next-poll cadence for a channel, instead of / ahead of the fixed 15-min idle.
2. Adaptive backoff for quiet channels — after K consecutive empty diffs, grow the interval (15 m → 1 h → … cap). Active channels stay push-driven and responsive; long-silent ones decay toward ~no polling.
3. Tie silent-channel resync to connection lifecycle (reconnect / common-state gap) rather than a per-channel wall-clock timer, since that's when updates can actually have been missed.
4. (Bigger) replace goroutine-per-channel with a single scheduler + a min-heap / timing-wheel of next-poll deadlines, so channel state is plain data and the goroutine count is O(1).
5. (Optional, niche) a Config hook letting the caller scope which channels the manager tracks at all, for clients that deliberately only care about a subset (default nil = today's behavior).

(1)–(3) would help any gotd user with many channels, not just me; (4) is the memory/goroutine axis; (5) is the niche extra.

Question

Is there a correctness reason the unconditional 15-min poll needs to stay — e.g. catching pushes that were silently dropped in a way gap detection wouldn't surface? If the broad direction sounds reasonable, I'm happy to prototype (1)/(2) and open a PR. Mostly I'd like your read on whether this fits where you want telegram/updates to go.

Thanks!

[ernado]

As far as I know, sometimes the Telegram server simply doesn't send updates for some reason.
The 15-minute poll is there to catch those situations.
However, this strategy isn't ideal. Your proposals seem reasonable — I'm open to them, and they would be a significant improvement.