You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
operator: re-measure resource sizing at scale; the equations under-size large clusters #839
The operator's sizing guidance, docs/operations/resources-at-scale.md and the chart's default resource calculation from estimatedNodeCount and estimatedPackageCount (chart/templates/deployment.yaml), is validated only up to 1k nodes. It no longer describes how the operator uses memory:
The equations take only node and package count. perf(operator): cut reconcile-path allocations on large clusters #797 profiled a cluster of about 2,700 nodes and 80,000 pods: after that change the live heap is about 2.5 GiB, and the Pod cache is the largest part of it. That cache is cluster-wide because drain has to see every pod, so pod count is a missing input.
The equations predate package execution as Jobs. The doc already says they were not re-measured with the Jobs informer in the cache.
Nothing sets GOMEMLIMIT, in the chart or in operator/config.
The chart sets the memory request to 0.4 × N; the doc says 0.45 × N.
Proposal
Redo the sizing investigation on top of #797, at several cluster sizes including at least one past 2k nodes, varying pod count and retained Jobs (jobTtlSucceeded) along with node and package count. The opt-in pprof endpoint in #795 would make the profiling repeatable. From the results:
Update the equations and the validated range in resources-at-scale.md.
Update the chart calculation to match. Add a pod-count input if the data shows it is needed.
Decide whether the chart should derive GOMEMLIMIT from the memory limit.
Activity