MEDIUM: ingress: support source-IP persistence failover

[MaxRink]

Summary

• Reconcile HAProxy localinstance peer entries from the current controller pod plus same-workload controller pods.
• Carry controller PodIP through pod informer events and store HAProxy controller pods as pod name -> PodIP.
• Add service/configmap annotations for HAProxy source-IP session persistence:
  • source-ip-persistence
  • source-ip-persistence-size
  • source-ip-persistence-expire
• Generate the HAProxy IP persistence pattern from the HAProxy session-persistence docs:
  • stick-table type ip size <size> expire <expire> peers localinstance
  • stick on src
• Keep backend session-persistence annotations service-scoped, falling back to the ConfigMap, and ignore ingress-level values for these backend-wide settings to avoid shared-backend config churn.
• Sort runtime endpoints before assigning SRV_<n> slots so every controller replica maps the same endpoint to the same server identity. This matters for MagLev/source-hash consistency and for peer-replicated stick-table server IDs during failover.
• Update the e2e config dump helper so tests can inspect a controller config when the local kind controller deployment has multiple running replicas.
• Document --localpeer-port, generated stick-table peer behavior, source-IP persistence annotations, and the deterministic server-slot requirement in documentation/doc.yaml; regenerate controller.md and annotations.md.

Behavior

• The controller still deletes the legacy local peer entry and ensures its own pod peer entry during startup.
• During normal HAProxy sync, it reconciles entries in the controller-managed localinstance peer section.
• Expected peer entries are built from:
  • the running controller pod name, pod IP, and --localpeer-port;
  • tracked pods from the same controller Deployment, DaemonSet, or ReplicaSet.
• Entries with names managed by this controller workload are deleted when they are no longer expected.
• Entries outside the controller workload naming scope are left alone.
• A reload is requested only when peer entries were created, updated, or deleted.
• When source-ip-persistence: "true" is set, the backend gets a source-IP stick table attached to peers localinstance and a stick on src rule.
• load-balance: "source" and CRD hash-type settings can still choose the initial server. The stick table then records the selected server and can replicate that state between controller pods through the local peer section.
• Runtime endpoint sorting keeps SRV_1, SRV_2, etc. stable across controller replicas, so MagLev/source-hash choices and replicated stick-table server IDs refer to the same backend pod on each controller.

Full failover setup

Service annotation example:

metadata:
  annotations:
    load-balance: "source"
    source-ip-persistence: "true"
    source-ip-persistence-size: "1m"
    source-ip-persistence-expire: "30m"

Generated HAProxy backend shape:

backend <service-backend>
  balance source
  stick on src
  stick-table type ip size 1m expire 1800000 peers localinstance
  server SRV_1 <pod-ip-1>:8888 enabled
  server SRV_2 <pod-ip-2>:8888 enabled

Topology requirements:

• Run more than one ingress-controller pod in the same Deployment, DaemonSet, or ReplicaSet lineage.
• Ensure --localpeer-port is reachable between ingress-controller pods. The default is 10000.
• Allow controller-pod-to-controller-pod traffic on that peer port in NetworkPolicy or equivalent firewall policy.
• Enable source-ip-persistence on the Service or globally in the ConfigMap.
• Use load-balance: "source" for standard Ingress source-hash behavior, or CRD hash-type settings where an environment needs explicit hash-type control.

Validation

Local Go/unit checks:

env GOCACHE=/tmp/tcaas-cli-go-cache go test ./pkg/store ./pkg/service ./pkg/haproxy/api ./test/annotations ./deploy/tests/ut/stick
env GOCACHE=/tmp/tcaas-cli-go-cache go test ./deploy/tests/e2e/cookie-persistence -tags=e2e_parallel -run TestCookiePersistenceTestSuite/Test_SourceIPPersistence_WithSourceHash -count=0

Local kind checks only:

deploy/tests/create.sh codex-haproxy-failover
env GOCACHE=/tmp/tcaas-cli-go-cache go test ./deploy/tests/e2e/cookie-persistence -tags=e2e_parallel -run TestCookiePersistenceTestSuite/Test_SourceIPPersistence_WithSourceHash -count=1 -v
env GOCACHE=/tmp/tcaas-cli-go-cache go test ./deploy/tests/e2e/cookie-persistence -tags=e2e_parallel -count=1 -v

Additional local two-controller proof in kind-codex-haproxy-failover:

• Scaled haproxy-controller/haproxy-kubernetes-ingress to 2 replicas.
• Rebuilt and loaded the local controller image into kind.
• Reran the full cookie-persistence e2e package successfully with both local controller replicas running.
• Verified both local controller pods rendered the same peer section with both pod peer entries.
• Verified both local controller pods rendered the same source-persistence backend server slot order:

balance source
stick on src
stick-table type ip size 1m expire 1800000 peers localinstance
server SRV_1 10.244.0.32:8888 enabled
server SRV_2 10.244.0.33:8888 enabled
server SRV_3 10.244.0.34:8888 enabled

No actual clusters were used for validation; all Kubernetes runtime checks above were against local kind only.

Scope and limitations

• This is intentionally a draft PR.
• This does not add a generic static peer mesh configuration surface.
• This does not reconcile arbitrary manually configured peer sections.
• This does not expose arbitrary HAProxy stick-table or stick-rule annotations.
• Peer replication still requires the controller pod peer port to be exposed and reachable between controller pods.

References

• HAProxy session persistence docs: https://www.haproxy.com/documentation/haproxy-configuration-tutorials/proxying-essentials/session-persistence/