Skip to content

Commit e9ce92c

Browse files
upgleupgle
authored
upgrade radix from v3 to v4 for improved pipeline handling (#1041)
* feat: upgrade radix from v3 to v4 Upgrade radix Redis client from v3.8.1 to v4.1.4. Main changes: - Import paths: radix/v3 -> radix/v4 - Pool/Cluster/Sentinel use Config.New() instead of New() - All client operations require context.Context parameter - Dialer setup changed from functional options to struct config - Pipelining uses radix.NewPipeline() and Append() - Write buffering via Dialer.WriteFlushInterval Breaking from v3: - Pool on-empty behavior (WAIT/CREATE/ERROR) not available - REDIS_PIPELINE_LIMIT setting deprecated (no effect in v4) Tested with existing test suite - all tests passing. Signed-off-by: seonghyun <seonghyunoh@gmail.com> * docs: update pipeline settings for radix v4 Update documentation to reflect radix v4's pipeline behavior: - REDIS_PIPELINE_WINDOW now sets WriteFlushInterval (auto-flush timing) - REDIS_PIPELINE_LIMIT deprecated - no effect in v4 - Add REDIS_USE_EXPLICIT_PIPELINE for manual pipeline control - Required for Redis Cluster: PIPELINE_WINDOW must be non-zero Update terminology from "implicit pipelining" to "write buffering" to better match radix v4's actual behavior. Signed-off-by: seonghyun <seonghyunoh@gmail.com> * test: update tests for radix v4 - Add useExplicitPipeline parameter to test client creation - Update error assertions for v4's error message format (v4 prefixes with "response returned from Conn:") - Handle different connection errors (EOF, connection reset, broken pipe) - Update radix.FlatCmd usage for v4 API Signed-off-by: seonghyun <seonghyunoh@gmail.com> * chore: add deprecation warning for REDIS_PIPELINE_LIMIT Signed-off-by: seonghyun <seonghyunoh@gmail.com> * test: fix Redis cluster test config for radix v4 Replace deprecated RedisPipelineLimit with RedisPipelineWindow in configRedisCluster function. Radix v4 requires WriteFlushInterval (RedisPipelineWindow) for cluster mode buffering instead of the deprecated pipeline limit setting. Signed-off-by: seonghyun <seonghyunoh@gmail.com> * style: fix gofmt struct field alignment in settings.go Signed-off-by: seonghyun <seonghyunoh@gmail.com> * fix: fail fast on unsupported REDIS_POOL_ON_EMPTY_BEHAVIOR settings Radix v4 does not support CREATE or ERROR behaviors for REDIS_POOL_ON_EMPTY_BEHAVIOR. Previously, these settings were logged as errors but the application would continue with blocking behavior, which could cause unexpected issues in production. Changes: - Panic at startup when CREATE or ERROR is detected - Prevent silent behavior changes that could cause blocking - Update tests to verify panic behavior - Improve migration documentation in comments This ensures users are immediately notified of incompatible configuration rather than experiencing unexpected blocking in production. Signed-off-by: seonghyun <seonghyunoh@gmail.com> * fix: change REDIS_POOL_ON_EMPTY_BEHAVIOR default to WAIT The default value 'CREATE' is not supported in radix v4 and causes integration tests to panic at startup. Changed default to 'WAIT' which matches radix v4's actual pool behavior (always blocks when empty). This fixes integration test failures where tests without explicit REDIS_POOL_ON_EMPTY_BEHAVIOR settings would panic during initialization with: "REDIS_POOL_ON_EMPTY_BEHAVIOR=CREATE is not supported in radix v4" Also updated documentation to clarify that CREATE/ERROR are not supported and marked RedisPoolOnEmptyWaitDuration as deprecated. Signed-off-by: seonghyun <seonghyunoh@gmail.com> * test: fix cluster connection timeout and context bug - Fix WaitForTcpPort to use timeoutCtx instead of ctx This ensures the timeout parameter is actually respected when dialing TCP connections. - Increase gRPC server startup timeout from 1s to 10s Radix v4 cluster connection initialization takes longer, especially when establishing connections to multiple cluster nodes. This prevents "connection refused" errors in integration tests. Signed-off-by: seonghyun <seonghyunoh@gmail.com> * refactor: extract dialer creation logic to reduce duplication Consolidates Redis and Sentinel dialer setup into a reusable createDialer helper function, eliminating ~30 lines of duplicated code. Improves logging by including connection target details (e.g., "sentinel(master,host1,host2)") instead of generic "sentinel" string. Signed-off-by: seonghyun <seonghyunoh@gmail.com> * refactor: remove deprecated REDIS_POOL_ON_EMPTY_WAIT_DURATION settings Remove the deprecated poolOnEmptyWaitDuration parameter and related configuration settings as they have no effect in radix v4. The pool always blocks until a connection is available when using WAIT behavior. Signed-off-by: seonghyun <seonghyunoh@gmail.com> * refactor: auto-select pipeline mode based on Redis type Remove REDIS_USE_EXPLICIT_PIPELINE configuration option and automatically determine pipeline mode based on Redis deployment type: - Cluster mode: uses grouped pipeline (groups same-key commands) - INCRBY + EXPIRE for same key are pipelined together (same slot) - Reduces round-trips from 2 to 1 per key in cluster mode - Single/Sentinel mode: uses explicit pipeline (batches all commands) - All commands in one pipeline for minimal latency - Optimal for non-cluster deployments This simplifies configuration by removing user-facing options while automatically choosing the optimal pipeline strategy for each Redis type. Breaking changes: - Remove REDIS_USE_EXPLICIT_PIPELINE env var - Remove REDIS_PERSECOND_USE_EXPLICIT_PIPELINE env var - Remove UseExplicitPipeline() interface method Signed-off-by: seonghyun <seonghyunoh@gmail.com> * docs: remove non-existent REDIS_USE_EXPLICIT_PIPELINE from README The REDIS_USE_EXPLICIT_PIPELINE and REDIS_PERSECOND_USE_EXPLICIT_PIPELINE settings were documented in README but do not exist in settings.go. Removed the documentation to match the actual implementation. Signed-off-by: seonghyun <seonghyunoh@gmail.com> * style: fix gofmt formatting in settings.go Signed-off-by: seonghyun <seonghyunoh@gmail.com> --------- Signed-off-by: seonghyun <seonghyunoh@gmail.com>
1 parent 167d0f8 commit e9ce92c

14 files changed

Lines changed: 290 additions & 260 deletions

File tree

‎README.md‎

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1308,14 +1308,12 @@ Controls what happens when all connections in the pool are in use and a new requ
13081308

13091309
By default, for each request, ratelimit will pick up a connection from pool, write multiple redis commands in a single write then reads their responses in a single read. This reduces network delay.
13101310

1311-
For high throughput scenarios, ratelimit also support [implicit pipelining](https://github.com/mediocregopher/radix/blob/v3.5.1/pool.go#L238) . It can be configured using the following environment variables:
1311+
For high throughput scenarios, ratelimit supports write buffering via [radix v4's WriteFlushInterval](https://pkg.go.dev/github.com/mediocregopher/radix/v4#Dialer). It can be configured using the following environment variables:
13121312

1313-
1. `REDIS_PIPELINE_WINDOW` & `REDIS_PERSECOND_PIPELINE_WINDOW`: sets the duration after which internal pipelines will be flushed.
1314-
If window is zero then implicit pipelining will be disabled.
1315-
1. `REDIS_PIPELINE_LIMIT` & `REDIS_PERSECOND_PIPELINE_LIMIT`: sets maximum number of commands that can be pipelined before flushing.
1316-
If limit is zero then no limit will be used and pipelines will only be limited by the specified time window.
1313+
1. `REDIS_PIPELINE_WINDOW` & `REDIS_PERSECOND_PIPELINE_WINDOW`: controls how often buffered writes are flushed to the network connection. When set to a non-zero value (e.g., 150us-500us), radix v4 will buffer multiple concurrent write operations and flush them together, reducing system calls and improving throughput. If zero, each write is flushed immediately. **Required for Redis Cluster mode.**
1314+
1. `REDIS_PIPELINE_LIMIT` & `REDIS_PERSECOND_PIPELINE_LIMIT`: **DEPRECATED** - These settings have no effect in radix v4. Write buffering is controlled solely by the window settings above.
13171315

1318-
`implicit pipelining` is disabled by default. To enable it, you can use default values [used by radix](https://github.com/mediocregopher/radix/blob/v3.5.1/pool.go#L278) and tune for the optimal value.
1316+
Write buffering is disabled by default (window = 0). For optimal performance, set `REDIS_PIPELINE_WINDOW` to 150us-500us depending on your latency requirements and load patterns.
13191317

13201318
## One Redis Instance
13211319

‎go.mod‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ require (
66
github.com/DataDog/datadog-go/v5 v5.5.0
77
github.com/alicebob/miniredis/v2 v2.33.0
88
github.com/bradfitz/gomemcache v0.0.0-20230905024940-24af94b03874
9+
github.com/cespare/xxhash/v2 v2.3.0
910
github.com/coocood/freecache v1.2.4
1011
github.com/envoyproxy/go-control-plane v0.13.4
1112
github.com/envoyproxy/go-control-plane/envoy v1.32.4
@@ -21,7 +22,7 @@ require (
2122
github.com/libp2p/go-reuseport v0.4.0
2223
github.com/lyft/goruntime v0.3.0
2324
github.com/lyft/gostats v0.4.14
24-
github.com/mediocregopher/radix/v3 v3.8.1
25+
github.com/mediocregopher/radix/v4 v4.1.4
2526
github.com/prometheus/client_golang v1.19.1
2627
github.com/prometheus/client_model v0.6.1
2728
github.com/prometheus/statsd_exporter v0.26.1
@@ -46,7 +47,6 @@ require (
4647
github.com/alicebob/gopher-json v0.0.0-20230218143504-906a9b012302 // indirect
4748
github.com/beorn7/perks v1.0.1 // indirect
4849
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
49-
github.com/cespare/xxhash/v2 v2.3.0 // indirect
5050
github.com/cncf/xds/go v0.0.0-20250501225837-2ac532fd4443 // indirect
5151
github.com/davecgh/go-spew v1.1.1 // indirect
5252
github.com/envoyproxy/protoc-gen-validate v1.2.1 // indirect
@@ -60,13 +60,13 @@ require (
6060
github.com/prometheus/common v0.48.0 // indirect
6161
github.com/prometheus/procfs v0.12.0 // indirect
6262
github.com/stretchr/objx v0.5.2 // indirect
63+
github.com/tilinna/clock v1.0.2 // indirect
6364
github.com/yuin/gopher-lua v1.1.1 // indirect
6465
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
6566
go.opentelemetry.io/otel/metric v1.36.0 // indirect
6667
go.opentelemetry.io/proto/otlp v1.3.1 // indirect
6768
golang.org/x/sys v0.34.0 // indirect
6869
golang.org/x/text v0.27.0 // indirect
69-
golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 // indirect
7070
google.golang.org/genproto/googleapis/api v0.0.0-20250528174236-200df99c418a // indirect
7171
google.golang.org/genproto/googleapis/rpc v0.0.0-20250728155136-f173205681a0 // indirect
7272
gopkg.in/yaml.v3 v3.0.1 // indirect

‎go.sum‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -102,8 +102,8 @@ github.com/lyft/goruntime v0.3.0/go.mod h1:BW1gngSpMJR9P9w23BPUPdhdbUWhpirl98TQh
102102
github.com/lyft/gostats v0.4.1/go.mod h1:Tpx2xRzz4t+T2Tx0xdVgIoBdR2UMVz+dKnE3X01XSd8=
103103
github.com/lyft/gostats v0.4.14 h1:xmP4yMfDvEKtlNZEcS2sYz0cvnps1ri337ZEEbw3ab8=
104104
github.com/lyft/gostats v0.4.14/go.mod h1:cJWqEVL8JIewIJz/olUIios2F1q06Nc51hXejPQmBH0=
105-
github.com/mediocregopher/radix/v3 v3.8.1 h1:rOkHflVuulFKlwsLY01/M2cM2tWCjDoETcMqKbAWu1M=
106-
github.com/mediocregopher/radix/v3 v3.8.1/go.mod h1:8FL3F6UQRXHXIBSPUs5h0RybMF8i4n7wVopoX3x7Bv8=
105+
github.com/mediocregopher/radix/v4 v4.1.4 h1:Uze6DEbEAvL+VHXUEu/EDBTkUk5CLct5h3nVSGpc6Ts=
106+
github.com/mediocregopher/radix/v4 v4.1.4/go.mod h1:ajchozX/6ELmydxWeWM6xCFHVpZ4+67LXHOTOVR0nCE=
107107
github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o=
108108
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
109109
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
@@ -144,6 +144,8 @@ github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO
144144
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
145145
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
146146
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
147+
github.com/tilinna/clock v1.0.2 h1:6BO2tyAC9JbPExKH/z9zl44FLu1lImh3nDNKA0kgrkI=
148+
github.com/tilinna/clock v1.0.2/go.mod h1:ZsP7BcY7sEEz7ktc0IVy8Us6boDrK8VradlKRUGfOao=
147149
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
148150
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
149151
github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
@@ -241,8 +243,6 @@ golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8T
241243
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
242244
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
243245
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
244-
golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 h1:+cNy6SZtPcJQH3LJVLOSmiC7MMxXNOb3PU/VUEz+EhU=
245-
golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028/go.mod h1:NDW/Ps6MPRej6fsCIbMTohpP40sJ/P/vI1MoTEGwX90=
246246
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
247247
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
248248
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=

‎src/redis/cache_impl.go‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,13 +19,13 @@ func NewRateLimiterCacheImplFromSettings(s settings.Settings, localCache *freeca
1919
if s.RedisPerSecond {
2020
perSecondPool = NewClientImpl(srv.Scope().Scope("redis_per_second_pool"), s.RedisPerSecondTls, s.RedisPerSecondAuth, s.RedisPerSecondSocketType,
2121
s.RedisPerSecondType, s.RedisPerSecondUrl, s.RedisPerSecondPoolSize, s.RedisPerSecondPipelineWindow, s.RedisPerSecondPipelineLimit, s.RedisTlsConfig, s.RedisHealthCheckActiveConnection, srv, s.RedisPerSecondTimeout,
22-
s.RedisPerSecondPoolOnEmptyBehavior, s.RedisPerSecondPoolOnEmptyWaitDuration, s.RedisPerSecondSentinelAuth)
22+
s.RedisPerSecondPoolOnEmptyBehavior, s.RedisPerSecondSentinelAuth)
2323
closer.Closers = append(closer.Closers, perSecondPool)
2424
}
2525

2626
otherPool := NewClientImpl(srv.Scope().Scope("redis_pool"), s.RedisTls, s.RedisAuth, s.RedisSocketType, s.RedisType, s.RedisUrl, s.RedisPoolSize,
2727
s.RedisPipelineWindow, s.RedisPipelineLimit, s.RedisTlsConfig, s.RedisHealthCheckActiveConnection, srv, s.RedisTimeout,
28-
s.RedisPoolOnEmptyBehavior, s.RedisPoolOnEmptyWaitDuration, s.RedisSentinelAuth)
28+
s.RedisPoolOnEmptyBehavior, s.RedisSentinelAuth)
2929
closer.Closers = append(closer.Closers, otherPool)
3030

3131
return NewFixedRateLimitCacheImpl(

‎src/redis/driver.go‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
package redis
22

3-
import "github.com/mediocregopher/radix/v3"
3+
import "github.com/mediocregopher/radix/v4"
44

55
// Errors that may be raised during config parsing.
66
type RedisError string
@@ -41,9 +41,13 @@ type Client interface {
4141

4242
// NumActiveConns return number of active connections, used in testing.
4343
NumActiveConns() int
44+
}
4445

45-
// ImplicitPipeliningEnabled return true if implicit pipelining is enabled.
46-
ImplicitPipeliningEnabled() bool
46+
// PipelineAction represents a single action in the pipeline along with its key.
47+
// The key is used for grouping commands in cluster mode.
48+
type PipelineAction struct {
49+
Action radix.Action
50+
Key string
4751
}
4852

49-
type Pipeline []radix.CmdAction
53+
type Pipeline []PipelineAction

0 commit comments

Comments
 (0)