[Proposal] Share Threshold for Wildcard Rate Limiting (#1016)

* Add share_threshold to make wild card values can share rate limit threshold

Signed-off-by: Nam Dang <xuannam230201@gmail.com>

* Implement lazy initilization based on reviews

Signed-off-by: Nam Dang <xuannam230201@gmail.com>

---------

Signed-off-by: Nam Dang <xuannam230201@gmail.com>

Author: xuannam230201

README.md

@@ -22,6 +22,7 @@
     - [ShadowMode](#shadowmode)
     - [Including detailed metrics for unspecified values](#including-detailed-metrics-for-unspecified-values)
     - [Including descriptor values in metrics](#including-descriptor-values-in-metrics)
+    - [Sharing thresholds for wildcard matches](#sharing-thresholds-for-wildcard-matches)
     - [Examples](#examples)
       - [Example 1](#example-1)
       - [Example 2](#example-2)
@@ -33,6 +34,7 @@
       - [Example 8](#example-8)
       - [Example 9](#example-9)
       - [Example 10](#example-10)
+      - [Example 11](#example-11)
   - [Loading Configuration](#loading-configuration)
     - [File Based Configuration Loading](#file-based-configuration-loading)
     - [xDS Management Server Based Configuration Loading](#xds-management-server-based-configuration-loading)
@@ -285,6 +287,7 @@ descriptors:
     shadow_mode: (optional)
     detailed_metric: (optional)
     value_to_metric: (optional)
+    share_threshold: (optional)
     descriptors: (optional block)
       - ... (nested repetition of above)
 ```
@@ -347,6 +350,20 @@ Setting `value_to_metric: true` (default: `false`) for a descriptor will include
 
 When combined with wildcard matching, the full runtime value is included in the metric key, not just the wildcard prefix. This feature works independently of `detailed_metric` - when `detailed_metric` is set, it takes precedence and `value_to_metric` is ignored.
 
+### Sharing thresholds for wildcard matches
+
+Setting `share_threshold: true` (default: `false`) for a descriptor with a wildcard value (ending with `*`) allows all values matching that wildcard to share the same rate limit threshold, instead of using isolated thresholds for each matching value.
+
+This is useful when you want to apply a single rate limit across multiple resources that match a wildcard pattern. For example, if you have a rule for `files/*`, both `files/a.pdf` and `files/b.csv` will share the same threshold when `share_threshold: true` is set.
+
+**Important notes:**
+
+- `share_threshold` can only be used with wildcard values (values ending with `*`)
+- When `share_threshold: true` is enabled, all matching values share the same cache key and rate limit counter
+- When `share_threshold: false` (or not set), each matching value has its own isolated threshold
+- When combined with `value_to_metric: true`, the metric key includes the wildcard prefix (the part before `*`) instead of the full runtime value, to reflect that values are sharing a threshold
+- When combined with `detailed_metric: true`, the metric key also includes the wildcard prefix for entries with `share_threshold` enabled
+
 ### Examples
 
 #### Example 1
@@ -692,6 +709,58 @@ descriptors:
 
 Note: When `detailed_metric: true` is set on a descriptor, it takes precedence and `value_to_metric` is ignored for that descriptor.
 
+#### Example 11
+
+Using `share_threshold: true` to share rate limits across wildcard matches:
+
+```yaml
+domain: example11
+descriptors:
+  # With share_threshold: true, all files/* matches share the same threshold
+  - key: files
+    value: files/*
+    share_threshold: true
+    rate_limit:
+      unit: hour
+      requests_per_unit: 10
+
+  # Without share_threshold, each files_no_share/* match has its own isolated threshold
+  - key: files_no_share
+    value: files_no_share/*
+    share_threshold: false
+    rate_limit:
+      unit: hour
+      requests_per_unit: 10
+```
+
+With this configuration:
+
+- Requests for `files/a.pdf`, `files/b.csv`, and `files/c.txt` all share the same threshold of 10 requests per hour
+- If 5 requests are made for `files/a.pdf` and 5 requests for `files/b.csv`, a request for `files/c.txt` will be rate limited (OVER_LIMIT) because the shared threshold of 10 has been reached
+- Requests for `files_no_share/a.pdf` and `files_no_share/b.csv` each have their own isolated threshold of 10 requests per hour
+- If 10 requests are made for `files_no_share/a.pdf` (exhausting its quota), requests for `files_no_share/b.csv` will still be allowed (up to 10 requests)
+
+Combining `share_threshold` with `value_to_metric`:
+
+```yaml
+domain: example11_metrics
+descriptors:
+  - key: route
+    value: api/*
+    share_threshold: true
+    value_to_metric: true
+    descriptors:
+      - key: method
+        rate_limit:
+          unit: minute
+          requests_per_unit: 60
+```
+
+- Request: `route=api/v1`, `method=GET`
+- Metric key: `example11_metrics.route_api.method_GET` (includes the wildcard prefix `api` instead of the full value `api/v1`)
+
+This reflects that all `api/*` routes share the same threshold, while still providing visibility into which API routes are being accessed.
+
 ## Loading Configuration
 
 Rate limit service supports following configuration loading methods. You can define which methods to use by configuring environment variable `CONFIG_TYPE`.

src/config/config.go

@@ -25,6 +25,9 @@ type RateLimit struct {
 	Name           string
 	Replaces       []string
 	DetailedMetric bool
+	// ShareThresholdKeyPattern is a slice of wildcard patterns for descriptor entries
+	// The slice index corresponds to the descriptor entry index.
+	ShareThresholdKeyPattern []string
 }
 
 // Interface for interacting with a loaded rate limit config.

src/config/config_impl.go

@@ -33,6 +33,7 @@ type YamlDescriptor struct {
 	ShadowMode     bool `yaml:"shadow_mode"`
 	DetailedMetric bool `yaml:"detailed_metric"`
 	ValueToMetric  bool `yaml:"value_to_metric"`
+	ShareThreshold bool `yaml:"share_threshold"`
 }
 
 type YamlRoot struct {
@@ -41,10 +42,12 @@ type YamlRoot struct {
 }
 
 type rateLimitDescriptor struct {
-	descriptors   map[string]*rateLimitDescriptor
-	limit         *RateLimit
-	wildcardKeys  []string
-	valueToMetric bool
+	descriptors     map[string]*rateLimitDescriptor
+	limit           *RateLimit
+	wildcardKeys    []string
+	valueToMetric   bool
+	shareThreshold  bool
+	wildcardPattern string // stores the wildcard pattern when share_threshold is true
 }
 
 type rateLimitDomain struct {
@@ -71,6 +74,7 @@ var validKeys = map[string]bool{
 	"replaces":          true,
 	"detailed_metric":   true,
 	"value_to_metric":   true,
+	"share_threshold":   true,
 }
 
 // Create a new rate limit config entry.
@@ -90,11 +94,12 @@ func NewRateLimit(requestsPerUnit uint32, unit pb.RateLimitResponse_RateLimit_Un
 			Unit:            unit,
 			Name:            name,
 		},
-		Unlimited:      unlimited,
-		ShadowMode:     shadowMode,
-		Name:           name,
-		Replaces:       replaces,
-		DetailedMetric: detailedMetric,
+		Unlimited:                unlimited,
+		ShadowMode:               shadowMode,
+		Name:                     name,
+		Replaces:                 replaces,
+		DetailedMetric:           detailedMetric,
+		ShareThresholdKeyPattern: nil,
 	}
 }
 
@@ -186,16 +191,38 @@ func (this *rateLimitDescriptor) loadDescriptors(config RateLimitConfigToLoad, p
 			}
 		}
 
-		logger.Debugf(
-			"loading descriptor: key=%s%s", newParentKey, rateLimitDebugString)
-		newDescriptor := &rateLimitDescriptor{map[string]*rateLimitDescriptor{}, rateLimit, nil, descriptorConfig.ValueToMetric}
-		newDescriptor.loadDescriptors(config, newParentKey+".", descriptorConfig.Descriptors, statsManager)
-		this.descriptors[finalKey] = newDescriptor
+		// Validate share_threshold can only be used with wildcards
+		if descriptorConfig.ShareThreshold {
+			if len(finalKey) == 0 || finalKey[len(finalKey)-1:] != "*" {
+				panic(newRateLimitConfigError(
+					config.Name,
+					fmt.Sprintf("share_threshold can only be used with wildcard values (ending with '*'), but found key '%s'", finalKey)))
+			}
+		}
+
+		// Store wildcard pattern if share_threshold is enabled
+		var wildcardPattern string = ""
+		if descriptorConfig.ShareThreshold && len(finalKey) > 0 && finalKey[len(finalKey)-1:] == "*" {
+			wildcardPattern = finalKey
+		}
 
 		// Preload keys ending with "*" symbol.
 		if finalKey[len(finalKey)-1:] == "*" {
 			this.wildcardKeys = append(this.wildcardKeys, finalKey)
 		}
+
+		logger.Debugf(
+			"loading descriptor: key=%s%s", newParentKey, rateLimitDebugString)
+		newDescriptor := &rateLimitDescriptor{
+			descriptors:     map[string]*rateLimitDescriptor{},
+			limit:           rateLimit,
+			wildcardKeys:    nil,
+			valueToMetric:   descriptorConfig.ValueToMetric,
+			shareThreshold:  descriptorConfig.ShareThreshold,
+			wildcardPattern: wildcardPattern,
+		}
+		newDescriptor.loadDescriptors(config, newParentKey+".", descriptorConfig.Descriptors, statsManager)
+		this.descriptors[finalKey] = newDescriptor
 	}
 }
 
@@ -265,7 +292,14 @@ func (this *rateLimitConfigImpl) loadConfig(config RateLimitConfigToLoad) {
 	}
 
 	logger.Debugf("loading domain: %s", root.Domain)
-	newDomain := &rateLimitDomain{rateLimitDescriptor{map[string]*rateLimitDescriptor{}, nil, nil, false}}
+	newDomain := &rateLimitDomain{rateLimitDescriptor{
+		descriptors:     map[string]*rateLimitDescriptor{},
+		limit:           nil,
+		wildcardKeys:    nil,
+		valueToMetric:   false,
+		shareThreshold:  false,
+		wildcardPattern: "",
+	}}
 	newDomain.loadDescriptors(config, root.Domain+".", root.Descriptors, this.statsManager)
 	this.domains[root.Domain] = newDomain
 }
@@ -320,6 +354,10 @@ func (this *rateLimitConfigImpl) GetLimit(
 	var valueToMetricFullKey strings.Builder
 	valueToMetricFullKey.WriteString(domain)
 
+	// Track share_threshold patterns for entries matched via wildcard (using indexes)
+	// This allows share_threshold to work when wildcard has nested descriptors
+	var shareThresholdPatterns map[int]string
+
 	for i, entry := range descriptor.Entries {
 		// First see if key_value is in the map. If that isn't in the map we look for just key
 		// to check for a default value.
@@ -350,11 +388,31 @@ func (this *rateLimitConfigImpl) GetLimit(
 			matchedUsingValue = false
 		}
 
+		// Track share_threshold pattern when matching via wildcard, even if no rate_limit at this level
+		if matchedViaWildcard && nextDescriptor != nil && nextDescriptor.shareThreshold && nextDescriptor.wildcardPattern != "" {
+			// Extract the value part from the wildcard pattern (e.g., "key_files*" -> "files*")
+			if shareThresholdPatterns == nil {
+				shareThresholdPatterns = make(map[int]string)
+			}
+
+			wildcardValue := strings.TrimPrefix(nextDescriptor.wildcardPattern, entry.Key+"_")
+			shareThresholdPatterns[i] = wildcardValue
+			logger.Debugf("tracking share_threshold for entry index %d (key %s), wildcard pattern %s", i, entry.Key, wildcardValue)
+		}
+
 		// Build value_to_metric metrics path for this level
 		valueToMetricFullKey.WriteString(".")
 		if nextDescriptor != nil {
+			// Check if share_threshold is enabled for this entry
+			hasShareThreshold := shareThresholdPatterns[i] != ""
 			if matchedViaWildcard {
-				if nextDescriptor.valueToMetric {
+				// When share_threshold is enabled AND value_to_metric is enabled, use the prefix of the wildcard pattern
+				if hasShareThreshold && nextDescriptor.valueToMetric {
+					wildcardPrefix := strings.TrimSuffix(shareThresholdPatterns[i], "*")
+					valueToMetricFullKey.WriteString(entry.Key)
+					valueToMetricFullKey.WriteString("_")
+					valueToMetricFullKey.WriteString(wildcardPrefix)
+				} else if nextDescriptor.valueToMetric {
 					valueToMetricFullKey.WriteString(entry.Key)
 					if entry.Value != "" {
 						valueToMetricFullKey.WriteString("_")
@@ -365,14 +423,28 @@ func (this *rateLimitConfigImpl) GetLimit(
 				}
 			} else if matchedUsingValue {
 				// Matched explicit key+value in config
-				valueToMetricFullKey.WriteString(entry.Key)
-				if entry.Value != "" {
+				// When share_threshold is enabled AND value_to_metric is enabled, use the prefix of the wildcard pattern
+				if hasShareThreshold && nextDescriptor.valueToMetric {
+					wildcardPrefix := strings.TrimSuffix(shareThresholdPatterns[i], "*")
+					valueToMetricFullKey.WriteString(entry.Key)
 					valueToMetricFullKey.WriteString("_")
-					valueToMetricFullKey.WriteString(entry.Value)
+					valueToMetricFullKey.WriteString(wildcardPrefix)
+				} else {
+					valueToMetricFullKey.WriteString(entry.Key)
+					if entry.Value != "" {
+						valueToMetricFullKey.WriteString("_")
+						valueToMetricFullKey.WriteString(entry.Value)
+					}
 				}
 			} else {
 				// Matched default key (no value) in config
-				if nextDescriptor.valueToMetric {
+				// When share_threshold is enabled AND value_to_metric is enabled, use the prefix of the wildcard pattern
+				if hasShareThreshold && nextDescriptor.valueToMetric {
+					wildcardPrefix := strings.TrimSuffix(shareThresholdPatterns[i], "*")
+					valueToMetricFullKey.WriteString(entry.Key)
+					valueToMetricFullKey.WriteString("_")
+					valueToMetricFullKey.WriteString(wildcardPrefix)
+				} else if nextDescriptor.valueToMetric {
 					valueToMetricFullKey.WriteString(entry.Key)
 					if entry.Value != "" {
 						valueToMetricFullKey.WriteString("_")
@@ -391,7 +463,31 @@ func (this *rateLimitConfigImpl) GetLimit(
 			logger.Debugf("found rate limit: %s", finalKey)
 
 			if i == len(descriptor.Entries)-1 {
-				rateLimit = nextDescriptor.limit
+				// Create a copy of the rate limit to avoid modifying the shared object
+				originalLimit := nextDescriptor.limit
+				rateLimit = &RateLimit{
+					FullKey:        originalLimit.FullKey,
+					Stats:          originalLimit.Stats,
+					Limit:          originalLimit.Limit,
+					Unlimited:      originalLimit.Unlimited,
+					ShadowMode:     originalLimit.ShadowMode,
+					Name:           originalLimit.Name,
+					Replaces:       originalLimit.Replaces,
+					DetailedMetric: originalLimit.DetailedMetric,
+					// Initialize ShareThresholdKeyPattern with correct length, empty strings for entries without share_threshold
+					ShareThresholdKeyPattern: nil,
+				}
+				// Apply all tracked share_threshold patterns when we find the rate_limit
+				// This works whether the rate_limit is at the wildcard level or deeper
+				// Only entries with share_threshold will have non-empty patterns
+				if len(shareThresholdPatterns) > 0 {
+					rateLimit.ShareThresholdKeyPattern = make([]string, len(descriptor.Entries))
+				}
+
+				for idx, pattern := range shareThresholdPatterns {
+					rateLimit.ShareThresholdKeyPattern[idx] = pattern
+					logger.Debugf("share_threshold enabled for entry index %d, using wildcard pattern %s", idx, pattern)
+				}
 			} else {
 				logger.Debugf("request depth does not match config depth, there are more entries in the request's descriptor")
 			}
@@ -402,7 +498,10 @@ func (this *rateLimitConfigImpl) GetLimit(
 			descriptorsMap = nextDescriptor.descriptors
 		} else {
 			if rateLimit != nil && rateLimit.DetailedMetric {
+				// Preserve ShareThresholdKeyPattern when recreating rate limit
+				originalShareThresholdKeyPattern := rateLimit.ShareThresholdKeyPattern
 				rateLimit = NewRateLimit(rateLimit.Limit.RequestsPerUnit, rateLimit.Limit.Unit, this.statsManager.NewStats(rateLimit.FullKey), rateLimit.Unlimited, rateLimit.ShadowMode, rateLimit.Name, rateLimit.Replaces, rateLimit.DetailedMetric)
+				rateLimit.ShareThresholdKeyPattern = originalShareThresholdKeyPattern
 			}
 
 			break
@@ -411,10 +510,39 @@ func (this *rateLimitConfigImpl) GetLimit(
 	}
 
 	// Replace metric with detailed metric, if leaf descriptor is detailed.
+	// When share_threshold is enabled, expose the prefix (before *) of the wildcard pattern
 	if rateLimit != nil && rateLimit.DetailedMetric {
-		detailedKey := detailedMetricFullKey.String()
-		rateLimit.Stats = this.statsManager.NewStats(detailedKey)
-		rateLimit.FullKey = detailedKey
+		// Check if any entry has share_threshold enabled
+		hasShareThreshold := rateLimit.ShareThresholdKeyPattern != nil && len(rateLimit.ShareThresholdKeyPattern) > 0
+		if hasShareThreshold {
+			// Build metric key with wildcard prefix for entries with share_threshold
+			var shareThresholdMetricKey strings.Builder
+			shareThresholdMetricKey.WriteString(domain)
+			for i, entry := range descriptor.Entries {
+				shareThresholdMetricKey.WriteString(".")
+				if i < len(rateLimit.ShareThresholdKeyPattern) && rateLimit.ShareThresholdKeyPattern[i] != "" {
+					// Use the prefix of the wildcard pattern (before *)
+					wildcardPrefix := strings.TrimSuffix(rateLimit.ShareThresholdKeyPattern[i], "*")
+					shareThresholdMetricKey.WriteString(entry.Key)
+					shareThresholdMetricKey.WriteString("_")
+					shareThresholdMetricKey.WriteString(wildcardPrefix)
+				} else {
+					// Include full key_value for entries without share_threshold
+					shareThresholdMetricKey.WriteString(entry.Key)
+					if entry.Value != "" {
+						shareThresholdMetricKey.WriteString("_")
+						shareThresholdMetricKey.WriteString(entry.Value)
+					}
+				}
+			}
+			shareThresholdKey := shareThresholdMetricKey.String()
+			rateLimit.FullKey = shareThresholdKey
+			rateLimit.Stats = this.statsManager.NewStats(shareThresholdKey)
+		} else {
+			detailedKey := detailedMetricFullKey.String()
+			rateLimit.FullKey = detailedKey
+			rateLimit.Stats = this.statsManager.NewStats(detailedKey)
+		}
 	}
 
 	// If not using detailed metric, but any value_to_metric path produced a different key,
@@ -423,7 +551,9 @@ func (this *rateLimitConfigImpl) GetLimit(
 		enhancedKey := valueToMetricFullKey.String()
 		if enhancedKey != rateLimit.FullKey {
 			// Recreate to ensure a clean stats struct, then set to enhanced stats
+			originalShareThresholdKeyPattern := rateLimit.ShareThresholdKeyPattern
 			rateLimit = NewRateLimit(rateLimit.Limit.RequestsPerUnit, rateLimit.Limit.Unit, this.statsManager.NewStats(rateLimit.FullKey), rateLimit.Unlimited, rateLimit.ShadowMode, rateLimit.Name, rateLimit.Replaces, rateLimit.DetailedMetric)
+			rateLimit.ShareThresholdKeyPattern = originalShareThresholdKeyPattern
 			rateLimit.Stats = this.statsManager.NewStats(enhancedKey)
 			rateLimit.FullKey = enhancedKey
 		}