Rate Limiter

Overview

A rate limiter serves as a fundamental protection mechanism for services. By intelligently controlling client request rates, it effectively prevents system overload and ensures stable service operation. The core principle involves setting access frequency thresholds, where excess requests beyond predefined limits are automatically intercepted.

The Sponge framework's adaptive rate-limiting solution innovatively implements a dynamic threshold adjustment mechanism based on system resources (particularly CPU utilization). This feature not only provides robust protection for web and gRPC services but also significantly simplifies rate-limiting configuration management in multi-server deployment scenarios. Addressing the reality of varying processing capacities across server nodes in a cluster, this real-time load-based adaptive algorithm completely eliminates the pain points of manual parameter tuning for individual servers in traditional solutions.

Enable Adaptive Rate Limiter

Configuration

In services created by sponge, the rate limiting component is disabled by default.

Set the enableLimit field in the YAML file under the configs directory:

app:
  enableLimit: true    # Whether to enable adaptive rate limiting, true: enable, false: disable

In Web Services Settings

For web services, if the default values do not meet the requirements, you can modify the threshold and system resource quota default values in the RateLimit middleware in internal/routers/routers.go. For example, the following is an example:

 // limit middleware
 if config.Get().App.EnableLimit {
    r.Use(middleware.RateLimit(
        //middleware.WithWindow(time.Second*5), // default 10s
        //middleware.WithBucket(200), // default 100
        //middleware.WithCPUThreshold(900), // default 800
    ))
 }

In gRPC Services Settings

For the gRPC service, if the default value does not meet the requirements, you can modify the threshold and the default value of system resource quota in the UnaryServerRateLimit interceptor in internal/server/grpc.go. For example, the following is an example:

 // limit interceptor
 if config.Get().App.EnableLimit {
    unaryServerInterceptors = append(unaryServerInterceptors, interceptor.UnaryServerRateLimit(
        //interceptor.WithWindow(time.Second*5), // default 10s
        //interceptor.WithBucket(200), // default 100
        //interceptor.WithCPUThreshold(900), // default 800
    ))
 }