Redis官方文档
Redis 使用单个 Lua 解释器去运行所有脚本,并且, Redis 也保证脚本会以原子性(atomic)的方式执行:当某个脚本正在运行的时候,不会有其他脚本或 Redis 命令被执行。因此,可以使用 lua 脚本进行限流操作,由于脚本操作是原子性的,分布式系统同样适用。
Redis限流实践
定义 Redis 配置,添加lua脚本支持。
import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.context.annotation.Scope;
import org.springframework.core.io.ClassPathResource;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.core.script.DefaultRedisScript;
import org.springframework.data.redis.serializer.Jackson2JsonRedisSerializer;
import org.springframework.scripting.support.ResourceScriptSource;
import org.springframework.stereotype.Component;
import redis.clients.jedis.JedisPoolConfig;
import java.util.*;
@Slf4j
@Configuration
@EnableAutoConfiguration
@Component
public class PubRedisConfig {
@Bean
public JedisPoolConfig getRedisConfig(){
JedisPoolConfig config = new JedisPoolConfig();
return config;
}
@Bean("pubRedisTemplate")
public RedisTemplate<Object, Object> redisTemplate(RedisConnectionFactory connectionFactory) {
RedisTemplate<Object, Object> redisTemplate = new RedisTemplate<>();
redisTemplate.setConnectionFactory(connectionFactory);
Jackson2JsonRedisSerializer<Object> jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer<>(Object.class);
redisTemplate.setKeySerializer(jackson2JsonRedisSerializer);
redisTemplate.setValueSerializer(jackson2JsonRedisSerializer);
redisTemplate.setHashKeySerializer(jackson2JsonRedisSerializer);
redisTemplate.setHashValueSerializer(jackson2JsonRedisSerializer);
return redisTemplate;
}
@Bean
public DefaultRedisScript<Long> limitScript() {
DefaultRedisScript<Long> redisScript = new DefaultRedisScript<>();
redisScript.setScriptSource(new ResourceScriptSource(new ClassPathResource("lua/limit.lua")));
redisScript.setResultType(Long.class);
return redisScript;
}
}
IP工具类,用于获取http请求的真实IP, 支持 IP 限流策略。
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.util.StringUtils;
import javax.servlet.http.HttpServletRequest;
public class IpUtils {
private static Logger logger = LoggerFactory.getLogger(IpUtils.class);
public static String getIpAddr(HttpServletRequest request) {
String ip = null;
//X-Forwarded-For:Squid 服务代理
String ipAddresses = request.getHeader("X-Forwarded-For");
if (ipAddresses == null || ipAddresses.length() == 0 || "unknown".equalsIgnoreCase(ipAddresses)) {
//Proxy-Client-IP:apache 服务代理
ipAddresses = request.getHeader("Proxy-Client-IP");
}
if (ipAddresses == null || ipAddresses.length() == 0 || "unknown".equalsIgnoreCase(ipAddresses)) {
//WL-Proxy-Client-IP:weblogic 服务代理
ipAddresses = request.getHeader("WL-Proxy-Client-IP");
}
if (ipAddresses == null || ipAddresses.length() == 0 || "unknown".equalsIgnoreCase(ipAddresses)) {
//HTTP_CLIENT_IP:某些代理服务器
ipAddresses = request.getHeader("HTTP_CLIENT_IP");
}
if (ipAddresses == null || ipAddresses.length() == 0 || "unknown".equalsIgnoreCase(ipAddresses)) {
//X-Real-IP:nginx服务代理
ipAddresses = request.getHeader("X-Real-IP");
}
//有些网络通过多层代理,那么获取到的ip就会有多个,一般都是通过逗号(,)分割开来,并且第一个ip为客户端的真实IP
if (ipAddresses != null && ipAddresses.length() != 0) {
ip = ipAddresses.split(",")[0];
}
//都没有获取到,通过request.getRemoteAddr()获取
if (ip == null || ip.length() == 0 || "unknown".equalsIgnoreCase(ipAddresses)) {
ip = request.getRemoteAddr();
}
return ip;
}
}
限流策略,目前提供全局策略以及IP策略,当然也可以根据头部信息增加其它策略, 比如鉴权使用的用户名。 默认为全局限流,即按接口名进行限流。
public enum LimitType {
/* 默认策略全局限流 */
DEFAULT,
/* 根据请求IP进行限流 */
IP
}
自定义注解, 在方法层添加后对该方法进行限流。
例如:3 秒内允许以同一 IP 访问 test 接口 2 次
@RateLimiter(key = "test", time = 3, count = 2)
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface RateLimiter {
/**
* 限流key,redis 缓存前缀
*/
String key() default "rate_limit:";
/**
* 限流时间,单位秒
*/
int time() default 60;
/**
* 限流次数
*/
int count() default 100;
/**
* 限流类型
*/
LimitType limitType() default LimitType.DEFAULT;
}
限流切面,在这里做出具体的限流操作, 在 getCombineKey(RateLimiter rateLimiter, JoinPoint point) 方法中 对不同策略生成不同类型的 key 以达到区分限流策略的目的。
@Aspect
@Component
public class RateLimiterAspect {
private static final Logger log = LoggerFactory.getLogger(RateLimiterAspect.class);
@Resource(name = "pubRedisTemplate")
private RedisTemplate<Object, Object> redisTemplate;
@Autowired
private RedisScript<Long> limitScript;
@Before("@annotation(rateLimiter)")
public void doBefore(JoinPoint point, RateLimiter rateLimiter) throws Throwable {
String key = rateLimiter.key();
int time = rateLimiter.time();
int count = rateLimiter.count();
String combineKey = getCombineKey(rateLimiter, point);
List<Object> keys = Collections.singletonList(combineKey);
try {
Long number = redisTemplate.execute(limitScript, keys, count, time);
if (number==null || number.intValue() > count) {
throw new RateLimiterException("访问过于频繁,请稍候再试");
}
log.info("限制请求次数'{}',当前请求次数'{}',缓存key'{}'", count, number.intValue(), key);
} catch (RateLimiterException e) {
throw e;
} catch (Exception e) {
e.printStackTrace();
throw new RuntimeException("服务器异常,请稍候再试");
}
}
public String getCombineKey(RateLimiter rateLimiter, JoinPoint point) {
StringBuffer stringBuffer = new StringBuffer(rateLimiter.key());
if (rateLimiter.limitType() == LimitType.IP) {
stringBuffer.append(IpUtils.getIpAddr(((ServletRequestAttributes) RequestContextHolder.currentRequestAttributes()).getRequest())).append("-");
}
MethodSignature signature = (MethodSignature) point.getSignature();
Method method = signature.getMethod();
Class<?> targetClass = method.getDeclaringClass();
stringBuffer.append(targetClass.getName()).append("-").append(method.getName());
return stringBuffer.toString();
}
}
自定义限流异常类,在发生异常时,抛出异常信息。便于接下来对限流异常的统一处理。
public class RateLimiterException extends Exception {
public RateLimiterException() {
}
public RateLimiterException(String message) {
super(message);
}
public RateLimiterException(String message, Throwable cause) {
super(message, cause);
}
public RateLimiterException(Throwable cause) {
super(cause);
}
public RateLimiterException(String message, Throwable cause, boolean enableSuppression, boolean writableStackTrace) {
super(message, cause, enableSuppression, writableStackTrace);
}
}
定义全局异常通知类, 捕获自定义限流异常, 统一返回格式。
@RestControllerAdvice
public class GlobalException {
@ExceptionHandler(RateLimiterException.class)
public ResultVo serviceException(RateLimiterException e) {
return new ResultVo().message(500, e.getMessage());
}
}
@Getter
public class ResultVo {
public Integer status = 200;
public String desc = "请求成功";
public ResultVo success(){
this.status = 200;
this.desc = "请求成功";
return this;
}
public ResultVo fail(){
this.status = 500;
this.desc = "请求失败";
return this;
}
public ResultVo message(String msg){
this.status = 200;
this.desc = msg;
return this;
}
public ResultVo message(Integer status, String msg){
this.status = status;
this.desc = msg;
return this;
}
}
编写 lua 脚本, lua/limit.lua ,实现限流次数的原子性操作。具体语法可以参考 Lua文档
local key = KEYS[1]
local count = tonumber(ARGV[1])
local time = tonumber(ARGV[2])
local current = redis.call('get', key)
if current and tonumber(current) > count then
return tonumber(current)
end
current = redis.call('incr', key)
if tonumber(current) == 1 then
redis.call('expire', key, time)
end
return tonumber(current)
测试
假设要对test 接口进行限制, 同一 IP 地址允许在 5 秒内访问接口 3 次。
@RateLimiter(key = "test", time = 5, count = 3, limitType = LimitType.IP)
@RequestMapping(value = "test", method = RequestMethod.POST)
@ResponseBody
public String test(){
return "success";
}
注意
1、如果是用代理服务器的话, 这里以 Nginx 为例, 需要设置请求的头部信息,否则后端拿到的会是代理服务器的ip, 那么IP 限流策略就失效了。
比如我要将 34821 端口过来的请求 全部转到 http://localhost:33123 。 在设置 proxy_pass 的同时, 还要设置 proxy_set_header X-Real-IP 、proxy_set_header Host 等属性。
server {
listen 34821;
server_name localhost;
location / {
proxy_pass http://localhost:33123;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_redirect off;
}
}