解决K8S多实例AI回答消息丢失及错乱问题

1. 现象描述

发布到生产环境[K8S server端服务2个pod, web端1个pod]后我们询问AI，发现AI回答卡死，查看日志是因为多实例问题导致SseEmitter异常:No emitter found for client

2. 分布式锁解决无法回复问题

主要通过Redis主题订阅消息监听机制实现

@Component
@Slf4j
public class SseServer {

    private final Map<String, SseEmitter> EMITTERS = new ConcurrentHashMap<>();
    private final RedissonClient redissonClient;
    private RTopic topic;

    @Autowired
    public SseServer(RedissonClient redissonClient) {
        this.redissonClient = redissonClient;
    }

    @PostConstruct
    public void init() {
        topic = redissonClient.getTopic("ai-price-sse-channel");

        // 订阅 Redis 频道，接收消息并推送给本地客户端
        topic.addListener(String.class, (channel, message) -> {
            EMITTERS.forEach((account, emitter) -> {
                try {
                    if (message.startsWith(account + "@:@")) {
                        emitter.send(message.replaceFirst(account + "@:@", ""));
                    }
                } catch (IOException e) {
                    emitter.completeWithError(e);
                    EMITTERS.remove(account);
                }
            });
        });
    }

    public SseEmitter subscribe(String token) {
        String account = TokenUtils.getClaimsFromToken(token).getAccount();
        SseEmitter emitter = new SseEmitter(0L);
        EMITTERS.put(account, emitter);

        emitter.onCompletion(() -> {
            System.out.println("SSE connection completed for client: " + account);
            EMITTERS.remove(account);
        });

        try {
            emitter.send(SysConstants.SSE_START);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }


        emitter.onTimeout(() -> {
            System.out.println("SSE connection timed out for client: " + account);
            emitter.complete();
            EMITTERS.remove(account);
        });
        return emitter;
    }

//    public static void sendMsg(String account, Object message) {
//        SseEmitter emitter = EMITTERS.get(account);
//        if (emitter != null) {
//            try {
//                emitter.send(message);
//            } catch (IOException e) {
//                EMITTERS.remove(account);
//                emitter.completeWithError(e);
//            }
//        } else {
//            System.out.println("No emitter found for client: " + account);
//        }
//    }

    public void sendMsg(String account, Object message) {
        try {
            // 发布消息到 Redis 频道，格式可以自定义，例如 "account:message"
            topic.publish(account + "@:@" + message.toString());
        } catch (Exception e) {
            log.error("Failed to publish message to Redis: {}", e.getMessage());
        }
    }

    public String closeConnection() {
        String account = TokenUtils.getClaimsFromToken().getAccount();
        SseEmitter emitter = EMITTERS.remove(account);
        if (emitter != null) {
            emitter.complete();
            return "Connection closed for client: " + account;
        }
        return "No active connection found for client: " + account;
    }
}

3. K8S会话粘滞机制

通过上述，多实例确实解决了无法回复问题，但是发现会出现AI重复回答问题，原因是什么同一客户端发送到不同的后端服务实例导致，我们可以通过配置K8S来解决

---
apiVersion: v1
kind: Service
metadata:
  name: fangjiatong-server
  namespace: ai
spec:
  type: NodePort
  # K8S会话粘滞，解决豆包乱回答问题
  sessionAffinity: ClientIP

上述是配置发布的K8S service中，下面的配置在同一集群的命名空间中，假设您的web与server端会与同一K8S集群

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: fangjiatong-ingress
  namespace: ai
  annotations:
    nginx.ingress.kubernetes.io/affinity: "cookie"
    nginx.ingress.kubernetes.io/session-cookie-name: "SESSIONID"
    nginx.ingress.kubernetes.io/session-cookie-expires: "3600"
spec:
  rules:
    - host: fangjiatong-server.ai
      http:
        paths:
          - path: /api
            pathType: Prefix
            backend:
              service:
                name: fangjiatong-server
                port:
                  number: 8080

OK,已解决，欢迎关注算法小生公众号~