consul envoy upstream connect error or disconnect

最新推荐文章于 2025-05-31 16:51:52 发布

nathan柠檬

最新推荐文章于 2025-05-31 16:51:52 发布

阅读量1k

点赞数

分类专栏：错误解决文章标签： consul envoy

本文链接：https://blog.youkuaiyun.com/qq_34884729/article/details/131394258

版权

错误解决专栏收录该内容

72 篇文章

订阅专栏

upstream connect error or disconnect/reset before headers. retried and the latest reset reason: connection failure, transport failure reason: TLS error: 268435581:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED

consul和envoy一起构建成的服务网格，上述的问题估计会经常发现，也头疼了我很久，还是总结列一下。

首先，应该从哪里开始检查？

1、看内容，就是tls的校验问题，根本不关你上游的服务什么事情。

2、要知道envoy之间，tls的控制来源是啥？consul！！！！！consul通过xds来动态配置envoy。所以，envoy出问题，很大程度是来源consul。不要单单看envoy的日志，其实感觉看envoy的日志，对于这种问题，感觉没啥卵用，直接去查一下，相关的envoy节点所关联的consul节点日志。

2023-06-26T11:25:35.211+0800 [ERROR] agent.client: RPC failed to server: method=ConnectCA.Sign server=10.70.51.58:47024 error="rpc error making call: rpc error making call: CA is uninitialized and unable to sign certificates yet: no root certificate"
2023-06-26T11:25:35.211+0800 [WARN]  agent.cache: handling error in Cache.Notify: cache-type=connect-ca-leaf error="rpc error making call: rpc error making call: CA is uninitialized and unable to sign certificates yet: no root certificate" index=0
2023-06-26T11:25:35.211+0800 [ERROR] agent.proxycfg: Failed to handle update from watch: kind=ingress-gateway proxy=ingress-service-eai service_id=ingress-service-eai id=leaf error="error filling agent cache: rpc error making call: rpc error making call: CA is uninitialized and unable to sign certificates yet: no root certificate"
2023-06-26T11:26:38.201+0800 [ERROR] agent.client: RPC failed to server: method=ConnectCA.Sign server=10.70.51.58:47004 error="rpc error making call: rpc error making call: CA is uninitialized and unable to sign certificates yet: no root certificate"
2023-06-26T11:26:38.201+0800 [WARN]  agent.cache: handling error in Cache.Notify: cache-type=connect-ca-leaf error="rpc error making call: rpc error making call: CA is uninitialized and unable to sign certificates yet: no root certificate" index=0
2023-06-26T11:26:38.201+0800 [ERROR] agent.proxycfg: Failed to handle update from watch: kind=ingress-gateway proxy=ingress-service-eai service_id=ingress-service-eai id=leaf error="error filling agent cache: rpc error making call: rpc error making call: CA is uninitialized and unable to sign certificates yet: no root certificate"

既然引期故障的日志找到了，就开始处理问题。

其实这个问题是因为consul connect 我采用的是vault，刚好昨天vault服务所在的服务器重启了，vault又没有配置自启，为啥又到了今天才报异常？因为刚好要更换leaf cert。consul感知到vault服务的异常，就所有的服务都gg了。

vault 这个蛋疼的服务，非必要，不要搞，老老实实用consul自带的cert。consul和envoy在国内的文档除了非常简单的，稍微深入一点的都没有，vault比它们更偏门，Google都不好使。

立刻把vault删了，把ca的配置改成用consul的，重启consul，完事