搭建k8s监控问题排查-9093: connect: connection refused

最新推荐文章于 2025-04-17 17:44:51 发布
最新推荐文章于 2025-04-17 17:44:51 发布
阅读量1.1w 收藏 4
点赞数 1
CC 4.0 BY-SA版权
分类专栏： k8s 云原生 kubernetes 文章标签： k8s 云原生 监控程序 alertmanager
原文链接：https://github.com/prometheus-operator/kube-prometheus/issues/653
本文聚焦于搭建k8s集群监控时Alertmanager的问题处理。当pod出现启动错误CrashLoopBackOff，意味着正常启动后异常退出。通过describe查看发现活性探测失败、连接遭拒，查看日志和statefulset，发现alertmanager - main未正常准备，还给出参考内容链接。

摘要生成于 C知道 ，由 DeepSeek-R1 满血版支持， 前往体验 >

搭建k8s集群监控-Alertmanager问题处理

pod启动错误-CrashLoopBackOff

在这里插入图片描述
CrashLoopBackOff说明pod正常启动后有异常退出了

describe查看

在这里插入图片描述

Events:
  Type     Reason     Age                    From                   Message
  ----     ------     ----                   ----                   -------
  Normal   Scheduled  <unknown>              default-scheduler      Successfully assigned monitoring/alertmanager-main-0 to 192.168.6.11
  Normal   Pulled     23m                    kubelet, 192.168.6.11  Container image "quay.mirrors.ustc.edu.cn/prometheus/alertmanager:v0.21.0" already present on machine
  Normal   Created    23m                    kubelet, 192.168.6.11  Created container alertmanager
  Normal   Started    23m                    kubelet, 192.168.6.11  Started container alertmanager
  Normal   Pulled     23m                    kubelet, 192.168.6.11  Container image "quay.mirrors.ustc.edu.cn/prometheus-operator/prometheus-config-reloader:v0.47.0" already present on machine
  Normal   Created    23m                    kubelet, 192.168.6.11  Created container config-reloader
  Normal   Started    23m                    kubelet, 192.168.6.11  Started container config-reloader
  Warning  Unhealthy  23m (x6 over 23m)      kubelet, 192.168.6.11  Liveness probe failed: Get http://172.17.25.5:9093/-/healthy: dial tcp 172.17.25.5:9093: connect: connection refused
  Warning  Unhealthy  8m53s (x148 over 23m)  kubelet, 192.168.6.11  Readiness probe failed: Get http://172.17.25.5:9093/-/ready: dial tcp 172.17.25.5:9093: connect: connection refused
  Warning  BackOff    3m51s (x34 over 12m)   kubelet, 192.168.6.11  Back-off restarting failed container

pod活性探测失败，无法连接，遭到拒绝

查看日志

在这里插入图片描述

[root@k8s-node1 ~]#  kubectl logs pod/alertmanager-main-0 alertmanager -n monitoring
level=info ts=2021-06-02T02:11:49.274Z caller=main.go:216 msg="Starting Alertmanager" version="(version=0.21.0, branch=HEAD, revision=4c6c03ebfe21009c546e4d1e9b92c371d67c021d)"
level=info ts=2021-06-02T02:11:49.274Z caller=main.go:217 build_context="(go=go1.14.4, user=root@dee35927357f, date=20200617-08:54:02)"
[root@k8s-node1 ~]# kubectl logs pod/alertmanager-main-0 config-reloader -n monitoring
level=info ts=2021-06-02T01:57:31.669430944Z caller=main.go:147 msg="Starting prometheus-config-reloader" version="(version=0.47.0, branch=refs/tags/pkg/client/v0.47.0, revision=539108b043e9ecc53c4e044083651e2ebfbd3492)"
level=info ts=2021-06-02T01:57:31.669531061Z caller=main.go:148 build_context="(go=go1.16.3, user=simonpasquier, date=20210413-15:46:43)"
level=info ts=2021-06-02T01:57:31.669664237Z caller=main.go:182 msg="Starting web server for metrics" listen=:8080
level=info ts=2021-06-02T01:57:31.67010267Z caller=reloader.go:214 msg="started watching config file and directories for changes" cfg= out= dirs=/etc/alertmanager/config
level=error ts=2021-06-02T01:57:32.81121586Z caller=runutil.go:101 msg="function failed. Retrying in next tick" err="trigger reload: reload request failed: Post \"http://localhost:9093/-/reload\": dial tcp 127.0.0.1:9093: connect: connection refused"
level=error ts=2021-06-02T01:57:37.811710125Z caller=runutil.go:101 msg="function failed. Retrying in next tick" err="trigger reload: reload request failed: Post \"http://localhost:9093/-/reload\": dial tcp 127.0.0.1:9093: connect: connection refused"
level=error ts=2021-06-02T01:57:42.811117367Z caller=runutil.go:101 msg="function failed. Retrying in next tick" err="trigger reload: reload request failed: Post \"http://localhost:9093/-/reload\": dial tcp 127.0.0.1:9093: connect: connection refused"
level=error ts=2021-06-02T01:57:47.810889541Z caller=runutil.go:101 msg="function failed. Retrying in next tick" err="trigger reload: reload request failed: Post \"http://localhost:9093/-/reload\": dial tcp 127.0.0.1:9093: connect: connection refused"

查看statefulset

在这里插入图片描述
发现alertmanager-main没有正常准备

导出statefulset

kubectl -n monitoring get statefulset.apps/alertmanager-main -o yaml > dump.yaml  
# spec.template.spec添加hostNetwork: true
# 删除原有的statefulset，重新创建
kubectl delete statefulsets.apps alertmanager-main -n monitoring
kubectl apply -f  dump.yaml

在这里插入图片描述

参考类容：
https://github.com/prometheus-operator/kube-prometheus/issues/653

Readiness probe failed: Get “http://x.x.x.x:80/“: dial tcp x.x.x.x:80: connect: connection refused
博客主要分享技术教程、工具教程、bug解决、前沿技术动态、编程经验、心得感悟等。 内容同步、干货获取、推广宣发请看侧栏推广信息
01-10 2211
每个人都有惰性，但不断学习是好好生活的根本，共勉！
k8s dial tcp 127.0.0.1:6443: connect: connection refused排查流程及解决思路
牛奔的博客
02-07 1382
前言 k8s 集群中，使用 kubelet 报错，如下： The connection to the server 127.0.0.1:6443 was refused - did you specify the right host or port? 排查思路 1. 检查环境是否正常 1.1 确认是否在 Master 节点排查 确保当前操作的机器是 Kubernetes 的 Master 节...
使用prometheus-operator部署prometheus服务,并对外提供访问
crontab_e的博客
04-17 554
说明一下这里默认拉取的是最新版本的代码版本.可以根据自己的k8s版本来获取对应的kube-prometheus代码版本。如果要使prometheus,grafana这些web服务可以使用浏览器访问的话,需要做一些配置修改。另外:说明一下有些镜像可能下载不下来,需要单独下载,并在相关deploy文件中做修改。代码拉取完成后会有一个kube-prometheus目录以下是相关的源码内容.这里我自己添加了个node-exporter,自己添加就行。看到一堆创建信息,表示配置文件没错,下面是部分兼容性示例图,
k8s集群-calico-报错-dial tcp *:10250: connect: connection refused
博然的宝藏库
05-23 2345
在 Kubernetes 集群中，确认节点已经重新启动并正常运行。错误，请确保与 Kubernetes API 的连接正常。确认节点与集群的网络连接正常。确认 Calico Pod 的状态为。
TCP/IP：连接服务器失败（错误原因：Connection refused）
热门推荐
test1280
06-10 29万+
TCP/IP：连接服务器失败（错误原因：Connection refused） Linux中，通过系统调用（system call） connect 连接指定服务器建立TCP连接。 connect 最常见的失败原因是 Connection refused。 假设服务器IP是192.168.44.148，且并 未有 进程监听端口是12500时： 若有TCP连接请求包到达192.16...
Fedora 31 k8s kubernetes kubeasz 防火墙 firewalld 导致 harbor pod 容器 实例 网络不通 connect: connection refused
hknaruto的专栏
12-29 1935
防火墙开启状态，harbor pod出错 Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning Unhealthy 19m (x319 over 7h28m) kubelet, 10.51.72.167
Prometheus健康检查(Readiness probe failed)失败问题解决
snowing
11-13 5139
Readiness probe failed: Get "http://10.244.40.157:3000/api/health": dial tcp 10.244.40.157:3000: connect: connection refused
K8s --The connection to the server 192.168.56.120:6443 was refused报错解决
xgysimida的博客
07-17 1615
自己本地虚拟机有一套使用kuebadm部署的3节点k8s集群，因为很长时间不使用了，今天登陆后在使用kubectl命令时一直报。既然已经定位到是证书过期的问题，那接下来就针对证书过期进行解决。再次使用kubectl命令，看看问题是否解决。错误，接下来就将解决的整个过程分享给大家。操作系统信息以及k8s集群版本信息请看下面。至此，问题得到解决。
k8s-问题：[root@master log]# kubectl get node The connection to the server 192.168.0.105:6443 was refus
Name_kongkong的博客
08-07 2136
6443: connect: connection refused
root@k8snode01-49 ~]# sudo kubeadm init --config=kubeadm-config.yaml [init] Using Kubernetes version: v1.25.0 [preflight] Running pre-flight checks [WARNING Hostname]: hostname "node" could not be reached [WARNING Hostname]: hostname "node": lookup node on 192.168.58.2:53: no such host [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' error execution phase preflight: [preflight] Some fatal errors occurred: [ERROR ImagePull]: failed to pull image registry.k8s.io/kube-apiserver:v1.25.0: output: E0625 10:49:37.036283 7096 remote_image.go:171] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/kube-apiserver:v1.25.0\": failed to resolve reference \"registry.k8s.io/kube-apiserver:v1.25.0\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/kube-apiserver/manifests/v1.25.0\": dial tcp 74.125.199.82:443: connect: connection refused" image="registry.k8s.io/kube-apiserver:v1.25.0" time="2025-06-25T10:49:37+08:00" level=fatal msg="pulling image: rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/kube-apiserver:v1.25.0\": failed to resolve reference \"registry.k8s.io/kube-apiserver:v1.25.0\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/kube-apiserver/manifests/v1.25.0\": dial tcp 74.125.199.82:443: connect: connection refused" , error: exit status 1 [ERROR ImagePull]: failed to pull image registry.k8s.io/kube-controller-manager:v1.25.0: output: E0625 10:51:34.160244 7175 remote_image.go:171] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/kube-controller-manager:v1.25.0\": failed to resolve reference \"registry.k8s.io/kube-controller-manager:v1.25.0\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/kube-controller-manager/manifests/v1.25.0\": dial tcp 74.125.199.82:443: connect: connection refused" image="registry.k8s.io/kube-controller-manager:v1.25.0" time="2025-06-25T10:51:34+08:00" level=fatal msg="pulling image: rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/kube-controller-manager:v1.25.0\": failed to resolve reference \"registry.k8s.io/kube-controller-manager:v1.25.0\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/kube-controller-manager/manifests/v1.25.0\": dial tcp 74.125.199.82:443: connect: connection refused" , error: exit status 1 [ERROR ImagePull]: failed to pull image registry.k8s.io/kube-scheduler:v1.25.0: output: E0625 10:54:05.266959 7219 remote_image.go:171] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/kube-scheduler:v1.25.0\": failed to resolve reference \"registry.k8s.io/kube-scheduler:v1.25.0\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/kube-scheduler/manifests/v1.25.0\": dial tcp 74.125.199.82:443: connect: connection refused" image="registry.k8s.io/kube-scheduler:v1.25.0" time="2025-06-25T10:54:05+08:00" level=fatal msg="pulling image: rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/kube-scheduler:v1.25.0\": failed to resolve reference \"registry.k8s.io/kube-scheduler:v1.25.0\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/kube-scheduler/manifests/v1.25.0\": dial tcp 74.125.199.82:443: connect: connection refused" , error: exit status 1 [ERROR ImagePull]: failed to pull image registry.k8s.io/kube-proxy:v1.25.0: output: E0625 10:55:55.302433 7311 remote_image.go:171] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/kube-proxy:v1.25.0\": failed to resolve reference \"registry.k8s.io/kube-proxy:v1.25.0\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/kube-proxy/manifests/v1.25.0\": dial tcp 142.251.188.82:443: connect: connection refused" image="registry.k8s.io/kube-proxy:v1.25.0" time="2025-06-25T10:55:55+08:00" level=fatal msg="pulling image: rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/kube-proxy:v1.25.0\": failed to resolve reference \"registry.k8s.io/kube-proxy:v1.25.0\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/kube-proxy/manifests/v1.25.0\": dial tcp 142.251.188.82:443: connect: connection refused" , error: exit status 1 [ERROR ImagePull]: failed to pull image registry.k8s.io/pause:3.8: output: E0625 10:57:46.470991 7353 remote_image.go:171] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/pause:3.8\": failed to resolve reference \"registry.k8s.io/pause:3.8\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/pause/manifests/3.8\": dial tcp 142.251.188.82:443: connect: connection refused" image="registry.k8s.io/pause:3.8" time="2025-06-25T10:57:46+08:00" level=fatal msg="pulling image: rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/pause:3.8\": failed to resolve reference \"registry.k8s.io/pause:3.8\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/pause/manifests/3.8\": dial tcp 142.251.188.82:443: connect: connection refused" , error: exit status 1 [ERROR ImagePull]: failed to pull image registry.k8s.io/etcd:3.5.4-0: output: E0625 10:59:37.312592 7393 remote_image.go:171] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/etcd:3.5.4-0\": failed to resolve reference \"registry.k8s.io/etcd:3.5.4-0\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/etcd/manifests/3.5.4-0\": dial tcp 142.251.188.82:443: connect: connection refused" image="registry.k8s.io/etcd:3.5.4-0" time="2025-06-25T10:59:37+08:00" level=fatal msg="pulling image: rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/etcd:3.5.4-0\": failed to resolve reference \"registry.k8s.io/etcd:3.5.4-0\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/etcd/manifests/3.5.4-0\": dial tcp 142.251.188.82:443: connect: connection refused" , error: exit status 1 [ERROR ImagePull]: failed to pull image registry.k8s.io/coredns/coredns:v1.9.3: output: E0625 11:01:26.957404 7476 remote_image.go:171] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.9.3\": failed to resolve reference \"registry.k8s.io/coredns/coredns:v1.9.3\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/coredns/coredns/manifests/v1.9.3\": dial tcp 142.251.188.82:443: connect: connection refused" image="registry.k8s.io/coredns/coredns:v1.9.3" time="2025-06-25T11:01:26+08:00" level=fatal msg="pulling image: rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.9.3\": failed to resolve reference \"registry.k8s.io/coredns/coredns:v1.9.3\": failed to do request: Head \"https://us-west2-docker.pkg.dev/v2/k8s-artifacts-prod/images/coredns/coredns/manifests/v1.9.3\": dial tcp 142.251.188.82:443: connect: connection refused" , error: exit status 1 [preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...` To see the stack trace of this error execute with --v=5 or higher
06-26
执行kubeadminit注意：在kubeadm1.25版本中，coredns镜像的仓库路径已经改变（从k8s.gcr.io/coredns/coredns变为k8s.gcr.io/coredns/coredns），但实际拉取时，coredns的镜像在阿里云上可能还是放在google_...
; write=0.006 s, sync=0.001 s, total=0.024 s; sync files=2, longest=0.001 s, average=0.001 s; distance=0 kB, estimate=0 kB 2025-07-25 11:25:49.627 UTC [1] LOG: database system is ready to accept connections 2025-07-25 11:25:49.628 UTC [11305] FATAL: could not open file "global/pg_filenode.map": No such file or directory 2025-07-25 11:25:49.628 UTC [11306] FATAL: could not open file "global/pg_filenode.map": No such file or directory 2025-07-25 11:25:49.629 UTC [1] LOG: autovacuum launcher process (PID 11305) exited with exit code 1 2025-07-25 11:25:49.629 UTC [1] LOG: terminating any other active server processes 2025-07-25 11:25:49.630 UTC [1] LOG: background worker "logical replication launcher" (PID 11306) exited with exit code 1 2025-07-25 11:25:49.631 UTC [1] LOG: all server processes terminated; reinitializing 2025-07-25 11:25:49.680 UTC [11307] LOG: database system was interrupted; last known up at 2025-07-25 11:25:49 UTC 2025-07-25 11:25:49.753 UTC [11307] LOG: database system was not properly shut down; automatic recovery in progress 2025-07-25 11:25:49.759 UTC [11307] LOG: invalid record length at 0/1478CF0: wanted 24, got 0 2025-07-25 11:25:49.759 UTC [11318] FATAL: the database system is in recovery mode 2025-07-25 11:25:49.759 UTC [11307] LOG: redo is not required 2025-07-25 11:25:49.765 UTC [11308] LOG: checkpoint starting: end-of-recovery immediate wait 2025-07-25 11:25:49.774 UTC [1] LOG: received smart shutdown request 2025-07-25 11:25:49.789 UTC [11308] LOG: checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.008 s, sync=0.001 s, total=0.026 s; sync files=2, longest=0.001 s, average=0.001 s; distance=0 kB, estimate=0 kB 2025-07-25 11:25:49.793 UTC [11308] LOG: shutting down 2025-07-25 11:25:49.794 UTC [11308] LOG: checkpoint starting: shutdown immediate 2025-07-25 11:25:49.810 UTC [11308] LOG: checkpoint complete: wrote 0 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.002 s, sync=0.001 s, total=0.017 s; sync files=0, longest=0.000 s, average=0.000 s; distance=0 kB, estimate=0 kB 2025-07-25 11:25:49.826 UTC [1] LOG: database system is shut down [root@k8s-master1 harbor]# [root@k8s-master1 harbor]# kubectl logs harbor-core-75cd4f54b5-b5stj -n harbor --previous Appending internal tls trust CA to ca-bundle ... find: '/etc/harbor/ssl': No such file or directory Internal tls trust CA appending is Done. init global config instance failed. If you do not use this, just ignore it. open conf/app.conf: no such file or directory 2025-07-25T11:29:52Z [INFO] [/controller/artifact/annotation/parser.go:85]: the annotation parser to parser artifact annotation version v1alpha1 registered 2025-07-25T11:29:52Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.cncf.helm.config.v1+json registered 2025-07-25T11:29:52Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.cnab.manifest.v1 registered 2025-07-25T11:29:52Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.cnai.model.manifest.v1+json registered 2025-07-25T11:29:52Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.oci.image.index.v1+json registered 2025-07-25T11:29:52Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.docker.distribution.manifest.list.v2+json registered 2025-07-25T11:29:52Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.docker.distribution.manifest.v1+prettyjws registered 2025-07-25T11:29:52Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.oci.image.config.v1+json registered 2025-07-25T11:29:52Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.docker.container.image.v1+json registered 2025-07-25T11:29:52Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.goharbor.harbor.sbom.v1 registered 2025-07-25T11:29:52Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.wasm.config.v1+json registered 2025-07-25T11:29:52Z [INFO] [/pkg/reg/adapter/native/adapter.go:36]: the factory for adapter docker-registry registered 2025-07-25T11:29:52Z [INFO] [/pkg/reg/adapter/aliacr/adapter.go:40]: the factory for adapter ali-acr registered 2025-07-25T11:29:52Z [INFO] [/pkg/reg/adapter/awsecr/adapter.go:44]: the factory for adapter aws-ecr registered 2025-07-25T11:29:52Z [INFO] [/pkg/reg/adapter/azurecr/adapter.go:29]: Factory for adapter azure-acr registered 2025-07-25T11:29:52Z [INFO] [/pkg/reg/adapter/dockerhub/adapter.go:40]: Factory for adapter docker-hub registered 2025-07-25T11:29:52Z [INFO] [/pkg/reg/adapter/dtr/adapter.go:36]: the factory of dtr adapter was registered 2025-07-25T11:29:52Z [INFO] [/pkg/reg/adapter/githubcr/adapter.go:43]: the factory for adapter github-ghcr registered 2025-07-25T11:29:52Z [INFO] [/pkg/reg/adapter/gitlab/adapter.go:33]: the factory for adapter gitlab registered 2025-07-25T11:29:52Z [INFO] [/pkg/reg/adapter/googlegcr/adapter.go:37]: the factory for adapter google-gcr registered 2025-07-25T11:29:52Z [INFO] [/pkg/reg/adapter/huawei/huawei_adapter.go:40]: the factory of Huawei adapter was registered 2025-07-25T11:29:52Z [INFO] [/pkg/reg/adapter/jfrog/adapter.go:42]: the factory of jfrog artifactory adapter was registered 2025-07-25T11:29:52Z [INFO] [/pkg/reg/adapter/quay/adapter.go:53]: the factory of Quay adapter was registered 2025-07-25T11:29:52Z [INFO] [/pkg/reg/adapter/tencentcr/adapter.go:55]: the factory for adapter tencent-tcr registered 2025-07-25T11:29:52Z [INFO] [/pkg/reg/adapter/volcenginecr/adapter.go:40]: the factory for adapter volcengine-cr registered 2025-07-25T11:29:52Z [INFO] [/pkg/reg/adapter/harbor/adaper.go:31]: the factory for adapter harbor registered 2025-07-25T11:29:52Z [INFO] [/core/controllers/base.go:187]: Config path: /etc/core/app.conf 2025-07-25T11:29:52Z [INFO] [/core/main.go:148]: initializing cache ... 2025-07-25T11:29:52Z [INFO] [/core/main.go:167]: initializing configurations... 2025-07-25T11:29:52Z [INFO] [/lib/config/systemconfig.go:178]: key path: /etc/core/key 2025-07-25T11:29:52Z [INFO] [/lib/config/config.go:92]: init secret store 2025-07-25T11:29:52Z [INFO] [/core/main.go:169]: configurations initialization completed 2025-07-25T11:29:52Z [INFO] [/common/dao/base.go:67]: Registering database: type-PostgreSQL host-harbor-database port-5432 database-registry sslmode-"disable" 2025-07-25T11:29:52Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:29:54Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:29:56Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:29:58Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:00Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:02Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:04Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:06Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:08Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:10Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:12Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:14Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:16Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:18Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:20Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:22Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:24Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:26Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:28Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:30Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:32Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:34Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:36Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:38Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:40Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:42Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:44Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:46Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:48Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:50Z [ERROR] [/common/utils/utils.go:108]: failed to connect to tcp://harbor-database:5432, retry after 2 seconds :dial tcp 10.0.192.44:5432: connect: connection refused 2025-07-25T11:30:52Z [FATAL] [/core/main.go:190]: failed to initialize database: failed to connect to tcp:harbor-database:5432 after 60 seconds [root@k8s-master1 harbor]# kubectl logs harbor-jobservice-6457b57477-7qgt2 -n harbor --previous Appending internal tls trust CA to ca-bundle ... find: '/etc/harbor/ssl': No such file or directory Internal tls trust CA appending is Done. 2025-07-25T11:30:54Z [INFO] [/controller/artifact/annotation/parser.go:85]: the annotation parser to parser artifact annotation version v1alpha1 registered 2025-07-25T11:30:54Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.cncf.helm.config.v1+json registered 2025-07-25T11:30:54Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.cnab.manifest.v1 registered 2025-07-25T11:30:54Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.cnai.model.manifest.v1+json registered 2025-07-25T11:30:54Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.oci.image.index.v1+json registered 2025-07-25T11:30:54Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.docker.distribution.manifest.list.v2+json registered 2025-07-25T11:30:54Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.docker.distribution.manifest.v1+prettyjws registered 2025-07-25T11:30:54Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.oci.image.config.v1+json registered 2025-07-25T11:30:54Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.docker.container.image.v1+json registered 2025-07-25T11:30:54Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.goharbor.harbor.sbom.v1 registered 2025-07-25T11:30:54Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.wasm.config.v1+json registered 2025-07-25T11:30:54Z [INFO] [/pkg/reg/adapter/native/adapter.go:36]: the factory for adapter docker-registry registered 2025-07-25T11:30:54Z [INFO] [/pkg/reg/adapter/aliacr/adapter.go:40]: the factory for adapter ali-acr registered 2025-07-25T11:30:54Z [INFO] [/pkg/reg/adapter/awsecr/adapter.go:44]: the factory for adapter aws-ecr registered 2025-07-25T11:30:54Z [INFO] [/pkg/reg/adapter/azurecr/adapter.go:29]: Factory for adapter azure-acr registered 2025-07-25T11:30:54Z [INFO] [/pkg/reg/adapter/dockerhub/adapter.go:40]: Factory for adapter docker-hub registered 2025-07-25T11:30:54Z [INFO] [/pkg/reg/adapter/dtr/adapter.go:36]: the factory of dtr adapter was registered 2025-07-25T11:30:54Z [INFO] [/pkg/reg/adapter/githubcr/adapter.go:43]: the factory for adapter github-ghcr registered 2025-07-25T11:30:54Z [INFO] [/pkg/reg/adapter/gitlab/adapter.go:33]: the factory for adapter gitlab registered 2025-07-25T11:30:54Z [INFO] [/pkg/reg/adapter/googlegcr/adapter.go:37]: the factory for adapter google-gcr registered 2025-07-25T11:30:54Z [INFO] [/pkg/reg/adapter/huawei/huawei_adapter.go:40]: the factory of Huawei adapter was registered 2025-07-25T11:30:54Z [INFO] [/pkg/reg/adapter/jfrog/adapter.go:42]: the factory of jfrog artifactory adapter was registered 2025-07-25T11:30:54Z [INFO] [/pkg/reg/adapter/quay/adapter.go:53]: the factory of Quay adapter was registered 2025-07-25T11:30:54Z [INFO] [/pkg/reg/adapter/tencentcr/adapter.go:55]: the factory for adapter tencent-tcr registered 2025-07-25T11:30:54Z [INFO] [/pkg/reg/adapter/volcenginecr/adapter.go:40]: the factory for adapter volcengine-cr registered 2025-07-25T11:30:54Z [INFO] [/pkg/reg/adapter/harbor/adaper.go:31]: the factory for adapter harbor registered 2025-07-25T11:30:54Z [INFO] [/pkg/config/rest/rest.go:47]: get configuration from url: http://harbor-core:80/api/v2.0/internalconfig 2025-07-25T11:30:54Z [ERROR] [/pkg/config/rest/rest.go:50]: Failed on load rest config err:Get "http://harbor-core:80/api/v2.0/internalconfig": dial tcp 10.5.232.220:80: connect: connection refused, url:http://harbor-core:80/api/v2.0/internalconfig panic: failed to load configuration, error: failed to load rest config goroutine 1 [running]: main.main() /harbor/src/jobservice/main.go:46 +0x3ae [root@k8s-master1 harbor]# 查看日子输出这些，我应该如何解决呢，目前正在用kubernetes集群搭建harbor仓库，想让root@k8s-master1 harbor]# kubectl get pod -n harbor NAME READY STATUS RESTARTS AGE harbor-core-75cd4f54b5-b5stj 0/1 CrashLoopBackOff 11 (118s ago) 26m harbor-database-0 0/1 Running 4 (3m16s ago) 26m harbor-jobservice-6457b57477-7qgt2 0/1 CrashLoopBackOff 13 (3m23s ago) 26m harbor-portal-5b6b5f7494-gcc8n 1/1 Running 1 (10m ago) 26m harbor-redis-0 1/1 Running 1 (10m ago) 26m harbor-registry-5fb967b497-d4r4r 2/2 Running 2 (10m ago) 26m harbor-trivy-0 1/1 Running 1 (10m ago) 26m全是running
最新发布
07-26
日志中可能会显示具体的数据库连接错误，如“connection refused”、“authentication failed”等，这有助于进一步定位问题。 ### 步骤6: 持久卷（PV）问题 引用[5]中提到，Harbor的数据库组件（如PostgreSQL）可能...
Readiness probe failed:connection refused
05-16 1万+
我的K8S集群在启动一个POD的时候说死起不来，然后就报下面的错误 Events: Type Reason Age From Message ---- ------ ---- ---- ...
已解决 【k8s】reconnect to server error: dial tcp : connect: connection refused
黑夜开发者的博客
05-23 1648
最近使用k8s构建项目，使用kuboard连接eks的时候，导入项目总是不成功，给出的指导建议是查看日志。
K8S常见报错处理
有勇气的牛排博客
04-23 2107
1 The connection to the server localhost:8080 was refused - did you specify the right host or port? 解决方法： mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config
socket通信显示连接被拒绝问题总结
qq_26105397的博客
09-17 3万+
socket通信方式 通常使用的是socket通信模式为c/s模式，就是通过服务端创建连接，并绑定监听相关的端口，客户端通过连接至相应的端口，实现使用tcp的三次握手来进行可靠性连接，从而达到数据传输。如果对应的客户端和服务端在同一台服务器上，因是数据内部通信方式，可直接相连接。但是当我们的客户端和服务端分别部署在不同的服务器上，有时候会出现客户端连接的时候提示connect r...
Kubernetes Pod 健康问题：CrashLoopBackoff
weixin_43495948的博客
06-20 415
搭建 Prometheus 的过程中，alertmanager-main 的 Status状态为. 我的alertmanager-main svc下有三个alertmanager-main-0 alertmanager-main-1 alertmanager-main-2。
部署 Prometheus Operator 监控 Kubernetes 集群
哎_小羊的博客
08-14 2万+
我们知道 Prometheus 是一套开源的系统监控、报警、时间序列数据库的组合，而 Prometheus Operator 是 CoreOS 开源的一套用于管理在 Kubernetes 集群上的 Prometheus 控制器，它是为了简化在 Kubernetes 上部署、管理和运行 Prometheus 和 Alertmanager 集群。本文介绍如何使用 Prometheus Operator 部署监控 Kubernetes 集群，并通过 Grafana 查看监控信息。
K8S - Shooting Trouble
c123m的专栏
08-12 567
记录K8S使用过程中的问题与解决方法
kubernetes云原生纪元：健康检查-高可用的守护者
张成基
01-20 2万+
kubernetes云原生纪元：健康检查-高可用的守护者 ????提前祝大家新年快乐???? 文章目录kubernetes云原生纪元：健康检查-高可用的守护者配置信息第一种健康检查方式如何使用第二种健康检查方式使用方式第三种tcp健康检查方式 健康检查是对容器的，健康检查的配置跟容器同一级别的 配置信息 第一种健康检查方式 通过命令方式检查 livenessProbe: #是否存活的探针，用来检查应用是不...
"Unable to register node with API server" err="Post \"https://192.168.49.131:6443/api/v1/nodes\": dial tcp 192.168.49.131:6443: connect: connection refused" node="k8s-master01"
03-30
### 问题分析 此错误表明 `kubelet` 尝试向 Kubernetes API Server 注册节点时无法建立 TCP 连接。具体表现为连接被拒绝 (`connection refused`)，这通常意味着目标服务未正常运行或网络配置存在问题。 以下是可能的原因及其解决方案： --- ### 可能原因及解决办法 #### 1. **API Server 未启动** 如果 API Server 没有正确启动，则会返回 `connection refused` 错误。可以通过以下命令检查 API Server 的状态： ```bash ps -aux | grep kube-apiserver ``` 如果没有找到相关进程，说明 API Server 已经停止工作[^4]。 **解决方法：** 尝试手动启动 API Server 并查看日志以排查问题： ```bash journalctl -u kube-apiserver.service ``` --- #### 2. **Etcd 数据库不可用** API Server 使用 Etcd 存储集群数据。如果 Etcd 不可用，API Server 将无法正常工作。可以验证 Etcd 是否正常运行： ```bash curl -s http://localhost:2379/health ``` 如果返回健康状态则表示 Etcd 正常；否则可能是 Etcd 出现问题[^2]。 **解决方法：** 重新启动 Etcd 或修复其配置文件 `/etc/kubernetes/manifests/kube-apiserver.yaml` 中的相关设置。 --- #### 3. **防火墙阻止访问** 某些情况下，防火墙可能会阻止到 API Server 端口 (默认为 6443) 的流量。可以通过以下命令检查是否存在阻塞规则： ```bash sudo iptables -L | grep 6443 firewall-cmd --list-all ``` **解决方法：** 允许端口 6443 上的入站和出站流量： ```bash sudo firewall-cmd --add-port=6443/tcp --permanent sudo firewall-cmd --reload ``` --- #### 4. **证书过期** Kubernetes 使用 TLS 加密通信，如果证书已过期也会导致类似的错误。可通过以下方式检查证书有效期： ```bash openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -text | grep Not ``` 如果发现证书已经过期，需按照官方文档更新证书[^5]。 **解决方法：** 执行以下操作之一： - 手动生成新证书并替换旧证书； - 使用 kubeadm 提供的一键更新工具刷新证书。 --- #### 5. **DNS 解析问题** 当主机名解析失败时也可能引发此类错误。确认当前节点能够成功解析 Master 节点地址： ```bash ping master001 nslookup master001 ``` **解决方法：** 编辑 `/etc/hosts` 文件添加正确的映射关系： ```plaintext 192.168.150.111 master001 ``` --- ### 总结代码片段 以下是一个简单的调试脚本用于快速定位上述问题： ```bash #!/bin/bash echo "Checking API Server Status..." ps aux | grep kube-apiserver || echo "[ERROR] API Server is not running." echo "Verifying etcd health..." curl -s http://localhost:2379/health && echo "[OK] Etcd is healthy." || echo "[ERROR] Etcd may be down." echo "Testing port connectivity..." nc -zv localhost 6443 && echo "[OK] Port 6443 is open." || echo "[ERROR] Connection to port 6443 failed." echo "Inspecting certificates expiration date..." openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -text | grep Not ``` --- ###
评论 4
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值