Prometheus监控主机进程

原创已于 2024-07-15 14:07:19 修改 · 1k 阅读

5 ·

CC 4.0 BY-SA版权

文章标签：

#prometheus

于 2024-07-15 11:19:31 首次发布

docker 专栏收录该内容

12 篇文章

订阅专栏

前言

此场景主要是利用process-export监控主机的进程存活、资源占用率，防止进程挂掉导致服务崩溃

gitlab地址：GitHub - ncabatoff/process-exporter: Prometheus exporter that mines /proc to report on selected processes

tgz包：

process-exporter 下载地址

process-exporter-0.7.5.linux-amd64.tar.gz

客户端安装及配置

1.将包上传至/opt目录

2.tar -zxvf解压

3.mv重命名为 process-exporter

4.touch一个规则文件：process_name.yaml

如下截图：

编辑规则文件：

当前要监控的进程是vsftpd

vim /opt/process-exporter/process_name.yaml

process_names:
  - name: "{{.Matches}}"
    cmdline:
    - 'vsftpd' #唯一标识
  - name: "{{.Matches}}"
    cmdline:
    - 'httpd' #唯一标识

规则参数解释：

启动脚本配置：

[root@bogon /opt/process-exporter]# cat /usr/lib/systemd/system/process_exporter.service



 
[Unit]
Description=Prometheus exporter for processors metrics, written in Go with pluggable metric collectors.
Documentation=https://github.com/ncabatoff/process-exporter
After=network.target
  
[Service]
Type=simple
User=root
WorkingDirectory=/opt/process-exporter
ExecStart=/opt/process-exporter/process-exporter -config.path=/opt/process-exporter/process_name.yaml
Restart=on-failure
  
[Install]
WantedBy=multi-user.target

加入开机自启：

systemctl daemon-reload
systemctl start process_exporter
systemctl enable process_exporter

Premetheus服务端配置

添加targets，端口为9256

测试端口：established表示建立连接

添加进程告警规则：

  - alert: 进程重启告警
    expr: ceil(time() - max by(cluster, job, instance, groupname) (namedprocess_namegroup_oldest_start_time_seconds)) < 60
    for: 30s
    labels:
      label: alert_once
      severity: warning
    annotations:
      value: 进程 {{ $labels.groupname }} 在 {{ $value }} 秒前发生重启

  - alert: 进程退出告警
    expr: up{export="process_exporter"} == 0 or max by(cluster, job, instance, groupname) (delta(namedprocess_namegroup_oldest_start_time_seconds{groupname=~"^map.*"}[10d])) < 0
    for: 60s
    labels:
      severity: warning
    annotations:
      value: 进程 {{ $labels.export}} 已退出

重启prometheus，我这边是docker部署，所以方式是：