promethus监控mysql指标_监控指标以及prometheus规则-不断完善中

最新推荐文章于 2025-04-21 08:49:07 发布

weixin_39975055

最新推荐文章于 2025-04-21 08:49:07 发布

阅读量881

点赞数

文章标签： promethus监控mysql指标

本文链接：https://blog.youkuaiyun.com/weixin_39975055/article/details/113478145

版权

本文详细介绍了如何使用Prometheus监控MySQL、系统资源及Pod性能。包括监控项如CPU、内存、磁盘使用率，以及Prometheus规则设置，如针对高CPU、内存使用、文件系统使用率、MySQL性能等的告警配置。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

1)监控项

cpu使用率: (100 - (avg by (instance)(irate(node_cpu_seconds_total{mode="idle"}[5m])) 100))

内存使用率：(100 - (((node_memory_MemFree_bytes+node_memory_Buffers_bytes+node_memory_Cached_bytes)/node_memory_MemTotal_bytes) 100))

磁盘使用率:(1- (node_filesystem_free_bytes{fstype=~"ext3|ext4|xfs"} / node_filesystem_size_bytes{fstype=~"ext3|ext4|xfs"}) ) * 100

2)prometheus规则

groups:

- name: alert-rule

rules:

- alert: NodeFilesystemUsage-high

expr: (1- (node_filesystem_free_bytes{fstype=~"ext3|ext4|xfs"} / node_filesystem_size_bytes{fstype=~"ext3|ext4|xfs"}) ) * 100 > 80

for: 2m

labels:

severity: warning

annotations:

summary: "{{$labels.instance}}: High Node Filesystem usage detected"

description: "{{$labels.instance}}: Node Filesystem usage is above 80% ,(current value is: {{ $value }})"

- alert: NodeMemoryUsage

expr: (100 - (((node_memory_MemFree_bytes+node_memory_Buffers_bytes+node_memory_Cached_bytes)/node_memory_MemTotal_bytes) * 100)) > 80

for: 2m

labels:

severity: warning

annotations:

summary: "{{$labels.instance}}: High Node Memory usage detected"

description: "{{$labels.instance}}: Node Memory usage is above 80% ,(current value is: {{ $value }})"

- alert: NodeCPUUsage

expr: (100 - (avg by (instance)(irate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)) > 80

for: 2m

labels:

severity: warning

annotations:

summary: "{{$labels.instance}}: Node High CPU usage detected"

description: "{{$labels.instance}}: Node CPU usage is above 80% ,(current value is: {{ $value }})"

(2)mysql 监控性能指标

1)mysql性能指标

mysql is down :mysql_up

每秒查询次数指标:rate(mysql_global_status_slow_queries[5m])

连接数指标：rate(mysql_global_status_threads_connected[5m]) > 200

或可用连接mysql_global_variables_max_connections - mysql_global_status_threads_connected <200

慢查询：rate(mysql_global_status_slow_queries[5m])

mysql主从复制 sql线程： mysql_slave_status_slave_sql_running

mysql主从延迟：rate(mysql_slave_status_seconds_behind_master[5m])

2)prometheus规则

groups:

- name: MySQLStatsAlert

rules:

- alert: MySQL is down

expr: mysql_up == 0

for: 1m

labels:

severity: critical

annotations:

summary: "Instance {{ $labels.instance }} MySQL is down"

description: "MySQL database is down. This requires immediate action!"

- alert: Mysql_High_QPS

expr: rate(mysql_global_status_questions[5m]) > 500

for: 2m

labels:

severity: warning

annotations:

summary: "{{$labels.instance}}: Mysql_High_QPS detected"

description: "{{$labels.instance}}: Mysql opreation is more than 500 per second ,(current value is: {{ $value }})"

- alert: Mysql_Too_Many_Connections

expr: rate(mysql_global_status_threads_connected[5m]) > 200

for: 2m

labels:

severity: warning

annotations:

summary: "{{$labels.instance}}: Mysql Too Many Connections detected"

description: "{{$labels.instance}}: Mysql Connections is more than 100 per second ,(current value is: {{ $value }})"

- alert: Mysql_Too_Many_slow_queries

expr: rate(mysql_global_status_slow_queries[5m]) > 3

for: 2m

labels:

severity: warning

annotations:

summary: "{{$labels.instance}}: Mysql_Too_Many_slow_queries detected"

description: "{{$labels.instance}}: Mysql slow_queries is more than 3 per second ,(current value is: {{ $value }})"

- alert: SQL thread stopped

expr: mysql_slave_status_slave_sql_running == 0

for: 1m

labels:

severity: critical

annotations:

summary: "Instance {{ $labels.instance }} SQL thread stopped"

description: "SQL thread has stopped. This is usually because it cannot apply a SQL statement received from the master."

- alert: Slave lagging behind Master

expr: rate(mysql_slave_status_seconds_behind_master[5m]) >30

for: 1m

labels:

severity: warning

annotations:

summary: "Instance {{ $labels.instance }} Slave lagging behind Master"

description: "Slave is lagging behind Master. Please check if Slave threads are running and if there are some performance issues!"

(3)pod性能指标

1)容器性能指标

pod的cpu使用率:container_memory_usage_bytes{container_name!=""} / container_spec_memory_limit_bytes{container_name!=""} *100 != +Inf

pod的内存使用率: sum by (pod_name)( rate(container_cpu_usage_seconds_total{image!=""}[1m] ) ) * 100

2)prometheus规则

groups:

- name: noah_pod.rules

rules:

- alert: PodMemUsage

expr: container_memory_usage_bytes{container_name!=""} / container_spec_memory_limit_bytes{container_name!=""} *100 != +Inf > 80

for: 2m

labels:

severity: warning

annotations:

summary: "{{$labels.name}}: Pod High Mem usage detected"

description: "{{$labels.name}}: Pod Mem is above 80% ,(current value is: {{ $value }})"

- alert: PodCpuUsage

expr: sum by (pod_name)( rate(container_cpu_usage_seconds_total{image!=""}[1m] ) ) * 100 > 80

for: 2m

labels:

severity: warning

annotations:

summary: "{{$labels.name}}: Pod High CPU usage detected"

description: "{{$labels.name}}: Pod CPU is above 80% ,(current value is: {{ $value }})"

参考文档：