alertmanager对接自定义webhook
1. 安装python模块
pip install Flask requests Jinja2
2. 编写flask代码
cat webhook.py
from flask import Flask, request, jsonify
import requests
from jinja2 import Environment, FileSystemLoader
import argparse
app = Flask(__name__)
# 钉钉机器人Webhook URL
DINGTALK_WEBHOOK = '换成你的钉钉机器人webhook'
# 设置Jinja2模板环境
env = Environment(loader=FileSystemLoader('.'))
# 命令行参数解析
parser = argparse.ArgumentParser(description='Start the Flask webhook server.')
parser.add_argument('--template-file', type=str, required=True, help='Path to the Jinja2 template file.')
args = parser.parse_args()
@app.route('/webhook', methods=['POST'])
def alertmanager_webhook():
if request.method == 'POST':
alerts = request.json.get('alerts', [])
print(alerts)
for alert in alerts:
status = alert.get('status')
labels = alert.get('labels', {})
annotations = alert.get('annotations', {})
startsAt = alert.get('startsAt')
endsAt = alert.get('endsAt') if status == 'resolved' else None
# 加载并渲染Jinja2模板
template = env.get_template(args.template_file)
message_content = template.render(
status=status,
alertname=labels.get('alertname'),
severity=labels.get('severity'),
startsAt=startsAt,
endsAt=endsAt,
description=annotations.get('description'),
summary=annotations.get('summary')
)
# 生成钉钉消息内容
message = {
"msgtype": "markdown",
"markdown": {
"text": message_content
}
}
# 发送消息到钉钉群机器人
headers = {'Content-Type': 'application/json'}
response = requests.post(DINGTALK_WEBHOOK, headers=headers, json=message)
# 检查消息是否成功发送
if response.status_code != 200:
return jsonify({'message': 'Failed to send message to DingTalk'}), 500
return jsonify({'message': 'Alerts processed successfully'}), 200
if __name__ == '__main__':
app.run(host='0.0.0.0', port=5000)
3. 编写模板
cat alert_template.tpl
{% if status == 'firing' %}
## flask测试告警
**告警状态**: {{ status }}<br>
**告警名称**: {{ alertname }}<br>
**告警级别**: <font color='red'>{{ severity }}</font><br>
**开始时间**: {{ startsAt }}<br>
**描述**: {{ description }}<br>
**摘要**: {{ summary }}<br>
<at user_id="-1">所有人</at>
{% elif status == 'resolved' %}
## flask测试告警恢复
**告警状态**: {{ status }}<br>
**告警名称**: {{ alertname }}<br>
**开始时间**: {{ startsAt }}<br>
**结束时间**: {{ endsAt }}<br>
**描述**: 告警已恢复<br>
{% endif %}
4. 启动程序
python webhook.py --templates-file alert_template.tpl
5. 编写测试代码模拟alertmanager告警数据
cat alertmanager.py
import requests
import json
headers = {'Content-Type': 'application/json'}
data={
"alerts": [{'status': 'resolved', 'labels': {'alert_type': 'metrics', 'alertgroup': 'Hanhai', 'alertname': '测试pod异常告警', 'container': 'ssrfproxy', 'instance': '10.244.25.57:8443', 'job': 'kube-state-metrics', 'namespace': 'lzy-test', 'origin_prometheus': 'hanhai-pre', 'pod': 'ssrfproxy-67989787cc-bq5wj', 'prometheus': 'monitoring/k8s', 'severity': 'critical', 'uid': '444aa780-5cc3-4f01-8bf4-502369b05e78'}, 'annotations': {'description': 'namespace: lzy-test status: Inference Service Status has failed', 'summary': 'lzy-test/ssrfproxy-67989787cc-bq5wj Inference Service Status has failed'}, 'startsAt': '2024-12-13T15:43:00+08:00', 'endsAt': '0001-01-01T00:00:00Z', 'generatorURL': 'http://k8s-master-1.ai.ksyun.com:8488/vmalert/alert?group_id=6001045321047810936&alert_id=9518171390601939944', 'fingerprint': 'ffd3eadf51e8b598'}]
}
result = requests.post("http://127.0.0.1:5000/webhook", headers=headers, data=json.dumps(data))
if result.status_code != 200:
print("失败")
6. 运行代码
告警有两种状态
resolved
:恢复
firing
:告警
测试告警,修改status为firing
data={
"alerts": [{'status': 'firing', 'labels': {'alert_type': 'metrics', 'alertgroup': 'Hanhai', 'alertname': '测试pod异常告警', 'container': 'ssrfproxy', 'instance': '10.244.25.57:8443', 'job': 'kube-state-metrics', 'namespace': 'lzy-test', 'origin_prometheus': 'hanhai-pre', 'pod': 'ssrfproxy-67989787cc-bq5wj', 'prometheus': 'monitoring/k8s', 'severity': 'critical', 'uid': '444aa780-5cc3-4f01-8bf4-502369b05e78'}, 'annotations': {'description': 'namespace: lzy-test status: Inference Service Status has failed', 'summary': 'lzy-test/ssrfproxy-67989787cc-bq5wj Inference Service Status has failed'}, 'startsAt': '2024-12-13T15:43:00+08:00', 'endsAt': '0001-01-01T00:00:00Z', 'generatorURL': 'http://k8s-master-1.ai.ksyun.com:8488/vmalert/alert?group_id=6001045321047810936&alert_id=9518171390601939944', 'fingerprint': 'ffd3eadf51e8b598'}]
}
测试恢复,修改status为resolved