TIDB-由飞书接收报警

一、背景

由于公司的办公软件由企业微信转到了飞书，所以报警也由企业微信转到飞书了，而altermanager的输出格式是飞书不接受的，飞书接受的消息格式可以从官网找到飞书链接
这时就有两种办法解决，第一种，自己写个服务，接收altermanager的消息，然后进行转换发给飞书，第二种是使用开源工具，如prometheusalert考虑到一些原因，最终选择了方案一

二、搭建

1.飞书操作

1.1 建群
点击下图所示加号建群,随意写个群组名

1.2创建机器人
点击群图标即可创建机器人，点击添加即可
1.3 生成webook
记录webook地址，是之后要用到的

2.写服务来接收处理消息

2.1 python服务文本到飞书示例

import requests
import arrow
from flask import Flask
from flask import request
import json
app1 = Flask(__name__)
@app1.route('/send', methods=['POST'])
def send():
    url = 'https://open.larksuite.com/open-apis/bot/v2/hook/*****'   # lark的weboook
    dict1 = dict()
    data = json.loads(request.data)
    #print(data)
    alerts = data['alerts'] 
    for output in alerts:
        try:
            message = output['annotations']['message']
        except KeyError:
            try:
                message = output['annotations']['description']
            except KeyError:
                message = 'null'
        info = "**告警类型**: %s \n\n" % output['labels']['alertname'] + "**告警详情** : %s \n\n" % message + "**告警状态**: %s \n\n" % output['status'] + "**触发时间**: %s \n\n" % arrow.get(output['startsAt']).to('Asia/Shanghai').format('YYYY-MM-DD HH:mm:ss ZZ') + "**触发结束时间**: %s \n" % arrow.get(output['endsAt']).to('Asia/Shanghai').format('YYYY-MM-DD HH:mm:ss ZZ')
        # 转换为飞书接受的格式（文本）
        send_data = {
			    "msg_type": "text",
                            "content": {
			                            "text":  info
			           }
                    }
        send_data = json.dumps(send_data)
        print(send_data)
        req = requests.post(url, data=send_data)
        result = req.json()
        #调用api提交信息给短信平台api
        # r = requests.post(url_test_api,data=j)
    return 'ok'


if __name__ == '__main__':
    app1.run(debug=True,host='0.0.0.0',port=5100)

2.1 python服务富文本到飞书示例

import requests
import arrow
from flask import Flask
from flask import request
import json
app1 = Flask(__name__)
print(3)
@app1.route('/send', methods=['POST'])
def send():
    url = 'https://open.larksuite.com/open-apis/bot/v2/hook/****'  
    dict1 = dict()   #定义一个字典用来存放清理过后的数据，最后需要把这个数据推送给其他应用
    data = json.loads(request.data)
    #print(data)
    alerts = data['alerts']         
    for output in alerts:
        try:
            message = output['annotations']['message']
        except KeyError:
            try:
                message = output['annotations']['description']
            except KeyError:
                message = 'null'
        title = "警告: %s" %  output['labels']['alertname']
        warning_info =  " %s \n" % message.replace(',', '\n').replace(':', ':  ')
        warning_status = " status:  %s \n" % output['status']
        warning_start_time = " start_time:  %s \n" % arrow.get(output['startsAt']).to('Asia/Shanghai').format('YYYY-MM-DD HH:mm:ss ZZ')
        warning_end_time = " end_time:  %s \n" % arrow.get(output['endsAt']).to('Asia/Shanghai').format('YYYY-MM-DD HH:mm:ss ZZ')

        send_data = {
        "msg_type": "post",
        "content": {
                "post": {
                        "zh_cn": {
                                "title": title,
                                "content": [[{
                                                        "tag": "text",
                                                        "text": warning_info
                                                },
                                               {
                                                        "tag": "text",
                                                        "text": warning_status
                                                },
                                             {
                                                        "tag": "text",
                                                        "text": warning_start_time
                                                },
                                             {
                                                        "tag": "text",
                                                        "text": warning_end_time
                                                }

                                           ]]
                        }
                }
        }
}
        send_data = json.dumps(send_data)
        print(send_data)
        req = requests.post(url, data=send_data)
        result = req.json()
        #调用api提交信息给短信平台api
        # r = requests.post(url_test_api,data=j)
    return 'ok'


if __name__ == '__main__':
    app1.run(debug=True,host='0.0.0.0',port=5100)

写这个服务用来接受altermanager的消息并格式化后发给飞书，启动服务即可

3.altermanager的处理

3.1 编辑conf下的alertmanager.yml

route:
  # A default receiver
  receiver: "feishubook"

  # The labels by which incoming alerts are grouped together. For example,
  # multiple alerts coming in for cluster=A and alertname=LatencyHigh would
  # be batched into a single group.
  group_by: ['env','instance','alertname','type','group','job']

  # When a new group of alerts is created by an incoming alert, wait at
  # least 'group_wait' to send the initial notification.
  # This way ensures that you get multiple alerts for the same group that start
  # firing shortly after another are batched together on the first
  # notification.
  group_wait:      30s

  # When the first notification was sent, wait 'group_interval' to send a batch
  # of new alerts that started firing for that group.
  group_interval:  3m

  # If an alert has successfully been sent, wait 'repeat_interval' to
  # resend them.
  repeat_interval: 1h


receivers:
- name: 'feishubook'
  webhook_configs:
  - url: 'http://python程序的ip:5100/send'

3.2 重启alterrmanager服务

重启后飞书即可收到报警消息了