rabbitmq-monitor.py

本文介绍了一个用于监控RabbitMQ消息队列的Python脚本。该脚本能够收集多种关键指标,如消息准备数、未确认消息数及各类速率等,并通过Falcon推送至监控系统。

转载git代码: https://github.com/iambocai/falcon-monit-scripts/tree/master/rabbitmq

#!/bin/env python
#-*- coding:utf-8 -*-

__author__ = 'iambocai'

import sys, urllib2, base64, json, time,socket


step = 60
#ip = socket.gethostname()
ip = "192.168.126.101"
ts = int(time.time())
keys = ('messages_ready', 'messages_unacknowledged')
rates = ('ack', 'deliver', 'deliver_get', 'publish')

request = urllib2.Request("http://%s:15672/api/queues" %ip)
# see #issue4
base64string = base64.b64encode('ops:123123')
request.add_header("Authorization", "Basic %s" % base64string)   
result = urllib2.urlopen(request)
data = json.loads(result.read())
tag = ''
#tag = sys.argv[1].replace('_',',').replace('.','=')

p = []
for queue in data:
    # ready and unack
    msg_total = 0
    for key in keys:
        q = {}
        q["endpoint"] = ip
        q['timestamp'] = ts
        q['step'] = step
        q['counterType'] = "GAUGE"
        q['metric'] = 'rabbitmq.%s' % key
        q['tags'] = 'name=%s,%s' % (queue['name'],tag)
        q['value'] = int(queue[key])
        msg_total += q['value']
        p.append(q)

    # total
    q = {}
    q["endpoint"] = ip
    q['timestamp'] = ts
    q['step'] = step
    q['counterType'] = "GAUGE"
    q['metric'] = 'rabbitmq.messages_total'
    q['tags'] = 'name=%s,%s' % (queue['name'],tag)
    q['value'] = msg_total
    p.append(q)

    # rates
    for rate in rates:
        q = {}
        q["endpoint"] = ip
        q['timestamp'] = ts
        q['step'] = step
        q['counterType'] = "GAUGE"
        q['metric'] = 'rabbitmq.%s_rate' % rate
        q['tags'] = 'name=%s,%s' % (queue['name'],tag)
        try:
            q['value'] = int(queue['message_stats']["%s_details" % rate]['rate'])
        except:
            q['value'] = 0
        p.append(q)

print json.dumps(p, indent=4)


method = "POST"
handler = urllib2.HTTPHandler()
opener = urllib2.build_opener(handler)
url = 'http://127.0.0.1:1988/v1/push'
request = urllib2.Request(url, data=json.dumps(p) )
request.add_header("Content-Type",'application/json')
request.get_method = lambda: method
try:
    connection = opener.open(request)
except urllib2.HTTPError,e:
    connection = e

# check. Substitute with appropriate HTTP code.
if connection.code == 200:
    print connection.read()
else:
    print '{"err":1,"msg":"%s"}' % connection
  • 假如有几个队列,那么作图的时候会有点麻烦,写个小脚本,生成falcon里面的规则
#!/bin/bash

for line in `cat 1`
do
    echo 
    echo 
    part1=`echo $line | awk '{print $1}'`
    for line2 in `cat 2`
    do
        part2=`echo $line2 | awk '{print $1}'`
        echo "$part1/$part2"
    done
done
  • rabbitmq-monitor可以提供的指标
rabbitmq.ack_rate
rabbitmq.deliver_get_rate
rabbitmq.deliver_rate
rabbitmq.messages_ready
rabbitmq.messages_total //这个指标重要,我们是高于5k就报警
rabbitmq.messages_unacknowledged
rabbitmq.publish_rate
[2025-08-20 19:43:04,809: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: timed out. Trying again in 2.00 seconds... (1/100) [2025-08-20 19:43:08,252: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672// [2025-08-20 19:43:12,258: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: timed out. Trying again in 2.00 seconds... (1/100) [2025-08-20 19:43:16,268: INFO/MainProcess] mingle: searching for neighbors /usr/local/lib/python3.11/site-packages/click/core.py:1193: UserWarning: The parameter --uid is used more than once. Remove its duplicate as parameters should be unique. parser = self.make_parser(ctx) /usr/local/lib/python3.11/site-packages/click/core.py:1186: UserWarning: The parameter --uid is used more than once. Remove its duplicate as parameters should be unique. self.parse_args(ctx, args) /usr/local/lib/python3.11/site-packages/celery/platforms.py:840: SecurityWarning: You're running the worker with superuser privileges: this is absolutely not recommended! Please specify a different user using the --uid option. User information: uid=0 euid=0 gid=0 egid=0 warnings.warn(SecurityWarning(ROOT_DISCOURAGED.format( -------------- celery@common_worker v5.2.7 (dawn-chorus) --- ***** ----- -- ******* ---- Linux-6.1.0-37-cloud-amd64-x86_64-with-glibc2.36 2025-08-20 19:43:24 - *** --- * --- - ** ---------- [config] - ** ---------- .> app: common:0x7fc58a87d990 - ** ---------- .> transport: amqp://guest:**@localhost:5672// - ** ---------- .> results: disabled:// - *** --- * --- .> concurrency: 2 (prefork) -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker) --- ***** ----- -------------- [queues] .> common exchange=common(direct) key=common [tasks] . tasks.common.common_tasks.process [2025-08-20 19:43:28,604: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: timed out. Trying again in 2.00 seconds... (1/100) [2025-08-20 19:43:32,253: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672// [2025-08-20 19:43:36,262: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: timed out. Trying again in 2.00 seconds... (1/100) [2025-08-20 19:43:40,260: INFO/MainProcess] mingle: searching for neighbors /usr/local/lib/python3.11/site-packages/click/core.py:1193: UserWarning: The parameter --uid is used more than once. Remove its duplicate as parameters should be unique. parser = self.make_parser(ctx) /usr/local/lib/python3.11/site-packages/click/core.py:1186: UserWarning: The parameter --uid is used more than once. Remove its duplicate as parameters should be unique. self.parse_args(ctx, args) /usr/local/lib/python3.11/site-packages/celery/platforms.py:840: SecurityWarning: You're running the worker with superuser privileges: this is absolutely not recommended! Please specify a different user using the --uid option. User information: uid=0 euid=0 gid=0 egid=0 warnings.warn(SecurityWarning(ROOT_DISCOURAGED.format( -------------- celery@common_worker v5.2.7 (dawn-chorus) --- ***** ----- -- ******* ---- Linux-6.1.0-37-cloud-amd64-x86_64-with-glibc2.36 2025-08-20 19:43:48 - *** --- * --- - ** ---------- [config] - ** ---------- .> app: common:0x7f17a9df4b10 - ** ---------- .> transport: amqp://guest:**@localhost:5672// - ** ---------- .> results: disabled:// - *** --- * --- .> concurrency: 2 (prefork) -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker) --- ***** ----- -------------- [queues] .> common exchange=common(direct) key=common [tasks] . tasks.common.common_tasks.process [2025-08-20 19:43:52,581: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: timed out. Trying again in 2.00 seconds... (1/100) [2025-08-20 19:43:56,254: INFO/MainProcess] Connected to amqp://guest:**@127.0.0.1:5672// [2025-08-20 19:44:00,265: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: timed out. Trying again in 2.00 seconds... (1/100)
08-21
(.venv) PS D:\celery-task> celery -A task_api worker -l info --pool=solo -------------- celery@LAPTOP-TOG2JO7D v5.5.3 (immunity) --- ***** ----- -- ******* ---- Windows-10-10.0.19045-SP0 2025-10-18 14:41:56 - *** --- * --- - ** ---------- [config] - ** ---------- .> app: progress_tasks:0x24fb66e7920 - ** ---------- .> transport: redis://localhost:6379/3 - ** ---------- .> results: redis://localhost:6379/4 - *** --- * --- .> concurrency: 12 (solo) -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker) --- ***** ----- -------------- [queues] .> celery exchange=celery(direct) key=celery [tasks] . task_api.progress_web_request [2025-10-18 14:41:56,078: INFO/MainProcess] Connected to redis://localhost:6379/3 [2025-10-18 14:41:56,080: INFO/MainProcess] mingle: searching for neighbors [2025-10-18 14:41:56,088: CRITICAL/MainProcess] Unrecoverable error: AttributeError("'NoneType' object has no attribute '_sock'") Traceback (most recent call last): File "D:\celery-task\.venv\Lib\site-packages\celery\worker\worker.py", line 203, in start self.blueprint.start(self) File "D:\celery-task\.venv\Lib\site-packages\celery\bootsteps.py", line 116, in start step.start(parent) File "D:\celery-task\.venv\Lib\site-packages\celery\bootsteps.py", line 365, in start return self.obj.start() ^^^^^^^^^^^^^^^^ File "D:\celery-task\.venv\Lib\site-packages\celery\worker\consumer\consumer.py", line 341, in start blueprint.start(self) File "D:\celery-task\.venv\Lib\site-packages\celery\bootsteps.py", line 116, in start step.start(parent) File "D:\celery-task\.venv\Lib\site-packages\celery\worker\consumer\mingle.py", line 37, in start self.sync(c) File "D:\celery-task\.venv\Lib\site-packages\celery\worker\consumer\mingle.py", line 41, in sync replies = self.send_hello(c) ^^^^^^^^^^^^^^^^^^ File "D:\celery-task\.venv\Lib\site-packages\celery\worker\consumer\mingle.py", line 54, in send_hello replies = inspect.hello(c.hostname, our_revoked._data) or {} ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\celery-task\.venv\Lib\site-packages\celery\app\control.py", line 389, in hello return self._request('hello', from_node=from_node, revoked=revoked) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\celery-task\.venv\Lib\site-packages\celery\app\control.py", line 106, in _request return self._prepare(self.app.control.broadcast( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\celery-task\.venv\Lib\site-packages\celery\app\control.py", line 777, in broadcast return self.mailbox(conn)._broadcast( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\celery-task\.venv\Lib\site-packages\kombu\pidbox.py", line 346, in _broadcast return self._collect(reply_ticket, limit=limit, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\celery-task\.venv\Lib\site-packages\kombu\pidbox.py", line 388, in _collect self.connection.drain_events(timeout=timeout) File "D:\celery-task\.venv\Lib\site-packages\kombu\connection.py", line 341, in drain_events return self.transport.drain_events(self.connection, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\celery-task\.venv\Lib\site-packages\kombu\transport\virtual\base.py", line 997, in drain_events get(self._deliver, timeout=timeout) File "D:\celery-task\.venv\Lib\site-packages\kombu\transport\redis.py", line 591, in get self._register_BRPOP(channel) File "D:\celery-task\.venv\Lib\site-packages\kombu\transport\redis.py", line 530, in _register_BRPOP self._register(*ident) File "D:\celery-task\.venv\Lib\site-packages\kombu\transport\redis.py", line 506, in _register if client.connection._sock is None: # not connected yet. ^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute '_sock'
最新发布
10-19
2025-09-16 21:03:54.696 73766 INFO os_vif [-] Loaded VIF plugins: ovs, linux_bridge, noop 2025-09-16 21:03:55.122 73766 CRITICAL nova [req-5a1a1b97-2981-4f83-83fd-69c8693f113d - - - - -] Unhandled error: AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN. For details see the broker logfile. 2025-09-16 21:03:55.122 73766 ERROR nova Traceback (most recent call last): 2025-09-16 21:03:55.122 73766 ERROR nova File "/usr/bin/nova-compute", line 10, in <module> 2025-09-16 21:03:55.122 73766 ERROR nova sys.exit(main()) 2025-09-16 21:03:55.122 73766 ERROR nova File "/usr/lib/python2.7/site-packages/nova/cmd/compute.py", line 57, in main 2025-09-16 21:03:55.122 73766 ERROR nova topic=compute_rpcapi.RPC_TOPIC) 2025-09-16 21:03:55.122 73766 ERROR nova File "/usr/lib/python2.7/site-packages/nova/service.py", line 259, in create 2025-09-16 21:03:55.122 73766 ERROR nova periodic_interval_max=periodic_interval_max) 2025-09-16 21:03:55.122 73766 ERROR nova File "/usr/lib/python2.7/site-packages/nova/service.py", line 130, in __init__ 2025-09-16 21:03:55.122 73766 ERROR nova conductor_api.wait_until_ready(context.get_admin_context()) 2025-09-16 21:03:55.122 73766 ERROR nova File "/usr/lib/python2.7/site-packages/nova/conductor/api.py", line 67, in wait_until_ready 2025-09-16 21:03:55.122 73766 ERROR nova timeout=timeout) 2025-09-16 21:03:55.122 73766 ERROR nova File "/usr/lib/python2.7/site-packages/nova/baserpc.py", line 58, in ping 2025-09-16 21:03:55.122 73766 ERROR nova return cctxt.call(context, 'ping', arg=arg_p) 2025-09-16 21:03:55.122 73766 ERROR nova File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 178, in call 2025-09-16 21:03:55.122 73766 ERROR nova retry=self.retry) 2025-09-16 21:03:55.122 73766 ERROR nova File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 128, in _send 2025-09-16 21:03:55.122 73766 ERROR nova retry=retry) 2025-09-16 21:03:55.122 73766 ERROR nova File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 645, in send 2025-09-16 21:03:55.122 73766 ERROR nova call_monitor_timeout, retry=retry) 2025-09-16 21:03:55.122 73766 ERROR nova File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 587, in _send 2025-09-16 21:03:55.122 73766 ERROR nova msg.update({'_reply_q': self._get_reply_q()}) 2025-09-16 21:03:55.122 73766 ERROR nova File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 568, in _get_reply_q 2025-09-16 21:03:55.122 73766 ERROR nova conn = self._get_connection(rpc_common.PURPOSE_LISTEN) 查找问题
09-18
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值