解决Sentry不收集爬虫产生的Exception问题

本文链接：https://blog.youkuaiyun.com/CleverCode/article/details/105880923

文章目录

技术交流

最近业务部门反馈说收到短信sentry报警数值和打开sentry页面看到的数据不一样。
我们报警数值是取的接口：

http://101.1.251.101:9100/api/0/projects/sentry/mail/stats/?stat=received

线下安装了sentry开始复现。《sentry安装与配置》：https://clevercode.blog.youkuaiyun.com/article/details/105880652 。

分析了nginx请求日志，发现sentry客户端请求的sentry服务的接口是: http://10.1.20.101:9100/api/4/store/ ,分析sentry请求的nginx日志，发现了http的状态码中有大量的403，查看源码403，对应的APIRateLimited产生的。但是源码抛出APIRateLimited发现有10处代码都可以产生。如何找到具体是哪一处代码抛出的这个APIRateLimited报错。

这时候想到抓包工具《流量录制与回放工具–GoReplay》：https://blog.youkuaiyun.com/CleverCode/article/details/101423570 。
对10.1.20.101的9100端口进行http抓取请求包和响应包。

/Data/apps/gor/gor --input-raw :9100 --output-file /tmp/sentry.gor --input-raw-track-response --http-allow-url /store/

分析sentry.gor日志。发现403的时候，响应包出现：{“error”:“Event dropped due to filter”}

2 eb2958ceb5974ace946df51e345cf0edaa8476bf 1588233398959704818 30501033
HTTP/1.0 403 FORBIDDEN
Content-Length: 39
Expires: Thu, 30 Apr 2020 07:56:38 GMT
X-Content-Type-Options: nosniff
Content-Language: en
X-Sentry-Error: Event dropped due to filter
Vary: Accept-Language, Cookie
Last-Modified: Thu, 30 Apr 2020 07:56:38 GMT
X-XSS-Protection: 1; mode=block
Cache-Control: max-age=0
X-Frame-Options: deny
Content-Type: application/json

{"error":"Event dropped due to filter"}

在源码中查：Event dropped due to filter。找到了位置：/Data/apps/ops4env/lib/python2.7/site-packages/sentry/web/api.py 找到 raise APIForbidden(‘Event dropped due to filter’)

vi /Data/apps/ops4env/lib/python2.7/site-packages/sentry/web/urls.py

url(r'^api/(?P<project_id>[\w_-]+)/store/$', api.StoreView.as_view(),

查看对应的视图
vi /Data/apps/ops4env/lib/python2.7/site-packages/sentry/web/api.py

class StoreView(APIView):
   
    def post(self, request, **kwargs):
        try:
            data = request.body
        except Exception as e:
            logger.exception(e)
            # We were unable to read the body.
            # This would happen if a request were submitted
            # as a multipart form for example, where reading
            # body yields an Exception. There's also not a more
            # sane exception to catch here. This will ultimately
            # bubble up as an APIError.
            data = None

        response_or_event_id = self.process(request, data=data,