awk分析nginx日志中响应时间的方法

关注

转载最新推荐文章于 2026-07-04 14:34:44 发布 6.8k 阅读 AI 写同款· GEO 优化 ›

linux用grep、awk分析nginx的access.log的日志，根据时间段截取日志、统计某接口/关键字访问成功数量及平均响应时间。衍生文章链接：https://blog.youkuaiyun.com/enjoyphp/article/details/100024220 项目中亲测，命令全部有效日志截取，从总日志文件中截取昨天已整天的数据出来，并覆盖到新文件中： cat access.log | awk '$4 >="[14/Mar/2019:00:00:00" && $4 <="[14/Ma... 阅读详情

针对响应时间慢的问题，我们在nginx日志格式中增加响应时间，现在需要针对响应时间进行分析，查找出相对较慢的响应时间。

1、确认下日志文件格式

日志格式：

log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                '$status $body_bytes_sent "$http_referer" '
                '"$http_user_agent" $http_x_forwarded_for "$request_time"';

2、输出日志内容：

118.187.188.188 - - [19/Jun/2015:09:26:08 +0800] "GET /wap/data/ipad/gsxw/1.json HTTP/1.1" 304 0 "-" "android-async-http/1.4.3 (http://loopj.com/android-async-http)" - "0.287"
61.179.109.132 - - [19/Jun/2015:09:26:08 +0800] "GET /wap/data/ipad/gsxw/1.json HTTP/1.1" 304 0 "-" "android-async-http/1.4.3 (http://loopj.com/android-async-http)" - "0.290"
61.179.107.103 - - [19/Jun/2015:09:26:08 +0800] "GET /wap/data/ipad/gsxw/1.json HTTP/1.1" 304 0 "-" "android-async-http/1.4.3 (http://loopj.com/android-async-http)" - "0.288"
218.92.220.57 - - [19/Jun/2015:09:26:08 +0800] "GET /wap/news/ipad2_gg.html HTTP/1.1" 200 1259 "-" "Mozilla/5.0 (iPad; CPU OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B206" - "0.292"
182.140.237.207 - - [19/Jun/2015:09:26:08 +0800] "GET /wap/data/advImg.json?keytime=20150619092610 HTTP/1.1" 200 589 "-" "Dalvik/1.6.0 (Linux; U; Android 4.3; GT-I9308 Build/JSS15J)" - "0.276"

3、分析nginx日志

　　3.1、打印响应时间并去除引号

＃awk '{print $NF}' news.log | awk -F "\"" '{print $2}'  >  time.txt

time.txt文件内容

　　3.2、合并日志文件，增加修改后的响应时间

paste  -d " " news.log time.txt > news1.log

合并后news1.log文件内容

218.92.220.156 - - [19/Jun/2015:09:58:25 +0800] "GET /wap/news/ipad2_kxcwsj.html HTTP/1.1" 304 0 "-" "Mozilla/5.0 (iPad; CPU OS 8_3 like Mac OS X) AppleWebKit/600.1.
4 (KHTML, like Gecko) Mobile/12F69" - "4.822" 4.822
61.179.107.29 - - [19/Jun/2015:09:58:25 +0800] "GET /wap/news/finance-dividends.html HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Linux; U; Android 4.3; zh-cn; SM-G7108V Buil
d/JLS36C) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30" - "4.805" 4.805
61.179.107.32 - - [19/Jun/2015:09:58:25 +0800] "GET /wap/data/1.json HTTP/1.1" 304 0 "-" "android-async-http/1.4.3 (http://loopj.com/android-async-htt
p)" - "0.000" 0.000
61.179.107.129 - - [19/Jun/2015:09:58:25 +0800] "GET /wap/data/index.json HTTP/1.1" 304 0 "-" "DZHTremereMobile/3.05 CFNetwork/609 Darwin/13.0.0" - "0.000"
0.000
101.254.206.41 - - [19/Jun/2015:09:58:25 +0800] "GET /wap/data/1.json HTTP/1.1" 304 0 "-" "android-async-http/1.4.3 (http://loopj.com/android-async-htt
p)" - "0.000" 0.000
61.179.107.132 - - [19/Jun/2015:09:58:25 +0800] "GET /wap/data/t_index.html?jsonname=rdht/200000441719.json&class=zxts HTTP/1.1" 200 2587 "/wap/news/1.html" "Mozilla/5.0 (iPad; CPU OS 8_2 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Mobile/12D508" - "4.806" 4.806
61.179.107.103 - - [19/Jun/2015:09:58:25 +0800] "GET /wap/data/1.json HTTP/1.1" 304 0 "-" "android-async-http/1.4.3 (http://loopj.com/android-async-htt
p)" - "4.801" 4.801

查询结果文件内容

218.92.220.156 [19/Jun/2015:09:58:25 "GET /wap/news/ipad2_kxcwsj.html 4.822
61.179.107.29 [19/Jun/2015:09:58:25 "GET /wap/news/finance-dividends.html 4.805
61.179.107.32 [19/Jun/2015:09:58:25 "GET /wap/data/t_index.html?jsonname=rdht/200000441719.json&class=zxts 4.806
61.179.107.103 [19/Jun/2015:09:58:25 "GET /wap/data/1.json 4.801
218.92.220.58 [19/Jun/2015:09:58:25 "GET /wap/news/ipad2_kxcwsj.html 4.781
115.231.82.33 [19/Jun/2015:09:58:25 "GET /wap/data/gsxw/1.json 4.777
61.179.107.129 [19/Jun/2015:09:58:25 "GET /wap/news/trader-index.html 4.773
222.88.95.136 [19/Jun/2015:09:58:25 "GET /wap/news/finance-indicators.html 4.771
218.92.220.59 [19/Jun/2015:09:58:25 "GET /wap/data/index.json 4.786
101.254.206.47 [19/Jun/2015:09:58:25 "GET /wap/data/1.json 4.767

4、可以将以上几个命令都加入到一个分析脚本

编写 loganalysis.sh 脚本

 1 #!/bin/bash
 2 
 3 if [ $1 ==" " ]
 4 then
 5        echo "Usage: ./loganalysis.sh logname"
 6 exit 0
 7 else
 8        cat $1 | awk '{print $NF}'  | awk -F "\"" '{print $2}'  >  time.txt
 9        echo "split request_time over!!!"
10 
11        paste  -d " " $1 time.txt > new.log
12        echo "build new logfile over!!!"
13 
14        awk '($NF>1){print$1" "$4" "$6" "$7" "$NF}' new.log > slowtime.txt
15        echo "please see slowtime in slowtime.txt!!!" 
16 
17        rm -f time.txt
18        rm -f new.log
19 fi