一、开发者的自测利器-Hprof命令
1、示例演示
例子程序:
<code class="hljs java has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;"><span class="hljs-javadoc" style="color: rgb(136, 0, 0); box-sizing: border-box;">/** * PROJECT_NAME: test * DATE: 16/7/22 * CREATE BY: chao.cheng **/</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-class" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">class</span> <span class="hljs-title" style="box-sizing: border-box; color: rgb(102, 0, 102);">HProfTest</span> {</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> <span class="hljs-title" style="box-sizing: border-box;">slowMethod</span>() { <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">try</span> { Thread.sleep(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1000</span>); } <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">catch</span> (Exception e) { e.printStackTrace(); } } <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> <span class="hljs-title" style="box-sizing: border-box;">slowerMethod</span>() { <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">try</span> { Thread.sleep(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10000</span>); } <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">catch</span> (Exception e) { e.printStackTrace(); } } <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">static</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> <span class="hljs-title" style="box-sizing: border-box;">main</span>(String[] args) { HProfTest test = <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">new</span> HProfTest(); test.slowerMethod(); test.slowMethod(); } }</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li></ul>
注:这是一段测试代码通过sleep方法进行延时,在程序运行过程中很慢,我想知道到底是哪段程序影响的整体性能呢?
我在这个Java程序中,加了如下运行参数:
<code class="hljs haml has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">-<span class="ruby" style="box-sizing: border-box;"><span class="hljs-symbol" style="color: rgb(0, 102, 102); box-sizing: border-box;">agentlib:</span>hprof=cpu=times,interval=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span> </span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">/* </span> times:java函数的执行时间 hprof=cpu是针对cpu统计时间 interval=10 采样10次 */</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>
再次运行这段程序显示如下图:
这时候还发现在工程目录里面,多了一个文本文件java.hprof.txt,如下图所示:
内容如下:
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">CPU TIME (ms) BEGIN (total = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11542</span>) Fri Jul <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">00</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">34</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2016</span> rank self accum count trace method <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">86.65</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">86.65</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">303422</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">com</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.test</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.HProfTest</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.slowerMethod</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8.66</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">95.31</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">303423</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">com</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.test</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.HProfTest</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.slowMethod</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.25</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">95.56</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">36</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">300745</span> java<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.util</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.zip</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ZipFile</span>.<init> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.20</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">95.76</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">36</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">300434</span> java<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.lang</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.String</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.equals</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.13</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">95.89</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">301138</span> java<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.net</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.URLStreamHandler</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.parseURL</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.11</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">96.01</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">301339</span> java<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.net</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.URLClassLoader</span>$1<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.run</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.10</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">96.10</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">14</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">301124</span> java<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.lang</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.String</span>.<init> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.09</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">96.19</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3407</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">300355</span> java<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.lang</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.String</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.charAt</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">9</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.08</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">96.27</span>% <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">36</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">300443</span> java<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.io</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.UnixFileSystem</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.normalize</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li></ul>
注:通过上面内容可以看到,哪个类的方法执行时间长,耗费了cpu时间,一目了然,方便我们快速定位问题。
2、命令的具体讲解
hprof不是独立的监控工具,它只是一个Java agent工具,它可以用在监控Java应用程序在运行时的CPU信息和堆内容,使用java -agentlib:hprof=help
命令可以查看hprof的使用文档。
通过上图可以看到这个工具非常强大,可以统计的东西很多,上面的例子统计的是cpu时间,同样我们还可以统计内存占用的dump信息。
如:-agentlib:hprof=heap,format=b,file=/test.hprof
这个hprof小工具,非常方便我们在用JUnit自测代码的时候结合使用,既可以解决业务上的BUG,又能够在一定程序上解决可发现的性能问题,非常实用。
二、性能排查工具-pidstat
1、示例演示
例子程序:
<code class="hljs java has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;"><span class="hljs-javadoc" style="color: rgb(136, 0, 0); box-sizing: border-box;">/** * PROJECT_NAME: test * DATE: 16/7/22 * CREATE BY: chao.cheng **/</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-class" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">class</span> <span class="hljs-title" style="box-sizing: border-box; color: rgb(102, 0, 102);">PidstatTest</span> {</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">static</span> <span class="hljs-class" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">class</span> <span class="hljs-title" style="box-sizing: border-box; color: rgb(102, 0, 102);">PidstatTask</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">implements</span> <span class="hljs-title" style="box-sizing: border-box; color: rgb(102, 0, 102);">Runnable</span> {</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> <span class="hljs-title" style="box-sizing: border-box;">run</span>() { <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">while</span>(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">true</span>) { <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">double</span> value = Math.random() * Math.random(); } } } <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">static</span> <span class="hljs-class" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">class</span> <span class="hljs-title" style="box-sizing: border-box; color: rgb(102, 0, 102);">LazyTask</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">implements</span> <span class="hljs-title" style="box-sizing: border-box; color: rgb(102, 0, 102);">Runnable</span> {</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> <span class="hljs-title" style="box-sizing: border-box;">run</span>() { <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">try</span> { <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">while</span> (<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">true</span>) { Thread.sleep(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1000</span>); } } <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">catch</span> (Exception e) { e.printStackTrace(); } } } <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">static</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> <span class="hljs-title" style="box-sizing: border-box;">main</span>(String[] args) { <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">new</span> Thread(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">new</span> PidstatTask()).start(); <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">new</span> Thread(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">new</span> LazyTask()).start(); <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">new</span> Thread(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">new</span> LazyTask()).start(); } }</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li></ul>
注:这是一段测试用的java程序,将其运行起来。
在命令行输入:
<code class="hljs haml has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">pidstat -p 843 1 3 -u -t <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">/* </span> -<span class="ruby" style="box-sizing: border-box;">u:代表对cpu使用率的监控 </span>参数1 3:表示每秒采样一次,一共三次 -<span class="ruby" style="box-sizing: border-box;">t:将监控级别细化到线程 </span>*/</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>
运行命令显示如下图所示:
注:其实中TID就是线程ID,%usr表示用户线程使用率,从图中可以看到855这个线程占用cpu非常的高。
再输入如下命令:
<code class="hljs bash has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">jstack <span class="hljs-operator" style="box-sizing: border-box;">-l</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">843</span> > /tmp/testlog.txt</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>
查看testlog.txt显示如下部分内容:
注:我们关注的是日志文件的NID这个字段,它对应的就是我们上面说的TID,NID是TID的16进制表示,将上面的十进制855转换成十六进制为357,在日志中进行搜索看到如下内容:
<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;"><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"Thread-0"</span> prio=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span> tid=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0x00007f7d90103800</span> nid=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0x357</span> runnable [<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0x00007f7d943d5000</span>] java<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.lang</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Thread</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.State</span>: RUNNABLE at PidstatTest$PidstatTask<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.run</span>(PidstatTest<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.java</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">13</span>) at java<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.lang</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Thread</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.run</span>(Thread<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.java</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">722</span>) Locked ownable synchronizers: - None</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li></ul>
以此可以推断出有性能瓶颈的程序点。
2、pidstat具体命令详解
pidstat是一个功能非常强大的性能监测工具,他是Sysstat的组件之一,可以从http://sebastien.godard.pagesperso-orange.fr/download.html 进行下载,下载后可以通过./configure
等命令进行安装,这个命令的强大之处在于不仅可以监控进程的性能情况,也可以监控线程的性能情况。
pidstat监控cpu常用显示字段内容如下:
<code class="hljs perl has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;"><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>、PID - 被监控的任务的进程号 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>、<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">%usr</span> - 当在用户层执行(应用程序)时这个任务的cpu使用率,和 nice 优先级无关。注意这个字段计算的cpu时间不包括在虚拟处理器中花去的时间。 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>、<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">%system</span> - 这个任务在系统层使用时的cpu使用率。 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span>、<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">%guest</span> - 任务花费在虚拟机上的cpu使用率(运行在虚拟处理器)。 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>、<span class="hljs-variable" style="color: rgb(102, 0, 102); box-sizing: border-box;">%CPU</span> - 任务总的cpu使用率。在SMP环境(多处理器)中,如果在命令行中输入-I参数的话,cpu使用率会除以你的cpu数量。 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span>、CPU - 正在运行这个任务的处理器编号。 <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7</span>、Command - 这个任务的命令名称。</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li></ul>
pidstat监控io常用的字段显示内容如下:
<code class="hljs has-numbering" style="display: block; padding: 0px; background-color: transparent; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-top-left-radius: 0px; border-top-right-radius: 0px; border-bottom-right-radius: 0px; border-bottom-left-radius: 0px; word-wrap: normal; background-position: initial initial; background-repeat: initial initial;">1、kB_rd/s - 任务从硬盘上的读取速度(kb) 2、kB_wr/s - 任务向硬盘中的写入速度(kb) 3、kB_ccwr/s - 任务写入磁盘被取消的速率(kb)</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>
三、一个内存溢出案例分析
1、内存溢出现象
系统共有8台服务器,每次随机只有一台服务器报java.lang.OutOfMemoryError: GC overhead limit exceeded
错误,然后接着就报内存溢出错误java.lang.OutOfMemoryError: Java heap space
。
2、理论支撑
我们先解释一下什么是GC overhead limit exceeded错误。
GC overhead limt exceed检查是Hotspot VM 1.6定义的一个策略,通过统计GC时间来预测是否要OOM了,提前抛出异常,防止OOM发生。Sun 官方对此的定义是:“并行/并发回收器在GC回收时间过长时会抛出OutOfMemroyError。过长的定义是,超过98%的时间用来做GC并且回收了不到2%的堆内存。用来避免内存过小造成应用不能正常工作。
可以看到当堆中的对象无法被收回的时候,就提前遇警报出这样的错误,此时内存并没有溢出,这个特性在JDK中是默认添加的。
3、DUMP文件分析
将dump文件导入VisualVM工具中,如下图所示:
通过上图可以看出类结构图中,最占用内存的是char[],LinkedHashMap和String三项。但是这三项的实例数并没有占满,看样子不会内存溢出,怎么才能具体分析呢?原因就在于GC overhead limt exceed,这个错并不会在内存真正溢出才会报,所以通过dump文件,我们只能自己去判断分析,哪些项有可能会造成溢出,我们进入char[]项具体来看,会发现里面有很多hessian的url字符被缓存,通过排除程序可以看到由于底层中间件程序为了提高“性能”,将每次调用的url都缓存起来,不用每次都生成,但没有相应缓存释放操作,于是造成了大量字符对象长期持有从而报错,在此就不截图来具体看代码,涉及一些公司信息。
4、问题解决方案
* 可以添加JVM的启动参数来去掉提前报警限制:-XX:-UseGCOverheadLimit,于其让应用每次都提前报警,还不如让暴风雨来的更猛些,直接内存溢出,因为服务器是集群,其中一台挂掉不会影响线上正常交易,同时也方便我们通过日志来排错。
- 通过排查程序,检查系统是否有使用大内存的代码不释放或死循环。