elasticsearch性能优化笔记(2)—— 单点压力测试(粗略)

本部分主要对ES的一套测试流程有一个熟悉,不断分析和学习各个指标和统计方法,找到性能瓶颈

一、装备

        Linux环境

        ElasticSearch单结点,无副本单分片

        Kibana + x-pack 监控集群

        jmeter压测

        nmon监测Linux性能

 

二、服务器环境:

# cat /proc/version
Linux version 3.10.0-514.2.2.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Tue Dec 6 23:06:41 UTC 2016

----------------------------------------------------------------------------------------------------------------

# lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                8
On-line CPU(s) list:   0-7
Thread(s) per core:    1
Core(s) per socket:    1
座:                 8
NUMA 节点:         1
厂商 ID:           GenuineIntel
CPU 系列:          6
型号:              42
型号名称:        Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
步进:              1
CPU MHz:             2394.454
BogoMIPS:            4788.90
超管理器厂商:  KVM
虚拟化类型:     完全
L1d 缓存:          32K
L1i 缓存:          32K
L2 缓存:           4096K

----------------------------------------------------------------------------------------------------------------

# free -m
              total        used        free      shared  buff/cache   available
Mem:          32013       16583        1027         198       14402       14461
Swap:             0           0           0

----------------------------------------------------------------------------------------------------------------

# lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vda    253:0    0   20G  0 disk
└─vda1 253:1    0   20G  0 part /
vdb    253:16   0  200G  0 disk
└─vdb1 253:17   0  200G  0 part

# fdisk -l

磁盘 /dev/vda:21.5 GB, 21474836480 字节,41943040 个扇区
Units = 扇区 of 1 * 512 = 512 bytes
扇区大小(逻辑/物理):512 字节 / 512 字节
I/O 大小(最小/最佳):512 字节 / 512 字节
磁盘标签类型:dos
磁盘标识符:0x000a0566

  设备 Boot      Start         End      Blocks   Id  System
/dev/vda1   *        2048    41943039    20970496   83  Linux

磁盘 /dev/vdb:214.7 GB, 214748364800 字节,419430400 个扇区
Units = 扇区 of 1 * 512 = 512 bytes
扇区大小(逻辑/物理):512 字节 / 512 字节
I/O 大小(最小/最佳):512 字节 / 512 字节
磁盘标签类型:dos
磁盘标识符:0x739398fb

   设备 Boot      Start         End      Blocks   Id  System
/dev/vdb1            2048   419430399   209714176   83  Linux

----------------------------------------------------------------------------------------------------------------

 

三、准备过程

1、elasticsearch5.4.2,kibana,x-pack

elasticsearch即插即用,下下来改改配置就能用;

kibana类似,在配置文件里修改为本地的ES地址即可;

原本是打算用marvel监控,但貌似自5.0之后的ES的HEAD啊,marvel啊这些插件全没了,全集成到x-pack里面了

elasticsearch中运行 elasticsearch-plugin install x-pack

kibana中运行kibana-plugin install x-pack(时间稍有些慢)

即把ES相关的性能图添加到kibana中

2、jmeter

jmeter是java开发的一块集成测试的软件,通过它可以构建测试用例,保存jmx测试用例并交到服务器上执行。

jmeter支持远程启动,在服务器上启动jmeter-server(可能需要参数指定ip),在本机jmeter.properites中添加remote_server:远程ip:1099(1099为默认端口),即可在jmeter运行脚本时使用远程启动。

由于服务器不在局域网内,因此上传jmx文件给服务器使用

操作:

010308_gSOD_2247638.png

右键测试计划,添加线程组,修改线程数,循环次数,运行时间等关键参数

010522_s5NA_2247638.png

在线程组中添加Sampler,选择HTTP请求

010517_qgsi_2247638.png

在HTTP请求中添加聚合报告和结果树

在文件=>保存为测试计划 中保存jmx文件

3、nmon

nmon是一款linux下的系统性能指标工具,提供CPU,内存,磁盘IO,网络等多项指标的界面化展示。

安装:

wget http://sourceforge.net/projects/nmon/files/nmon16d_x86.tar.gz

tar xvzf  nmon16d_x86.tar.gz

得到一系列nmon对应的各种版本的文件

011019_55Rt_2247638.png

找到相应的版本的nmon文件

chmod +x 赋予可执行权限

运行:

010844_azmi_2247638.png

其中每个参数可控制展示的开关,按下c,m,d,n

011137_IGT1_2247638.png

即可监控系统性能

四、开始测试

 ES结点数分片数副本数ES配置  线程数ES数据索引settingJVM参数逐条insert写入速率范围(条/s)5分钟写入数据条数/数据量 
Sample1110A1   AA   
Sample2110A200AA   
Sample3110A1000BAA   
Sample4110A200BAA   
Sample5110A200CAA   
Sample6           
Sample7           

 

 

样本项目描述

——ES配置:

A:

http.cors.enabled: true
http.cors.allow-origin: "*"
xpack.security.enabled: false
indices.memory.index_buffer_size: "50%"
indices.memory.min_index_buffer_size: "1024mb"
http.max_initial_line_length: "64kb"
http.max_header_size: "64kb"
action.search.shard_count.limit: 50000
script.inline: true
script.stored: true
thread_pool.bulk.queue_size: 5000

——JVM参数:

A:

-Xms2g -Xmx2g -XX:+UseConcMarkSweepGC

-XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -XX:+AlwaysPreTouch -server -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8

-Djna.nosys=true

-Djdk.io.permissionsUseCanonicalPath=true

-Dio.netty.noUnsafe=true

-Dio.netty.noKeySetOptimization=true

-Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false

-Dlog4j2.disable.jmx=true -Dlog4j.skipJansi=true -XX:+HeapDumpOnOutOfMemoryError

 

——ES数据:

A:

{
    "_MCSJnlNo": "45139477"
}

B:

{
    "_MCSJnlNo": "45139477",
    "_raw": "2017-05-01 04:51:10.723 [[ACTIVE] ExecuteThread: '15' for queue: 'weblogic.kernel.Default (self-tuning)'] INFO  c.c.p.s.c.t.TransformTransport - recv:<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n<Message>\r\n<Head>\r\n<_MCSJnlNo>45139477</_MCSJnlNo>\r\n<_ResponseSystemId>MCA</_ResponseSystemId>\r\n<_RejCode>000000</_RejCode>\r\n<_RejMsg></_RejMsg>\r\n<_MCHJnlNo>3973735</_MCHJnlNo>\r\n<_TransactionId>per.MCLoanInterestRateQuery</_TransactionId>\r\n<_MCHTimestamp>2017-05-01 04:51:10.61</_MCHTimestamp>\r\n</Head>\r\n<Body>\r\n<InterestRate>10.00</InterestRate>\r\n</Body>\r\n</Message>\r",
    "_context_id": 472,
    "_host": "192.168.0.34",
    "type": "recv",
    "_MCHTimestamp": "2017-05-01 04:51:10.61",
    "_MCHJnlNo": "3973735",
    "_RejCode": "000000",
    "_ResponseSystemId": "MCA",
    "InterestRate": "10.00",
    "_TransactionId": "per.MCLoanInterestRateQuery",
    "_ts": 1493585470000
}

C:

{
    "_raw": "[2017-07-23 22:09:49,435]  [elasticsearch]   [DEBUG]   [< {\"took\":12,\"timed_out\":false,\"_shards\":{\"total\":10,\"successful\":10,\"failed\":0},\"hits\":{\"total\":196699,\"max_score\":null,\"hits\":[{\"_index\":\"ls_749bc4e976f24f6f9fee21e4bc6c3e5b\",\"_type\":\"20170723\",\"_id\":\"AV1vyDSJVct_8CDGz3QT\",\"_score\":null,\"_source\":{\"_raw\":\"2017-07-23 22:09:35 test_longtime1/longtimetxt.0 three line log for one 296963 \",\"_context_id\":1088331,\"_host\":\"192.168.0.33\",\"_ts\":1500818975000},\"highlight\":{\"_raw\":[\"2017-07-23 22:09:35 test_longtime1/longtimetxt.0 three line log for <em>one</em> 296963 \"]},\"sort\":[1500818975000]},{\"_index\":\"ls_749bc4e976f24f6f9fee21e4bc6c3e5b\",\"_type\":\"20170723\",\"_id\":\"AV1vyDSJVct_8CDGz3QQ\",\"_score\":null,\"_source\":{\"_raw\":\"2017-07-23 22:09:34 test_longtime1/longtimetxt.0 three line log for one 296962 \",\"_context_id\":1088328,\"_host\":\"192.168.0.33\",\"_ts\":1500818974000},\"highlight\":{\"_raw\":[\"2017-07-23 22:09:34 test_longtime1/longtimetxt.0 three line log for <em>one</em> 296962 \"]},\"sort\":[1500818974000]},{\"_index\":\"ls_749bc4e976f24f6f9fee21e4bc6c3e5b\",\"_type\":\"20170723\",\"_id\":\"AV1vyDSJVct_8CDGz3QN\",\"_score\":null,\"_source\":{\"_raw\":\"2017-07-23 22:09:33 test_longtime1/longtimetxt.0 three line log for one 296961 \",\"_context_id\":1088325,\"_host\":\"192.168.0.33\",\"_ts\":1500818973000},\"highlight\":{\"_raw\":[\"2017-07-23 22:09:33 test_longtime1/longtimetxt.0 three line log for <em>one</em> 296961 \"]},\"sort\":[1500818973000]},{\"_index\":\"ls_749bc4e976f24f6f9fee21e4bc6c3e5b\",\"_type\":\"20170723\",\"_id\":\"AV1vyDSJVct_8CDGz3OF\",\"_score\":null,\"_source\":{\"_raw\":\"2017-07-23 22:09:32 test_longtime1/longtimetxt.0 three line log for one 296960 \",\"_context_id\":1088322,\"_host\":\"192.168.0.33\",\"_ts\":1500818972000},\"highlight\":{\"_raw\":[\"2017-07-23 22:09:32 test_longtime1/longtimetxt.0 three line log for <em>one</em> 296960 \"]},\"sort\":[1500818972000]},{\"_index\":\"ls_749bc4e976f24f6f9fee21e4bc6c3e5b\",\"_type\":\"20170723\",\"_id\":\"AV1vyDSJVct_8CDGz3OC\",\"_score\":null,\"_source\":{\"_raw\":\"2017-07-23 22:09:31 test_longtime1/longtimetxt.0 three line log for one 296959 \",\"_context_id\":1088319,\"_host\":\"192.168.0.33\",\"_ts\":1500818971000},\"highlight\":{\"_raw\":[\"2017-07-23 22:09:31 test_longtime1/longtimetxt.0 three line log for <em>one</em> 296959 \"]},\"sort\":[1500818971000]},{\"_index\":\"ls_749bc4e976f24f6f9fee21e4bc6c3e5b\",\"_type\":\"20170723\",\"_id\":\"AV1vyDSJVct_8CDGz3N_\",\"_score\":null,\"_source\":{\"_raw\":\"2017-07-23 22:09:30 test_longtime1/longtimetxt.0 three line log for one 296958 \",\"_context_id\":1088316,\"_host\":\"192.168.0.33\",\"_ts\":1500818970000},\"highlight\":{\"_raw\":[\"2017-07-23 22:09:30 test_longtime1/longtimetxt.0 three line log for <em>one</em> 296958 \"]},\"sort\":[1500818970000]},{\"_index\":\"ls_749bc4e976f24f6f9fee21e4bc6c3e5b\",\"_type\":\"20170723\",\"_id\":\"AV1vyDSJVct_8CDGz3Lx\",\"_score\":null,\"_source\":{\"_raw\":\"2017-07-23 22:09:29 test_longtime1/longtimetxt.0 three line log for one 296957 \",\"_context_id\":1088313,\"_host\":\"192.168.0.33\",\"_ts\":1500818969000},\"highlight\":{\"_raw\":[\"2017-07-23 22:09:29 test_longtime1/longtimetxt.0 three line log for <em>one</em> 296957 \"]},\"sort\":[1500818969000]},{\"_index\":\"ls_749bc4e976f24f6f9fee21e4bc6c3e5b\",\"_type\":\"20170723\",\"_id\":\"AV1vyDSJVct_8CDGz3Lu\",\"_score\":null,\"_source\":{\"_raw\":\"2017-07-23 22:09:28 test_longtime1/longtimetxt.0 three line log for one 296956 \",\"_context_id\":1088310,\"_host\":\"192.168.0.33\",\"_ts\":1500818968000},\"highlight\":{\"_raw\":[\"2017-07-23 22:09:28 test_longtime1/longtimetxt.0 three line log for <em>one</em> 296956 \"]},\"sort\":[1500818968000]},{\"_index\":\"ls_749bc4e976f24f6f9fee21e4bc6c3e5b\",\"_type\":\"20170723\",\"_id\":\"AV1vyDSJVct_8CDGz3Lr\",\"_score\":null,\"_source\":{\"_raw\":\"2017-07-23 22:09:27 test_longtime1/longtimetxt.0 three line log for one 296955 \",\"_context_id\":1088307,\"_host\":\"192.168.0.33\",\"_ts\":1500818967000},\"highlight\":{\"_raw\":[\"2017-07-23 22:09:27 test_longtime1/longtimetxt.0 three line log for <em>one</em> 296955 \"]},\"sort\":[1500818967000]},{\"_index\":\"ls_749bc4e976f24f6f9fee21e4bc6c3e5b\",\"_type\":\"20170723\",\"_id\":\"AV1vyDSJVct_8CDGz3Jr\",\"_score\":null,\"_source\":{\"_raw\":\"2017-07-23 22:09:26 test_longtime1/longtimetxt.0 three line log for one 296954 \",\"_context_id\":1088304,\"_host\":\"192.168.0.33\",\"_ts\":1500818966000},\"highlight\":{\"_raw\":[\"2017-07-23 22:09:26 test_longtime1/longtimetxt.0 three line log for <em>one</em> 296954 \"]},\"sort\":[1500818966000]},{\"_index\":\"ls_749bc4e976f24f6f9fee21e4bc6c3e5b\",\"_type\":\"20170723\",\"_id\":\"AV1vyDSJVct_8CDGz3Jo\",\"_score\":null,\"_source\":{\"_raw\":\"2017-07-23 22:09:25 test_longtime1/longtimetxt.0 three line log for one 296953 \",\"_context_id\":1088301,\"_host\":\"192.168.0.33\",\"_ts\":1500818965000},\"highlight\":{\"_raw\":[\"2017-07-23 22:09:25 test_longtime1/longtimetxt.0 three line log for <em>one</em> 296953 \"]},\"sort\":[1500818965000]},{\"_index\":\"ls_749bc4e976f24f6f9fee21e4bc6c3e5b\",\"_type\":\"20170723\",\"_id\":\"AV1vyDSJVct_8CDGz3Jl\",\"_score\":null,\"_source\":{\"_raw\":\"2017-07-23 22:09:24 test_longtime1/longtimetxt.0 three line log for one 296952 \",\"_context_id\":1088298,\"_host\":\"192.168.0.33\",\"_ts\":1500818964000},\"highlight\":{\"_raw\":[\"2017-07-23 22:09:24 test_longtime1/longtimetxt.0 three line log for <em>one</em> 296952 \"]},\"sort\":[1500818964000]},{\"_index\":\"ls_749bc4e976f24f6f9fee21e4bc6c3e5b\",\"_type\":\"20170723\",\"_id\":\"AV1vyAW0Vct_8CDGz3Hl\",\"_score\":null,\"_source\":{\"_raw\":\"2017-07-23 22:09:23 test_longtime1/longtimetxt.0 three line log for one 296951 \",\"_context_id\":1088295,\"_host\":\"192.168.0.33\",\"_ts\":1500818963000},\"highlight\":{\"_raw\":[\"2017-07-23 22:09:23 test_longtime1/longtimetxt.0 three line log for <em>one</em> 296951 \"]},\"sort\":[1500818963000]},{\"_index\":\"ls_749bc4e976f24f6f9fee21e4bc6c3e5b\",\"_type\":\"20170723\",\"_id\":\"AV1vyAW0Vct_8CDGz3Hi\",\"_score\":null,\"_source\":{\"_raw\":\"2017-07-23 22:09:22 test_longtime1/longtimetxt.0 three line log for one 296950 \",\"_context_id\":1088292,\"_host\":\"192.168.0.33\",\"_ts\":1500818962000},\"highlight\":{\"_raw\":[\"2017-07-23 22:09:22 test_longtime1/longtimetxt.0 three line log for <em>one</em> 296950 \"]},\"sort\":[1500818962000]},{\"_index\":\"ls_749bc4e976f24f6f9fee21e4bc6c3e5b\",\"_type\":\"20170723\",\"_id\":\"AV1vyAW0Vct_8CDGz3Hf\",\"_score\":null,\"_source\":{\"_raw\":\"2017-07-23 22:09:21 test_longtime1/longtimetxt.0 three line log for one 296949 \",\"_context_id\":1088289,\"_host\":\"192.168.0.33\",\"_ts\":1500818961000},\"highlight\":{\"_raw\":[\"2017-07-23 22:09:21 test_longtime1/longtimetxt.0 three line log for <em>one</em> 296949 \"]},\"sort\":[1500818961000]},{\"_index\":\"ls_749bc4e976f24f6f9fee21e4bc6c3e5b\",\"_type\":\"20170723\",\"_id\":\"AV1vyAW0Vct_8CDGz3FU\",\"_score\":null,\"_source\":{\"_raw\":\"2017-07-23 22:09:20 test_longtime1/longtimetxt.0 three line log for one 296948 \",\"_context_id\":1088286,\"_host\":\"192.168.0.33\",\"_ts\":1500818960000},\"highlight\":{\"_raw\":[\"2017-07-23 22:09:20 test_longtime1/longtimetxt.0 three line log for <em>one</em> 296948 \"]},\"sort\":[1500818960000]},{\"_index\":\"ls_749bc4e976f24f6f9fee21e4bc6c3e5b\",\"_type\":\"20170723\",\"_id\":\"AV1vyAW0Vct_8CDGz3FR\",\"_score\":null,\"_source\":{\"_raw\":\"2017-07-23 22:09:19 test_longtime1/longtimetxt.0 three line log for one 296947 \",\"_context_id\":1088283,\"_host\":\"192.168.0.33\",\"_ts\":1500818959000},\"highlight\":{\"_raw\":[\"2017-07-23 22:09:19 test_longtime1/longtimetxt.0 three line log for <em>one</em> 296947 \"]},\"sort\":[1500818959000]},{\"_index\":\"ls_749bc4e976f24f6f9fee21e4bc6c3e5b\",\"_type\":\"20170723\",\"_id\":\"AV1vyAW0Vct_8CDGz3FO\",\"_score\":null,\"_source\":{\"_raw\":\"2017-07-23 22:09:18 test_longtime1/longtimetxt.0 three line log for one 296946 \",\"_context_id\":1088280,\"_host\":\"192.168.0.33\",\"_ts\":1500818958000},\"highlight\":{\"_raw\":[\"2017-07-23 22:09:18 test_longtime1/longtimetxt.0 three line log for <em>one</em> 296946 \"]},\"sort\":[1500818958000]},{\"_index\":\"ls_749bc4e976f24f6f9fee21e4bc6c3e5b\",\"_type\":\"20170723\",\"_id\":\"AV1vyAW0Vct_8CDGz3Cr\",\"_score\":null,\"_source\":{\"_raw\":\"2017-07-23 22:09:17 test_longtime1/longtimetxt.0 three line log for one 296945 \",\"_context_id\":1088277,\"_host\":\"192.168.0.33\",\"_ts\":1500818957000},\"highlight\":{\"_raw\":[\"2017-07-23 22:09:17 test_longtime1/longtimetxt.0 three line log for <em>one</em> 296945 \"]},\"sort\":[1500818957000]},{\"_index\":\"ls_749bc4e976f24f6f9fee21e4bc6c3e5b\",\"_type\":\"20170723\",\"_id\":\"AV1vyAW0Vct_8CDGz3Co\",\"_score\":null,\"_source\":{\"_raw\":\"2017-07-23 22:09:16 test_longtime1/longtimetxt.0 three line log for one 296944 \",\"_context_id\":1088274,\"_host\":\"192.168.0.33\",\"_ts\":1500818956000},\"highlight\":{\"_raw\":[\"2017-07-23 22:09:16 test_longtime1/longtimetxt.0 three line log for <em>one</em> 296944 \"]},\"sort\":[1500818956000]}]}}]",
    "_context_id": 1247368,
    "_host": "192.168.0.34",
    "_ts": 1500818989000,

    "data1": 重复raw,

    "data2": 重复raw,

    "data3": 重复raw,

    "data4": 重复raw,

    "data5": 重复raw,

    "data6": 重复raw,

    "data7": 重复raw,

    "data8": 重复raw,

    "data9": 重复raw,

    "data10": 重复raw,
  }

——索引settings:

A:

PUT index1


{
        "template": "*",
        "order": 1,
        "settings": {
            "number_of_shards": 1,
            "number_of_replicas": 0,
            "index.query.default_field": "_raw"
        },
        "mappings": {
            "_default_": {
                "date_detection": false,
                "numeric_detection": false,
                "_all": {
                    "enabled": false
                },
                "properties": {
                    "_host": {
                        "type": "keyword",
                        "norms": false
                    },
                    "_raw": {
                        "type": "text"
                    },
                    "_ts": {
                        "type": "long"
                    },
                    "_context_id": {
                        "type": "long"
                    }
                },
                "dynamic_templates": [
                    {
                        "strings_as_keywords": {
                            "match_mapping_type": "string",
                            "mapping": {
                                "type": "keyword"
                            }
                        }
                    }
                ]
            }
        }
    }

----------------------------------------------------------------------------------------------------------------

Sample 1:

 

16:53:07 —— 16:58:07

性能抽样:

165343_lP4p_2247638.png

165555_iCTZ_2247638.png

写入日志:

165829_4gs6_2247638.png

数据量:

165854_x3nR_2247638.png

x-pack图:

170129_kTpw_2247638.png

----------------------------------------------------------------------------------------------------------------

Sample 2:

17:07:15 —— 17:12:15

性能抽样:

170822_nijT_2247638.png

170857_MKq2_2247638.png

写入日志:

171235_ceFq_2247638.png

数据量:

171319_kf7K_2247638.png

x-pack图:

171300_rQZW_2247638.png

----------------------------------------------------------------------------------------------------------

Sample 3:

17:24:03——17:29:03

性能抽样:

172433_Ld4l_2247638.png

172510_ViRd_2247638.png

写入日志:

172934_gG7z_2247638.png

数据量:

172945_V1V5_2247638.png

x-pack图:

172955_rv26_2247638.png

 

----------------------------------------------------------------------------------------------------------

Sample 4:

23:07:04——23:12:04

性能抽样:

231308_Rngy_2247638.png

231338_sWZ2_2247638.png

 

写入日志

232942_siEc_2247638.png

数据量:

233119_qa8m_2247638.png

x-pack图:

233003_cH1l_2247638.png

----------------------------------------------------------------------------------------------------------

Sample 5:

23:35:06——23:40:06

性能抽样:

233522_upFy_2247638.png

 

233642_sPkB_2247638.png

写入日志:

234049_Xhsk_2247638.png

数据量

234123_IRS7_2247638.png

x-pack图:

234144_pBVD_2247638.png

转载于:https://my.oschina.net/u/2247638/blog/1528050

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值