elasticsearch ：介绍文档

 http://www.cnblogs.com/chowmin/articles/4629220.html 
一、下载文件：

wget https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.3.5/elasticsearch-2.3.5.tar.gz



二、拷贝到三个节点

mv elasticsearch-2.3.5 /wdzj/elasticsearch-node  
cd /wdzj/elasticsearch-node1
cp -rf /wdzj/elasticsearch-node1 /wdzj/elasticsearch-node2
cp -rf /wdzj/elasticsearch-node1 /wdzj/elasticsearch-node3

三、配置文件修改

节点1部署：
配置elasticsearch，放开注释并且修改如下配置：
[root@lvnian wdzj]# egrep -v '#|^$' elasticsearch_node1/config/elasticsearch.yml  
cluster.name: wdzj-elasticsearch    #配置es的集群名称，默认是elasticsearch
node.name: node-1                   #该节点的名字 
bootstrap.mlockall: true            #设置为true来锁住内存
network.host: 192.168.11.46            #该参数用于同时设置bind_host和publish_host
http.port: 9200                        #设置对外服务的http端口号
transport.tcp.port: 9300            #设置节点之间交互的端口号
discovery.zen.ping.unicast.hosts: ["192.168.11.46:9500" , "192.168.11.46:9700"]  #设置集群中的Master节点的初始列表，可以通过这些节点来自动发现其他新加入集群的节点
节点2部署：
[root@lvnian wdzj]# egrep -v '#|^$' elasticsearch_node2/config/elasticsearch.yml  
cluster.name: wdzj-elasticsearch
node.name: node-2
bootstrap.mlockall: true
network.host: 192.168.11.46
http.port: 9400
transport.tcp.port: 9500
discovery.zen.ping.unicast.hosts: ["192.168.11.46:9300" , "192.168.11.46:9700"]
节点3部署：
[root@lvnian wdzj]# egrep -v '#|^$' elasticsearch_node3/config/elasticsearch.yml  
cluster.name: wdzj-elasticsearch
node.name: node-3
bootstrap.mlockall: true
network.host: 192.168.11.46
http.port: 9600
transport.tcp.port: 9700
discovery.zen.ping.unicast.hosts: ["192.168.11.46:9300" , "192.168.11.46:9500"]


三、启动以及检查

普通用户启动：

bin/elasticsearch -d    #（后台启动）  普通用户启动！

Root用户启动：

bin/elasticsearch -Des.insecure.allow.root=true -d

然后访问http://192.168.11.46:9200/_cat/health查看节点状态
{"cluster_name":"wdzj-elasticsearch","status":"green","timed_out":false,"number_of_nodes":2,"number_of_data_nodes":2,"active_primary_shards":5,"active_shards":10,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":100.0}
复制文件夹到其他节点，相应修改配置后查看节点http://192.168.11.46:9200/_cat/nodes

查看端口是否启动

[root@lvnian wdzj]# netstat -lntp|grep 9[2-7]00
tcp        0      0 192.168.11.46:9200          0.0.0.0:*                   LISTEN      15717/java          
tcp        0      0 192.168.11.46:9300          0.0.0.0:*                   LISTEN      15717/java          
tcp        0      0 192.168.11.46:9400          0.0.0.0:*                   LISTEN      15853/java          
tcp        0      0 192.168.11.46:9500          0.0.0.0:*                   LISTEN      15853/java  
tcp        0      0 192.168.11.46:9600          0.0.0.0:*                   LISTEN      15853/java  
tcp        0      0 192.168.11.46:9700          0.0.0.0:*                   LISTEN      15853/java

检查进程是否正常

[root@lvnian wdzj]# ps -ef|grep elk
root     26034 20476  0 16:52 pts/0    00:00:00 grep elk
[root@iZ2360rplyoZ wdzj]# ps -ef|grep el
root        44     2  0 May16 ?        00:00:08 [khelper]
root      1542     1  0 May16 ?        00:00:00 /usr/sbin/gshelld
root     15717     1  1 16:25 pts/0    00:00:24 /wdzj/java1.8/bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/wdzj/elasticsearch_node1 -cp /wdzj/elasticsearch_node1/lib/elasticsearch-2.3.5.jar:/wdzj/elasticsearch_node1/lib/* org.elasticsearch.bootstrap.Elasticsearch start -Des.insecure.allow.root=true -d
root     15853     1  1 16:26 pts/0    00:00:24 /wdzj/java1.8/bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/wdzj/elasticsearch_node2 -cp /wdzj/elasticsearch_node2/lib/elasticsearch-2.3.5.jar:/wdzj/elasticsearch_node2/lib/* org.elasticsearch.bootstrap.Elasticsearch start -Des.insecure.allow.root=true -d
root     15853     1  1 16:26 pts/0    00:00:24 /wdzj/java1.8/bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/wdzj/elasticsearch_node3 -cp /wdzj/elasticsearch_node3/lib/elasticsearch-2.3.5.jar:/wdzj/elasticsearch_node2/lib/* org.elasticsearch.bootstrap.Elasticsearch start -Des.insecure.allow.root=true -d


注意：
1.java 需要时1.8以上，否则出现类似如下错误：

[2016-08-25 15:41:17,507][ERROR][bootstrap                ] Exception
java.lang.RuntimeException: Java version: Oracle Corporation 1.7.0_40 [Java HotSpot(TM) 64-Bit Server VM 24.0-b56] suffers from critical bug https://bugs.openjdk.java.net/browse/JDK-8024830 which can cause data corruption.
Please upgrade the JVM, see http://www.elastic.co/guide/en/elasticsearch/reference/current/_installation.html for current recommendations.
If you absolutely cannot upgrade, please add -XX:-UseSuperWord to the JAVA_OPTS environment variable.
Upgrading is preferred, this workaround will result in degraded performance.

2，每行配置文件的冒号必须需要一个空格再写值，已经discovery.zen.ping.unicast.hosts: ["192.168.11.46:9300" , "192.168.11.46:9500"]中的主机之间的逗号前后也必须要一个逗号分隔

3，启动的时候，如果是root用户启动。必须是：
bin/elasticsearch -Des.insecure.allow.root=true -d
不能缺少-Des.insecure.allow.root=true 参数

4.其他坑请参考下面网址：
http://www.cnblogs.com/jiu0821/p/5624908.html



故障解决：

安装elasticsearch
参考：https://my.oschina.net/u/2607319/blog/818688
故障1：
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) ~[elasticsearch-5.3.0.jar:5.3.0]
Caused by: java.lang.RuntimeException: can not run elasticsearch as root
    at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:106) ~[elasticsearch-5.3.0.jar:5.3.0]
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:204) ~[elasticsearch-5.3.0.jar:5.3.0]
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:360) ~[elasticsearch-5.3.0.jar:5.3.0]
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) ~[elasticsearch-5.3.0.jar:5.3.0]
    ... 6 more

    不能使用root运行elasticsearch
    
    
故障2：
[2017-03-30T03:59:15,241][WARN ][i.n.u.i.MacAddressUtil   ] Failed to find a usable hardware address from the network interfaces; using random bytes: 4c:2a:6b:03:3d:63:03:a6
[2017-03-30T03:59:15,303][INFO ][o.e.t.TransportService   ] [rela-elk-node1] publish_address {172.31.1.79:9300}, bound_addresses {[::]:9300}
[2017-03-30T03:59:15,311][INFO ][o.e.b.BootstrapChecks    ] [rela-elk-node1] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
ERROR: bootstrap checks failed
max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2017-03-30T03:59:15,325][INFO ][o.e.n.Node               ] [rela-elk-node1] stopping ...


    vim /etc/security/limits.conf //添加


    * soft nofile 300000
    * hard nofile 300000
    * soft nproc 102400
    * soft memlock unlimited
    * hard memlock unlimited




报错三：

    max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144]

    解决方案：

    vim /etc/sysctl.conf    //添加

    fs.file-max = 1645037
    vm.max_map_count=655360


    执行：sysctl -p



* hard nproc 102400



集群配置文件

cluster.name: rela-elk
node.name: rela-elk-node1
path.data: /data
path.logs: /work/opt/elasticsearch/logs
bootstrap.memory_lock: false
network.host: 172.31.1.79
http.port: 9200
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["172.31.1.116:9300" , "172.31.6.155:9300"]
http.cors.enabled: true 
http.cors.allow-origin: "*"


cluster.name: rela-elk
node.name: rela-elk-node2
path.data: /data
path.logs: /work/opt/elasticsearch/logs
bootstrap.memory_lock: false
network.host: 172.31.6.155
http.port: 9200
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["172.31.1.116:9300" , "172.31.1.79:9300"]
http.cors.enabled: true 
http.cors.allow-origin: "*"



cluster.name: rela-elk
node.name: rela-elk-node3
path.data: /data
path.logs: /work/opt/elasticsearch/logs
bootstrap.memory_lock: false
network.host: 172.31.1.116
http.port: 9200
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["172.31.1.79:9300" , "172.31.6.155:9300"]
http.cors.enabled: true 
http.cors.allow-origin: "*"

###################################



部署单点的elasticsearch
cluster.name: rela-elk
node.name: rela-elk-node1
path.data: /data
path.logs: /work/opt/elasticsearch/logs
bootstrap.memory_lock: false
#network.host: 172.31.1.79
network.host: 0.0.0.0
http.port: 9200
transport.tcp.port: 9300
##discovery.zen.ping.unicast.hosts: ["172.31.1.116:9300" , "172.31.6.155:9300"]
http.cors.enabled: true
http.cors.allow-origin: "*"

如果在用单点的elasticsearch过程中希望该为集群，之需要在新家的服务器上面，修改node.name名称，已经启动下面的参数，其中ip和端口就是除本机以为的其他服务器的ip和对应的端口：
discovery.zen.ping.unicast.hosts: ["172.31.1.116:9300" , "172.31.6.155:9300"]


下面是dockerfile

FROM ubuntusshd##这个是ubuntu服务器，仅仅开启ssh端口，4201
RUN   mkdir -p /work/source  /work/opt ;rm -rf  /work/source/*
COPY  elasticsearch-5.3.0.tar.gz  /work/source/
COPY  elasticsearch-head.tar.gz  /work/source/
COPY  jdk-8u121-linux-x64.tar.gz  /work/source/
COPY  node-v6.10.1-linux-x64.tar.xz  /work/source/
COPY  run.sh /run.sh
WORKDIR /work/source
RUN  ls -l ;pwd
RUN apt-get  install bzip2 gzip xz-utils  -y
RUN  tar xf  elasticsearch-5.3.0.tar.gz -C /work/opt/;mv /work/opt/elasticsearch-5.3.0 /work/opt/elasticsearch
RUN  tar xf  elasticsearch-head.tar.gz  -C /work/opt/
RUN  tar xf  jdk-8u121-linux-x64.tar.gz -C /work/opt/  ; mv /work/opt/jdk1.8.0_121  /work/opt/jdk
RUN  tar xf  node-v6.10.1-linux-x64.tar.xz  -C /work/opt/  ;mv /work/opt/node-v6.10.1-linux-x64 /work/opt/node
RUN  ls  -l /work/opt/jdk/
RUN  ls  -l /work/opt/
RUN  chown -R root.root  /work/opt/
ENV  JAVA_HOME /work/opt/jdk
ENV  JAVA_BIN $JAVA_HOME/bin
ENV  JRE_HOME ${JAVA_HOME}/jre
ENV  CLASSPATH $JAVA_HOME/jre/lib/ext:$JAVA_HOME/lib/tools.jar
ENV  PATH $JAVA_BIN:$PATH
RUN  echo $PATH;java -version
ADD  elasticsearch.yml  /work/opt/elasticsearch/config/elasticsearch.yml
ENV NODE_HOME /work/opt/node/
ENV PATH $PATH:$NODE_HOME/bin
RUN node --version ; echo $PATH ; echo "export PATH=$PATH" >>/etc/profile
RUN npm config set registry https://registry.npm.taobao.org 
RUN cd /work/opt/node/lib/node_modules ; npm install grunt
RUN grunt -version
RUN cd /work/opt/elasticsearch-head ;npm install ;cp Gruntfile.js Gruntfile.js.ori ;sed -i '/port: 9100,/i\hostname: "0.0.0.0",' Gruntfile.js
RUN mkdir /data/ ; chown -R ubuntu.ubuntu /work/opt/elasticsearch /data
RUN sed -i 's#http://localhost:9200#http://127.0.0.1:9200#g'   /work/opt/elasticsearch-head/_site/app.js
RUN chmod +x /run.sh
EXPOSE 4201 
EXPOSE 9100
EXPOSE 9200
EXPOSE 9300
CMD ["/run.sh"]

run.sh 文件内容

#!/bin/sh
nohup su -  ubuntu /work/opt/elasticsearch/bin/elasticsearch >>/tmp/elasticsearch.log  &   #启动9200
sleep 10 
cd /work/opt/elasticsearch-head ;grunt server &            ## 启动9100   lasticsearch-head
/usr/sbin/sshd -D



简单脚本

#!/bin/bash 

# 在本脚本目录下，需要有下面安装包
# jdk-8u121-linux-x64.tar.gz elasticsearch-5.3.0.tar.gz node-v6.10.1-linux-x64.tar.gz
# elasticsearch-head.tar.gz elasticsearch-analysis-ik-5.3.0.zip

ip=`ifconfig |awk -F'[: ]+' '/inet addr:/{print $4}' |egrep  -v '127.0.'`
if [ ! -d /work/opt/ ];then
   mkdir -p /work/opt/
fi

##install  java 1.8
function java1.8(){
    cd /work/source
    tar xf jdk-8u121-linux-x64.tar.gz 
    mkdir /work/opt -p
    mv  jdk1.8.0_121  /work/opt/jdk
    ll /work/opt/jdk/
    chown -R root.root  /work/opt
    cp /etc/profile /etc/profile.`date +%F`
    echo -e  "\n\n###########java#############"     >>/etc/profile
    echo -e "JAVA_HOME=/work/opt/jdk\nJAVA_BIN=\$JAVA_HOME/bin\nJRE_HOME=\${JAVA_HOME}/jre\nCLASSPATH=\$JAVA_HOME/jre/lib/ext:\$JAVA_HOME/lib/tools.jar\nPATH=\$JAVA_BIN:\$PATH\n"     >>/etc/profile
    export JAVA_HOME=/work/opt/jdk
    export JAVA_BIN=$JAVA_HOME/bin
    export JRE_HOME=${JAVA_HOME}/jre
    export CLASSPATH=$JAVA_HOME/jre/lib/ext:$JAVA_HOME/lib/tools.jar
    export PATH=$JAVA_BIN:$PATH
}

## install elasticsearch-5.3.0
function elasticsearch5.3(){
    cd /work/source
    tar xf elasticsearch-5.3.0.tar.gz 
    mv elasticsearch-5.3.0 /work/opt/elasticsearch
    
    ip=`ifconfig |awk -F'[: ]+' '/inet addr:/{print $4}' |egrep  -v '127.0.'`
cat > /work/opt/elasticsearch/config/elasticsearch.yml<<EOF
cluster.name: rela-elasticsearch
node.name: rela-elasticsearch-node-$ip
path.data: /data 
path.logs: /work/opt/elasticsearch/logs   
bootstrap.memory_lock: false
network.host: 0.0.0.0
http.port: 9200
http.cors.enabled: true 
http.cors.allow-origin: "*"

EOF

    ulimit -n 655360
    cp /etc/security/limits.conf /etc/security/limits.conf.`date +%F`
echo -e "* soft nofile 300000\n* hard nofile 300000\n* soft nproc 102400\n* soft memlock unlimited\n* hard memlock unlimited" >> /etc/security/limits.conf
    cp /etc/sysctl.conf /etc/sysctl.conf.`date +%F`
    echo -e "fs.file-max = 1645037\nvm.max_map_count=655360\n" >> /etc/sysctl.conf 
    sysctl -p
    if [ ! -d  /work/opt/elasticsearch/logs ];then
        mkdir -p  /work/opt/elasticsearch/logs
    fi
    if [ ! -d  /data ];then
        mkdir -p  /data
    fi
    chown -R ubuntu.ubuntu /work/opt/elasticsearch /data 
    nohup su - ubuntu /work/opt/elasticsearch/bin/elasticsearch >> /tmp/elasticsearch.log  &
    
    cat >/etc/init.d/elasticsearch  <<EOF
#!/bin/bash 
#
# by gaogd on 20170605
server=\`basename \$0\`
echo \$server
case \$1 in
start)
    echo "start \$server "
    nohup su - ubuntu /work/opt/elasticsearch/bin/elasticsearch >> /tmp/elasticsearch.log  &
    sleep 3
    ps -ef|grep '/work/opt/elasticsearch'
;;

stop) 
    echo "stop \$server "
    ps -ef|grep '/work/opt/elasticsearch' |egrep -v grep |awk '{print \$2}' |xargs kill -9
    sleep 3
    ps -ef|grep '/work/opt/elasticsearch'
;;

*)
    echo "\$0 start|stop"
esac 
EOF
    
    chmod +x /etc/init.d/elasticsearch

}


function node(){
    cd /work/source
    tar xf node-v6.10.1-linux-x64.tar.xz 
    mv node-v6.10.1-linux-x64 /work/opt/node
    echo -e "export NODE_HOME=/work/opt/node/ \n export PATH=\$PATH:\$NODE_HOME/bin" >> /etc/profile
    export NODE_HOME=/work/opt/node/
    export PATH=$PATH:$NODE_HOME/bin
    echo '测试node是否安装成功，以及查看其版本号'
    /work/opt/node/bin/node --version
    npm config set registry https://registry.npm.taobao.org 
    cd /work/opt/node/lib/node_modules
    npm install grunt 
}


function elasticsearch-head(){
    cd /work/source
    cp phantomjs-2.1.1-linux-x86_64.tar.bz2 /tmp/phantomjs
    tar xf elasticsearch-head.tar.gz -C /work/opt/
    cd /work/opt/elasticsearch-head
    npm install
    sed -i '/port: 9100,/i\hostname: "0.0.0.0",' Gruntfile.js
    sed -i 's#http://localhost:9200#http://127.0.0.1:9200#g'   /work/opt/elasticsearch-head/_site/app.js
    grunt server &   ## 必须在 /work/opt/elasticsearch-head 这个目录下执行
    echo "查看安装 elasticsearch-head 是否安装成功 ，如果出现9100 则安装成功"
    netstat -lntp |grep 00
}


function elasticsearch-analysis-ik(){
    cd /work/source
    mkdir -p analysis-ik
    \cp elasticsearch-analysis-ik-5.3.0.zip analysis-ik/
    \cd analysis-ik
    apt-get install zip -y 
    unzip elasticsearch-analysis-ik-5.3.0.zip
    mkdir -p /work/opt/elasticsearch/plugins/ik
    \cp -rfp ../analysis-ik/* /work/opt/elasticsearch/plugins/ik/
    ls -l /work/opt/elasticsearch/plugins/ik/
    ## 重启 elasticsearch
    /etc/init.d/elasticsearch stop 
    /etc/init.d/elasticsearch start
}



java1.8
elasticsearch5.3
node 
elasticsearch-head
elasticsearch-analysis-ik








故障:

```

max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]

```

解决:


```

## centos 6 

1. [root@BBB ~]# cat /etc/security/limits.d/90-nproc.conf 

# Default limit for number of user's processes to prevent

# accidental fork bombs.

# See rhbz #432903 for reasoning.


*          soft    nproc     65536

*      soft    nproc     unlimited

[root@BBB ~]#

```

```

2. [root@BBB ~]# cat /etc/security/limits.conf 

# /etc/security/limits.conf


#@student        -       maxlogins       4

* soft nofile 300000

* hard nofile 300000

* soft nproc 102400

* soft memlock unlimited

* hard memlock unlimited

```



问题2

```

system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk

```


解决:

```

bootstrap.memory_lock: false

bootstrap.system_call_filter: false


```