线上环境elasticsearch5.0.1集群的配置部署
es集群的规划： 硬件： 7台8核、64G内存、2T ssd硬盘加1台8核16G的阿里云服务器 其中一台作为kibana+kafka连接查询的服务器 其他6台都作为node和master两种角色 操作系统：centos7.2 x86_64 为方便磁盘扩容建议将磁盘进行lvm逻辑卷配置，可以参考： aliyun添加数据盘后的物理分区和lvm逻辑卷两种挂载方式 http://blog.csdn.net/reblue520/article/details/541741781.安装jdk1.8和elasticsearch5.0.1
rpm -ivh jdk-8u111-linux-x64.rpm
tar -zxvf elasticsearch-5.0.1.tar.gz2.添加yunva这个运行elasticsearch的用户(es必须使用非root用户启动)
useradd yunva -d /home/yunva
echo 'pass'|passwd --stdin yunva chown -R yunva.yunva /data 修改默认端口 sed -i 's/#Port 22/Port 2222/' /etc/ssh/sshd_config sed -i 's/^GSSAPIAuthentication yes$/GSSAPIAuthentication no/' /etc/ssh/sshd_config service sshd restart3.针对es做的一些系统的优化配置
swapoff -a
echo "fs.file-max = 1000000" >> /etc/sysctl.conf echo "vm.max_map_count=262144" >> /etc/sysctl.conf echo "vm.swappiness = 1" >> /etc/sysctl.conf sysctl -p sed -i 's/* soft nofile 65535/* soft nofile 655350/g' /etc/security/limits.conf sed -i 's/* hard nofile 65535/* hard nofile 655350/g' /etc/security/limits.conf 将java_home加入环境变量 cat >> /etc/profile <<EOF export JAVA_HOME=/usr/java/jdk1.8.0_111 export PATH=\$JAVA_HOME/bin:\$PATH EOF source /etc/profile4.es内存调整配置文件(建议配置为物理内存的一半或者更多最好不要超过32G，超过了也可能不会增强性能)：
/data/elasticsearch-5.0.1/config/jvm.options sed -i 's/-Xms2g/-Xms32g/' /data/elasticsearch-5.0.1/config/jvm.options sed -i 's/-Xmx2g/-Xmx32g/' /data/elasticsearch-5.0.1/config/jvm.options echo "-Xss256k" >>/data/elasticsearch-5.0.1/config/jvm.options sed -i 's/-XX:+UseConcMarkSweepGC/-XX:+UseG1GC/' /data/elasticsearch-5.0.1/config/jvm.options
5.集群的主要配置文件
修改elasticsearch的参数
vim /etc/elasticsearch/elasticsearch.yml(rpm安装方式的配置文件位置) vim /data/elasticsearch-5.0.1/config/elasticsearch.yml es节点的配置： # 节点名 cluster.name: yunva-es # 集群的名称，可以不写 discovery.zen.ping.unicast.hosts: ["node-1","yunva_etl_es2", "yunva_etl_es3","yunva_etl_es4","yunva_etl_es5","yunva_etl_es6","yunva_etl_es7"] node.name: yunva_etl_es6 node.master: true node.data: true path.data: /data/es/data path.logs: /data/es/logs action.auto_create_index: false indices.fielddata.cache.size: 12g bootstrap.memory_lock: false # 内网地址，可以加快速度 network.host: 192.168.1.10 http.port: 9200 # 增加新的参数head插件可以访问es http.cors.enabled: true http.cors.allow-origin: "*" gateway.recover_after_time: 8m gateway.expected_nodes: 3 cluster.routing.allocation.node_initial_primaries_recoveries: 8# 以下配置可以减少当es节点短时间宕机或重启时shards重新分布带来的磁盘io读写浪费
discovery.zen.fd.ping_timeout: 180sdiscovery.zen.fd.ping_retries: 8discovery.zen.fd.ping_interval: 30sdiscovery.zen.ping_timeout: 120s 针对kibana的es配置(非node和master节点) # cat /etc/elasticsearch/elasticsearch.yml cluster.name: yunva-es node.name: yunva_etl_es1 node.master: false node.data: false node.ingest: false action.auto_create_index: false path.data: /data/es/data path.logs: /data/es/logs bootstrap.memory_lock: false network.host: 0.0.0.0 http.port: 9200 http.cors.enabled: true http.cors.allow-origin: "*"# 以下配置可以减少当es节点短时间宕机或重启时shards重新分布带来的磁盘io读写浪费
discovery.zen.fd.ping_timeout: 180sdiscovery.zen.fd.ping_retries: 8discovery.zen.fd.ping_interval: 30sdiscovery.zen.ping_timeout: 120s 注意修改配置文件vim /etc/hosts 列出集群节点名称和对应ip地址的对应关系(有内网dns并且配置的就不需要再次配置了) echo "10.28.50.131 node-1" >> /etc/hosts echo "10.26.241.239 yunva_etl_es3" >> /etc/hosts echo "10.25.135.215 yunva_etl_es2" >> /etc/hosts echo "10.26.241.237 yunva_etl_es4" >> /etc/hosts echo "10.27.78.228 yunva_etl_es5" >> /etc/hosts echo "10.27.65.121 yunva_etl_es6" >> /etc/hosts echo "10.27.35.94 yunva_etl_es7" >> /etc/hosts6.创建日志和数据存放目录
mkdir -p /data/es/data
mkdir /data/es/logs chown -R yunva.yunva /data7.启动es服务：
# su - yunva
[yunva]$ cd /data/elasticsearch-5.0.1/bin/ ./elasticsearch &8.检查单台服务是否正常：
$ curl http://ip:9200/ { "name" : "yunva_etl_es5", "cluster_name" : "yunva-es", "cluster_uuid" : "2shAg8u3SjCRNJ4mEUBzBQ", "version" : { "number" : "5.0.1", "build_hash" : "080bb47", "build_date" : "2016-11-11T22:08:49.812Z", "build_snapshot" : false, "lucene_version" : "6.2.1" }, "tagline" : "You Know, for Search" } # 查看集群状态 $ curl http://ip:9200/_cluster/health/?pretty { "cluster_name" : "yunva-es", "status" : "green", "timed_out" : false, "number_of_nodes" : 5, "number_of_data_nodes" : 4, "active_primary_shards" : 66, "active_shards" : 132, "relocating_shards" : 2, "initializing_shards" : 0, "unassigned_shards" : 0, "delayed_unassigned_shards" : 0, "number_of_pending_tasks" : 0, "number_of_in_flight_fetch" : 0, "task_max_waiting_in_queue_millis" : 0, "active_shards_percent_as_number" : 100.0 } 然后将配置好的es程序拷贝到其他服务器中，注意修改以下内容(network.host为内网地址，速度更快，节省互相复制、分片的时候处理带宽)： 1.elasticsearch.yml文件的配置修改 node.name: 节点名称 network.host: es节点的内网IP地址 2./etc/hosts文件中内网ip和node.name的对应关系 后续添加对集群服务的监控，可以参考：
elasticsearch6.0的安装
一、确定服务器配置，新建硬件资源需要配置es集群作为热数据，数据量不大，两台8核16G，200G磁盘的服务器做es集群即可二、初始化系统1、关闭firewall，并安装iptables服务systemctl stop firewalld.service #停止firewallsystemctl disable firewalld.service #禁止firewall开机启动yum install iptables-services -y# 添加防火墙规则vim /usr/local/worksh/FireWall.sh#!/bin/bashiptables -A INPUT -p udp --dport 1701 -j ACCEPT################################################################### Required modules/sbin/modprobe ip_tables/sbin/modprobe ip_conntrack/sbin/modprobe iptable_mangle/sbin/modprobe iptable_nat/sbin/modprobe ipt_LOG/sbin/modprobe ipt_limit/sbin/modprobe ipt_state/sbin/modprobe ip_conntrack_ftp/sbin/modprobe ip_nat_ftp/sbin/modprobe ipt_owner/sbin/modprobe ipt_REJECT### Clean Rulesiptables -P INPUT ACCEPTiptables -P FORWARD ACCEPTiptables -P OUTPUT ACCEPTiptables -t nat -P PREROUTING ACCEPTiptables -t nat -P POSTROUTING ACCEPTiptables -t nat -P OUTPUT ACCEPTiptables -t mangle -P PREROUTING ACCEPTiptables -t mangle -P OUTPUT ACCEPTiptables -Fiptables -t nat -F#iptables -t mangle -Fiptables -Xiptables -t nat -X#iptables -t mangle -X### Drop all pocket,firstiptables -P INPUT DROP#iptables -P OUTPUT DROPiptables -P FORWARD DROP### Create New chainsiptables -N bad_tcp_packets#iptables -N allowediptables -N icmp_packets### Bad_tcp_packets chain/sbin/iptables -A bad_tcp_packets -p tcp ! --syn -m state --state NEW -j DROP/sbin/iptables -A bad_tcp_packets -p tcp --tcp-flags ALL ALL -j DROP/sbin/iptables -A bad_tcp_packets -p tcp --tcp-flags ALL NONE -j DROP/sbin/iptables -A bad_tcp_packets -p tcp --tcp-flags SYN,FIN SYN,FIN -j DROP/sbin/iptables -A bad_tcp_packets -p tcp --tcp-flags SYN,RST SYN,RST -j DROP/sbin/iptables -A bad_tcp_packets -p tcp --tcp-flags FIN,RST FIN,RST -j DROP/sbin/iptables -A bad_tcp_packets -p tcp --tcp-flags ACK,FIN FIN -j DROP/sbin/iptables -A bad_tcp_packets -p tcp --tcp-flags ACK,PSH PSH -j DROP/sbin/iptables -A bad_tcp_packets -p tcp --tcp-flags ACK,URG URG -j DROP### ICMP Rulesiptables -A icmp_packets -p icmp --icmp-type 8 -j ACCEPTiptables -A icmp_packets -p icmp --icmp-type 11 -j ACCEPT#iptables -A icmp_packets -p icmp -j DROP##keepalivediptables -A INPUT -i eth1 -p vrrp -s 63.159.217.139 -j ACCEPT### LookBack and Private interfaceiptables -A INPUT -p ALL -i lo -j ACCEPTiptables -A INPUT -p ALL -i eth0 -j ACCEPT### INPUT chainiptables -A INPUT -p tcp -j bad_tcp_packetsiptables -A INPUT -p icmp -j icmp_packetsiptables -A INPUT -p ALL -m state --state ESTABLISHED,RELATED -j ACCEPT### 5666 EOPiptables -A INPUT -p tcp -i eth1 --dport 5666 -s 47.91.75.80 -j ACCEPT##repiptables -A INPUT -p tcp -i eth1 --dport 3306 -s 47.91.90.28 -j ACCEPT# Count Limit#iptables -A INPUT -m limit --limit 3/minute --limit-burst 3 -j LOG --log-level INFO --log-prefix "IPT INPUT PACKET DIED:"############################################################iptables -I INPUT -p udp --dport 1701 -j ACCEPT### Open PortsPublic_access="80 443 8080"Server_access="20 21 873 3306 2008 8899 9200 9300"Company_access="20 21 873 9200 9300 9100"aliyun_access="3306"### Allow IpsServers_ip="192.168.254.0/24"Company_ip="1.1.1.1"aliyun_ip="2.2.2.2"Eus_server_ip="3.3.3.3"### Public access Rulesfor port in $Public_accessdo iptables -A INPUT -p tcp -i eth1 --dport $port -j ACCEPTdone### Servers access Rulesfor port in $Server_accessdo for ip in $Servers_ip do iptables -A INPUT -p tcp -i eth1 --dport $port -s $ip -j ACCEPT donedone### Eus_server access Rulesfor port in $Server_accessdo for ip in $Eus_server_ip do iptables -A INPUT -p tcp -i eth1 --dport $port -s $ip -j ACCEPT donedone### Company access Rulesfor port in $Company_accessdo for ip in $Company_ip do iptables -A INPUT -p tcp -i eth1 --dport $port -s $ip -j ACCEPT donedone# chmod +x /usr/local/worksh/FireWall.sh# sh -x /usr/local/worksh/FireWall.sh# 开机运行防火墙规则[root@cms_elasticsearch01 elasticsearch-head]# tail -n 1 /etc/rc.local /usr/local/worksh/FireWall.sh*********************# centos7设置洛杉矶时间timedatectl set-timezone America/Los_Angeles# 修改文件句柄sed -i 's#root soft nofile 65535#root soft nofile 655350#g' /etc/security/limits.confsed -i 's#root hard nofile 65535#root hard nofile 655350#g' /etc/security/limits.confsed -i 's#* soft nofile 65535#* soft nofile 655350#g' /etc/security/limits.confsed -i 's#* hard nofile 65535#* hard nofile 655350#g' /etc/security/limits.confsed -i 's#* soft nproc 4096##g' /etc/security/limits.d/20-nproc.conf# 最终效果root soft nofile 655350root hard nofile 655350* soft nofile 655350* hard nofile 655350# 安装依赖yum install -y gcc gcc-c++ htop telnet iotop iptraf iftop make logrotate xinetd ntsysv sysstat perl autoconf libjpeg libjpeg-devel libpng libpng-devel glibc glibc-devel glib2 glib2-devel bzip2 bzip2-devel ncurses ncurses-devel e2fsprogs e2fsprogs-devel krb5-devel libidn libidn-devel libxml2 libxml2-devel libxslt-devel libevent-devel libtool libtool-ltdl bison ntpdate patch vim wget openssh-clients bind-utils# 系统升级到最新yum update -y# 安装jdk1.8_181rpm -ivh jdk-8u181-linux-x64.rpm# vim /etc/profileexport JAVA_HOME=/usr/java/jdk1.8.0_181-amd64export PATH=$JAVA_HOME/bin:$PATH4.添加运行elasticsearch的普通用户##### 安装elasticsearchelasticsearch wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.2.tar.gztar -zxf elasticsearch-6.3.2.tar.gz#### 创建elasticsearch的普通账号groupadd elasticsearchuseradd elasticsearch -g elasticsearchchown -R elasticsearch.elasticsearch /datachown -R elasticsearch.elasticsearch /usr/local/elasticsearch-6.3.2/# 一些优化配置swapoff -aecho "fs.file-max = 1000000" >> /etc/sysctl.confecho "vm.max_map_count=262144" >> /etc/sysctl.confecho "vm.swappiness = 1" >> /etc/sysctl.conf# 让配置生效sysctl -p# 创建数据和日志目录mkdir -p /data/es/datamkdir -p /data/es/logses节点的配置：主节点 master 配置：[root@cms_elasticsearch01 config]# cat elasticsearch.yml# 节点名cluster.name: cms-uat-es# 集群的名称，可以不写discovery.zen.ping.unicast.hosts: ["cms_elasticsearch01","cms_elasticsearch02"]node.name: cms_elasticsearch01node.master: truenode.data: truepath.data: /data/es/datapath.logs: /data/es/logs#action.auto_create_index: falseindices.fielddata.cache.size: 1g# 内网地址，可以加快速度#network.host: 192.168.254.36network.host: 0.0.0.0http.port: 9200# 增加新的参数head插件可以访问eshttp.cors.enabled: truehttp.cors.allow-origin: "*"gateway.recover_after_time: 8m# 以下配置可以减少当es节点短时间宕机或重启时shards重新分布带来的磁盘io读写浪费discovery.zen.fd.ping_timeout: 300sdiscovery.zen.fd.ping_retries: 8discovery.zen.fd.ping_interval: 30sdiscovery.zen.ping_timeout: 180s数据节点配置[root@cms_elasticsearch02 ~]# cat /usr/local/elasticsearch-6.3.2/config/elasticsearch.yml# 节点名cluster.name: cms-uat-es# 集群的名称，可以不写discovery.zen.ping.unicast.hosts: ["cms_elasticsearch01","cms_elasticsearch02"]node.name: cms_elasticsearch02node.master: falsenode.data: truepath.data: /data/es/datapath.logs: /data/es/logs#action.auto_create_index: falseindices.fielddata.cache.size: 1gbootstrap.memory_lock: false# 内网地址，可以加快速度#network.host: 192.168.254.37network.host: 0.0.0.0http.port: 9200# 增加新的参数head插件可以访问eshttp.cors.enabled: truehttp.cors.allow-origin: "*"gateway.recover_after_time: 8m# 以下配置可以减少当es节点短时间宕机或重启时shards重新分布带来的磁盘io读写浪费discovery.zen.fd.ping_timeout: 300sdiscovery.zen.fd.ping_retries: 8discovery.zen.fd.ping_interval: 30sdiscovery.zen.ping_timeout: 180s# 修改hosts文件echo "192.168.254.36 cms_elasticsearch01" >> /etc/hostsecho "192.168.254.37 cms_elasticsearch02" >> /etc/hosts# 启动程序su elasticsearch -c "cd /usr/local/elasticsearch-6.3.2 && bin/elasticsearch -d"三、添加zabbix监控硬件资源的监控es的监控# 集群状态监控脚本UserParameter=es_status,curl -sXGET http://192.168.254.37:9200/_cluster/health/?pretty | grep "status"|awk -F '[ "]+' '{print $4}'|grep -c 'green'# 如果出现错误的监控UserParameter=es_debug,sudo /bin/find /usr/local/elasticsearch-6.3.2 -name hs_err_pid*.log -o -name java_pid*.hprof|wc -l# 监控触发自动启动elasticsearch脚本：# vim /usr/local/zabbix_agents_3.2.0/scripts/start_es.sh#!/bin/bash# if elasticsearch process exists kill itsource /etc/profilecount_es=`ps -ef|grep elasticsearch|grep -v grep|wc -l`if [ $count_es -ge 1 ];thenps -ef|grep elasticsearch|grep -v grep|/bin/kill `awk '{print $2}'`firm -f /usr/local/elasticsearch-6.3.2/bin/java_pid*.hprof# start itsu elasticsearch -c "cd /usr/local/elasticsearch-6.3.2 && bin/elasticsearch -d"# chmod +x /usr/local/zabbix_agents_3.2.0/scripts/start_es.sh########################################## 安装head插件# yum install -y epel-release# yum install -y nodejs[root@cms_elasticsearch01 elasticsearch-head]# npm install -g cnpm --registry=https://registry.npm.taobao.orgnpm WARN deprecated socks@1.1.10: If using 2.x branch, please upgrade to at least 2.1.6 to avoid a serious bug with socket data flow and an import issue introduced in 2.1.0/usr/bin/cnpm -> /usr/lib/node_modules/cnpm/bin/cnpm/usr/lib└─┬ cnpm@6.0.0 ├── auto-correct@1.0.0 ├── bagpipe@0.3.5 [root@cms_elasticsearch01 elasticsearch-head]# npm install -g grunt/usr/bin/grunt -> /usr/lib/node_modules/grunt/bin/grunt/usr/lib└─┬ grunt@1.0.3 [root@cms_elasticsearch01 elasticsearch-head]# npm install -g grunt-cli --registry=https://registry.npm.taobao.org --no-proxy/usr/bin/grunt -> /usr/lib/node_modules/grunt-cli/bin/grunt/usr/lib└─┬ grunt-cli@1.3.1[root@cms_elasticsearch01 elasticsearch-head]# grunt -versiongrunt-cli v1.3.1（6）下载依赖 进入elasticsearch-head-master目录，执行下面命令[root@cms_elasticsearch01 elasticsearch-head]# npm install(7)修改配置，当有外网和内网IP的时候需要修改配置 elasticsearch-head/Gruntfile.js 文件 connect: { server: { options: { hostname: '*', port: 9100, base: '.', keepalive: true# 修改 elasticsearch-head/_site/app.js 将localhost修改为本机外网ip地址this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://70.39.189.23:9200";# 防火墙要互相放行# 启动head插件su elasticsearch -c "cd /usr/local/elasticsearch-head && nohup npm run start >> /usr/local/elasticsearch-head/logs/npm-start.log 2>&1 &"# chown -R elasticsearch.elasticsearch /usr/local/elasticsearch-head/# 启动head插件脚本#!/bin/sh# stop elasticsearch master functionfunction stopelkmaster(){ count=`ps -ef | grep -v grep |grep grunt | wc -l` if [ $count -gt 0 ];then for i in `ps -ef | grep -v grep |grep grunt|awk '{print $2}'` do #echo "is exists:" kill -9 $i done fi}# start elasticsearch master functionfunction startelkmaster(){ count=`ps -ef | grep -v grep |grep grunt | wc -l` if [ $count -gt 0 ];then for i in `ps -ef | grep -v grep |grep grunt|awk '{print $2}'` do kill -9 $i done sleep 1 cd /opt/elasticsearch-head su elasticsearch -c "cd /usr/local/elasticsearch-head && nohup npm run start >> /usr/local/elasticsearch-head/logs/npm-start.log 2>&1 &" else cd /opt/elasticsearch-head su elasticsearch -c "cd /usr/local/elasticsearch-head && nohup npm run start >> /usr/local/elasticsearch-head/logs/npm-start.log 2>&1 &" fi}isExistsElk=`ps -ef | grep -v grep |grep grunt | wc -l`if [ $isExistsElk -eq 0 ]then #echo "start process....." startelkmasterelif [ $isExistsElk -gt 0 ]then stopelkmaster sleep 1 startelkmasterelse echo "error!"fi