ElasticSearch 规划-集群规划
ElasticSearch 规划-集群规划
ElasticSearch 规划-用户规划
ElasticSearch 规划-目录规划
ElasticSearch 集群环境检查-时钟同步
1.设置本地时间
cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
2.集群时间日期同步NTP
yum install ntp ntpdate pool.ntp.org
ElasticSearch 集群环境检查-hosts文件检查
1.配置ElasticSearch集群ip与hostname之间的对应关系
vi /etc/hosts 192.168.86.135 masternode 192.168.86.136 slavenode1 192.168.86.137 slavenode2
2.ElasticSearch 集群环境检查-禁用防火墙
查看防火墙状态
service iptables status
关闭防火墙
chkconfig iptables off //永久关闭防火墙service iptables stop //临时关闭防火墙
ElasticSearch 集群安装
1.下载
https://www.elastic.co/
2.上传并解压
[hadoop@masternode es]$ cd /home/hadoop/app/[hadoop@masternode app]$ lshadoop jdk1.8.0_60 zookeeper[hadoop@masternode app]$ rz[hadoop@masternode app]$ rz[hadoop@masternode app]$ lselasticsearch-2.4.0.tar.gz hadoop jdk1.8.0_60 kibana-4.6.3-linux-x86_64.tar.gz zookeeper[hadoop@masternode app]$ tar -xvf elasticsearch-2.4.0.tar.gz[hadoop@masternode app]$ rm -rf elasticsearch-2.4.0.tar.gz
3.修改config/elasticsearch.yml配置文件
[hadoop@masternode app]$ cd elasticsearch-2.4.0/[hadoop@masternode elasticsearch-2.4.0]$ lsbin config lib LICENSE.txt modules NOTICE.txt README.textile[hadoop@masternode elasticsearch-2.4.0]$ cd config/[hadoop@masternode config]$ lselasticsearch.yml logging.yml[hadoop@masternode config]$ vim elasticsearch.yml
4.路径创建
[hadoop@masternode data]$ mkdir es/[hadoop@masternode data]$ cd es[hadoop@masternode es]$ mkdir data[hadoop@masternode es]$ mkdir datalog[hadoop@masternode es]$ mkdir pid[hadoop@masternode es]$ lsdata datalog pid
5.分发到其他节点并修改以下配置
[hadoop@masternode app]$ scp -r elasticsearch-2.4.0/ slavenode1:/home/hadoop/app/[hadoop@masternode app]$ scp -r elasticsearch-2.4.0/ slavenode2:/home/hadoop/app/
Slavenode1：
Slavenode2：
其它配置不变。
ElasticSearch 插件安装
1.安装marvel插件（所有节点都需要安装）
[hadoop@masternode app]$ cd elasticsearch-2.4.0[hadoop@masternode elasticsearch-2.4.0]$ suPassword: [root@masternode elasticsearch-2.4.0]# bin/plugin install license-> Installing license...Plugins directory [/home/hadoop/app/elasticsearch-2.4.0/plugins] does not exist. Creating...Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.4.0/license-2.4.0.zip ...Downloading .......DONEVerifying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/license/2.4.0/license-2.4.0.zip checksums if available ...Downloading .DONEInstalled license into /home/hadoop/app/elasticsearch-2.4.0/plugins/license[root@masternode elasticsearch-2.4.0]# bin/plugin install marvel-agent-> Installing marvel-agent...Trying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/marvel-agent/2.4.0/marvel-agent-2.4.0.zip ...Downloading ..........DONEVerifying https://download.elastic.co/elasticsearch/release/org/elasticsearch/plugin/marvel-agent/2.4.0/marvel-agent-2.4.0.zip checksums if available ...Downloading .DONE@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ WARNING: plugin requires additional permissions @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@* java.lang.RuntimePermission setFactory* javax.net.ssl.SSLPermission setHostnameVerifierSee http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.htmlfor descriptions of what these permissions allow and the associated risks.Continue with installation? [y/N]yInstalled marvel-agent into /home/hadoop/app/elasticsearch-2.4.0/plugins/marvel-agent
2.安装head插件（选择一个节点安装即可）
[root@masternode elasticsearch-2.4.0]# bin/plugin install mobz/elasticsearch-head-> Installing mobz/elasticsearch-head...Trying https://github.com/mobz/elasticsearch-head/archive/master.zip ...Downloading ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................DONEVerifying https://github.com/mobz/elasticsearch-head/archive/master.zip checksums if available ...NOTE: Unable to verify checksum for downloaded plugin (unable to find .sha1 or .md5 file to verify)Installed head into /home/hadoop/app/elasticsearch-2.4.0/plugins/head[root@masternode elasticsearch-2.4.0]# lsbin config lib LICENSE.txt modules NOTICE.txt plugins README.textile[root@masternode elasticsearch-2.4.0]# cd plugins/[root@masternode plugins]# lshead license marvel-agent
Kibana 安装
1.下载
https://www.elastic.co/
2.解压
[root@masternode plugins]# cd ..[root@masternode elasticsearch-2.4.0]# cd ..[root@masternode app]# tar –xvf kibana-4.6.3-linux-x86_64.tar.gz[root@masternode app]# rm -rf kibana-4.6.3-linux-x86_64.tar.gz [root@masternode app]# lselasticsearch-2.4.0 hadoop jdk1.8.0_60 kibana-4.6.3-linux-x86_64 zookeeper
3.修改kibana.yml配置文件
[root@masternode app]# cd kibana-4.6.3-linux-x86_64/[root@masternode kibana-4.6.3-linux-x86_64]# lsbin config data installedPlugins LICENSE.txt node node_modules optimize package.json README.txt src webpackShims[root@masternode kibana-4.6.3-linux-x86_64]# cd config/[root@masternode config]# lskibana.yml[root@masternode config]# vim kibana.yml
4.安装marvel插件
[root@masternode config]# cd ..[root@masternode kibana-4.6.3-linux-x86_64]# bin/kibana plugin --install elasticsearch/marvel/latestInstalling marvelAttempting to transfer from https://download.elastic.co/elasticsearch/marvel/marvel-latest.tar.gzTransferring 2400332 bytes....................Transfer completeExtracting plugin archiveExtraction completeOptimizing and caching browser bundles...Plugin installation complete[root@masternode kibana-4.6.3-linux-x86_64]# lsbin config data installedPlugins LICENSE.txt node node_modules optimize package.json README.txt src webpackShims[root@masternode kibana-4.6.3-linux-x86_64]# cd installedPlugins/[root@masternode installedPlugins]# lsMarvel
ElasticSearch、Kibana 启动
前台启动
1.启动ElasticSearch
[root@masternode kibana-4.6.3-linux-x86_64]# su Hadoop[hadoop@masternode elasticsearch-2.4.0]$ bin/elasticsearch //在三个节点执行相同的操作[2018-08-17 16:38:02,342][WARN ][bootstrap ] unable to install syscall filter: seccomp unavailable: requires kernel 3.5+ with CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER compiled in[2018-08-17 16:38:03,057][INFO ][node ] [node-0] version[2.4.0], pid[5421], build[ce9f0c7/2016-08-29T09:14:17Z][2018-08-17 16:38:03,057][INFO ][node ] [node-0] initializing ...[2018-08-17 16:38:04,741][INFO ][plugins ] [node-0] modules [reindex, lang-expression, lang-groovy], plugins [head, license, marvel-agent], sites [head][2018-08-17 16:38:04,827][INFO ][env ] [node-0] using [1] data paths, mounts [[/home (/dev/sda2)]], net usable_space [1.7gb], net total_space [2.8gb], spins? [possibly], types [ext4][2018-08-17 16:38:04,827][INFO ][env ] [node-0] heap size [1015.6mb], compressed ordinary object pointers [true][2018-08-17 16:38:04,827][WARN ][env ] [node-0] max file descriptors [4096] for elasticsearch process likely too low, consider increasing to at least [65536][2018-08-17 16:38:10,101][INFO ][node ] [node-0] initialized[2018-08-17 16:38:10,102][INFO ][node ] [node-0] starting ...[2018-08-17 16:38:10,394][INFO ][transport ] [node-0] publish_address { 192.168.86.135:9300}, bound_addresses { 192.168.86.135:9300}[2018-08-17 16:38:10,426][INFO ][discovery ] [node-0] escluster/pT7mHJglTw6euS87IbvMGg[2018-08-17 16:38:20,117][INFO ][marvel.agent.exporter ] [node-0] skipping exporter [default_local] as it isn't ready yet[2018-08-17 16:38:30,120][INFO ][marvel.agent.exporter ] [node-0] skipping exporter [default_local] as it isn't ready yet[2018-08-17 16:38:40,120][INFO ][marvel.agent.exporter ] [node-0] skipping exporter [default_local] as it isn't ready yet[2018-08-17 16:38:40,430][WARN ][discovery ] [node-0] waited for 30s and no initial state was set by the discovery[2018-08-17 16:38:40,441][INFO ][http ] [node-0] publish_address { 192.168.86.135:9200}, bound_addresses { 192.168.86.135:9200}[2018-08-17 16:38:40,447][INFO ][node ] [node-0] started 启动之后打开网页可以查看其状态：
2.启动kibana
[hadoop@masternode kibana-4.6.3-linux-x86_64]$ bin/kibana log [16:53:07.744] [info][status][plugin:kibana@1.0.0] Status changed from uninitialized to green - Ready log [16:53:07.792] [info][status][plugin:elasticsearch@1.0.0] Status changed from uninitialized to yellow - Waiting for Elasticsearch log [16:53:07.816] [info][status][plugin:marvel@2.4.6] Status changed from uninitialized to yellow - Waiting for Elasticsearch log [16:53:07.856] [info][status][plugin:kbn_vislib_vis_types@1.0.0] Status changed from uninitialized to green - Ready log [16:53:09.699] [info][status][plugin:markdown_vis@1.0.0] Status changed from uninitialized to green - Ready log [16:53:09.713] [info][status][plugin:metric_vis@1.0.0] Status changed from uninitialized to green - Ready log [16:53:09.719] [info][status][plugin:spyModes@1.0.0] Status changed from uninitialized to green - Ready log [16:53:09.723] [info][status][plugin:statusPage@1.0.0] Status changed from uninitialized to green - Ready log [16:53:09.727] [info][status][plugin:table_vis@1.0.0] Status changed from uninitialized to green - Ready log [16:53:09.736] [info][listening] Server running at http://0.0.0.0:5601 log [16:53:14.719] [info][status][plugin:elasticsearch@1.0.0] Status changed from yellow to yellow - No existing Kibana index found log [16:53:17.640] [info][status][plugin:marvel@2.4.6] Status changed from yellow to green - Marvel ready log [16:53:17.644] [info][status][plugin:elasticsearch@1.0.0] Status changed from yellow to green - Kibana index ready
后台启动
bin/elasticsearch -d -p /home/hadoop/data/es/pid //三个节点都执行nohup /home/ybxiang/kibana/kibana-4.5.1-linux-x64/bin/kibana & exit //主节点执行
以上就是博主为大家介绍的这一板块的主要内容，这都是博主自己的学习过程，希望能给大家带来一定的指导作用，有用的还望大家点个支持，如果对你没用也望包涵，有错误烦请指出。如有期待可关注博主以第一时间获取更新哦，谢谢！