手机版
你好,游客 登录 注册
背景:
阅读新闻

Elasticsearch单机多实例环境部署

[日期:2016-11-20] 来源:Linux社区  作者:shihuc [字体: ]

Elasticsearch的功能,主要用在搜索领域,这里,我来研究这个,也是项目需要,为公司开发了一款CMS系统,网站上的搜索栏功能,我打算采用Elasticsearch来实现。 Elasticsearch的高性能,低延时是最大的吸引力。

系统环境:

CentOS6.8, X86_64。

Elasticsearch的版本: 2.3.5,下载地址来自官网,选择的是RPM包。

es安装完毕后,默认安装的路径是/usr/share/elasticsearch, 配置文件,默认是在/etc/elasticsearch目录。这里要重点说说配置文件的信息:

 1 [root@CloudGame bin]# cd /etc/elasticsearch/
 2 [root@CloudGame elasticsearch]# ll
 3 total 12
 4 -rwxr-x---. 1 root elasticsearch 3189 Jul 27 18:34 elasticsearch.yml
 5 -rwxr-x---. 1 root elasticsearch 2571 Jul 27 18:34 logging.yml
 6 drwxr-x---. 2 root elasticsearch 4096 Jul 27 18:44 scripts
 7 [root@CloudGame elasticsearch]# tree
 8 .
 9 ├── elasticsearch.yml     #es的系统配置文件
10 ├── logging.yml        #es的日志配置文件
11 └── scripts
12 
13 1 directory, 2 files

单实例的启动很简单,安装完成后,直接service elasticsearch start即可启动,日志在/var/log/elasticsearch目录,数据在/var/lib/elasticsearch目录。

这里重点放在如何配置单机多实例上面。为了简单起见,多实例就选择启动2个实例,更多的实例,和2个实例是一样的配置,只是简单的相应改改参数即可。

1. 在/etc目录下创建esconf目录,然后在esconf下面创建esins1以及esins2目录。将原本/etc/elasticsearch目录的内容copy到esins1以及esins2目录下。

 1 [root@CloudGame esconf]# pwd
 2 /etc/esconf
 3 [root@CloudGame esconf]# tree
 4 .
 5 ├── esins1
 6 │   ├── elasticsearch.yml
 7 │   ├── logging.yml
 8 │   └── scripts
 9 ├── esins2
10     ├── elasticsearch.yml
11     ├── logging.yml
12     └── scripts
13 
14 
15 4 directories, 6 files 

2. 修改配置文件elasticsearch.yml,对esins1以及esins2下面的这个文件都做修改。主要是修改其节点信息,网络链接信息,以及日志数据路径信息。下面直接上修改好的配置文件内容。

实例1: esins1目录下的elasticsearch.yml内容

 1 [root@CloudGame esins1]# cat elasticsearch.yml 
 2 # ======================== Elasticsearch Configuration =========================
 3 #
 4 # NOTE: Elasticsearch comes with reasonable defaults for most settings.
 5 #       Before you set out to tweak and tune the configuration, make sure you
 6 #       understand what are you trying to accomplish and the consequences.
 7 #
 8 # The primary way of configuring a node is via this file. This template lists
 9 # the most important settings you may want to configure for a production cluster.
10 #
11 # Please see the documentation for further information on configuration options:
12 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html>
13 #
14 # ---------------------------------- Cluster -----------------------------------
15 #
16 # Use a descriptive name for your cluster:
17 #
18  cluster.name: tksearch        #es默认的集群名称是elasticsearch,注意,集群中是以name来区分节点属于那个集群的。
19 #
20 # ------------------------------------ Node ------------------------------------
21 #
22 # Use a descriptive name for the node:
23 #
24  node.name: node1             #节点的名称
25 #
26 # Add custom attributes to the node:
27 #
28 # node.rack: r1
29 #
30  node.master: true            #是否让这个节点作为默认的master,若不是,默认会选择集群里面的第一个作为master,es有一套选择那个节点作为master的机制
31 # ----------------------------------- Paths ------------------------------------
32 #
33 # Path to directory where to store the data (separate multiple locations by comma):
34 #
35  path.data: /home/AppData/es/esins1/data #配置节点数据存放的目录
36 #
37 # Path to log files:
38 #
39  path.logs: /home/AppData/es/esins1/logs #配置节点日志存放的目录
40 #
41 # ----------------------------------- Memory -----------------------------------
42 #
43 # Lock the memory on startup:
44 #
45 # bootstrap.mlockall: true
46 #
47 # Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory
48 # available on the system and that the owner of the process is allowed to use this limit.
49 #
50 # Elasticsearch performs poorly when the system is swapping the memory.
51 #
52 # ---------------------------------- Network -----------------------------------
53 #
54 # Set the bind address to a specific IP (IPv4 or IPv6):
55 #
56  network.host: 0.0.0.0   #配置节点绑定的地址,全0表示可以绑定任何地址,当然这里,本机可以是127.0.0.1回还地址,也可以是ifconfig看到的eth1的地址。
57 #
58 # Set a custom port for HTTP:
59 #
60  http.port: 9200   #配置当前节点对外http访问的端口号,默认是9200,不配的话,es会从9200-9299当中找一个未用过的。
61 #
62 # For more information, see the documentation at:
63 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html>
64 #
65  transport.tcp.port: 9300  #es集群节点之间的通信端口号。默认9300.
66 # --------------------------------- Discovery ----------------------------------
67 #
68 # Pass an initial list of hosts to perform discovery when new node is started:
69 # The default list of hosts is ["127.0.0.1", "[::1]"]
70 #
71  discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300"]  #集群多播时发现其他节点的主机列表, 真实多机集群环境下,这里会是多个主机的IP列表,默认格式“host:port”的数组
72 #
73 # Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1):
74 #
75 # discovery.zen.minimum_master_nodes: 3
76 #
77 # For more information, see the documentation at:
78 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html>
79 #
80 # ---------------------------------- Gateway -----------------------------------
81 #
82 # Block initial recovery after a full cluster restart until N nodes are started:
83 #
84 # gateway.recover_after_nodes: 3
85 #
86 # For more information, see the documentation at:
87 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html>
88 #
89 # ---------------------------------- Various -----------------------------------
90 #
91 # Disable starting multiple nodes on a single system:
92 #
93  node.max_local_storage_nodes: 2  #默认情况下,是不建议单机启动多个node的,这里这个参数,就是告知es单机上启动了几个实例,这里我们配置2个,若是要配置3个或者更多实例,修改这个数字即可
94 #
95 # Require explicit names when deleting indices:
96 #
97 # action.destructive_requires_name: true

实例2: esins2目录下的elasticsearch.yml内容,配置和esins1几乎一样。

 
 1 [root@CloudGame esins2]# cat elasticsearch.yml 
 2 # ======================== Elasticsearch Configuration =========================
 3 #
 4 # NOTE: Elasticsearch comes with reasonable defaults for most settings.
 5 #       Before you set out to tweak and tune the configuration, make sure you
 6 #       understand what are you trying to accomplish and the consequences.
 7 #
 8 # The primary way of configuring a node is via this file. This template lists
 9 # the most important settings you may want to configure for a production cluster.
10 #
11 # Please see the documentation for further information on configuration options:
12 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html>
13 #
14 # ---------------------------------- Cluster -----------------------------------
15 #
16 # Use a descriptive name for your cluster:
17 #
18  cluster.name: tksearch
19 #
20 # ------------------------------------ Node ------------------------------------
21 #
22 # Use a descriptive name for the node:
23 #
24  node.name: node2
25 #
26 # Add custom attributes to the node:
27 #
28 # node.rack: r1
29 #
30  node.master: false
31 # ----------------------------------- Paths ------------------------------------
32 #
33 # Path to directory where to store the data (separate multiple locations by comma):
34 #
35  path.data: /home/AppData/es/esins2/data
36 #
37 # Path to log files:
38 #
39  path.logs: /home/AppData/es/esins2/logs
40 #
41 # ----------------------------------- Memory -----------------------------------
42 #
43 # Lock the memory on startup:
44 #
45 # bootstrap.mlockall: true
46 #
47 # Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory
48 # available on the system and that the owner of the process is allowed to use this limit.
49 #
50 # Elasticsearch performs poorly when the system is swapping the memory.
51 #
52 # ---------------------------------- Network -----------------------------------
53 #
54 # Set the bind address to a specific IP (IPv4 or IPv6):
55 #
56  network.host: 0.0.0.0
57 #
58 # Set a custom port for HTTP:
59 #
60  http.port: 9201
61 #
62 # For more information, see the documentation at:
63 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html>
64 #
65  transport.tcp.port: 9301
66 # --------------------------------- Discovery ----------------------------------
67 #
68 # Pass an initial list of hosts to perform discovery when new node is started:
69 # The default list of hosts is ["127.0.0.1", "[::1]"]
70 #
71  discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300"]
72 #
73 # Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1):
74 #
75 # discovery.zen.minimum_master_nodes: 3
76 #
77 # For more information, see the documentation at:
78 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html>
79 #
80 # ---------------------------------- Gateway -----------------------------------
81 #
82 # Block initial recovery after a full cluster restart until N nodes are started:
83 #
84 # gateway.recover_after_nodes: 3
85 #
86 # For more information, see the documentation at:
87 # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html>
88 #
89 # ---------------------------------- Various -----------------------------------
90 #
91 # Disable starting multiple nodes on a single system:
92 #
93  node.max_local_storage_nodes: 2
94 #
95 # Require explicit names when deleting indices:
96 #
97 # action.destructive_requires_name: true

3. 由于配置中指定了配置文件和数据的路径了,所以,要在相应的路径下创建所需的目录。

1 /home/AppData/es/esins1/data
2 /home/AppData/es/esins1/logs
3 /home/AppData/es/esins2/data
4 /home/AppData/es/esins2/logs

4. 启动es, 先启动es实例1. 注意,es启动时指定的配置文件时,要指定配置文件所在的路径,这个路径包含elasticsearch.yml以及logging.yml

1 [root@CloudGame bin]# ./elasticsearch -d -Des.path.conf=/etc/esconf/esins1 -p /etc/esconf/esins1.pid
2 [root@CloudGame bin]# Exception in thread "main" Java.lang.RuntimeException: don't run elasticsearch as root.
3     at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:93)
4     at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:144)
5     at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
6     at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
7 Refer to the log for complete error details.

哟,错了啊,不能用root启动es,见鬼,再来一次吧。

 1 [water@CloudGame bin]$ ./elasticsearch -d -Des.path.conf=/etc/esconf/esins1 -p /etc/esconf/esins1.pid
 2 [water@CloudGame bin]$ Exception in thread "main" SettingsException[Failed to open stream for url [/etc/esconf/esins1/elasticsearch.yml]]; nested: AccessDeniedException[/etc/esconf/esins1/elasticsearch.yml];
 3 Likely root cause: java.nio.file.AccessDeniedException: /etc/esconf/esins1/elasticsearch.yml
 4     at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
 5     at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
 6     at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
 7     at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
 8     at java.nio.file.Files.newByteChannel(Files.java:317)
 9     at java.nio.file.Files.newByteChannel(Files.java:363)
10     at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:380)
11     at java.nio.file.Files.newInputStream(Files.java:108)
12     at org.elasticsearch.common.settings.Settings$Builder.loadFromPath(Settings.java:1067)
13     at org.elasticsearch.node.internal.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:88)
14     at org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:202)
15     at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
16     at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
17 Refer to the log for complete error details.

哟嘿,还是错??? 访问etc/esconf/esins1/elasticsearch.yml被阻止了???想起来了,这个文件是在root用户下创建配置的。修改一下用户组吧: chown -R water:water /etc/esconf/esins1。再来一次。。。

 1 [water@CloudGame bin]$ ./elasticsearch -d -Des.path.conf=/etc/esconf/esins1 -p /etc/esconf/esins1.pid
 2 [water@CloudGame bin]$ Exception in thread "main" SettingsException[Failed to open stream for url [/etc/esconf/esins1/elasticsearch.yml]]; nested: AccessDeniedException[/etc/esconf/esins1/elasticsearch.yml];
 3 Likely root cause: java.nio.file.AccessDeniedException: /etc/esconf/esins1/elasticsearch.yml
 4     at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
 5     at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
 6     at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
 7     at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
 8     at java.nio.file.Files.newByteChannel(Files.java:317)
 9     at java.nio.file.Files.newByteChannel(Files.java:363)
10     at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:380)
11     at java.nio.file.Files.newInputStream(Files.java:108)
12     at org.elasticsearch.common.settings.Settings$Builder.loadFromPath(Settings.java:1067)
13     at org.elasticsearch.node.internal.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:88)
14     at org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:202)
15     at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
16     at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
17 Refer to the log for complete error details.

怎么还是一样的错啊,去看看日志吧,靠,/home/AppData/es/esins1/logs下面什么也没有啊???????倒,这个目录也是在root权限下创建的,water没有写的权限。。。。。。再给这数据和日志目录改属组。。。。。

 1 [root@CloudGame es]# ll
 2 total 8
 3 drwxr-xr-x. 4 root root 4096 Oct 18 11:20 esins1
 4 drwxr-xr-x. 4 root root 4096 Oct 18 11:21 esins2
 5 [root@CloudGame es]# chown -R water:water *
 6 [root@CloudGame es]# ll
 7 total 8
 8 drwxr-xr-x. 4 water water 4096 Oct 18 11:20 esins1
 9 drwxr-xr-x. 4 water water 4096 Oct 18 11:21 esins2
10 [root@CloudGame es]# cd esins1/
11 [root@CloudGame esins1]# ll
12 total 8
13 drwxr-xr-x. 2 water water 4096 Oct 18 11:20 data
14 drwxr-xr-x. 2 water water 4096 Oct 18 11:49 logs

再来启动一次,应该可以了吧。。。。

 1 [water@CloudGame bin]$ ./elasticsearch -d -Des.path.conf=/etc/esconf/esins1 -p /etc/esconf/esins1.pid
 2 [water@CloudGame bin]$ Exception in thread "main" java.nio.file.AccessDeniedException: /etc/esconf/esins1.pid
 3     at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
 4     at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
 5     at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
 6     at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
 7     at java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:430)
 8     at java.nio.file.Files.newOutputStream(Files.java:172)
 9     at org.elasticsearch.common.PidFile.create(PidFile.java:76)
10     at org.elasticsearch.common.PidFile.create(PidFile.java:55)
11     at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:247)
12     at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
13 Refer to the log for complete error details.

去你的,怎么还是权限的问题,来pid文件没有办法写入。。。。。再搞一次权限配置。。。。

1 [root@CloudGame etc]# chown -R water:water esconf

再启动,再不好,是不是就不玩了。。。。

1 [water@CloudGame bin]$ ./elasticsearch -d -Des.path.conf=/etc/esconf/esins1 -p /etc/esconf/esins1.pid
2 [water@CloudGame bin]$ 

哈哈哈,很好,这次启动不错了,说明八九不离十了。

同样的,参照启动esins1的过程,启动esins2.一切都ok。  esconf下面有两个pid文件

1 [root@CloudGame esconf]# ll -al
2 total 32
3 drwxr-xr-x.   4 water water  4096 Oct 18 12:00 .
4 drwxr-xr-x. 126 root  root  12288 Oct 18 13:25 ..
5 drwxr-xr-x.   3 water water  4096 Oct 18 11:38 esins1
6 -rw-rw-r--.   1 water water     5 Oct 18 11:55 esins1.pid
7 drwxr-xr-x.   3 water water  4096 Oct 18 11:39 esins2
8 -rw-rw-r--.   1 water water     5 Oct 18 12:00 esins2.pid

到此,单机双实例的启动完成。

真的完成了么?要不访问看看,那就验证一下吧,下面来几张截图demo一下最后的结果!

esins1的基本信息:

esins2的基本信息:

esins1的健康状态:

esins2的健康状态:

ElasticSearch 最新版本 2.20 发布下载了  http://www.linuxidc.com/Linux/2016-02/128166.htm

Linux上安装部署ElasticSearch全程记录  http://www.linuxidc.com/Linux/2015-09/123241.htm

Elasticsearch安装使用教程 http://www.linuxidc.com/Linux/2015-02/113615.htm

ElasticSearch 配置文件译文解析 http://www.linuxidc.com/Linux/2015-02/114244.htm

ElasticSearch集群搭建实例  http://www.linuxidc.com/Linux/2015-02/114243.htm

分布式搜索ElasticSearch单机与服务器环境搭建  http://www.linuxidc.com/Linux/2012-05/60787.htm

ElasticSearch的工作机制  http://www.linuxidc.com/Linux/2014-11/109922.htm 

Elasticsearch的安装,运行和基本配置 http://www.linuxidc.com/Linux/2016-07/133057.htm

使用Elasticsearch + Logstash + Kibana搭建日志集中分析平台实践  http://www.linuxidc.com/Linux/2015-12/126587.htm

Ubuntu 14.04搭建ELK日志分析系统(Elasticsearch+Logstash+Kibana) http://www.linuxidc.com/Linux/2016-06/132618.htm

Elasticsearch1.7升级到2.3实践总结  http://www.linuxidc.com/Linux/2016-11/137282.htm

ElasticSearch 的详细介绍请点这里
ElasticSearch 的下载地址请点这里 

本文永久更新链接地址http://www.linuxidc.com/Linux/2016-11/137359.htm

linux
本文评论   查看全部评论 (0)
表情: 表情 姓名: 字数

       

评论声明
  • 尊重网上道德,遵守中华人民共和国的各项有关法律法规
  • 承担一切因您的行为而直接或间接导致的民事或刑事法律责任
  • 本站管理人员有权保留或删除其管辖留言中的任意内容
  • 本站有权在网站内转载或引用您的评论
  • 参与本评论即表明您已经阅读并接受上述条款