0
点赞
收藏
分享

微信扫一扫

solrCloud 4.9 分布式集群部署及注意事项


环境搭建

一.zookeeper

参考:http://blog.chinaunix.net/uid-25135004-id-4214399.html

现有4台机器 10.14.2.201 10.14.2.202 10.14.2.203 10.14.2.204         


          安装zookeeper集群         


                    


          在所有机器上进行         


          1、下载安装包解压         


                    tar xvf zookeeper-3.4.5.tar.gz  -C /export/         


                    cd /export/         


                    ln -s zookeeper-3.4.5 zookeeper         


                    


          2、设置环境变量         


                    vim /etc/profile         


          #Set ZooKeeper Enviroment         


          export ZOOKEEPER_HOME=/export/zookeeper         


          export PATH=$PATH:$ZOOKEEPER_HOME/bin         


                    


          3、加载环境变量         


                    source   /etc/profile         


          4、配置zookeeper         


          cd /export/zookeeper/conf         


          cp zoo_sample.cfg  zoo.cfg         


          vim zoo.cfg         


                    


          tickTime=2000         


          initLimit=10         


          syncLimit=5         


          dataDir=/export/zookeeper/data         


          dataLogDir=/export/zookeeper/log         


          clientPort=2181         


          server.1=10.14.2.201:2888:3888         


          server.2=10.14.2.202:2888:3888         


          server.3=10.14.2.203:2888:3888         


          server.4=10.14.2.204:2888:3888         


                    


          5、dataDir和日志目录dataLogDir         


          mkdir -p /export/zookeeper/data         


          mkdir -p /export/zookeeper/log         


                    


          6、在10.14.2.201上         


          在dataDir目录下创建myid文件         


          echo 1 > /export/zookeeper/data/myid         


                    


          myid中的数字与zoo.cfg中的          "server.数字=ip或者域名:port1:port2"          相对应,         


          配置文件中10.14.2.201对应的server.数字是1 所以myid的数字应为1         


                    


          7、在10.14.2.202上         


          在dataDir目录下创建myid文件         


          echo 2 > /export/zookeeper/data/myid         


                    


          8、在10.14.2.203上         


          在dataDir目录下创建myid文件         


          echo 3 > /export/zookeeper/data/myid         


                    


          9、在10.14.2.204上         


          在dataDir目录下创建myid文件         


          echo 4 > /export/zookeeper/data/myid         


                    


          在所用机器上执行         


          10、修改权限         


          chown hadoop.hadoop -R /export/zookeeper/         


          chown hadoop.hadoop -R /export/zookeeper         


                    


          11、启动zookeeper         


          /export/zookeeper/bin/zkServer.sh start         


                    


          12、观察zookeeper状态         


                    1> 10.14.2.201         


                    # /export/zookeeper/bin/zkServer.sh status         


                    JMX enabled           by           default         


                    Using config: /export/zookeeper/bin/../conf/zoo.cfg         


                    Mode: follower         


                    


                    2> 10.14.2.202         


                    # /export/zookeeper/bin/zkServer.sh status         


                    JMX enabled           by           default         


                    Using config: /export/zookeeper/bin/../conf/zoo.cfg         


                    Mode: follower         


                    


                    3> 10.14.2.203         


                    # /export/zookeeper/bin/zkServer.sh status         


                    JMX enabled           by           default         


                    Using config: /export/zookeeper/bin/../conf/zoo.cfg         


                    Mode: leader         


                    


                    4> 10.14.2.204         


                    # /export/zookeeper/bin/zkServer.sh status         


                    JMX enabled           by           default         


                    Using config: /export/zookeeper/bin/../conf/zoo.cfg         


                    Mode: follower         


                    


          13、此时zookeeper已安装并启动完毕



  

二.solrCloud

参考:http://lucien-zzy.iteye.com/blog/2002463

1.安装tomcat,配置端口。部署solr.war,将solr\example\lib\ext下的jar拷贝到solr\WEB-INF\lib下。

2.创建

/home/solr/solrcloud

/home/solr/solrcloud/config-files (放置solr\example\solr\collection1\conf 下的所有文件)

/home/solr/solrcloud/solr-lib (放置solr\WEB-INF\lib下的所有jar包)

3.配置中文分词器

4.创建solr数据目录

/home/solr/solr-cores

生成solr.xml,如下:



<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="true">
<logging enabled="true">
<watcher size="100" threshold="INFO" />
</logging>
<cores defaultCoreName="collection1" adminPath="/admin/cores" host="${host:}" hostPort="8080" hostContext="${hostContext:solr}" zkClientTimeout="${zkClientTimeout:15000}">
</cores>
</solr>



5.创建

tomcat/conf/Catalina

tomcat/conf/Catalina/localhost

并创建solr.xml



<?xml version="1.0" encoding="UTF-8"?>
<Context docBase="/soft/apache-tomcat-7.0.37/webapps/solr" debug="0" crossContext="true">
<Environment name="solr/home" type="java.lang.String" value="/home/solr/solr-cores" override="true"/>
</Context>



docBase根据实际情况配置;

solr/home为之前创建的目录:/home/solr/solr-cores

6.修改tomcat/bin/cataina.sh 文件,在最上方加入:JAVA_OPTS="-DzkHost=master:2181,slave1:2181,slave2:2181",或直接使用ip。

将以上配置发布到集群其他机器上。

7.SolrCloud是通过ZooKeeper集群来保证配置文件的变更及时同步到各个节点上,所以,需要将配置文件上传到ZooKeeper集群中。执行如下操作:



java -classpath .:/home/solr/solrcloud/solr-lib/* org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost 192.168.91.128:2181,192.168.91.129:2181,192.168.91.130:2181 -confdir /home/solr/solrcloud/config-files/ -confname myconf



-cmd upconfig:上传配置文件

-confdir:配置文件的目录

-confname:指定对应的名称



java -classpath .:/home/solr/solrcloud/solr-lib/* org.apache.solr.cloud.ZkCLI -cmd linkconfig -collection collection1 -confname myconf -zkhost 192.168.91.128:2181,192.168.91.129:2181,192.168.91.130:2181



-cmd linkconfig:为指定collection"绑定"配置文件

-collection

-confname

8.检查zookeeper的配置信息

zkCli.sh -server ip:port

ls /configs/myconf

ls /live_nodes

9.启动tomcat,创建collection



curl 'http://192.168.91.128:8080/solr/admin/collections?action=CREATE&name=mycollection&numShards=3&replicationFactor=1'



action:CREATE

name:collection名称

numShards:分片数量

replicationFactor:副本集数量

也可以通过zk查询collection的情况:ls /collections/mycollection

 

-------------------------------

1.ZK CLI

ZooKeeper's Command Line Utility (CLI).

zkcli Parameters



Short

Parameter Usage

Meaning

 

-cmd <arg>

CLI Command to be executed: bootstrapupconfigdownconfiglinkconfigmakepathgetgetfileput,putfilelist or clear
This parameter is mandatory

-z

-zkhost <locations>

ZooKeeper host address. 
This parameter is mandatory for all CLI commands.

-c

-collection <name>

For linkconfig: name of the collection.

-d

-confdir <path>

For upconfig: a directory of configuration files.

-h

-help

Display help text.

-n

-confname <arg>

For upconfiglinkconfig: name of the configuration set.

-r

-runzk <port>

Run ZooKeeper internally by passing the Solr run port; only for clusters on one machine.

-s

-solrhome <path>

For bootstrap or when using -runzk: the mandatory solrhome location.



ZooKeeper CLI Examples

Below are some examples of using the zkcli CLI:
Uploading a Configuration Directory



java -classpath example/solr-webapp/WEB-INF/lib/*
org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost 127.0.0.1:9983
-confdir example/solr/collection1/conf -confname conf1 -solrhome example/solr



Put arbitrary data into a new ZK file



java -classpath example/solr-webapp/WEB-INF/lib/*
org.apache.solr.cloud.ZkCLI -zkhost 127.0.0.1:9983 -put /data.txt 'some data'



Put a local file into a new ZK file



java -classpath example/solr-webapp/WEB-INF/lib/*
org.apache.solr.cloud.ZkCLI -zkhost 127.0.0.1:9983 -putfile /data.txt /some/local/file.txt



Linking a Collection to a Configuration Set



java -classpath example/solr-webapp/webapp/WEB-INF/lib/*
org.apache.solr.cloud.ZkCLI -cmd linkconfig -zkhost 127.0.0.1:9983
-collection collection1 -confname conf1 -solrhome example/solr



Bootstrapping All the Configuration Directories in solr.xml



java -classpath example/solr-webapp/webapp/WEB-INF/lib/*
org.apache.solr.cloud.ZkCLI -cmd bootstrap -zkhost 127.0.0.1:9983
-solrhome example/solr



Scripts

There are scripts in example/cloud-scripts that handle the classpath and class name for you if you are using Solr out of the box with Jetty. Commands then become:



sh zkcli.sh -cmd linkconfig -zkhost 127.0.0.1:9983
-collection collection1 -confname conf1 -solrhome example/solr



 



 



2.Managing collections via the collections API

CREATE---



http://localhost:8983/solr/admin/collections?action=CREATE&name=mycollection&numShards=3&replicationFactor=4



name:collection的名称

numShards:指定分片数量(slices)

replicationFactor:副本数量

maxShardsPerNode:默认值为1,注意三个数值:numShards、replicationFactor、liveSolrNode,一个正常的solrCloud集群不容许同一个liveSolrNode上部署同一个shard的多个replic,因此当maxShardsPerNode=1时,numShards*replicationFactor>liveSolrNode时,报错。因此正确时因满足以下条件:
numShards*replicationFactor<liveSolrNode*maxShardsPerNode

createNodeSet:

collection.configName:指定该collection使用那份config,这份config必须存在于zk中。

DELETE---



http://localhost:8983/solr/admin/collections?action=DELETE&name=mycollection



RELOAD---



http://localhost:8983/solr/admin/collections?action=RELOAD&name=mycollection



SPLIT SHARD---



http://localhost:8983/solr/admin/collections?action=SPLITSHARD&collection=<collection_name>&shard=shardId



 

3.Creating cores via CoreAdmin



curl 'http://localhost:8983/solr/admin/cores?action=CREATE&name=mycore&collection=collection1&shard=shard2'



 

 

-------------------------------

Q1:Multiple schemas in the same SolrCloud ?



java -classpath .:/home/solrcloud/solr-lib/* org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost 10.200.51.190:2224,10.200.51.192:2224,10.200.51.194:2224 -confdir /home/solrcloud/fox-config-files/ -confname foxconf



指定配置文件,上传到zk。



curl 'http://10.200.51.194:2223/solr/admin/collections?action=CREATE&name=fox4&numShards=1&replicationFactor=5&collection.configName=foxconf'



为创建的collection指定collection.configName。

举报

相关推荐

0 条评论