0
点赞
收藏
分享

微信扫一扫

CentOS7下简单搭建zookeeper+kafka集群

笑望叔叔 2022-03-10 阅读 79

CentOS7下简单搭建zookeeper+kafka集群_java

CentOS7下简单搭建zookeeper+kafka集群_java_02

CentOS7下简单搭建zookeeper+kafka集群_zookeeper_03CentOS7下简单搭建zookeeper+kafka集群_zookeeper_04

如下图所示

CentOS7下简单搭建zookeeper+kafka集群_java_05

三台CentOS7的机器来搭建zookeeper+kafka集群

192.168.31.221 node1 node1.walkingcloud.cn

192.168.31.222 node2 node2.walkingcloud.cn

192.168.31.223 node3 node3.walkingcloud.cn

一、环境搭建

1、三台机器均修改/etc/hosts

CentOS7下简单搭建zookeeper+kafka集群_kafka_06

2、配置互信

ssh-keygen

ssh-copy-id root@node2

ssh-copy-id root@node3

如果有必要的话,三台机器间两两互信

CentOS7下简单搭建zookeeper+kafka集群_zookeeper_07

CentOS7下简单搭建zookeeper+kafka集群_kafka_08

3、提前下载JDK,zookeeper,kakfa的相关安装包,并上传到/opt目录

如下图所示

scp /opt/* root@node2:/opt

scp /opt/* root@node3:/opt

CentOS7下简单搭建zookeeper+kafka集群_kafka_09

二、每台机器都安装JDK环境,并配置好环境变量

1、安装JDKrpm -ivh jdk-8u251-linux-x64.rpm 

CentOS7下简单搭建zookeeper+kafka集群_zookeeper_10

2、配置环境变量vi /etc/profile,将如下三行加到最后export JAVA_HOME=/usr/java/jdk1.8.0_251-amd64export PATH=$PATH:$JAVA_HOME/binexport CLASSPATH=.:$JAVA_HOME/jre/lib:$JAVA_HOME/lib:$JAVA_HOME/lib/tools.jar3、使​环境变量​生效source /etc/profile

CentOS7下简单搭建zookeeper+kafka集群_zookeeper_11

三、安装zookeeper,配置zookeeper

1、三个节点均解压zookeeper安装包

cd /opt

tar -zxf apache-zookeeper-3.6.0-bin.tar.gz

mv apache-zookeeper-3.6.0-bin zookeeper

cd zookeeper

mkdir data

cd conf

cp zoo_sample.cfg zoo.cfg

CentOS7下简单搭建zookeeper+kafka集群_java_12

2、配置zoo.cfg

vi zoo.cfg

修改日志保存目录

dataDir=/opt/zookeeper/data

#server.服务编号=服务地址、LF通信端口、选举端口

server.221=node1:2888:3888

server.222=node2:2888:3888

server.223=node3:2888:3888

CentOS7下简单搭建zookeeper+kafka集群_java_13

配置同步到node2和node3

scp zoo.cfg root@node2:/opt/zookeeper/conf/

scp zoo.cfg root@node3:/opt/zookeeper/conf/

CentOS7下简单搭建zookeeper+kafka集群_zookeeper_14

3、配置/data/myid文件,写入服务器ID

CentOS7下简单搭建zookeeper+kafka集群_zookeeper_15

node1的id设置为221

node2的id设置为222

node3的id设置为223

4、三台均修改变量/etc/profile

vi /etc/profile,在最后面加一行

export PATH=$PATH:/opt/zookeeper/bin

然后 source /etc/profile

并验证zkServer.sh命令是否可以正常调出来

CentOS7下简单搭建zookeeper+kafka集群_zookeeper_16

5、三个节点均启动zookeeper

zkServer.sh start

CentOS7下简单搭建zookeeper+kafka集群_kafka_17

查看状态zkServer.sh status

CentOS7下简单搭建zookeeper+kafka集群_zookeeper_18

可以看到node1和node2为follower

node3为leader

四、三个节点均安装kafka,配置kakfa

1、node1上解压kafka安装包

cd /opt

tar -zxf kafka_2.13-2.5.0.tgz

mv kafka_2.13-2.5.0 kafka

cd kafka

cd config

cp server.properties server.properties_default

CentOS7下简单搭建zookeeper+kafka集群_zookeeper_19

2、创建data目录

CentOS7下简单搭建zookeeper+kafka集群_kafka_20

3、修改kakfa配置文件

vi server.properties

CentOS7下简单搭建zookeeper+kafka集群_java_21

1)broker.id

CentOS7下简单搭建zookeeper+kafka集群_java_22

2)log.dir目录指定

log.dirs=/opt/kafka/data

CentOS7下简单搭建zookeeper+kafka集群_kafka_23

3)zookeeper连接信息

zookeeper.connect=node1:2181,node2:2181,node3:2181

CentOS7下简单搭建zookeeper+kafka集群_zookeeper_24

4、node1将kafka整个目录进行文件scp同步,同步到node2和node3上

cd /opt/

scp -r kafka root@node2:/opt/

scp -r kafka root@node3:/opt/

然后再去node2,node3上手动修改server.properties中broker.id分为222和223

5、修改环境变量,加入kafkar的bin目录

vi /etc/profile最后一行添加:/opt/kafka/bin

export PATH=$PATH:/opt/zookeeper/bin:/opt/kafka/bin

source /etc/profile

CentOS7下简单搭建zookeeper+kafka集群_zookeeper_25

 scp /etc/profile root@node2:/etc

 scp /etc/profile root@node3:/etc

在节点node2,node3上source /etc/profile

6、三个节点启动kafka

kafka-server-start.sh /opt/kafka/config/server.properties

先前台的方式启动kakfa-server,后期改用后台或者服务脚本的方式启动kafka

CentOS7下简单搭建zookeeper+kafka集群_zookeeper_26

7、再开其它会话窗口测试topic

CentOS7下简单搭建zookeeper+kafka集群_kafka_27

创建topic

kafka-topics.sh --create --zookeeper node1:2181 --partitions 2 --replication-factor 3 --topic yuanfantest

列出topic

 kafka-topics.sh --list --zookeeper node1:2181 

查看topic描述

kafka-topics.sh  --zookeeper node1:2181  --topic yuanfantest --describe 

CentOS7下简单搭建zookeeper+kafka集群_java_28

8、测试主题的生产与消费

如下图所示,利用console在node1上生产,在node3消费的效果

node1上生产:kafka-console-producer.sh --bootstrap-server node1:9092 --topic yuanfantest

node3上消费:kafka-console-consumer.sh --bootstrap-server node3:9092 --topic yuanfantest

CentOS7下简单搭建zookeeper+kafka集群_kafka_29


举报

相关推荐

0 条评论