一、搭建zookeeper
步骤忽略
二、使用openssl和keytool生成客户端和服务端证书
步骤忽略
三、zookeeper配置ssl
1、简单证书生成
keytool -genkeypair -alias certificatekey -keyalg RSA -validity 3650 -keystore keystore.jks
keytool -list -v -keystore keystore.jks
keytool -export -alias certificatekey -keystore keystore.jks -rfc -file selfsignedcert.cer
keytool -import -alias certificatekey -file selfsignedcert.cer -keystore truststore.jks
keytool -list -v -keystore truststore.jks
firstname必须填写当前主机的hostname
2、服务端添加ssl
有两种方式
1、添加到配置文件
在zoo.cfg里面添加
serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory
ssl.keyStore.location=/data/zookeeper/cert/keystore.jks
ssl.keyStore.password=123456
ssl.trustStore.location=/data/zookeeper/cert/truststore.jks
ssl.trustStore.password=123456
2、以变量的形式添加
在zkServer.sh开头添加
export SERVER_JVMFLAGS="
-Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory
-Dzookeeper.ssl.keyStore.location=/data/zookeeper/cert/keystore.jks
-Dzookeeper.ssl.keyStore.password=123456
-Dzookeeper.ssl.trustStore.location=/data/zookeeper/cert/truststore.jks
-Dzookeeper.ssl.trustStore.password=123456"
2、配置文件添加安全端口
zoo.cfg需要额外添加安全端口
secureClientPort=2183
为了防止全网监听
secureClientPortAddress=192.168.10.133
3、启动服务
./zkServer.sh start4、配置zkCli.sh连接
首先测试连接普通端口
./zkCli.sh -server 192.168.10.133:2181
没有问题以后在zkCli.sh开头添加配置
export CLIENT_JVMFLAGS="
-Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty
-Dzookeeper.client.secure=true
-Dzookeeper.ssl.hostnameVerification=false
-Dzookeeper.ssl.keyStore.location=/data/zookeeper/cert/keystore.jks
-Dzookeeper.ssl.keyStore.password=123456
-Dzookeeper.ssl.trustStore.location=/data/zookeeper/cert/truststore.jks
-Dzookeeper.ssl.trustStore.password=123456"
###注意
此处keyStore、trustStore都是S大写，后面kafka配置必须是小写
然后测试连接
./zkCli.sh -server 192.168.10.133:2183
第一次日志报错：
"Cannot support TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 with currently installed providers"
怀疑是jdk版本太低引起的
当前版本为jdk1.8.0_151，升级到jdk1.8.0_221再次测试
./zkCli.sh -server 192.168.10.133:2181
日志不在报错，正常连接
四、配置kafka连接zookeeper
zookeeper.connect=192.168.10.133:2183
zookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty
zookeeper.ssl.client.enable=true
zookeeper.ssl.hostnameVerification=false
zookeeper.ssl.keystore.location=/data/zookeeper/cert/zookeeper.server.keystore.p12
zookeeper.ssl.keystore.password=123456
zookeeper.ssl.truststore.location=/data/zookeeper/cert/zookeeper.server.truststore.p12
zookeeper.ssl.truststore.password=123456
五、问题汇总
1、jdk版本导致加密不支持
"Cannot support TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 with currently installed providers"
当前：java version "1.8.0_151"
升级到java version "1.8.0_221"
重启zookeeper，重新连接测试
2、kafka连接配置引起报错
[2022-07-23 17:53:00,002] WARN Session 0x0 for sever k8s03/192.168.10.133:2183, Closing socket connection. Attempting reconnect except it is a SessionExpiredException. (org.apache.zookeeper.ClientCnxn)
EndOfStreamException: Unable to read additional data from server sessionid 0x0, likely server has closed socket
at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1290)
如果zookeeper的zkCli.sh正常连接，而kafka连接报错，可以考虑排查连接配置
zkCli.sh有一行配置：zookeeper.client.secure=true
但是kafka配置是：zookeeper.ssl.client.enable=true
千万别搞混了，搞混了就会报这个错
3、证书域名问题
"Certificate for <k8s03> doesn't match common name of the certificate subject: localhost"
"javax.net.ssl.SSLHandshakeException: General SSLEngine problem"
"Failed to verify both host address and host name"
证书设置的firstname为localhost，和hostname（k8s03）不对应
重新签发证书，并配置firstname为hostname
4、证书配置错误
"PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException:
unable to find valid certification path to requested target"
注意zkCli.sh配置
-Dzookeeper.ssl.keyStore.location=
-Dzookeeper.ssl.keyStore.password=
-Dzookeeper.ssl.trustStore.location=
-Dzookeeper.ssl.trustStore.password=
keyStore和trustStore中Store的S是大写，如果换成小写就报错连不上zookeeper服务
注意kafka的server.properties
zookeeper.ssl.keystore.location=
zookeeper.ssl.keystore.password=
zookeeper.ssl.truststore.location=
zookeeper.ssl.truststore.password=
keystore和truststore中store的s都是小写，如果写成大写会启动先报以下警告：
"WARN zookeeper.ssl.keyStore.location not specified (org.apache.zookeeper.common.X509Util)"
"WARN zookeeper.ssl.trustStore.location not specified (org.apache.zookeeper.common.X509Util)"
