0
点赞
收藏
分享

微信扫一扫

Centos7.9部署MySQL8+keepalived双主热备高可用

_LEON_ 2023-04-14 阅读 77
mysqlsqlkeepalivedmysql8mysql监控运维私藏项目实操分享

前言

本文借鉴和参考：https://blog.51cto.com/u_15445837/6081230，如有侵权，请与我联系，感谢~

架构图

说明：

如上图，VIP地址是10.0.x.126，如果主机10.0.x.125宕机，会自动切换到备机​​10.0.x.23，由于是双主，数据可以进行同步恢复。

部署流程

部署大致流程如下：

  • 在10.0.x.125服务器上部署docker、master1
  • 在​​10.0.x.23​​服务器上部署docker、master2
  • 在两台服务器上安装keeplived形成高可用
  • 10.0.x.126是vip

配置第一台Master

准备Master配置文件my.cnf

## 创建相关目录和文件
mkdir -p /data/mysql
mkdir -p /data/mysql/master1
mkdir -p /data/mysql/master1/config
touch /data/mysql/master1/config/my.cnf
chmod +x /data/mysql/master1/config/my.cnf

cat > /data/mysql/master1/config/my.cnf <<EOF
[mysqld]
server_id=110
user=mysql
character-set-server=utf8mb4
collation-server=utf8mb4_general_ci
default-authentication-plugin=mysql_native_password
max_connections=1000
max_connect_errors=1000

log_bin=mysql_binlog_master1
slow_query_log=ON
slow_query_log_file=/var/lib/mysql/slow.log

log_error=/var/lib/mysql/mysql.err
log_replica_updates=ON
enforce_gtid_consistency=ON
gtid_mode=ON

plugin-load="rpl_semi_sync_source=semisync_source.so;rpl_semi_sync_replica=semisync_replica.so"
#rpl_semi_sync_source_enabled=ON
#rpl_semi_sync_replica_enabled=ON

binlog-ignore-db=information_schema
binlog-ignore-db=mysql
binlog-ignore-db=sys
binlog-ignore-db=performance_schema

[client]
default-character-set=utf8mb4
EOF

运行第一个master

cat > /data/mysql/master1/start-master1.sh <<EOF
#!/bin/bash
docker run -d \
--name mysql_master1 \
--restart always \
-p 3306:3306 \
-e MYSQL_ROOT_PASSWORD=test#123 \
-e TZ="Asia/Shanghai" \
-v /data/mysql/master1/config:/etc/mysql/conf.d \
-v /data/mysql/master1/data:/var/lib/mysql \
mysql:8.0.29 --lower-case-table-names=1 --authentication_policy=mysql_native_password
EOF

chmod +x /data/mysql/master1/start-master1.sh
sh /data/mysql/master1/start-master1.sh

启动半同步插件

此步主要是为了将​​rpl_semi_sync_source_enabled=ON​​​和​rpl_semi_sync_replica_enabled=ON​​的前#去掉。

验证半同步插件是否安装：

mysql> INSTALL PLUGIN rpl_semi_sync_source SONAME 'semisync_source.so';
mysql> INSTALL PLUGIN rpl_semi_sync_replica SONAME 'semisync_replica.so';

说明：mysql会检测​​semisync_source.so​​​和​​semisync_replica.so​​​插件的情况，如果没有安装就不能设置​​rpl_semi_sync_source_enabled​​​和​​rpl_semi_sync_replica_enabled​​。

cat > /data/mysql/master1/config/my.cnf <<EOF
[mysqld]
server_id=110
user=mysql
character-set-server=utf8mb4
collation-server=utf8mb4_general_ci
default-authentication-plugin=mysql_native_password
max_connections=1000
max_connect_errors=1000

log_bin=mysql_binlog_master1
slow_query_log=ON
slow_query_log_file=/var/lib/mysql/slow.log

log_error=/var/lib/mysql/mysql.err
log_replica_updates=ON
enforce_gtid_consistency=ON
gtid_mode=ON

plugin-load="rpl_semi_sync_source=semisync_source.so;rpl_semi_sync_replica=semisync_replica.so"
rpl_semi_sync_source_enabled=ON
rpl_semi_sync_replica_enabled=ON

binlog-ignore-db=information_schema
binlog-ignore-db=mysql
binlog-ignore-db=sys
binlog-ignore-db=performance_schema

[client]
default-character-set=utf8mb4
EOF

重启mysql

docker restart mysql_master1
docker exec -it mysql_master1 mysql -uroot -ptest#123 -e "show variables like 'rpl_semi_sync_%';"
+---------------------------------------------+------------+
| Variable_name                               | Value      |
+---------------------------------------------+------------+
| rpl_semi_sync_replica_enabled               | ON         |
| rpl_semi_sync_replica_trace_level           | 32         |
| rpl_semi_sync_source_enabled                | ON         |
| rpl_semi_sync_source_timeout                | 10000      |
| rpl_semi_sync_source_trace_level            | 32         |
| rpl_semi_sync_source_wait_for_replica_count | 1          |
| rpl_semi_sync_source_wait_no_replica        | ON         |
| rpl_semi_sync_source_wait_point             | AFTER_SYNC |
+---------------------------------------------+------------+

配置第二台Master

准备Master配置文件my.cnf

## 创建相关目录和文件
mkdir -p /data/mysql
mkdir -p /data/mysql/master2
mkdir -p /data/mysql/master2/config
touch /data/mysql/master2/config/my.cnf
chmod +x /data/mysql/master2/config/my.cnf

cat > /data/mysql/master2/config/my.cnf <<EOF
[mysqld]
server_id=111
user=mysql
character-set-server=utf8mb4
collation-server=utf8mb4_general_ci
default-authentication-plugin=mysql_native_password
max_connections=1000
max_connect_errors=1000

log_bin=mysql_binlog_master2
slow_query_log=ON
slow_query_log_file=/var/lib/mysql/slow.log

log_error=/var/lib/mysql/mysql.err
log_replica_updates=ON
enforce_gtid_consistency=ON
gtid_mode=ON

plugin-load="rpl_semi_sync_source=semisync_source.so;rpl_semi_sync_replica=semisync_replica.so"
#rpl_semi_sync_source_enabled=ON
#rpl_semi_sync_replica_enabled=ON

binlog-ignore-db=information_schema
binlog-ignore-db=mysql
binlog-ignore-db=sys
binlog-ignore-db=performance_schema

[client]
default-character-set=utf8mb4

运行第二个master

cat > /data/mysql/master2/start-master2.sh <<EOF
#!/bin/bash
docker run -d \
--name mysql_master2 \
--restart always \
-p 3306:3306 \
-e MYSQL_ROOT_PASSWORD=test#123 \
-e TZ="Asia/Shanghai" \
-v /data/mysql/master2/config:/etc/mysql/conf.d \
-v /data/mysql/master2/data:/var/lib/mysql \
mysql:8.0.29 --lower-case-table-names=1 --authentication_policy=mysql_native_password
EOF

chmod +x /data/mysql/master2/start-master2.sh
sh /data/mysql/master2/start-master2.sh

启动半同步插件

此步主要是为了将​​rpl_semi_sync_source_enabled=ON​​​和​rpl_semi_sync_replica_enabled=ON​​的前#去掉。

验证半同步插件是否安装：

mysql> INSTALL PLUGIN rpl_semi_sync_source SONAME 'semisync_source.so';
mysql> INSTALL PLUGIN rpl_semi_sync_replica SONAME 'semisync_replica.so';

说明：mysql会检测​​semisync_source.so​​​和​​semisync_replica.so​​​插件的情况，如果没有安装就不能设置​​rpl_semi_sync_source_enabled​​​和​​rpl_semi_sync_replica_enabled​​。

cat > /data/mysql/master2/config/my.cnf <<EOF
[mysqld]
server_id=111
user=mysql
character-set-server=utf8mb4
collation-server=utf8mb4_general_ci
default-authentication-plugin=mysql_native_password
max_connections=1000
max_connect_errors=1000

log_bin=mysql_binlog_master2
slow_query_log=ON
slow_query_log_file=/var/lib/mysql/slow.log

log_error=/var/lib/mysql/mysql.err
log_replica_updates=ON
enforce_gtid_consistency=ON
gtid_mode=ON

plugin-load="rpl_semi_sync_source=semisync_source.so;rpl_semi_sync_replica=semisync_replica.so"
rpl_semi_sync_source_enabled=ON
rpl_semi_sync_replica_enabled=ON

binlog-ignore-db=information_schema
binlog-ignore-db=mysql
binlog-ignore-db=sys
binlog-ignore-db=performance_schema

[client]
default-character-set=utf8mb4
EOF

重启mysql

docker restart mysql_master2
docker exec -it mysql_master2 mysql -uroot -ptest#123 -e "show variables like 'rpl_semi_sync_%';"
+---------------------------------------------+------------+
| Variable_name                               | Value      |
+---------------------------------------------+------------+
| rpl_semi_sync_replica_enabled               | ON         |
| rpl_semi_sync_replica_trace_level           | 32         |
| rpl_semi_sync_source_enabled                | ON         |
| rpl_semi_sync_source_timeout                | 10000      |
| rpl_semi_sync_source_trace_level            | 32         |
| rpl_semi_sync_source_wait_for_replica_count | 1          |
| rpl_semi_sync_source_wait_no_replica        | ON         |
| rpl_semi_sync_source_wait_point             | AFTER_SYNC |
+---------------------------------------------+------------+

双主配置

在进行双主配置时，需要进行一些准备工作，如mysql客户端安装、创建同步账号等。

mysql客户端安装

具体步骤，可参考Centos7.9仅安装mysql8客户端工具。

两台机器上创建主从同步账号

登录其中一台master上操作即可。

mysql> create user 'rpl_admin'@'%' identified with mysql_native_password by 'rpl_admin123456';
mysql> grant replication slave on *.* to 'rpl_admin'@'%';
mysql> flush privileges;

Master1作为主，Master2作为备份

Master1的IP地址为：10.0.x.125，Master2的IP地址为：10.0.x.23。

编写配置文件：

touch /data/mysql/rplShell.sh
chmod +x /data/mysql/rplShell.sh

cat > /data/mysql/rplShell.sh <<EOF
#!/bin/bash
MASTER_RPL_USER=rpl_admin
MASTER_RPL_PASSWORD=rpl_admin123456
# master and slaves admin user
ADMIN_USER=root
ADMIN_PASSWORD=test#123

MASTER_HOST=10.0.x.125
MASTER_PORT=3306
BACKUP_HOST=10.0.x.23
BACKUP_PORT=3306

RESULT=`mysql -h$MASTER_HOST -P$MASTER_PORT -u"$ADMIN_USER" -p"$ADMIN_PASSWORD" -e "SHOW MASTER STATUS;" | grep -v grep |tail -n +2| awk '{print $1,$2}'`
LOG_FILE_NAME=`echo $RESULT | grep -v grep | awk '{print $1}'`
LOG_FILE_POS=`echo $RESULT | grep -v grep | awk '{print $2}'`

SYNC_SQL="change master to master_host='$MASTER_HOST', master_port=$MASTER_PORT, master_user='$MASTER_RPL_USER',master_password='$MASTER_RPL_PASSWORD',master_log_file='$LOG_FILE_NAME',master_log_pos=$LOG_FILE_POS;"
START_SYNC_SQL="start REPLICA;"
STATUS_SQL="show REPLICA status\G;"

mysql -h$BACKUP_HOST -P$BACKUP_PORT -u"$ADMIN_USER" -p"$ADMIN_PASSWORD" -e"$SYNC_SQL"
mysql -h$BACKUP_HOST -P$BACKUP_PORT -u"$ADMIN_USER" -p"$ADMIN_PASSWORD" -e"$START_SYNC_SQL"
mysql -h$BACKUP_HOST -P$BACKUP_PORT -u"$ADMIN_USER" -p"$ADMIN_PASSWORD" -e"$STATUS_SQL"
EOF

执行脚本并检查：

sh /data/mysql/rplShell.sh
*************************** 1. row ***************************
...
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
...

Master2作为主，Master1作为备份

Master1的IP地址为：10.0.x.125，Master2的IP地址为：10.0.x.23。

编写配置文件：

touch /data/mysql/rplShell.sh
chmod +x /data/mysql/rplShell.sh

cat > /data/mysql/rplShell.sh <<EOF
#!/bin/bash
MASTER_RPL_USER=rpl_admin
MASTER_RPL_PASSWORD=rpl_admin123456
# master and slaves admin user
ADMIN_USER=root
ADMIN_PASSWORD=test#123

MASTER_HOST=10.0.x.23
MASTER_PORT=3306
BACKUP_HOST=10.0.x.125
BACKUP_PORT=3306

RESULT=`mysql -h$MASTER_HOST -P$MASTER_PORT -u"$ADMIN_USER" -p"$ADMIN_PASSWORD" -e "SHOW MASTER STATUS;" | grep -v grep |tail -n +2| awk '{print $1,$2}'`
LOG_FILE_NAME=`echo $RESULT | grep -v grep | awk '{print $1}'`
LOG_FILE_POS=`echo $RESULT | grep -v grep | awk '{print $2}'`

SYNC_SQL="change master to master_host='$MASTER_HOST', master_port=$MASTER_PORT, master_user='$MASTER_RPL_USER',master_password='$MASTER_RPL_PASSWORD',master_log_file='$LOG_FILE_NAME',master_log_pos=$LOG_FILE_POS;"
START_SYNC_SQL="start REPLICA;"
STATUS_SQL="show REPLICA status\G;"

mysql -h$BACKUP_HOST -P$BACKUP_PORT -u"$ADMIN_USER" -p"$ADMIN_PASSWORD" -e"$SYNC_SQL"
mysql -h$BACKUP_HOST -P$BACKUP_PORT -u"$ADMIN_USER" -p"$ADMIN_PASSWORD" -e"$START_SYNC_SQL"
mysql -h$BACKUP_HOST -P$BACKUP_PORT -u"$ADMIN_USER" -p"$ADMIN_PASSWORD" -e"$STATUS_SQL"
EOF

执行脚本并检查：

sh /data/mysql/rplShell.sh
*************************** 1. row ***************************
...
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
...

安装 keepsalived

在安装keepalived出现问题：需要：libmysqlclient.so.18

主要步骤：

wget https://dev.mysql.com/get/Downloads/MySQL-5.7/mysql-community-libs-compat-5.7.25-1.el7.x86_64.rpm
rpm -ivh mysql-community-libs-compat-5.7.25-1.el7.x86_64.rpm
## 再次安装
yum install keepalived -y

Master1主机配置

编写keepalived配置文件：

cat > /etc/keepalived/keepalived.conf <<EOF
! Configuration File for keepalived

global_defs {
    router_id HA-M1
    script_user root
}

vrrp_script chk_mysql {
    script /etc/keepalived/chk_mysql.sh
    interval 3
    fall 3
    rise 2
}

vrrp_instance VI_MYSQL {
    state MASTER
    interface ens192
    virtual_router_id 100
    priority 100
    advert_int 1
    unicast_src_ip  10.0.x.125
    unicast_peer {
            10.0.x.23
    }
    
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    
    virtual_ipaddress {
        10.0.x.126/24 dev ens192 label ens192:vip
    }
    
    track_script {
        chk_mysql
    }
}
EOF

说明：

  • router_id​​：标识，两台需不一样

  • ​​state MASTER​​：表明这台是Master角色

​​* priority​​：优先级，MASTER角色比BACKUP高

​​* virtual_router_id​​：虚拟路由编号，两台需要一致，两个节点不能一样

​​* 10.0.x.126/24 dev ens192 label ens192:vip​​：VIP地址是10.0.x.126

​​* script /etc/keepalived/chk_mysql.sh​​：MySQL的检测脚本，定时去检测MySQL进程是否挂掉，会重启，interval是为间隔时间（2s检测一次）

配置Keepalived的MySQL状态监测脚本：

touch /etc/keepalived/chk_mysql.sh
chmod +x /etc/keepalived/chk_mysql.sh

cat > /etc/keepalived/chk_mysql.sh <<EOF
#!/bin/bash
MYSQL=/usr/bin/mysql
MYSQL_HOST=10.0.x.125
MYSQL_USER=root
MYSQL_PASSWORD=test#123
CHECK_TIME=3
  
#mysql  is working MYSQL_OK is 1 , mysql down MYSQL_OK is 0
  
MYSQL_OK=1
  
function check_mysql_helth (){
    $MYSQL -h $MYSQL_HOST -u$MYSQL_USER -p${MYSQL_PASSWORD} -e "show status;" >/dev/null 2>&1
    if [ $? = 0 ] ;then
    MYSQL_OK=1
    else
    MYSQL_OK=0
    fi
    return $MYSQL_OK
}
while [ $CHECK_TIME -ne 0 ]
do
    let "CHECK_TIME -= 1"
    check_mysql_helth
if [ $MYSQL_OK = 1 ] ; then
    CHECK_TIME=0
    exit 0
fi
if [ $MYSQL_OK -eq 0 ] &&  [ $CHECK_TIME -eq 0 ]
then
    systemctl stop keepalived
    exit 1
fi
sleep 1
done
EOF

Master2备机配置

cat > /etc/keepalived/keepalived.conf <<EOF
! Configuration File for keepalived

global_defs {
    router_id HA-M2
    script_user root
}

vrrp_script chk_mysql {
    script /etc/keepalived/chk_mysql.sh
    interval 3
    fall 3
    rise 2
}

vrrp_instance VI_MYSQL {
    state BACKUP
    interface ens192
    virtual_router_id 100
    nopreempt
    priority 50
    advert_int 1
    unicast_src_ip  10.0.x.23
    unicast_peer {
            10.0.x.125
    }
    
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    
    virtual_ipaddress {
        10.0.x.126/24 dev ens192 label ens192:vip
    }
    
    track_script {
        chk_mysql
    }
}
EOF

注意：注意router_id需要不一样，state BACKUP表明是备机，virtual_router_id需要一致，priority要比MASTER低！

配置Keepalived的MySQL状态监测脚本：

touch /etc/keepalived/chk_mysql.sh
chmod +x /etc/keepalived/chk_mysql.sh

cat > /etc/keepalived/chk_mysql.sh <<EOF
#!/bin/bash
MYSQL=/usr/bin/mysql
MYSQL_HOST=10.0.x.23
MYSQL_USER=root
MYSQL_PASSWORD=test#123
CHECK_TIME=3
  
#mysql  is working MYSQL_OK is 1 , mysql down MYSQL_OK is 0

MYSQL_OK=1
  
function check_mysql_helth (){
    $MYSQL -h $MYSQL_HOST -u$MYSQL_USER -p${MYSQL_PASSWORD} -e "show status;" >/dev/null 2>&1
    if [ $? = 0 ] ;then
    MYSQL_OK=1
    else
    MYSQL_OK=0
    fi
    return $MYSQL_OK
}
while [ $CHECK_TIME -ne 0 ]
do
    let "CHECK_TIME -= 1"
    check_mysql_helth
if [ $MYSQL_OK = 1 ] ; then
    CHECK_TIME=0
    exit 0
fi
if [ $MYSQL_OK -eq 0 ] &&  [ $CHECK_TIME -eq 0 ]
then
    systemctl stop keepalived
    exit 1
fi
sleep 1
done
EOF

启动Keepalived

systemctl start keepalived
systemctl enable keepalived

双主高可用测试

数据同步测试

进入​​Master1​​​，创建数据库​​test_db​​​，表​​test_user ​​，并插入一条数据：

docker exec -it mysql_master1 mysql -uroot -ptest#123 -e "CREATE DATABASE test_db;"
docker exec -it mysql_master1 mysql -uroot -ptest#123 -e "CREATE TABLE test_db.test_user (id INT(11),name VARCHAR(25),age int(11) );"
docker exec -it mysql_master1 mysql -uroot -ptest#123 -e "INSERT INTO test_db.test_user values (1,'zhangsan',18);"
docker exec -it mysql_master1 mysql -uroot -ptest#123 -e "SELECT * FROM test_db.test_user;"

查看​​Master1​​​及Master2​​​的数据同步情况：

docker exec -it mysql_master1 mysql -uroot -ptest#123 -e "SELECT * FROM test_db.test_user;"
+------+----------+------+
| id   | name     | age  |
+------+----------+------+
|    1 | zhangsan |   18 |
+------+----------+------+

docker exec -it mysql_master2 mysql -uroot -ptest#123 -e "SELECT * FROM test_db.test_user;"
+------+----------+------+
| id   | name     | age  |
+------+----------+------+
|    1 | zhangsan |   18 |
+------+----------+------+

高可用测试

停止master1主机上的mysql：

docker stop mysql_master1

注意: Master备机上做写操作，第一次写会发生半同步等待（10s左右），第二次写会退化成异步复制。

master1主机恢复测试：

docker start mysql_master1
systemctl start keepalived

小结

如果杀掉master1主机上的mysql：

  • VIP会漂移到master2主机上，ifconfig能看到VIP信息。

  • master2主机的Keepalived从BACKUP状态转换为了MASTER状态。

  • master2主机上做写操作，第一次写会发生半同步等待（10s左右），第二次写会退化成异步复制。

恢复master1主机mysql：

  • ifconfig看下VIP会漂移回来，并且备机上的VIP会消失。

  • 刚才在master2主机上写入的数据同步过来了。

  • Keepalived的状态是从FAULT转变为BACKUP，最后变成MASTER。

添加 mysql 监控

待监控mysql服务需先创建 mysql 监控专用账号，登录其中一台master主机即可：

mysql> CREATE USER 'monitoring'@'%'  IDENTIFIED BY 'test#0131';
Query OK, 0 rows affected (0.09 sec)

mysql> GRANT SELECT, PROCESS, SUPER, REPLICATION CLIENT, RELOAD ON *.* TO 'monitoring'@'%';
Query OK, 0 rows affected (0.01 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.02 sec)

登入consulmanager点击“ 云资源管理 ”，“ MySQL管理 ”，“ 自建MySQL管理 ”，点击新增或导入来添加待监控mysql服务信息。

添加完成后，我们可以通过Mysqld Exporter Dashboard数据平台看到新增的 mysql 资源已经被监控起来了。

参考文档

  • 部署MySQL8+keepalived双主热备高可用.md
举报

相关推荐

MYSQL双主双从，使用Keepalived双机热备+LVS高可用群集

物联网

Centos7.9部署nexus3.70

nexusnexus3.70运维

7.高可用集群架构Keepalived双主热备原理

spring bootspringcloudspringjava服务发现架构

Nginx Keepalived双主热备配置

Nginx

MYSQL集群双主双备双VIP读写分离高可用优化版KEEPALIVED

mysql优先级ipadLinux系统/运维

Nginx+keepalived双主配置(双机双主热备)

NginxNetwork管理nginx

MySql双主+高可用

日记本

两台Nginx keepalived双主热备配置（三）

服务器ipad权重Linux系统/运维

基于Keepalived实现LVS双主高可用集群

服务器nginxkeepalivedlvsLinux系统/运维

CentOS 7.9部署go环境及代码验证

go运行环境搭建可执行二进制程序go代码Go语言编程语言yyds干货盘点
0 条评论