通过ansible自动化安装openstack-victoria
1、准备工作
1.1 实验拓扑
1.2 主机信息
系统版本CentOS8.5
主机名 | IP地址1 | IP地址2 | 说明 | 配置 |
OpenStack-controller | 192.168.0.100 | 172.16.24.100 | 控制节点 | 4核8G |
OpenStack-compute1 | 192.168.0.101 | 172.16.24.101 | 计算节点1 | 4核8G |
OpenStack-compute2 | 192.168.0.102 | 172.16.24.102 | 计算节点2 | 4核8G |
OpenStack-compute3 | 192.168.0.103 | 172.16.24.103 | 计算节点3 | 4核8G |
1.3 控制节点准备的软件仓库
控制节点软件仓库设置
[BaseOS]
name=CentOS-$releasever - Base - repo.huaweicloud.com
baseurl=https://repo.huaweicloud.com/centos-vault/8.5.2111/BaseOS/$basearch/os/
#mirrorlist=https://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=BaseOS&infra=$infra
gpgcheck=1
gpgkey=https://repo.huaweicloud.com/centos/RPM-GPG-KEY-CentOS-Official
#released updates
[AppStream]
name=CentOS-$releasever - AppStream - repo.huaweicloud.com
baseurl=https://repo.huaweicloud.com/centos-vault/8.5.2111/AppStream/$basearch/os/
#mirrorlist=https://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=AppStream&infra=$infra
gpgcheck=1
gpgkey=https://repo.huaweicloud.com/centos/RPM-GPG-KEY-CentOS-Official
[PowerTools]
name=CentOS-$releasever - PowerTools - repo.huaweicloud.com
baseurl=https://repo.huaweicloud.com/centos-vault/8.5.2111/PowerTools/$basearch/os/
#mirrorlist=https://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=PowerTools&infra=$infra
gpgcheck=1
gpgkey=https://repo.huaweicloud.com/centos/RPM-GPG-KEY-CentOS-Official
[cloud]
name=cloud
baseurl=https://repo.huaweicloud.com/centos-vault/8.5.2111/cloud/$basearch/openstack-victoria/
#mirrorlist=https://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=cloud&infra=$infra
gpgchecke=1
enbaled=1
gpgkey=https://repo.huaweicloud.com/centos/RPM-GPG-KEY-CentOS-Official
[rabbitmq-38]
name=rabbitmq-38
baseurl=https://repo.huaweicloud.com/centos-vault/8.5.2111/messaging/$basearch/rabbitmq-38/
#mirrorlist=https://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=messaging&infra=$infra
gpgchecke=1
enbaled=1
gpgkey=https://repo.huaweicloud.com/centos/RPM-GPG-KEY-CentOS-Official
[epel]
name=epel
baseurl=https://repo.huaweicloud.com/epel/8/Everything/$basearch/
gpgchecke=1
enbaled=1
gpgkey=https://repo.huaweicloud.com/centos/RPM-GPG-KEY-CentOS-Official1.4 hosts文件设置
template/hosts.j2
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.0.100 controller
192.168.0.101 compute1
192.168.0.102 compute2
192.168.0.103 compute31.5 bashrc文件设置
template.bash_temp.j2
# .bashrc
# User specific aliases and functions
alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'
alias ping='ping -c4 -i 0.1'
alias egrep="egrep -v '^#|^$' "
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
export PS1='[\[\033[31m\]\u\[\033[0m\]@\[\033[32m\]\h\[\033[0m\] \[\033[35m\]\w\[\033[0m\]]\$ '1.6 初始化ansible配置
yum -y install ansible rhel-system-roles
mkdir -p /root/Desktop/ansible/Openstack_ansible
cp -a /usr/share/ansible/roles /root/Desktop/ansible/Openstack_ansible/roles
cd /root/Desktop/ansible/Openstack_ansible
cat > inventory.yml <<END
---
OpenStack-victoria:
children:
controller:
hosts:
controller:
ansible_host: 192.168.0.100
ansible_ssh_password: '1'
ansible_user: root
host_name: "OpenStack-controller"
computes:
hosts:
compute1:
ansible_host: 192.168.0.101
ansible_ssh_password: '1'
ansible_user: root
host_name: "OpenStack-compute1"
compute2:
ansible_host: 192.168.0.102
ansible_ssh_password: '1'
ansible_user: root
host_name: "OpenStack-compute2"
compute3:
ansible_host: 192.168.0.103
ansible_ssh_password: '1'
ansible_user: root
host_name: "OpenStack-compute3"
END
cat > ansible.cfg <<END
[defaults]
inventory=./inventory.yml
remote_user=root
roles_path=roles
END1.7 ansible初始化测试
ansible-inventory --graph
@all:
|--@ungrouped:
|--@OpenStack-yoga:
| |--@controller:
| | |--controller
| |--@computes:
| | |--compute1
| | |--compute2
| | |--compute3ansible-galaxy list
# /root/Desktop/ansible-code/OpenStack-yoga版本自动化安装/roles
- rhel-system-roles.firewall, (unknown version)
- rhel-system-roles.ha_cluster, (unknown version)
- rhel-system-roles.kdump, (unknown version)
- rhel-system-roles.metrics, (unknown version)
- rhel-system-roles.selinux, (unknown version)
- rhel-system-roles.vpn, (unknown version)
- linux-system-roles.metrics, (unknown version)
- linux-system-roles.nbde_client, (unknown version)
- linux-system-roles.nbde_server, (unknown version)
- linux-system-roles.network, (unknown version)
...1.8 ssh服务初始化（取消指纹）
ssh_auth.yml
---
- name: 初始化配置
hosts: localhost
tasks:
- name: 关闭SSH指纹
shell: >
sshpass -p 1
ssh -o StrictHostKeyChecking=no root@{{ item }}
"sed -i 's/^#.*StrictHostKeyChecking.*/StrictHostKeyChecking no/' /etc/ssh/ssh_config"
loop:
- 192.168.0.100
- 192.168.0.101
- 192.168.0.102
- 192.168.0.1031.9 系统初始化
set_init.yml
---
- name: 初始化配置
hosts: all
tasks:
- name: Set a hostname
ansible.builtin.hostname:
name: "{{ host_name }}"
- name: Remove /etc/yum.repos.d/ file
ansible.builtin.file:
path: /etc/yum.repos.d
state: absent
- name: Create a directory if it does not exist
ansible.builtin.file:
path: /etc/yum.repos.d
state: directory
mode: '0755'
- name: Create a repos file
ansible.builtin.template:
src: template/openstack_victoria.repo.j2
dest: /etc/yum.repos.d/openstack_victoria.repo
mode: '0644'
- name: Remove /root/.bashrc file
ansible.builtin.file:
path: /root/.bashrc
state: absent
- name: set .bashrc
ansible.builtin.template:
src: template/bash_temp.j2
dest: /root/.bashrc
mode: '0644'
- name: Remove /etc/hosts
ansible.builtin.file:
path: /etc/hosts
state: absent
- name: set hosts
ansible.builtin.template:
src: template/hosts.j2
dest: /etc/hosts
mode: '0644'
- name: 加载rpm key
rpm_key:
state: present
key: https://repo.huaweicloud.com/centos/RPM-GPG-KEY-CentOS-Official
- name: 重新加载yum缓存
shell: |
yum makecache
- name: 安装常用软件
ansible.builtin.yum:
name:
- bash-completion
- bind-utils
- net-tools
state: latest
- name: 安装一个过滤小工具
ansible.builtin.template:
src: template/quick_grep.j2
dest: /usr/local/bin/quick_grep
setype: bin_t
mode: 755模板文件template/quick_grep.j2
# 过滤工具
#！/bin/bash
grep -Ev '^$|^#' $1 > /var/log/temp.txt
cat /var/log/temp.txt > $11.10 网络初始化(手动)
192.168.0.0/24这个网段为DHCP自动获取地址，也可以手动设置
nmcli connection modify eth0 ipv4.addresses 192.168.0.100/24 ipv4.gateway 192.168.0.1 ipv4.dns 114.114.114.114 ipv4.method manual connection.autoconnect yes
nmcli connection modify eth1 ipv4.addresses 172.16.24.100/24 ipv4.method manual connection.autoconnect yesnmcli connection modify eth0 ipv4.addresses 192.168.0.101/24 ipv4.gateway 192.168.0.1 ipv4.dns 114.114.114.114 ipv4.method manual connection.autoconnect yes
nmcli connection modify eth1 ipv4.addresses 172.16.24.101/24 ipv4.method manual connection.autoconnect yesnmcli connection modify eth0 ipv4.addresses 192.168.0.102/24 ipv4.gateway 192.168.0.1 ipv4.dns 114.114.114.114 ipv4.method manual connection.autoconnect yes
nmcli connection modify eth1 ipv4.addresses 172.16.24.102/24 ipv4.method manual connection.autoconnect yesnmcli connection modify eth0 ipv4.addresses 192.168.0.103/24 ipv4.gateway 192.168.0.1 ipv4.dns 114.114.114.114 ipv4.method manual connection.autoconnect yes
nmcli connection modify eth1 ipv4.addresses 172.16.24.103/24 ipv4.method manual connection.autoconnect yes1.11 网络初始化（ansible）
当存在多块网卡需要配置IP地址的时候，手动方式就显得很蛮烦了，而且一旦数量过多就可能出错，这个时候通过编辑主机变量文件+自动化配置网络role会极大的提高工作效率！
host_vars/controller
---
eth0_ip: 192.168.0.100/24
eth1_ip: 172.24.1.100/24host_vars/node1
---
eth0_ip: 192.168.0.101/24
eth1_ip: 172.24.1.101/24host_vars/node2
---
eth0_ip: 192.168.0.102/24
eth1_ip: 172.24.1.102/24host_vars/node3
---
eth0_ip: 192.168.0.103/24
eth1_ip: 172.24.1.103/24network_init.yml
# 注意：只能指定其他网卡的IP，原因也很简单，如果修改连接网卡IP会导致网络故障，ansible失去连接
---
- name: 设置node1节点eth2单独ip地址
hosts: all
vars:
network_connections:
- name: eth1
state: up
type: ethernet
interface_name: eth1
ip:
dhcp4: no
auto6: no
address:
- "{{ eth1_ip }}"
roles:
- rhel-system-roles.network1.12 执行ansible
ansible-playbook ssh_auth.yml
ansible-playbook set_init.yml
ansible-playbook network_init.yml2、OpenStack环境准备
2.1 安全配置
密码变量名 | 密码 | 说明 |
| 49e74247c02bd869eafa | 数据库root密码 |
| 6f82d8fe44c25f9cf8ee | admin服务密码 |
| df864567e439adea5882 | 数据库CINDER（块存储）密码 |
| 89c966ebcc7d5bfa91e5 | CINDER服务密码 |
| 13c2b11fda1b48df489e | Dashboard密码 |
| 025e448d0d205969fb24 | DEMO服务密码 |
| 9686aaab1bc6714fe37f | 数据库GLANCE（镜像服务）密码 |
| 4e4fd3eb169ff4eb074c | GLANCE服务密码 |
| ff076b3d276f749f54c0 | 数据库KEYSTONE（认证服务）密码 |
| 762b19300910aea1b774 | 元数据代理密码 |
| 6f259e345bcf8c0596a1 | 数据库NEUTRON（网络服务）密码 |
| 0024c6530c6d010c366b | 数据库NEUTRON密码 |
| fedeb6d02a53c7aabaff | NOVA（计算服务）服务密码 |
| 4a89e97c0967150fafd1 | NOVA服务密码（计算服务） |
| 7d2b2d2a08742b8eedb9 | Placement服务密码 |
| 4449f6d8a81c7c15c853 | RabbitMQ服务密码（消息队列服务） |
密码生成方式，以下是一条命令直接复制即可
for i in {SQL_PASS,ADMIN_PASS,CINDER_DBPASS,CINDER_PASS,DASH_DBPASS,DEMO_PASS,GLANCE_DBPASS,GLANCE_PASS,KEYSTONE_DBPASS,METADATA_SECRET,NEUTRON_DBPASS,NEUTRON_PASS,NOVA_DBPASS,NOVA_PASS,PLACEMENT_PASS,RABBIT_PASS}
do echo $i: `openssl rand -hex 10`
done2.2 定义密码文件
vars/service_password.yml
---
SQL_PASS: 49e74247c02bd869eafa
ADMIN_PASS: 6f82d8fe44c25f9cf8ee
CINDER_DBPASS: df864567e439adea5882
CINDER_PASS: 89c966ebcc7d5bfa91e5
DASH_DBPASS: 13c2b11fda1b48df489e
DEMO_PASS: 025e448d0d205969fb24
GLANCE_DBPASS: 9686aaab1bc6714fe37f
GLANCE_PASS: 4e4fd3eb169ff4eb074c
KEYSTONE_DBPASS: ff076b3d276f749f54c0
METADATA_SECRET: 762b19300910aea1b774
NEUTRON_DBPASS: 6f259e345bcf8c0596a1
NEUTRON_PASS: 0024c6530c6d010c366b
NOVA_DBPASS: fedeb6d02a53c7aabaff
NOVA_PASS: 4a89e97c0967150fafd1
PLACEMENT_PASS: 7d2b2d2a08742b8eedb9
RABBIT_PASS: 4449f6d8a81c7c15c8532.3 配置NTP服务器
2.3.1 编辑playbook
要正确地同步节点间的服务，可以安装NTP的实现Chrony服务进行时间同步
# openstack_ntp_controller.yml
---
- name: 控制节点配置NTP服务
hosts: controller
tasks:
- name: 安装软件
ansible.builtin.yum:
name:
- chrony
state: latest
- name: 编辑controller chronyd配置文件
ansible.builtin.template:
src: template/chrony.conf_controller.j2
dest: /etc/chrony.conf
mode: '0644'
- name: 开机启动/开启chronyd服务
ansible.builtin.service:
name: chronyd
enabled: yes
state: started
# openstack_ntp_compute.yml
---
- name: 计算节点配置NTP服务
hosts: computes
tasks:
- name: 安装软件
ansible.builtin.yum:
name:
- chrony
state: latest
- name: 编辑compute chronyd配置文件
ansible.builtin.template:
src: template/chrony.conf_compute.j2
dest: /etc/chrony.conf
mode: '0644'
- name: 开机启动/开启chronyd服务
ansible.builtin.service:
name: chronyd
enabled: yes
state: started2.3.2 准备模板文件
模板文件template/chrony.conf_compute.j2
pool 2.centos.pool.ntp.org iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
allow 0.0.0.0
keyfile /etc/chrony.keys
leapsectz right/UTC
logdir /var/log/chrony模板文件template/chrony.conf_controller.j2
pool controller iburst
driftfile /var/lib/chrony/drift
makestep 1.0 3
rtcsync
allow 0.0.0.0
keyfile /etc/chrony.keys
leapsectz right/UTC
logdir /var/log/chrony2.3.3 执行playbook
ansible-playbook ntp_controller.yml
ansible-playbook ntp_compute.yml2.3.4 检查NTP是否配置完成
检查命令：chronyc sources
[root@OpenStack-controller ~]# chronyc sources
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^+ ntp7.flashdance.cx 2 6 377 31 -13ms[ -13ms] +/- 187ms
^- time.cloudflare.com 3 6 141 99 +73ms[ +79ms] +/- 174ms
^* localhost 2 6 377 39 +4831us[ +11ms] +/- 63ms
^- ntp8.flashdance.cx 2 6 277 34 -7506us[-7506us] +/- 186ms
[root@OpenStack-compute1 ~]# chronyc sources
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^? controller 0 7 0 - +0ns[ +0ns] +/- 0ns2.4 安装openstack软件包
openstack_packages.yml
---
- name: 安装OpenStack软件包
hosts: all
tasks:
- name: 安装软件
ansible.builtin.yum:
name:
- python3-openstackclient
- openstack-selinux
state: latest
disable_gpg_check: true2.5 安装数据库
2.5.1 编辑playbook
openstack_mariadb.yml
---
- name: 安装OpenStack软件包
hosts: controller
vars_files:
- vars/service_password.yml
tasks:
- name: 安装软件
ansible.builtin.yum:
name:
- mariadb
- mariadb-server
# - python2-PyMySQL
- python3-PyMySQL
state: latest
- name: openstack数据库配置文件
ansible.builtin.template:
src: template/my.cnf.d/openstack.cnf.j2
dest: /etc/my.cnf.d/openstack.cnf
mode: 0644
vars:
# bind_address: "{{ ansible_default_ipv4['address'] }}/24"
# 监听所有IP地址，除非ansible在控制节点上，可以监听一个地址，我ansible在其他主机上……
bind_address: "*"
- name: 开机启动/开启mariadb
ansible.builtin.service:
name: mariadb
enabled: yes
state: started
- name: 更新mariadb root 密码
mysql_user:
name: root
host_all: yes
update_password: always
password: "{{ SQL_PASS }}"
- name: 身份验证凭据复制到根主目录
template:
src: template/my.cnf.d/my.cnf.j2
dest: /root/.my.cnf
- name: 移除匿名用户登录(anonymous)
community.mysql.mysql_user:
login_user: root
name: ''
host: localhost
state: absent
- name: 移除测试数据库（test）
mysql_db:
login_user: root
name: 'test'
state: absent2.5.2 mariadb测试
检查是否存在.my.cnf文件并且能够免密登录mysql
[root@OpenStack-controller ~]# cat .my.cnf
[client]
user=root
password="49e74247c02bd869eafa"
[root@OpenStack-controller ~]# mysql
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 12
Server version: 10.3.28-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> \q
Bye
[root@OpenStack-controller ~]#2.6 配置消息队列
OpenStack通过消息队列来协调服务之间的操作和状态信息。消息队列服务通常运行在控制节点上。OpenStack支持RabbitMQ、Qpid、ZeroMQ等多种消息队列服务。但是，大多数打包OpenStack的发行版都支持特定的消息队列服务。
openstack_RabbitMQ.yml
---
- name: 配置消息队列
hosts: controller
vars_files:
- vars/service_password.yml
tasks:
- name: 安装软件
ansible.builtin.yum:
name:
- rabbitmq-server
state: latest
disable_gpg_check: true
- name: 开机启动/开启rabbitmq-server
ansible.builtin.service:
name: rabbitmq-server
enabled: yes
state: started
- name: 添加openstack用户
shell: |
rabbitmqctl add_user openstack {{ RABBIT_PASS }};
rabbitmqctl set_permissions openstack ".*" ".*" ".*"
ignore_errors: yes2.7 配置Memcached
服务的身份服务身份验证机制使用Memcached缓存令牌。memcached服务通常运行在控制节点上。
对于生产部署，建议启用防火墙、身份验证和加密的组合来保护它。
2.7.1 编辑playbook
openstack_memcache.yml
---
- name: 配置缓存服务
hosts: controller
vars_files:
- vars/service_password.yml
tasks:
- name: 安装软件
ansible.builtin.yum:
name:
- memcached
- python3-memcached
state: latest
disable_gpg_check: true
- name: 编辑配置文件/etc/sysconfig/memcached
ansible.builtin.template:
src: template/memcached.j2
dest: /etc/sysconfig/memcached
mode: 0644
- name: 开机启动/开启memcached
ansible.builtin.service:
name: memcached
enabled: yes
state: started2.7.2 准备模板文件
模板文件template/memcached.j2
PORT="11211"
USER="memcached"
MAXCONN="1024"
CACHESIZE="64"
OPTIONS="-l 127.0.0.1,::1,{{ inventory_hostname }}"2.8 配置etcd
OpenStack服务可能会使用Etcd，这是一种分布式可靠的key-value存储，用于分布式密钥锁定、存储配置、跟踪服务寿命等场景。
2.8.1 编辑playbook
openstack_etcd.yml
---
- name: 配置etcd服务
hosts: controller
vars_files:
- vars/service_password.yml
tasks:
- name: 安装软件
ansible.builtin.yum:
name:
- etcd
state: latest
disable_gpg_check: true
- name: 编辑配置文件/etc/sysconfig/memcached
ansible.builtin.template:
src: template/etcd.conf.j2
dest: /etc/etcd/etcd.conf
mode: 0644
vars:
controller_ip: "{{ ansible_default_ipv4['address'] }}"
- name: 开机启动/开启etcd
ansible.builtin.service:
name: etcd
enabled: yes
state: started2.8.2 准备模板文件
模板文件template/etcd.conf.j2
#[Member]
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="http://{{ controller_ip }}:2380"
ETCD_LISTEN_CLIENT_URLS="http://{{ controller_ip }}:2379"
ETCD_NAME="controller"
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://{{ controller_ip }}:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://{{ controller_ip }}:2379"
ETCD_INITIAL_CLUSTER="controller=http://{{ controller_ip }}:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster-01"
ETCD_INITIAL_CLUSTER_STATE="new"
