OpenStack Train 多节点部署实战：3节点（1控制+2计算）环境搭建与核心服务验证

OpenStack Train 多节点生产级部署实战指南

1. 环境规划与基础配置

在开始部署OpenStack Train版本之前，合理的环境规划至关重要。我们采用1控制节点+2计算节点的架构，这种配置既满足生产环境的高可用需求，又保持适度的资源消耗。

硬件建议配置：

• 控制节点：16核CPU/32GB内存/200GB系统盘+100GB数据盘
• 计算节点：根据虚拟机密度需求配置，建议32核CPU/64GB内存/200GB系统盘

网络拓扑设计：

+-------------------+ +-------------------+ | 控制节点 | | 计算节点1 | | eth0:管理网络 |<----->| eth0:管理网络 | | eth1:Provider网络 | | eth1:Provider网络 | +-------------------+ +-------------------+ ^ | v +-------------------+ | 计算节点2 | | eth0:管理网络 | | eth1:Provider网络 | +-------------------+

关键提示：生产环境中建议将管理网络与Provider网络物理隔离，使用不同网卡和交换机

基础服务配置步骤：

1. 所有节点配置静态IP并更新主机名：

# 控制节点 hostnamectl set-hostname controller # 计算节点1 hostnamectl set-hostname compute1 # 计算节点2 hostnamectl set-hostname compute2

2. 统一配置/etc/hosts解析：

10.0.19.133 controller 10.0.19.134 compute1 10.0.19.135 compute2

3. 关闭防火墙和SELinux：

systemctl stop firewalld systemctl disable firewalld setenforce 0 sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

2. 核心服务部署与优化

2.1 时间同步服务配置

多节点环境中时间同步至关重要，我们采用chrony实现纳秒级时间同步：

控制节点配置：

yum install -y chrony cat > /etc/chrony.conf <<EOF server ntp.aliyun.com iburst allow 10.0.19.0/24 local stratum 10 EOF systemctl enable --now chronyd

计算节点配置：

yum install -y chrony cat > /etc/chrony.conf <<EOF server controller iburst EOF systemctl enable --now chronyd

验证时间同步状态：

# 在所有节点执行 chronyc sources -v chronyc tracking

2.2 数据库与消息队列

MariaDB集群配置：

yum install -y mariadb-server mariadb-backup cat > /etc/my.cnf.d/openstack.cnf <<EOF [mysqld] bind-address = 10.0.19.133 default-storage-engine = innodb innodb_file_per_table = on max_connections = 4096 collation-server = utf8_general_ci character-set-server = utf8 EOF systemctl enable --now mariadb

安全加固：

mysql_secure_installation

RabbitMQ消息队列：

yum install -y rabbitmq-server systemctl enable --now rabbitmq-server rabbitmqctl add_user openstack openstack rabbitmqctl set_permissions openstack ".*" ".*" ".*" rabbitmq-plugins enable rabbitmq_management

2.3 Memcached与Etcd

# Memcached yum install -y memcached python-memcached sed -i 's/OPTIONS="-l 127.0.0.1,::1"/OPTIONS="-l 127.0.0.1,::1,controller"/g' /etc/sysconfig/memcached systemctl enable --now memcached # Etcd yum install -y etcd cat > /etc/etcd/etcd.conf <<EOF [Member] ETCD_DATA_DIR="/var/lib/etcd/default.etcd" ETCD_LISTEN_PEER_URLS="http://10.0.19.133:2380" ETCD_LISTEN_CLIENT_URLS="http://10.0.19.133:2379" ETCD_NAME="controller" [Clustering] ETCD_INITIAL_ADVERTISE_PEER_URLS="http://10.0.19.133:2380" ETCD_ADVERTISE_CLIENT_URLS="http://10.0.19.133:2379" ETCD_INITIAL_CLUSTER="controller=http://10.0.19.133:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster-01" ETCD_INITIAL_CLUSTER_STATE="new" EOF systemctl enable --now etcd

3. 认证服务(Keystone)部署

Keystone作为OpenStack的认证核心，需要特别注意安全配置：

# 创建数据库 mysql -uroot -p -e "CREATE DATABASE keystone;" mysql -uroot -p -e "GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'localhost' IDENTIFIED BY 'Mss123456';" mysql -uroot -p -e "GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'%' IDENTIFIED BY 'Mss123456';" # 安装服务 yum install -y openstack-keystone httpd mod_wsgi

配置/etc/keystone/keystone.conf关键参数：

[database] connection = mysql+pymysql://keystone:Mss123456@controller/keystone [token] provider = fernet

初始化数据库和密钥：

su -s /bin/sh -c "keystone-manage db_sync" keystone keystone-manage fernet_setup --keystone-user keystone --keystone-group keystone keystone-manage credential_setup --keystone-user keystone --keystone-group keystone

引导认证服务：

keystone-manage bootstrap --bootstrap-password Mss123456 \ --bootstrap-admin-url http://controller:5000/v3/ \ --bootstrap-internal-url http://controller:5000/v3/ \ --bootstrap-public-url http://controller:5000/v3/ \ --bootstrap-region-id RegionOne

配置Apache服务：

echo "ServerName controller" >> /etc/httpd/conf/httpd.conf ln -s /usr/share/keystone/wsgi-keystone.conf /etc/httpd/conf.d/ systemctl enable --now httpd

4. 计算服务(Nova)部署

4.1 控制节点配置

创建Nova数据库和用户：

mysql -uroot -p -e "CREATE DATABASE nova_api;" mysql -uroot -p -e "CREATE DATABASE nova;" mysql -uroot -p -e "CREATE DATABASE nova_cell0;" mysql -uroot -p -e "GRANT ALL PRIVILEGES ON nova_api.* TO 'nova'@'localhost' IDENTIFIED BY 'Mss123456';" mysql -uroot -p -e "GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'localhost' IDENTIFIED BY 'Mss123456';" mysql -uroot -p -e "GRANT ALL PRIVILEGES ON nova_cell0.* TO 'nova'@'localhost' IDENTIFIED BY 'Mss123456';"

安装服务组件：

yum install -y openstack-nova-api openstack-nova-conductor \ openstack-nova-novncproxy openstack-nova-scheduler

关键配置/etc/nova/nova.conf：

[api_database] connection = mysql+pymysql://nova:Mss123456@controller/nova_api [database] connection = mysql+pymysql://nova:Mss123456@controller/nova [DEFAULT] transport_url = rabbit://openstack:openstack@controller [api] auth_strategy = keystone [keystone_authtoken] www_authenticate_uri = http://controller:5000/ auth_url = http://controller:5000/ memcached_servers = controller:11211 auth_type = password project_domain_name = Default user_domain_name = Default project_name = service username = nova password = Mss123456 [placement] region_name = RegionOne auth_url = http://controller:5000/v3 auth_type = password project_domain_name = Default user_domain_name = Default project_name = service username = placement password = Mss123456

4.2 计算节点配置

安装基础组件：

yum install -y openstack-nova-compute

配置/etc/nova/nova.conf：

[DEFAULT] transport_url = rabbit://openstack:openstack@controller [compute] [keystone_authtoken] www_authenticate_uri = http://controller:5000/ auth_url = http://controller:5000/ memcached_servers = controller:11211 auth_type = password project_domain_name = Default user_domain_name = Default project_name = service username = nova password = Mss123456 [vnc] enabled = true server_listen = 0.0.0.0 server_proxyclient_address = $my_ip novncproxy_base_url = http://controller:6080/vnc_auto.html

检查虚拟化支持：

egrep -c '(vmx|svm)' /proc/cpuinfo # 返回0则需要配置 echo "[libvirt] virt_type = qemu" >> /etc/nova/nova.conf

启动服务：

systemctl enable --now libvirtd openstack-nova-compute

5. 网络服务(Neutron)部署

5.1 控制节点配置

创建数据库：

mysql -uroot -p -e "CREATE DATABASE neutron;" mysql -uroot -p -e "GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'localhost' IDENTIFIED BY 'Mss123456';" mysql -uroot -p -e "GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'%' IDENTIFIED BY 'Mss123456';"

安装组件：

yum install -y openstack-neutron openstack-neutron-ml2 \ openstack-neutron-linuxbridge ebtables

配置/etc/neutron/neutron.conf：

[DEFAULT] core_plugin = ml2 service_plugins = router allow_overlapping_ips = true transport_url = rabbit://openstack:openstack@controller auth_strategy = keystone [keystone_authtoken] www_authenticate_uri = http://controller:5000 auth_url = http://controller:5000 memcached_servers = controller:11211 auth_type = password project_domain_name = default user_domain_name = default project_name = service username = neutron password = Mss123456

配置ML2插件/etc/neutron/plugins/ml2/ml2_conf.ini：

[ml2] type_drivers = flat,vlan,vxlan tenant_network_types = vxlan mechanism_drivers = linuxbridge,l2population extension_drivers = port_security [ml2_type_flat] flat_networks = provider [ml2_type_vxlan] vni_ranges = 1:1000 [securitygroup] enable_ipset = true

6. 部署验证与排错

6.1 服务健康检查

创建检查脚本/root/check_openstack.sh：

#!/bin/bash services=( "nova-api" "nova-scheduler" "nova-conductor" "nova-compute" "neutron-server" "neutron-linuxbridge-agent" "neutron-dhcp-agent" "neutron-metadata-agent" "keystone" "glance-api" ) for service in "${services[@]}"; do status=$(systemctl is-active $service) if [ "$status" != "active" ]; then echo "[ERROR] $service is $status" journalctl -u $service -n 30 --no-pager else echo "[OK] $service is running" fi done # 检查端点 openstack endpoint list

6.2 常见问题解决

问题1：计算节点无法注册

# 在控制节点执行 nova-manage cell_v2 discover_hosts --verbose

问题2：网络服务异常

# 检查OVS状态 ovs-vsctl show # 检查网络命名空间 ip netns list

问题3：镜像上传失败

# 检查存储权限 chown glance:glance /var/lib/glance/images/ # 检查Quota设置 openstack quota show

7. 生产环境优化建议

1. 数据库优化：

   • 配置MariaDB Galera集群
   • 调整innodb_buffer_pool_size为物理内存的70%

2. 消息队列优化：

   rabbitmqctl set_policy HA-all "^" '{"ha-mode":"all"}'

3. 网络性能优化：

   • 启用巨帧(Jumbo Frame)
   • 配置SR-IOV或DPDK加速

4. 监控方案：

   • 使用Prometheus+Granfana监控集群
   • 配置告警规则关键指标：

     - alert: HighCPUUsage expr: 100 - (avg by(instance)(irate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 80 for: 10m

通过以上步骤，我们完成了OpenStack Train版本的三节点生产级部署。实际运维中建议定期执行备份操作，并建立完善的监控告警体系。