Ansible + Docker 部署 Zookeeper 集群
1. 服务器和集群信息
1.1 主机列表
| IP | 主机名 | 内存（GB） | CPU核数 | 磁盘 | 操作系统 | CPU 架构 | 已安装软件 |
|---|---|---|---|---|---|---|---|
| 10.0.0.13 | arc-pro-dc01 my.registry.com | 8 | 1 | 500GB | CentOS 7.9.2009 | x86_64 | ansible 2.9.27 Docker version 28.1.1 Docker Compose version v2.39.2 harbor v2.13.2 |
| 10.0.0.14 | arc-pro-dc02 | 8 | 1 | 500GB | CentOS 7.9.2009 | x86_64 | Docker version 28.1.1 Docker Compose version v2.39.2 |
| 10.0.0.15 | arc-pro-dc03 | 8 | 1 | 500GB | CentOS 7.9.2009 | x86_64 | Docker version 28.1.1 Docker Compose version v2.39.2 |
说明：
- 每个服务器均存在一个管理员用户 admin，该用户可以免密码执行 sudo 命令；
- 在 arc-pro-dc01 机器上，可以使用 admin 用户免密码 ssh 到其他机器；
- 在 arc-pro-dc01 机器上，已经安装好了 Ansible 命令；
- 所有操作均使用 admin 用户完成，Zookeeper 集群的所属用户为 admin；
- 已完成集群基础配置；
- 私有镜像仓库地址：https://my.registry.com:10443。
以上配置参考：
- 使用 VMware Workstation 安装 CentOS-7 虚拟机
- 用 Ansible 批量完成 CentOS 7 操作系统基础配置
- 使用 Ansible 批量安装 Docker
- Docker 私有镜像仓库 Harbor 安装部署带签名认证
1.2 集群规划
| IP | 主机名 | 角色 |
|---|---|---|
| 10.0.0.13 | arc-pro-dc01 | Zookeeper Server |
| 10.0.0.14 | arc-pro-dc02 | Zookeeper Server |
| 10.0.0.15 | arc-pro-dc03 | Zookeeper Server |
1.3 镜像准备
找一个可以连接互联网的、已经安装了 docker 的服务器，下载镜像
docker pull zookeeper:3.6.4
docker image save zookeeper:3.6.4 -o zookeeper.3.6.4.tar.gz
将 zookeeper.3.6.4.tar.gz 上传到本集群任意一台服务器，导入镜像：
docker load -i zookeeper.3.6.4.tar.gz
docker tag zookeeper:3.6.4 my.registry.com:10443/library/zookeeper:3.6.4
# 上传到私服
docker push my.registry.com:10443/library/zookeeper:3.6.4
2. Ansible 文件
2.1 Ansible 目录结构
说明：在 arc-pro-dc01 机器上，执行 ansible 命令的基础目录为 /home/admin/ansible
$ tree /home/admin/ansible/
/home/admin/ansible/
├── ansible.cfg
├── hosts
└── zookeeper├── docker-compose.yml.j2└── start-zookeeper-container.yml
2.2 ansible.cfg
[defaults]
inventory=./hosts
host_key_checking=False
2.3 hosts
[zookeeper]
arc-pro-dc01 zk_ip=10.0.0.13
arc-pro-dc02 zk_ip=10.0.0.14
arc-pro-dc03 zk_ip=10.0.0.15
2.4 docker-compose.yml.j2
services:zoo:image: my.registry.com:10443/library/zookeeper:3.6.4restart: unless-stoppedcontainer_name: zookeeperhostname: {{ inventory_hostname }}network_mode: hostenvironment:ZOO_MY_ID: {{ zoo_id }}ZOO_SERVERS: {% for host in groups['zookeeper'] %}server.{{ loop.index }}={{ hostvars[host].zk_ip }}:2888:3888;2181 {% endfor %}ZOO_LOG4J_PROP: "INFO,ROLLINGFILE"volumes:- /etc/zookeeper:/conf- /data/zookeeper/logs:/logs- /data/zookeeper/data:/data
2.5 start-zookeeper-container.yml
---
- name: Start Zookeeper Containerhosts: zookeeperbecome: truegather_facts: truevars:zk_owner: adminzk_group: admincompose_file_dir: /opt/app/zookeeperzk_conf_dir: /etc/zookeeperzk_log_dir: /data/zookeeper/logszk_data_dir: /data/zookeeper/datatasks:- name: Remove old Zookeeper container if existscommand: docker rm -f zookeeperignore_errors: true- name: Remove Zookeeper directories if they existfile:path: "{{ item }}"state: absentloop:- "{{ compose_file_dir }}"- "{{ zk_conf_dir }}"- "{{ zk_log_dir }}"- "{{ zk_data_dir }}"- name: Create Zookeeper directoriesfile:path: "{{ item }}"state: directoryowner: "{{ zk_owner }}"group: "{{ zk_group }}"mode: '0755'loop:- "{{ compose_file_dir }}"- "{{ zk_conf_dir }}"- "{{ zk_log_dir }}"- "{{ zk_data_dir }}"- name: Set zoo_id based on group indexset_fact:zoo_id: "{{ groups['zookeeper'].index(inventory_hostname) + 1 }}"- name: Deploy docker-compose.ymltemplate:src: docker-compose.yml.j2dest: "{{ compose_file_dir }}/docker-compose.yml"owner: "{{ zk_owner }}"group: "{{ zk_group }}"mode: '0644'- name: Start Zookeeper containercommand: docker-compose -f {{ compose_file_dir }}/docker-compose.yml up -d
3. 部署
$ pwd
/home/admin/ansible$ ansible-playbook zookeeper/start-zookeeper-container.yml# 验证
$ ansible zookeeper -a "docker exec zookeeper zkServer.sh status"
arc-pro-dc01 | CHANGED | rc=0 >>
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: followerZooKeeper JMX enabled by default
Using config: /conf/zoo.cfg
arc-pro-dc03 | CHANGED | rc=0 >>
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: leaderZooKeeper JMX enabled by default
Using config: /conf/zoo.cfg
arc-pro-dc02 | CHANGED | rc=0 >>
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: followerZooKeeper JMX enabled by default
Using config: /conf/zoo.cfg