Ansible + Docker 部署 Zookeeper 集群

1. 服务器和集群信息

1.1 主机列表

IP	主机名	内存（GB）	CPU核数	磁盘	操作系统	CPU 架构	已安装软件
10.0.0.13	arc-pro-dc01 my.registry.com	8	1	500GB	CentOS 7.9.2009	x86_64	ansible 2.9.27 Docker version 28.1.1 Docker Compose version v2.39.2 harbor v2.13.2
10.0.0.14	arc-pro-dc02	8	1	500GB	CentOS 7.9.2009	x86_64	Docker version 28.1.1 Docker Compose version v2.39.2
10.0.0.15	arc-pro-dc03	8	1	500GB	CentOS 7.9.2009	x86_64	Docker version 28.1.1 Docker Compose version v2.39.2

说明：

• 每个服务器均存在一个管理员用户 admin，该用户可以免密码执行 sudo 命令；
• 在 arc-pro-dc01 机器上，可以使用 admin 用户免密码 ssh 到其他机器；
• 在 arc-pro-dc01 机器上，已经安装好了 Ansible 命令；
• 所有操作均使用 admin 用户完成，Zookeeper 集群的所属用户为 admin；
• 已完成集群基础配置；
• 私有镜像仓库地址：https://my.registry.com:10443。

以上配置参考：

• 使用 VMware Workstation 安装 CentOS-7 虚拟机
• 用 Ansible 批量完成 CentOS 7 操作系统基础配置
• 使用 Ansible 批量安装 Docker
• Docker 私有镜像仓库 Harbor 安装部署带签名认证

1.2 集群规划

IP	主机名	角色
10.0.0.13	arc-pro-dc01	Zookeeper Server
10.0.0.14	arc-pro-dc02	Zookeeper Server
10.0.0.15	arc-pro-dc03	Zookeeper Server

1.3 镜像准备

找一个可以连接互联网的、已经安装了 docker 的服务器，下载镜像

docker pull zookeeper:3.6.4
docker image save zookeeper:3.6.4 -o zookeeper.3.6.4.tar.gz

将 zookeeper.3.6.4.tar.gz 上传到本集群任意一台服务器，导入镜像：

docker load -i zookeeper.3.6.4.tar.gz
docker tag zookeeper:3.6.4 my.registry.com:10443/library/zookeeper:3.6.4
# 上传到私服
docker push my.registry.com:10443/library/zookeeper:3.6.4

2. Ansible 文件

2.1 Ansible 目录结构

说明：在 arc-pro-dc01 机器上，执行 ansible 命令的基础目录为 /home/admin/ansible

$ tree /home/admin/ansible/
/home/admin/ansible/
├── ansible.cfg
├── hosts
└── zookeeper├── docker-compose.yml.j2└── start-zookeeper-container.yml

2.2 ansible.cfg

[defaults]
inventory=./hosts
host_key_checking=False

2.3 hosts

[zookeeper]
arc-pro-dc01 zk_ip=10.0.0.13
arc-pro-dc02 zk_ip=10.0.0.14
arc-pro-dc03 zk_ip=10.0.0.15

2.4 docker-compose.yml.j2

services:zoo:image: my.registry.com:10443/library/zookeeper:3.6.4restart: unless-stoppedcontainer_name: zookeeperhostname: {{ inventory_hostname }}network_mode: hostenvironment:ZOO_MY_ID: {{ zoo_id }}ZOO_SERVERS: {% for host in groups['zookeeper'] %}server.{{ loop.index }}={{ hostvars[host].zk_ip }}:2888:3888;2181 {% endfor %}ZOO_LOG4J_PROP: "INFO,ROLLINGFILE"volumes:- /etc/zookeeper:/conf- /data/zookeeper/logs:/logs- /data/zookeeper/data:/data

2.5 start-zookeeper-container.yml

---
- name: Start Zookeeper Containerhosts: zookeeperbecome: truegather_facts: truevars:zk_owner: adminzk_group: admincompose_file_dir: /opt/app/zookeeperzk_conf_dir: /etc/zookeeperzk_log_dir:  /data/zookeeper/logszk_data_dir: /data/zookeeper/datatasks:- name: Remove old Zookeeper container if existscommand: docker rm -f zookeeperignore_errors: true- name: Remove Zookeeper directories if they existfile:path: "{{ item }}"state: absentloop:- "{{ compose_file_dir }}"- "{{ zk_conf_dir }}"- "{{ zk_log_dir }}"- "{{ zk_data_dir }}"- name: Create Zookeeper directoriesfile:path: "{{ item }}"state: directoryowner: "{{ zk_owner }}"group: "{{ zk_group }}"mode: '0755'loop:- "{{ compose_file_dir }}"- "{{ zk_conf_dir }}"- "{{ zk_log_dir }}"- "{{ zk_data_dir }}"- name: Set zoo_id based on group indexset_fact:zoo_id: "{{ groups['zookeeper'].index(inventory_hostname) + 1 }}"- name: Deploy docker-compose.ymltemplate:src: docker-compose.yml.j2dest: "{{ compose_file_dir }}/docker-compose.yml"owner: "{{ zk_owner }}"group: "{{ zk_group }}"mode: '0644'- name: Start Zookeeper containercommand: docker-compose -f {{ compose_file_dir }}/docker-compose.yml up -d

3. 部署

$ pwd
/home/admin/ansible$ ansible-playbook zookeeper/start-zookeeper-container.yml# 验证
$ ansible zookeeper -a "docker exec zookeeper zkServer.sh status"
arc-pro-dc01 | CHANGED | rc=0 >>
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: followerZooKeeper JMX enabled by default
Using config: /conf/zoo.cfg
arc-pro-dc03 | CHANGED | rc=0 >>
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: leaderZooKeeper JMX enabled by default
Using config: /conf/zoo.cfg
arc-pro-dc02 | CHANGED | rc=0 >>
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: followerZooKeeper JMX enabled by default
Using config: /conf/zoo.cfg