要配置 Kafka 集群,需要以下步骤:

  1. 安装 Docker 和 Docker Compose。

  2. 创建一个 Docker Compose 文件,指定 Kafka 和 ZooKeeper 的容器。

  3. 配置 Kafka 和 ZooKeeper 的环境变量,包括集群 ID、节点 ID、监听端口等。

  4. 在 Docker Compose 文件中指定网络,确保 Kafka 和 ZooKeeper 可以相互通信。

  5. 运行 Docker Compose 文件,启动 Kafka 和 ZooKeeper 容器。

以下是一个简单的 Docker Compose 文件示例,用于配置 Kafka 和 ZooKeeper 集群:

version: '2'
services:
  zookeeper-1:
    image: 'zookeeper:3.4'
    ports:
      - '2181:2181'
    environment:
      ZOO_MY_ID: 1
      ZOO_SERVERS: server.1=zookeeper-1:2888:3888,server.2=zookeeper-2:2888:3888,server.3=zookeeper-3:2888:3888
    networks:
      - kafka-net
  zookeeper-2:
    image: 'zookeeper:3.4'
    ports:
      - '2182:2181'
    environment:
      ZOO_MY_ID: 2
      ZOO_SERVERS: server.1=zookeeper-1:2888:3888,server.2=zookeeper-2:2888:3888,server.3=zookeeper-3:2888:3888
    networks:
      - kafka-net
  zookeeper-3:
    image: 'zookeeper:3.4'
    ports:
      - '2183:2181'
    environment:
      ZOO_MY_ID: 3
      ZOO_SERVERS: server.1=zookeeper-1:2888:3888,server.2=zookeeper-2:2888:3888,server.3=zookeeper-3:2888:3888
    networks:
      - kafka-net
  kafka-1:
    image: 'wurstmeister/kafka:2.12-2.5.0'
    ports:
      - '9092:9092'
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:2181,zookeeper-2:2182,zookeeper-3:2183/kafka
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-1:9092
    networks:
      - kafka-net
  kafka-2:
    image: 'wurstmeister/kafka:2.12-2.5.0'
    ports:
      - '9093:9092'
    environment:
      KAFKA_BROKER_ID: 2
      KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:2181,zookeeper-2:2182,zookeeper-3:2183/kafka
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-2:9093
    networks:
      - kafka-net
  kafka-3:
    image: 'wurstmeister/kafka:2.12-2.5.0'
    ports:
      - '9094:9092'
    environment:
      KAFKA_BROKER_ID: 3
      KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:2181,zookeeper-2:2182,zookeeper-3:2183/kafka
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka-3:9094
    networks:
      - kafka-net
networks:
  kafka-net:

这个 Docker Compose 文件定义了三个 ZooKeeper 节点和三个 Kafka 节点,并将它们连接到一个名为 kafka-net 的网络中。每个节点都有自己的 ID 和监听端口。在 Kafka 节点中,KAFKA_ZOOKEEPER_CONNECT 变量指定了 ZooKeeper 集群的连接信息。

要启动这个集群,只需要在命令行中导航到包含 Docker Compose 文件的目录,并运行以下命令:

docker-compose up -d

这将启动所有容器并在后台运行。要停止集群,请运行以下命令:

docker-compose down

这将停止并删除所有容器和网络

docker 如何配置 kafka 集群

原文地址: https://www.cveoy.top/t/topic/hb1y 著作权归作者所有。请勿转载和采集!

免费AI点我,无需注册和登录