Posted on

Apache Kafka에 λŒ€ν•œ ν•™μŠ΅μ„ μœ„ν•΄ 도컀 컴포즈둜 카프카에 λŒ€ν•œ ν™˜κ²½μ„ ꡬ성할 λ•Œ λŒ€λΆ€λΆ„ wurstmeister/kafka-docker λ˜λŠ” confluentinc/cp-kafka μ‚¬μš©ν•˜μ—¬ ZooKeeper 기반의 μΉ΄ν”„μΉ΄λ₯Ό μ‹€ν–‰ν•˜λŠ” 방법에 λŒ€ν•΄ μ„€λͺ…ν•˜λŠ” 것을 λ³Ό 수 μžˆλ‹€. λ³Έ κΈ€μ—μ„œλŠ” μ»¨ν”Œλ£¨μ–ΈνŠΈμ—μ„œ μ œκ³΅ν•˜λŠ” confluentinc/confluent-localλ₯Ό μ‚¬μš©ν•΄ KRaft λͺ¨λ“œμ˜ Apache Kafka에 λŒ€ν•œ ν•™μŠ΅ ν™˜κ²½μ„ λ§Œλ“œλŠ” 것을 κ³΅μœ ν•˜κ³ μž ν•œλ‹€.

도컀 컴포즈 λ¬Έμ„œ μž‘μ„±

compose.yaml
version: '3'
services:
  kafka:
    image: confluentinc/confluent-local:7.5.6
    hostname: kafka
    container_name: kafka
    ports:
      - "9092:9092"
      - "8082:8082"
    environment:
      CLUSTER_ID: "EaJs_OcfTB6oWCOPTuY64Q"
      KAFKA_NODE_ID: 1
      KAFKA_PROCESS_ROLES: "broker,controller"
      KAFKA_CONTROLLER_LISTENER_NAMES: "CONTROLLER"
      KAFKA_CONTROLLER_QUORUM_VOTERS: "1@kafka:29093"
      KAFKA_INTER_BROKER_LISTENER_NAME: "PLAINTEXT"
      KAFKA_LISTENERS: "PLAINTEXT://kafka:29092,CONTROLLER://kafka:29093,PLAINTEXT_HOST://0.0.0.0:9092"
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: "CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT"
      KAFKA_ADVERTISED_LISTENERS: "PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092"
      KAFKA_LOG_DIRS: "/tmp/kraft-combined-logs"
      KAFKA_REST_HOST_NAME: rest-proxy
      KAFKA_REST_BOOTSTRAP_SERVERS: "kafka:29092"
      KAFKA_REST_LISTENERS: "http://0.0.0.0:8082"

  kafka-ui: # Optional
    container_name: kafka-ui
    image: provectuslabs/kafka-ui:latest
    ports:
      - "8080:8080"
    depends_on:
      - kafka
    environment:
      KAFKA_CLUSTERS_0_NAME: local
      KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka:29092
      KAFKA_CLUSTERS_0_AUDIT_TOPICAUDITENABLED: 'true'
      KAFKA_CLUSTERS_0_AUDIT_CONSOLEAUDITENABLED: 'true'

confluent-local μ΄λ―Έμ§€λŠ” 개발 ν™˜κ²½μ„ μœ„ν•œ κ²ƒμ΄λ―€λ‘œ ν”„λ‘œλ•μ…˜ ν™˜κ²½μ—μ„œλŠ” cp-all-in-one-kraft λ₯Ό μ°Έκ³ ν•΄μ„œ cp-kafka 둜 직접 κ΅¬μ„±ν•΄μ•Όν•©λ‹ˆλ‹€.

Kafka ν΄λŸ¬μŠ€ν„° 아이디

kafka-storage둜 μΉ΄ν”„μΉ΄μ—μ„œ μ‚¬μš©ν•  ν΄λŸ¬μŠ€ν„° 아이디λ₯Ό λ§Œλ“€μ–΄μ•Όν•˜λ©° kafka-storage λŠ” confluent-local 이미지에 ν¬ν•¨λ˜μ–΄ μžˆμ–΄ μ•„λž˜μ™€ 같이 λͺ…λ Ήμ–΄λ₯Ό μ‹€ν–‰ν•˜μ—¬ 생성할 수 μžˆλ‹€. How to generate Kafka cluster ID μ—μ„œλŠ” kafka-storage λ₯Ό μ‚¬μš©ν•˜μ§€ μ•Šκ³ λ„ Base64둜 μΈμ½”λ”©λœ UUIDλ₯Ό λ§Œλ“œλŠ” 방법에 λŒ€ν•΄μ„œ μ•Œλ €μ£Όκ³  μžˆμœΌλ‹ˆ μ°Έκ³ ν•˜λ©΄ μ’‹κ² λ‹€.

Terminal
# docker run --rm confluentinc/confluent-local:7.5.6 /bin/sh -c "/bin/uuidgen --time | tr -d '-' | base64 | cut -b 1-22"
docker run --rm confluentinc/confluent-local:7.5.6 /bin/kafka-storage random-uuid
EaJs_OcfTB6oWCOPTuY64Q

참고자료