Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems consuming with kafkacat running in docker on WSL2 #301

Open
mikaello opened this issue Mar 3, 2021 · 1 comment
Open

Problems consuming with kafkacat running in docker on WSL2 #301

mikaello opened this issue Mar 3, 2021 · 1 comment

Comments

@mikaello
Copy link

mikaello commented Mar 3, 2021

I got some troubles consuming with kafkacat running in docker on WSL2.

Software versions:

  • Linux distro: Ubuntu 20.04.2 LTS
  • Windows version: 20H2
  • Windows OS build: 19042.804

Example:

# Start Kafka and create a topic, e.g. :
$ confluent local services kafka start
$ kafka-topics --create --topic test --bootstrap-server localhost:9092

$ PRIVATE_IP=$(ip addr show eth0 | grep 'inet\b' | awk '{print $2}' | cut -d/ -f1)

# Metadata with kafkacat local (does also work with broker localhost:9092), SUCCESS:
$ /usr/bin/kafkacat -L -b $PRIVATE_IP:9092 -t test
Metadata for test (from broker -1: 172.25.113.126::9092/bootstrap):
 1 brokers:
  broker 0 at my-computer-mikaello:9092 (controller)
 1 topics:
  topic "test" with 1 partitions:
    partition 0, leader 0, replicas: 0, isrs: 0

# Metadata with kafkacat docker, SUCCESS:
$ docker run -it --network=host edenhill/kafkacat:1.6.0 -L -b $PRIVATE_IP:9092 -t test
Metadata for test (from broker -1: 172.25.113.126:9092/bootstrap):
 1 brokers:
  broker 0 at my-computer-mikaello:9092 (controller)
 1 topics:
  topic "test" with 1 partitions:
    partition 0, leader 0, replicas: 0, isrs: 0

# Consume with kafkacat local, SUCCESS:
$ /usr/bin/kafkacat -C -b $PRIVATE_IP:9092 -t test -e
% Reached end of topic test [0] at offset 0: exiting

# Consume with kafkacat docker, FAILURE:
$ docker run -it --network=host edenhill/kafkacat:1.6.0 -C -b $PRIVATE_IP:9092 -t test -e
%3|1614806333.215|FAIL|rdkafka#consumer-1| [thrd:my-computer-mikaello:9092/0]: my-computer-mikaello:9092/0: Failed to resolve 'my-computer-mikaello:9092': Name does not resolve (after 3ms in state CONNECT)
% ERROR: Topic test [0] error: Failed to query logical offset END: Local: Host resolution failure

Is there any obvious reasons why this fails?

Since I'm trying to consume Avro, I need the docker version of kafkacat. Producing with kafkacat running in docker does not work either.

@whatsupbros
Copy link

In fact, you don't need docker version to consume Avro, but you need to build kafkacat with libavro and libserdes support.
The bootstrap.sh script does this all for you, but you will have to track that all dependencies were built successfully.

Bu trial and error method, I found out that you'll need some additional packages for that, listed in #303

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants