There are two very good client for Apache Kafka written for Python.
which can be installed with
pip install kafka
And then we have the client written by Confluent, the developers of Kafka, https://github.com/confluentinc/confluent-kafka-python which like all their clients are a wrapper around the excellent librdkafka
We recommend to use the client from Confluent since it supports authentication with SASL SCRAM which is what we use at CloudKaraka.
To get started, you first need all the connection variables, which you can get from the provider you used to create the CloudKarafka instance.
c = Consumer(**conf) c.subscribe(topics) while True: msg = c.poll(timeout=1.0) if msg.error(): print(msg.error()) else: sys.stderr.write('%% %s [%d] at offset %d with key %s:\n' % (msg.topic(), msg.partition(), msg.offset(), str(msg.key()))) print(msg.value())
The complete code example, with configuration, can be found here: https://github.com/CloudKarafka/python-kafka-example
Please check the answer to this issue in the FAQ: https://www.cloudkarafka.com/docs/faq.html#connect--failed-to-verify