How to delete a Kafka topic

A Kafka topic is basically an 'entity' where messages are sent. It is essentially the only detail producers and consumers have to agree upon. Topics are not deleted by default; however, this blog post describes how to delete them.

During the lifetime of a Kafka cluster, topics are primarily written and recycled. The data contained in the topic may expire, and the topic will be left empty. However, even when empty, the topic itself will remain. The reasons why the topic is not deleted, even if it is empty, are simple. First, the topic might come in handy in the future. Second, it could have some special settings imprinted by the creator, and Kafka does not have the means to know if a topic is still helpful or not.

Kafka-topics.sh

The utility Kafka-topics.sh is recommended when deleting Kafka topics. If you do not know how to install or use Kafka utils, you can refer to How to view Kafka messages.

Use the utility to view all topics:

Kafka-topics.sh --bootstrap-server [broker0:9092,broker1:9092..] --describe

Then you can pick your topic and delete it:

Kafka-topics.sh --bootstrap-server [broker0:9092,broker1:9092..] --delete --topic my_topic

After hitting enter, Kafka will receive the order to delete the topic in concertation with Zookeeper.

It might take a little while before your topic actually disappears, but it usually happens within a matter of seconds, depending on the size of the topic and how busy the cluster is.

Is the topic not deleted?

If the topic is still there after a long period of time, it probably means that the setting delete.topic.enable https://kafka.apache.org/documentation/#brokerconfigs_delete.topic.enable is set to the default false instead of true.

If that is the case, Kafka will refuse to delete topics. This is easily fixed by changing this setting to true and restarting the brokers.