产生该问题的原因主要是zookeeper
中存在旧版本的kafka-connect topic
信息,导致新版本的kafka-connect
启动异常:
ERROR Unexpected exception in Thread[KafkaBasedLog Work Thread - connect-configs,5,main] (org.apache.kafka.connect.util.KafkaBasedLog:334)
java.lang.IllegalStateException: Consumer is not subscribed to any topics or assigned any partitions
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1109)
at org.apache.kafka.connect.util.KafkaBasedLog.poll(KafkaBasedLog.java:256)
at org.apache.kafka.connect.util.KafkaBasedLog.access$500(KafkaBasedLog.java:69)
at org.apache.kafka.connect.util.KafkaBasedLog$WorkThread.run(KafkaBasedLog.java:327)
解决办法:
(1) 使用kafka
命令列出所有与connect
相关的topic
:
bin/kafka-topics.sh --list --zookeeper 10.255.8.102:2181
输出:
__consumer_offsets
ambari_kafka_service_check
connect-configs
connect-offsets
connect-status
(2)使用kafka
命令删除所有与connect
相关的topic
:
bin/kafka-topics.sh --delete --zookeeper 10.255.8.102:2181 --topic connect-configs
bin/kafka-topics.sh --delete --zookeeper 10.255.8.102:2181 --topic connect-offsets
bin/kafka-topics.sh --delete --zookeeper 10.255.8.102:2181 --topic connect-status
最后验证是否删除:
bin/kafka-topics.sh --list --zookeeper 10.255.8.102:2181
输出:
__consumer_offsets
ambari_kafka_service_check
connect-configs - marked for deletion
connect-offsets - marked for deletion
connect-status - marked for deletion
输出信息中显示三个connect topic
已被标记删除了,要想彻底删除,需要在kafka
的server.properties
配置文件里设置delete.topic.enable=true