using kafkacat reset kafka offset
2024-09-01 00:21:00
1. install kafkacat
Ubuntu
apt-get install kafkacat
CentOS
install deepenency
yum install librdkafka-devel
download source from github
build source on centos
./configure <usual-configure-options>
make
sudo make install
2. watch the target topic data
lenmom@M1701:~/workspace/software/confluent-community-5.1.-2.11$ bin/kafka-console-consumer --bootstrap-server localhost: --from-beginning --topic connect-offsets --property print.key=true
["jdbc_source_inventory_customers",{"query":"query"}] {"incrementing":}
there is only one record in kafka topic connect-offsets.
3. dump the record from topic
lenmom@M1701:~/workspace/software/confluent-community-5.1.-2.11$ kafkacat -b localhost: -t connect-offsets -C -K# -o-
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
["jdbc_source_inventory_customers",{"query":"query"}]#{"incrementing":}
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
% Reached end of topic connect-offsets [] at offset
the value:
["jdbc_source_inventory_customers",{"query":"query"}]#{"incrementing":1005}
is what we want!
4. use the value get in step 3 as template to and send to the topic again
lenmom@M1701:~/workspace/software/confluent-community-5.1.-2.11$ echo '["jdbc_source_inventory_customers",{"query":"query"}]#{"incrementing":1}' | \
> kafkacat -b localhost: -t connect-offsets -P -Z -K#
here, we modify the incrementing value from 1005 to 1.
for timestamp+increment
echo '["jdbc_source_inventory_orders",{"query":"query"}]#{"timestamp_nanos":0,"incrementing":0,"timestamp":0}' | \
kafkacat -b localhost: -t connect-offsets -P -Z -K#
5. watch the topic again
lenmom@M1701:~/workspace/software/confluent-community-5.1.-2.11$ bin/kafka-console-consumer --bootstrap-server localhost: --from-beginning --topic connect-offsets --property print.key=true
["jdbc_source_inventory_customers",{"query":"query"}] {"incrementing":}
["jdbc_source_inventory_customers",{"query":"query"}] {"incrementing":}
we can see, there are two values with the same key in the topic now.
refernce
https://docs.confluent.io/current/app-development/kafkacat-usage.html
最新文章
- 扩展JS Date对象时间格式化功能
- vs快捷键
- Java EE开发平台随手记6——Mybatis扩展4
- Asp.net操作Excel（终极方法NPOI）(转)
- 理解OAuth 2.0--转
- -_-#【JS】isFinite
- git merge的recursive策略和merge-base
- 当cpu飙升时，找出php中可能有问题的代码行
- JavaSE学习总结第01天_Java概述
- Unity3D ——强大的跨平台3D游戏开发工具（三）
- 关于Makefile,Makefile.in,Makefile.am,Configure功能及相互关系的问题
- CodeForces 1151E Number of Components
- 4、爬虫之mongodb
- linux chkconfig添加开机启动服务
- 文本处理三剑客之 sed
- BZOJ5305 [Haoi2018]苹果树 【组合数学】
- Android-bindService本地服务-音乐播放-上
- vs快捷键代码格式化或代码对齐名字
- 【Sql Server】Sql语句整理
- ambari关于ranger的一个大坑----端口永远是3306，需要手动修改