jmeter中执行kafka topic指令
前置条件
kafka版本:2.2.1
jmeter版本:5.3
插件:ApacheJMeter_ssh-1.2.0.jar
1.拷贝 ApacheJMeter_ssh-1.2.0.jar 到/lib/ext目录下
2.拷贝 jsch-0.1.55.jar 到/lib目录下
3.启动jmeter后,在Sampler下找到SSH Command和SSH SFTP进行配置后,可以让jmeter与linux进行交互
kafka 常用指令:
- 查看offset
使用kafka bin路径下kafka-run-class.sh可实现topic 偏移量查看,需要填写的参数如下:
--broker-list | <String: hostname: REQUIRED: The list of hostname and port,...,hostname:port> port of the server to connect to. |
--max-wait-ms | <Integer: ms> DEPRECATED AND IGNORED: The max amount of time each fetch request waits. (default: 1000) |
--offsets | <Integer: count> DEPRECATED AND IGNORED: number of offsets returned (default: 1) |
--partitions | <String: partition ids> comma separated list of partition ids. If not specified, it will find offsets for all partitions (default:) |
--time | <Long: timestamp/-1(latest)/-2 timestamp of the offsets before that (earliest)> (default: -1) |
--topic | <String: topic> REQUIRED: The topic to get offset from. |
查看指定topic偏移量—— --time=-1
./bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list service-kafka-admin-broker-0:9092 -topic test_by_cheliang --time -1
test_by_cheliang:0:0
test_by_cheliang:1:2
test_by_cheliang:2:0
test_by_cheliang:3:0
test_by_cheliang:4:0
test_by_cheliang:5:0
test_by_cheliang:6:0
test_by_cheliang:7:0
test_by_cheliang:8:0
test_by_cheliang:9:0
返回kafka topic所有分区的最大偏移量值
查看指定分区topic偏移量—— --time=-1
./bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list service-kafka-admin-broker-0:9092 --partitions 1 -topic test_by_cheliang -time -1
test_by_cheliang:1:2
返回kafka topic指定分区的最大偏移量值
查看topic偏移量—— --time=-2
./bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list service-kafka-admin-broker-0:9092 --topic test_by_cheliang --time -2
test_by_cheliang:0:0
test_by_cheliang:1:0
test_by_cheliang:2:0
test_by_cheliang:3:0
test_by_cheliang:4:0
test_by_cheliang:5:0
test_by_cheliang:6:0
test_by_cheliang:7:0
test_by_cheliang:8:0
test_by_cheliang:9:0
返回kafka topic指定分区的最小偏移量值
查看topic偏移量——不指定time参数
./bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list service-kafka-admin-broker-0:9092 --topic test_by_cheliang
test_by_cheliang:0:0
test_by_cheliang:1:2
test_by_cheliang:2:0
test_by_cheliang:3:0
test_by_cheliang:4:0
test_by_cheliang:5:0
test_by_cheliang:6:0
test_by_cheliang:7:0
test_by_cheliang:8:0
test_by_cheliang:9:0
不指定time参数,默认返回所有topic分区最大偏移量
./kafka-topic.sh相关
--alter |
Alter the number of partitions,replica assignment, and/orconfiguration for the topic. |
--bootstrap-server <String: server to connect to> |
kafka连接信息 |
--command-config <String: command config property file> |
Property file containing configs to be passed to Admin Client. This is vused only with --bootstrap-server option for describing nd altering broker configs. |
--config <String: name=value> |
A topic configuration override for the topic being created or altered.The following is a list of valid configurations:cleanup.policy compression.type delete.retention.ms |
--create | Create a new topic. |
--delete | Delete a topic. |
--delete-config <String: name> |
A topic configuration override to be removed for an existing topic (see the list of configurations under the --config option). Not supported with the --bootstrap-server option. |
--describe | List details for the given topics. |
--disable-rack-aware | Disable rack aware replica assignment |
--exclude-internal |
exclude internal topics when running list or describe command. The internal topics will be listed by default |
--force | Suppress console prompts |
--help | Print usage information. |
--if-exists |
if set when altering or deleting or describing topics, the action will only execute if the topic exists. Not supported with the --bootstrap- server option. |
--if-not-exists |
if set when creating topics, the action will only execute if the topic does not already exist. Not supported with the --bootstrap- server option. |
--list | List all available topics. |
--replica-assignment <String: |
A list of manual partition-to-broker assignments for the topic being created or altered. |
--replication-factor <Integer: |
The replication factor for each partition in the topic being |
--topic <String: topic> |
The topic to create, alter, describe or delete. It also accepts a regular expression, except for --create option. Put topic name in double quotes and use the '\' prefix to escape regular expression symbols; e.g. "test\.topic". |
--topics-with-overrides |
if set when describing topics, only show topics that have overridden configs |
--unavailable-partitions |
if set when describing topics, only show partitions whose leader is not available |
--under-replicated-partitions |
if set when describing topics, only show under replicated partitions |
--zookeeper <String: hosts> |
DEPRECATED, The connection string for the zookeeper connection in the form host:port. Multiple hosts can be given to allow fail-over. |
--partitions <Integer: # of partitions> |
The number of partitions for the topic being created or altered (WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected |
查看kafka topic
返回所有kafka topic
# ./kafka-topics.sh --bootstrap-server service-kafka-admin-broker-0:9092 --list
test_by_cheliang_mutil
test_by_cheliang_single
__consumer_offsets
返回用户手动创建topic
# ./kafka-topics.sh --bootstrap-server service-kafka-admin-broker-0:9092 --list --exclude-internal
test_by_cheliang_mutil
test_by_cheliang_single
- 创建kafka topic
./kafka-topics.sh --create --bootstrap-server service-kafka-admin-broker-0:9092 --topic hxf_test --replication-factor 1 --partitions 1
WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
查看topic情况
./kafka-topics.sh --bootstrap-server service-kafka-hxfaaa-broker-0:9092 --describe --topic hxf_test
Topic:hxf_test PartitionCount:1 ReplicationFactor:1 Configs:min.insync.replicas=1,segment.bytes=1073741824,max.message.bytes=1000000
Topic: hxf_test Partition: 0 Leader: 0 Replicas: 0 Isr: 0
修改kafka topic属性
# ./kafka-topics.sh --bootstrap-server service-kafka-admin-broker-0:9092 --alter --partitions 10 --topic hxf_test
./kafka-topics.sh --bootstrap-server service-kafka-admin-broker-0:9092 --describe --topic hxf_test
Topic:hxf_test PartitionCount:10 ReplicationFactor:1 Configs:min.insync.replicas=1,segment.bytes=1073741824,max.message.bytes=1000000
Topic: hxf_test Partition: 0 Leader: 0 Replicas: 0 Isr: 0
Topic: hxf_test Partition: 1 Leader: 0 Replicas: 0 Isr: 0
Topic: hxf_test Partition: 2 Leader: 0 Replicas: 0 Isr: 0
Topic: hxf_test Partition: 3 Leader: 0 Replicas: 0 Isr: 0
Topic: hxf_test Partition: 4 Leader: 0 Replicas: 0 Isr: 0
Topic: hxf_test Partition: 5 Leader: 0 Replicas: 0 Isr: 0
Topic: hxf_test Partition: 6 Leader: 0 Replicas: 0 Isr: 0
Topic: hxf_test Partition: 7 Leader: 0 Replicas: 0 Isr: 0
Topic: hxf_test Partition: 8 Leader: 0 Replicas: 0 Isr: 0
Topic: hxf_test Partition: 9 Leader: 0 Replicas: 0 Isr: 0
consumer-group相关
# 查看某消费组(consumer_group)具体消费情况(活跃的消费者以及lag情况等等)
bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group test_group --describe
# 列出当前所有的消费组
bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list
# 旧版
bin/kafka-consumer-groups.sh --zookeeper 127.0.0.1:2181 --group test_group --describe
consumer相关
# 消费数据(从latest消费)
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test_topic
# 消费数据(从头开始消费)
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test_topic --from-beginning
# 消费数据(最多消费多少条就自动退出消费)
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test_topic --max-messages 1
# 消费数据(同时把key打印出来)
bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test_topic --property print.key=true
# 旧版
bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test_topic
producer相关
# 生产数据
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test_topic
# 生产数据(写入带有key的message)
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test_topic --property "parse.key=true" --property "key.separator=:"
最新文章
- ERROR ITMS-90167: ";No .app bundles found in the package";错误
- ViEmu 3.6.0 过期 解除30天限制的方法
- C#泛型文章汇总
- codeforces 727F. Polycarp&#39;s problems
- JavaScript开发原生App模式能否突出重围?
- selenium如何高亮某元素和操作隐藏的内容
- PHP---TP框架---添加数据-----有三种方式
- Android牟利之道(一)--界面嵌入有米广告
- IMAQdx和IMAQ
- 推荐安卓开发神器(里面有各种UI特效和实例)
- Web程序发布后显示个性化图标
- DevExpress控件学习总结 z
- AsyncTask理解- Day36or37
- Two-phase Termination
- linux于test 订购具体解释
- 组件Prop验证
- nyoj 仿射密码
- ES6中新增字符串方法,字符串模板
- xamarin 遇到的奇葩问题
- .net4.0切换2.0时,SplitContainer”的对象强制转换为类型