confluent部署:

confluent介绍
https://www.cnblogs.com/dadadechengzi/p/9506964.html

kafka connect:
https://www.cnblogs.com/dadadechengzi/p/9506216.html

官网文档:
https://docs.confluent.io/current/

部署:

https://blog.csdn.net/zzq900503/article/details/83419245 GG

https://www.jianshu.com/p/87ae3a822f2f

官网安装:

https://docs.confluent.io/current/quickstart/cos-quickstart.html#cos-quickstart

tail -500 /data1/confluent-5.2.2/logs/connect/kafka-connect.log |grep error

confluent status
confluent start
confluent stop

平台自带监控平台: 适用期30天

Control Center

https://docs.confluent.io/current/control-center/installation/install-apache-kafka.html

Confluent的组件

Confluent Platform 包括更多的工具和服务，使构建和管理数据流平台更加容易。

Confluent Control Center（闭源）。管理和监控Kafka最全面的GUI驱动系统。

Confluent Kafka Connectors（开源）。连接SQL数据库/Hadoop/Hive

Confluent Kafka Clients（开源）。对于其他编程语言，包括C/C++,Python

Confluent Kafka REST Proxy（开源）。允许一些系统通过HTTP和kafka之间发送和接收消息。

Confluent Schema Registry（开源）。帮助确定每一个应用使用正确的schema当写数据或者读数据到kafka中。

confluent:
安装JDK
https://www.cnblogs.com/hongfeng2019/p/11331688.html
下载部署
https://docs.confluent.io/current/installation/installing_cp/zip-tar.html

curl -O http://packages.confluent.io/archive/5.3/confluent-5.3.1-2.12.tar.gz
tar xzf confluent-5.3.1-2.12.tar.gz -C /data1/

添加环境变量:
JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64
JRE_HOME=$JAVA_HOME/jre
CLASS_PATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib
PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin:/data1/confluent-5.2.2/bin

source /etc/profile

可能还配zK

启动
confluent status

测试:
curl localhost:8083/connectors

#如果场景是指用connector,做一个systemctl启动服务

cd /data1/confluent-5.2.2/bin

cd /usr/lib/systemd/system/

ll | grep confluent

[root@bigdata-confluent-1 system]# cat confluent-kafka-connect.service
[Unit]
Description=Apache Kafka Connect - distributed
Documentation=http://docs.confluent.io/
After=network.target

[Service]
Type=simple
Environment="KAFKA_HEAP_OPTS=-Xms1G -Xmx6G"
ExecStart=/data1/confluent-5.2.2/bin/connect-distributed /data1/confluent-5.2.2/etc/schema-registry/connect-avro-distributed.properties
TimeoutStopSec=30
Restart=always

[Install]
WantedBy=multi-user.target

---------------
systemctl daemon-reload
systemctl enable confluent-kafka-connect.service
systemctl star confluent-kafka-connect

/data1/confluent-5.2.2/etc/schema-registry
vim connect-avro-distributed.properties
bootstrap.servers=152.32.140.94:9092,152.32.140.94:19092,152.32.140.143:9092,152.32.140.143:19092

/data1/confluent-5.2.2/bin/connect-distributed /data1/confluent-5.2.2/etc/schema-registry/connect-avro-distributed.properties

巴特西

confluent部署:

最新文章

热门文章