这是一个验证贴,因为社区文档是错误的。

先说结论

Flink1.15仅支持ZooKeeper3.5/3.6,不再支持3.4。FLINK-25146 Drop support for Zookeeper 3.4

社区文档

其中的ZooKeeper Versions描述默认使用3.4,可以自行调整为3.5。从文档上来看自1.11版本就添加了这段描述,到目前最新稳定版本(1.16.0)依然是这么描述的。但是实际我们在flink-1.15.0/lib目录中只看到了flink-shaded-zookeeper-3.5.9.jar,在opt目录中看到了flink-shaded-zookeeper-3.6.3.jar,即默认使用3.5版本,并在opt目录中放置了3.6版本。

其实在1.15的 ReleaseNote 中明确写了,不再支持3.4,只是还是有点不死心,做了如下的验证。

Release notes - Flink 1.15

Drop support for Zookeeper 3.4 #

FLINK-25146 #

Support for using Zookeeper 3.4 for HA has been dropped. Users relying on Zookeeper need to upgrade to 3.5/3.6. By default Flink now uses a Zookeeper 3.5 client.

验证一

那么我们是否可以将3.4的版本(flink-1.13.3/lib/flink-shaded-zookeeper-3.4.14.jar)放置到flink-1.15.3/lib目录中,在1.15版本中使用3.4版本呢?

不能

如果我们这么做,将会遇到 java.lang.ClassNotFoundException: org.apache.flink.shaded.curator5.org.apache.curator.framework.api.ACLProvider ,完整堆栈如下:

 1 2022-12-08 18:18:05,693 INFO  org.apache.flink.runtime.rpc.akka.AkkaRpcService             [] - Stopping Akka RPC service.
2 2022-12-08 18:18:05,734 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator [] - Shutting down remote daemon.
3 2022-12-08 18:18:05,736 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator [] - Remote daemon shut down; proceeding with flushing remote transports.
4 2022-12-08 18:18:05,758 INFO akka.remote.RemoteActorRefProvider$RemotingTerminator [] - Remoting shut down.
5 2022-12-08 18:18:05,782 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService [] - Stopped Akka RPC service.
6 2022-12-08 18:18:05,782 ERROR org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Could not start cluster entrypoint YarnApplicationClusterEntryPoint.
7 org.apache.flink.runtime.entrypoint.ClusterEntrypointException: Failed to initialize the cluster entrypoint YarnApplicationClusterEntryPoint.
8 at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:250) ~[flink-dist-1.15.3.jar:1.15.3]
9 at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:711) [flink-dist-1.15.3.jar:1.15.3]
10 at org.apache.flink.yarn.entrypoint.YarnApplicationClusterEntryPoint.main(YarnApplicationClusterEntryPoint.java:112) [flink-dist-1.15.3.jar:1.15.3]
11 Caused by: java.lang.NoClassDefFoundError: org/apache/flink/shaded/curator5/org/apache/curator/framework/api/ACLProvider
12 at org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createZooKeeperHaServices(HighAvailabilityServicesUtils.java:90) ~[flink-dist-1.15.3.jar:1.15.3]
13 at org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createHighAvailabilityServices(HighAvailabilityServicesUtils.java:140) ~[flink-dist-1.15.3.jar:1.15.3]
14 at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.createHaServices(ClusterEntrypoint.java:427) ~[flink-dist-1.15.3.jar:1.15.3]
15 at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.initializeServices(ClusterEntrypoint.java:376) ~[flink-dist-1.15.3.jar:1.15.3]
16 at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:277) ~[flink-dist-1.15.3.jar:1.15.3]
17 at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$1(ClusterEntrypoint.java:227) ~[flink-dist-1.15.3.jar:1.15.3]
18 at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_202]
19 at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_202]
20 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875) ~[hadoop-common-3.0.0-cdh6.3.2.jar:?]
21 at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) ~[flink-dist-1.15.3.jar:1.15.3]
22 at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:224) ~[flink-dist-1.15.3.jar:1.15.3]
23 ... 2 more
24 Caused by: java.lang.ClassNotFoundException: org.apache.flink.shaded.curator5.org.apache.curator.framework.api.ACLProvider
25 at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_202]
26 at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_202]
27 at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[?:1.8.0_202]
28 at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_202]
29 at org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createZooKeeperHaServices(HighAvailabilityServicesUtils.java:90) ~[flink-dist-1.15.3.jar:1.15.3]
30 at org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createHighAvailabilityServices(HighAvailabilityServicesUtils.java:140) ~[flink-dist-1.15.3.jar:1.15.3]
31 at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.createHaServices(ClusterEntrypoint.java:427) ~[flink-dist-1.15.3.jar:1.15.3]
32 at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.initializeServices(ClusterEntrypoint.java:376) ~[flink-dist-1.15.3.jar:1.15.3]
33 at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:277) ~[flink-dist-1.15.3.jar:1.15.3]
34 at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$1(ClusterEntrypoint.java:227) ~[flink-dist-1.15.3.jar:1.15.3]
35 at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_202]
36 at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_202]
37 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875) ~[hadoop-common-3.0.0-cdh6.3.2.jar:?]
38 at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) ~[flink-dist-1.15.3.jar:1.15.3]
39 at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:224) ~[flink-dist-1.15.3.jar:1.15.3]
40 ... 2 more

验证二

如果使用3.5的客户端访问3.4版本服务端,将会提示 org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for 异常,完整堆栈如下:

 1 2022-12-08 15:10:14,935 ERROR org.apache.flink.runtime.entrypoint.ClusterEntrypoint        [] - Fatal error occurred in the cluster entrypoint.
2 org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for /flink/jm_ha_metadata/application_1666841155139_0173
3 at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException.create(KeeperException.java:106) ~[flink-shaded-zookeeper-3-3.5.9-15.0.jar:3.5.9-15.0]
4 at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.KeeperException.create(KeeperException.java:54) ~[flink-shaded-zookeeper-3-3.5.9-15.0.jar:3.5.9-15.0]
5 at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:1538) ~[flink-shaded-zookeeper-3-3.5.9-15.0.jar:3.5.9-15.0]
6 at org.apache.flink.shaded.curator5.org.apache.curator.utils.ZKPaths.mkdirs(ZKPaths.java:351) ~[flink-shaded-zookeeper-3-3.5.9-15.0.jar:3.5.9-15.0]
7 at org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.NamespaceImpl$1.call(NamespaceImpl.java:90) ~[flink-shaded-zookeeper-3-3.5.9-15.0.jar:3.5.9-15.0]
8 at org.apache.flink.shaded.curator5.org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:93) ~[flink-shaded-zookeeper-3-3.5.9-15.0.jar:3.5.9-15.0]
9 at org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.NamespaceImpl.fixForNamespace(NamespaceImpl.java:83) ~[flink-shaded-zookeeper-3-3.5.9-15.0.jar:3.5.9-15.0]
10 at org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.NamespaceImpl.newNamespaceAwareEnsurePath(NamespaceImpl.java:109) ~[flink-shaded-zookeeper-3-3.5.9-15.0.jar:3.5.9-15.0]
11 at org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.CuratorFrameworkImpl.newNamespaceAwareEnsurePath(CuratorFrameworkImpl.java:618) ~[flink-shaded-zookeeper-3-3.5.9-15.0.jar:3.5.9-15.0]
12 at org.apache.flink.runtime.util.ZooKeeperUtils.useNamespaceAndEnsurePath(ZooKeeperUtils.java:729) ~[flink-runtime-1.15.2.jar:1.15.2]
13 at org.apache.flink.runtime.highavailability.zookeeper.ZooKeeperMultipleComponentLeaderElectionHaServices.<init>(ZooKeeperMultipleComponentLeaderElectionHaServices.java:85) ~[flink-runtime-1.15.2.jar:1.15.2]
14 at org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createZooKeeperHaServices(HighAvailabilityServicesUtils.java:96) ~[flink-runtime-1.15.2.jar:1.15.2]
15 at org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createHighAvailabilityServices(HighAvailabilityServicesUtils.java:140) ~[flink-runtime-1.15.2.jar:1.15.2]
16 at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.createHaServices(ClusterEntrypoint.java:427) ~[flink-runtime-1.15.2.jar:1.15.2]
17 at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.initializeServices(ClusterEntrypoint.java:376) ~[flink-runtime-1.15.2.jar:1.15.2]
18 at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:277) ~[flink-runtime-1.15.2.jar:1.15.2]
19 at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$1(ClusterEntrypoint.java:227) ~[flink-runtime-1.15.2.jar:1.15.2]
20 at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_202]
21 at javax.security.auth.Subject.doAs(Subject.java:422) [?:1.8.0_202]
22 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875) [hadoop-common-3.0.0-cdh6.3.2.jar:?]
23 at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41) [flink-runtime-1.15.2.jar:1.15.2]
24 at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:224) [flink-runtime-1.15.2.jar:1.15.2]
25 at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:711) [flink-runtime-1.15.2.jar:1.15.2]
26 at org.apache.flink.yarn.entrypoint.YarnApplicationClusterEntryPoint.main(YarnApplicationClusterEntryPoint.java:112) [flink-dist-1.15.3.jar:1.15.3]
27 2022-12-08 15:10:14,943 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Shutting YarnApplicationClusterEntryPoint down with application status UNKNOWN. Diagnostics Cluster entrypoint has been closed externally..

参考资料:

  1. FLINK-25146Drop support for Zookeeper 3.4
  2. ZooKeeper Version 3.4.x Compatibility

  3. https://nightlies.apache.org/flink/flink-docs-release-1.16/release-notes/flink-1.15/

最新文章

  1. C#[抽象类,接口]
  2. 为什么我们的web前端变的越来越复杂
  3. TextMate 小小心得
  4. Spring监听器配置
  5. [转]JSOM绘制地图区块及航道路径
  6. Java的跨平台原理
  7. Hidden Word
  8. C# Datatable.Select()用法简介
  9. (81)Wangdao.com第十六天_JavaScript 严格模式
  10. htmlUnit加持,网络小蜘蛛的超级进化
  11. java栈的实现复习
  12. 解决video标签在微信中强制全屏、微信全屏播放(Android和IOS)
  13. 未能加载文件或程序集&ldquo;Newtonsoft.Json&rdquo;或它的某一个依赖项。找到的程序集清单定义与程序集引用不匹配。 (异常来自 HRESULT:0x80131040)
  14. Android MarsDaemon实现进程及Service常驻
  15. 列表中不限制宽度,hover时,字体font-weight:bold,防止抖动
  16. Docker swarm 使用服务编排部署lnmp
  17. eval函数的作用
  18. SQLite学习手册(转)
  19. 好几次的CSS存档
  20. CF 1005B Delete from the Left 【模拟数组操作/正难则反】

热门文章

  1. Jenkins用户管理
  2. JUC学习笔记——进程与线程
  3. Unity之&quot;诡异&quot;的协程
  4. Windows Server 2019 安装 Oracle 19C RAC(VMWare虚拟机环境)
  5. 篇(18)-Asp.Net Core入门实战-文章管理之文章内容管理(下拉框二级结构递归)
  6. Linux C++目标中添加git版本信息
  7. 腾讯会议如何在Linux下的安装与使用
  8. 螺旋矩阵II-LeetCode59 考验代码能力
  9. js中数组追加和删除
  10. bug处理记录:java.util.UnknownFormatConversionException: Conversion = &#39;Y&#39;