Flink升级到1.4版本遇到的坑
2024-08-24 14:33:48
Flink 1.4没出来以前,一直使用Flink 1.3.2,感觉还算稳定,最近将运行环境升级到1.4,遇到了一些坑:
1.需要将可运行程序,基于1.4.0重新编译一次
2.对比了一下flink-conf.yml中的配置,发现一处不同点:
# The port under which the web-based runtime monitor listens.
# A value of -1 deactivates the web server.
1.3.2:jobmanager.web.port: 8081
1.4.0:web.port: 8081
1.4.0 少了前面的“jobmanager”,但是查看官网1.4.0的文档,配置项却是:jobmanager.web.port(https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/config.html)
查看源代码,原来是两个都行,不过后面应该建议都用web开头的,改了还不少:
3.在Configuration的Common Options部分,1.4多了如下的东西:
classloader.resolve-order: Whether Flink should use a child-first ClassLoader when loading
user-code classes or a parent-first ClassLoader. Can be one of parent-first or child-first. (default: child-first) classloader.parent-first-patterns: A (semicolon-separated) list of patterns that specifies
which classes should always be resolved through the parent ClassLoader first.
A pattern is a simple prefix that is checked against the fully qualified class name.
By default, this is set to java.;org.apache.flink.;javax.annotation;org.slf4j;org.apache.log4j;org.apache.logging.log4j;ch.qos.logback.
If you want to change this setting you have to make sure to also include the default patterns in your list of patterns if you want to keep that default behaviour.
Flink程序,原来在1.3.2上跑着没问题,换到1.4后,总是报:
java.lang.LinkageError: loader constraint violation:
loader (instance of org/apache/flink/runtime/execution/librarycache/FlinkUserCodeClassLoaders$ChildFirstClassLoader)
previously initiated loading for a different type with name "scala/collection/Iterable"
查看发布说明,才知道默认类加载顺序变了:
https://flink.apache.org/news/2017/12/12/release-1.4.0.html
在flink-conf.yml中配置了
classloader.resolve-order: parent-first
后问题解决
所以每个版本的发布说明,得仔细看清楚。
最新文章
- Windows Phone开发需要了解的背景
- mvc-5视图和模版
- java中的类实现comparable接口 用于排序
- [C#]Winform下回车或Tab键自动切换下一个控件焦点
- 转:Yii 常量的轻松管理
- uploadify在IE6下的问题
- ViewSwitcher的功能与用法
- python网络爬虫与信息提取 学习笔记day1
- IntelliJ IDEA使用maven-javadoc-plugin生成Java Doc控制台乱码
- SNP (Single Nucleotide Polymorphism), SNV ( single nucleotide variants ) , Indel (insertion-deletion) 的区别
- Java知识汇总——思维导图
- Opencv 视频保存为图像
- CentOS代理设置
- java正则表达式学习
- 模块常用snippet
- 【数组】Container With Most Water
- Lunix/Mac下根据最后修改时间复制文件和文件夹,保持原有的目录结构
- [bzoj3714] [PA2014] Kuglarz(最小生成树)
- ASP.NET MVC4 新手入门教程特别篇之一----Code First Migrations更新数据库结构(数据迁移)修改Entity FrameWork 数据结构(不删除数据)
- 【Docker 命令】- top命令