spark 运行报错：java.lang.AbstractMethodError
报错日志如下：
Caused by: java.lang.AbstractMethodError: sparkCore.JavaWordCount$2.call(Ljava/lang/Object;)Ljava/lang/Iterable;
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.apply(JavaRDDLike.scala:129)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.apply(JavaRDDLike.scala:129)
at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371)
at scala.collection.Iterator$$anon$14.hasNext(Iterator.scala:388)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:189)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:316)
at org.apache.spark.rdd.RDD.saveAsTextFile(RDD.scala:1436)
at org.apache.spark.api.java.JavaRDDLike$class.saveAsTextFile(JavaRDDLike.scala:507)
at org.apache.spark.api.java.AbstractJavaRDDLike.saveAsTextFile(JavaRDDLike.scala:46)
at sparkCore.JavaWordCount.main(JavaWordCount.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.AbstractMethodError: sparkCore.JavaWordCount$2.call(Ljava/lang/Object;)Ljava/lang/Iterable;
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.apply(JavaRDDLike.scala:129)
at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.apply(JavaRDDLike.scala:129)
at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371)
at scala.collection.Iterator$$anon$14.hasNext(Iterator.scala:388)
at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:327)
at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:189)
at org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:64)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
错误原因：
从错误日志看java.lang.AbstractMethodError:sparkCore.JavaWordCount$2.call(Ljava/lang/Object;)Ljava/lang/Iterable;大致可以猜测是存在函数定义冲突。
具体的原因基本上是因为spark执行程序的版本和集群spark版本不一样，如集群spark版本是1.6.*，而执行程序使用的是2.1.*；由于版本差异大，部分函数的
定义发生变化，会造成以上问题。
最新文章
- String类型，Function类型
- SQL Server数据库ROW_NUMBER()函数使用详解
- eclipse出现每次修改代码报heap over错误，然后必须重启tomcat问题，修改过程
- android 反编译操作
- 浅谈C#委托和事件
- Servlet之cookie处理
- Ubuntu18安装Pycharm时遇到的几个问题
- LimeSDR 上手指南
- 远程桌面连接：出现身份验证错误，要求的函数不受支持，可能是由于CredSSP加密Oracle修正的解决方法
- Linux下FastDFS分布式存储-总结及部署记录
- 根据href给当前导航添加样式
- 架构：The Onion Architecture : part 1（洋葱架构：第一篇）（转载）
- 使用eclipse在linux下开发C/C++
- 服务器IP安全策略限制网络访问
- java中Double的isInfinite()和isNaN()
- Bipolar transistor boosts switcher's current by 12 times
- poj1065-Wooden Sticks
- Linux 远程登录——（九）
- linux上安装gitolite和windows上安装tortoisegit及msysgit
- 状压DP【洛谷P1879】 [USACO06NOV]玉米田Corn Fields