1 详细异常

ERROR scheduler.JobScheduler: Error running job streaming job  ms.
org.apache.spark.SparkException: Job aborted due to stage failure: Task in stage 0.0 failed times,
most recent failure: Lost task 0.3 in stage 0.0 (TID , , executor ): ExecutorLostFailure (executor exited caused by one of the running tasks) Reason: Executor heartbeat timed out after ms
Driver stacktrace:
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$failJobAndIndependentStages(DAGScheduler.scala:)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$.apply(DAGScheduler.scala:)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$.apply(DAGScheduler.scala:)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:)
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$.apply(DAGScheduler.scala:)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$.apply(DAGScheduler.scala:)
at scala.Option.foreach(Option.scala:)
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:)
at org.apache.spark.util.EventLoop$$anon$.run(EventLoop.scala:)
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$.apply(RDD.scala:)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$.apply(RDD.scala:)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:)
at org.apache.spark.rdd.RDD.foreachPartition(RDD.scala:)
at com.wm.bigdata.phoenix.etl.WmPhoniexEtlToHbase$$anonfun$main$.apply(WmPhoniexEtlToHbase.scala:)
at com.wm.bigdata.phoenix.etl.WmPhoniexEtlToHbase$$anonfun$main$.apply(WmPhoniexEtlToHbase.scala:)
at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$$$anonfun$apply$mcV$sp$.apply(DStream.scala:)
at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$$$anonfun$apply$mcV$sp$.apply(DStream.scala:)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$$$anonfun$apply$mcV$sp$.apply$mcV$sp(ForEachDStream.scala:)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$$$anonfun$apply$mcV$sp$.apply(ForEachDStream.scala:)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$$$anonfun$apply$mcV$sp$.apply(ForEachDStream.scala:)
at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$.apply$mcV$sp(ForEachDStream.scala:)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$.apply(ForEachDStream.scala:)
at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$.apply(ForEachDStream.scala:)
at scala.util.Try$.apply(Try.scala:)
at org.apache.spark.streaming.scheduler.Job.run(Job.scala:)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$.apply$mcV$sp(JobScheduler.scala:)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$.apply(JobScheduler.scala:)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$.apply(JobScheduler.scala:)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:)
at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:)
at java.lang.Thread.run(Thread.java:)

2 查询Stack Overflow里面问答

 
3 解决
提交spark submit任务的时候,加大超时时间设置
--conf spark.network.timeout  --conf spark.executor.heartbeatInterval=   --conf spark.driver.maxResultSize=4g

最新文章

  1. sql练习(mysql版)
  2. Flapper Bird的学习笔记(一)
  3. ini_set("display_errors","On");和error_reporting(E_ALL);
  4. 【Bootstrap】2.作品展示站点
  5. JDBC常用接口详解
  6. 模拟 POJ 1573 Robot Motion
  7. PHP写一段代码,确保多个进程同时写入一个文件成功
  8. 点击TableView中某行进入下一级界面(Swift)
  9. VC++编程中为程序加入启动画面功能
  10. 转: 透过CAT,来看分布式实时监控系统的设计与实现
  11. PHP生成表格
  12. POJ3608(旋转卡壳--求两凸包的最近点对距离)
  13. cocos 水果机,老Tiger虎机流水灯,博彩大转盘效果
  14. 下载jar包的网站
  15. Delphi之Exception获得错误信息
  16. ADO.NET基础学习 一(连接数据库)
  17. UrlDecode
  18. vs2008快捷键一览表
  19. P1279 字串距离
  20. Android应用开发-Activity

热门文章

  1. Postman系列之发送请求(一)
  2. OpenStack 的单元测试
  3. Python进阶学习之特殊方法实例详析
  4. elk、kafka、zookeeper 安装
  5. Python列表排序
  6. Autofac实现AOP拦截
  7. MFC中png格式图片贴图的实现
  8. [转载]Jupyter Notebook 的快捷键
  9. EVE-NG TOOLKIT在线安装
  10. 51CTO下载-html+javascript+css学习宝典