0
点赞
收藏
分享

微信扫一扫

Spark java.lang.ClassCastException

闲嫌咸贤 2022-07-27 阅读 120


详细报错信息:

Driver stacktrace:
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1588)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1576)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1575)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1575)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:833)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:833)
at scala.Option.foreach(Option.scala:257)
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:833)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1809)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1758)
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1747)
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:641)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2070)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2091)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2110)
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2135)
at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:943)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
at org.apache.spark.rdd.RDD.collect(RDD.scala:942)
at org.apache.spark.sql.execution.SparkPlan.executeCollect(SparkPlan.scala:278)
at org.apache.spark.sql.Dataset$$anonfun$count$1.apply(Dataset.scala:2434)
at org.apache.spark.sql.Dataset$$anonfun$count$1.apply(Dataset.scala:2433)
at org.apache.spark.sql.Dataset$$anonfun$55.apply(Dataset.scala:2841)
at org.apache.spark.sql.execution.SQLExecution$.withNewQueryId(SQLExecution.scala:57)
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:101)
at org.apache.spark.sql.Dataset.withAction(Dataset.scala:2840)
at org.apache.spark.sql.Dataset.count(Dataset.scala:2433)

其实报错的位置不准,

实际是​​row.getAs[]()​​的变量类型错了


举报

相关推荐

0 条评论