[Scala进阶]-- Spark累加器（单例模式应用）

一、举例创建单例模式的累加器

import org.apache.spark.{Accumulator, SparkContext}

/**
  * Created by root on 20170618.
  * Update date:
  * Time: 12:04 PM
  *
  */
object SuccessRecordCounter {
  @volatile private var instance: Accumulator[Long] = null //共享变量

  def getInstance(sc: SparkContext): Accumulator[Long] = {
    if (instance == null) {
      synchronized {
        if (instance == null) {
          instance = sc.accumulator(0L, "SuccessRecordCounter")
        }
      }
    }
    instance
  }
}

二、使用

val successCounter = SuccessRecordCounter.getInstance(rdd.sparkContext)
successCounter.add(1L);

参考：

​​https://spark.apache.org/docs/1.6.1/streaming-programming-guide.html​​

​​https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/streaming/RecoverableNetworkWordCount.scala​​