0
点赞
收藏
分享

微信扫一扫

Spark Action算子->takeSample(boolean,num,seed)

刘员外__ 2022-07-04 阅读 47

takeSample 可以对 RDD 中的数据随机获取 num 个,第一个参数是有 无放回,第二个参数是随机获取几个元素,第三个参数如果固定,那么每次获取的数据固定。抽取到的数据放入Driver端集合

  1. scala
package action

import org.apache.spark.rdd.RDD
import org.apache.spark.{SparkConf, SparkContext}

/**
* @Author yqq
* @Date 2021/12/10 12:09
* @Version 1.0
*/
object TaskSampleTest1 {
def main(args: Array[String]): Unit = {
val context = new SparkContext(
new SparkConf()
.setMaster("local")
.setAppName("TaskSampleTest1")
)
context.setLogLevel("Error")
val value: RDD[String] = context.parallelize(Array[String]("a", "b", "c", "d", "e", "f", "g"))
val strings: Array[String] = value.takeSample(false, 3, 100L)
strings.foreach(println)
}
}

Spark Action算子->takeSample(boolean,num,seed)_apache
2. java

package action;

import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.api.java.JavaSparkContext;

import java.util.Arrays;

/**
* @Author yqq
* @Date 2021/12/10 11:59
* @Version 1.0
*/
public class TaskSampleTest {
public static void main(String[] args) {
JavaSparkContext context = new JavaSparkContext(
new SparkConf()
.setMaster("local")
.setAppName("TaskSampleTest")
);
context.setLogLevel("Error");
JavaRDD<String> rdd = context.parallelize(Arrays.asList("a", "b", "c", "d", "e", "f", "g"));
rdd.takeSample(false,3,100L).forEach(e-> System.out.print(e+"\t"));
}
}

Spark Action算子->takeSample(boolean,num,seed)_scala_02


举报

相关推荐

0 条评论