0
点赞
收藏
分享

微信扫一扫

Flink读取Iceberg数据

脱下愤怒的小裤衩 2022-03-20 阅读 38
javaflink

1.依赖

参考:https://iceberg.apache.org/docs/latest/flink/#reading-with-datastream

<dependency>
      <groupId>org.apache.flink</groupId>
      <artifactId>flink-java</artifactId>
      <version>1.12.0</version>
      <!--      <scope>provided</scope>-->
    </dependency>
    <dependency>
      <groupId>org.apache.flink</groupId>
      <artifactId>flink-streaming-java_2.12</artifactId>
      <version>1.12.0</version>
      <!--      <scope>provided</scope>-->
    </dependency>
    <dependency>
      <groupId>org.apache.flink</groupId>
      <artifactId>flink-clients_2.12</artifactId>
      <version>1.12.0</version>
      <!--      <scope>provided</scope>-->
    </dependency>
    <dependency>
      <groupId>org.apache.iceberg</groupId>
      <artifactId>iceberg-core</artifactId>
      <version>0.13.1</version>
    </dependency>

    <dependency>
      <groupId>org.apache.iceberg</groupId>
      <artifactId>iceberg-flink-runtime</artifactId>
      <version>0.12.1</version>
    </dependency>

2.代码

public class ReadIceberg {
    public static void main(String[] args) throws Exception {
        //创建flink执行环境
        StreamExecutionEnvironment env = StreamExecutionEnvironment.createLocalEnvironmentWithWebUI(new Configuration());
        TableLoader tableLoader = TableLoader.fromHadoopTable("hdfs://hadoop01:9000/warehouse/iceberg/ods_db/test");
        DataStream<RowData> batch = FlinkSource.forRowData()
                .env(env)
                .tableLoader(tableLoader)
                .streaming(false)
                .build();
        //打印
        batch.print();
        //提交任务
        env.execute("Test Iceberg Batch Read");
    }
}
举报

相关推荐

0 条评论