一、背景
需把MaxCompute的结果采用FLink DataSet 读取数据,并sink到目标表,采用离线批处理读取数据并
二、全量同步代码
MaxCompute源表和结果表依赖 VVR 4.0.7(对应Flink 1.13)
<dependency>
<groupId>com.alibaba.ververica</groupId>
<artifactId>ververica-connector-odps</artifactId>
<version>1.13-vvr-4.0.7</version>
</dependency>
离线对某一个分区全量读取数据
ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
//字段信息
ODPSColumn[] oDPSColumns=new ODPSColumn[1];
ODPSColumn ODPSColumn=new ODPSColumn("id", OdpsType.STRING,false);
oDPSColumns[0]=ODPSColumn;
//表名
String tableName ="";
//分区信息
String[] partitions=new String[1];
partitions[0]="dt=";
OdpsConf odpsConf =new OdpsConf(accessid,accesskey, endpoint, project, null);
ODPSInputFormat ODPSInputFormat =new ODPSInputFormat(odpsConf, tableName,oDPSColumns,true,partitions);
DataSource<RowData> dataSources=env.createInput(ODPSInputFormat);
source.addSink(new PrintSinkFunction<>());
env.execute("test_odps");
之前查找官方文档没有这块说明,查看源代码发现,记录一下