编译flink-shaded-hadoop-2-uber.jar包

编译flink-shaded-hadoop-2-uber.jar包

编译环境

• Flink1.10
• Hadoop2.7.7

1. 下载flink-shaded

前往github下载flink-shaded 下载relese-10.0

2. 解压至linux

cd /home/bigdata/soft/

tar -zxvf flink-shaded-release-10.0.tar.gz

mv flink-shaded-release-10.0/ /home/bigdata/apps/

3. 进入flink-shaded-hadoop-2-parent，修改pom

cd   /home/bigdata/apps/flink-shaded-release-10.0/flink-shaded-hadoop-2-parent/
vim pom.xml

改成自己的hadoop版本然后保存退出

4. 进入flink-shaded-hadoop-2 修改pom

cd flink-shaded-hadoop-2/
vim pom.xml

5.进入flink-shaded-hadoop-2-uber 修改pom

cd /home/bigdata/apps/flink-shaded-release-10.0/flink-shaded-hadoop-2-parent/flink-shaded-hadoop-2-uber/
vim pom.xml

如果没有这两个配置 请添加

    <dependencies>
            <dependency>
                    <groupId>org.apache.flink</groupId>
                    <artifactId>flink-shaded-hadoop-2</artifactId>
                    <version>${hadoop.version}-10.0</version>
            </dependency>
            <dependency>
                    <groupId>commons-cli</groupId>
                    <artifactId>commons-cli</artifactId>
                    <version>1.3.1</version>
            </dependency>

    </dependencies>

6. 返回进入flink-shaded-release-10.0下 修改pom

cd /home/bigdata/apps/flink-shaded-release-10.0/
vim pom.xml

添加阿里云镜像

     <profile>
        <id>vendor-repos</id>
        <activation>
            <property>
                <name>vendor-repos</name>
            </property>
        </activation>   <!-- Add vendor maven repositories -->
        <repositories>       <!-- Cloudera -->
            <repository>
                <id>cloudera-releases</id>
                <url>https://maven.aliyun.com/repository/central</url>
                <releases>
                    <enabled>true</enabled>
                </releases>
                <snapshots>
                    <enabled>false</enabled>
                </snapshots>
            </repository>
        </repositories>
    </profile>

7. 配置完毕，运行前提是需要linux配置maven

如果你还没有配置maven，可以参考这篇文章
Linux下安装maven环境

如果你有配置maven 请继续下一步

8. 执行编译

cd /home/bigdata/apps/flink-shaded-release-10.0/

mvn clean install -DskipTests -Dhadoop.version=2.7.7 -Drat.skip=true

全程没ERROR 就是成功

9. 找到jar 放入flink1.10的lib下

cp flink-shaded-hadoop-2-uber-2.7.7-10.0.jar /home/bigdata/apps/flink-1.10.1/lib/