【Flink问题】The number of requested virtual cores per node 8 exceeds the maximum number of virtual core

问题描述

运行以下命令时出现问题

./bin/flink run-application -t yarn-application ./examples/streaming/TopSpeedWindowing.jar

具体错误

org.apache.flink.client.deployment.ClusterDeploymentException: Couldn't deploy Yarn Application Cluster
        at org.apache.flink.yarn.YarnClusterDescriptor.deployApplicationCluster(YarnClusterDescriptor.java:465)
        .....
Caused by: org.apache.flink.configuration.IllegalConfigurationException: The number of requested virtual cores per node 8 exceeds the maximum number of virtual cores 6 available in the Yarn Cluster. Please note that the number of virtual cores is set to the number of task slots by default unless configured in the Flink config with 'yarn.containers.vcores.'
        at org.apache.flink.yarn.YarnClusterDescriptor.isReadyForDeployment(YarnClusterDescriptor.java:338)
        at org.apache.flink.yarn.YarnClusterDescriptor.deployInternal(YarnClusterDescriptor.java:534)
        at org.apache.flink.yarn.YarnClusterDescriptor.deployApplicationCluster(YarnClusterDescriptor.java:458)
        ... 9 more

解决方案

有以下错误可以看出

The number of requested virtual cores per node 8 exceeds the maximum number of virtual cores 6 available in the Yarn Cluster. 
Please note that the number of virtual cores is set to the number of task slots by default unless configured 
in the Flink config with 'yarn.containers.vcores.'

yarn配置中的​​yarn.containers.vcores​​设置了虚拟的cores=6，taskManager的slot我设置了8个，资源不够

解决方法：

• 方法一：调大vcores参数
• 方法二：减少slot个数taskmanager.numberOfTaskSlots: 5
  在flink安装目录下​​conf/flink-conf.yaml​​文件修改以下内容

taskmanager.numberOfTaskSlots: 5