Exception in thread “main“ org.apache.hadoop.security.AccessControlException: Permission denied: use

报错详情

执行 SparkStreaming 程序的Scala代码时报错：

Exception in thread "main" org.apache.hadoop.security.AccessControlException: 
Permission denied: user=jason, access=WRITE, inode="/":root:supergroup:drwxr-xr-x

报错原因

代码中设置了 checkpoint 检查点，其路径为hdfs的，但没有对其配置权限导致

解决方法

1. 临时解决方案：设置代码执行用户

• 点击代码变量，选择 Edit，VM options 添加： -DHADOOP_USER_NAME=root

2. 永久解决方案：修改hdfs配置文件
进入Hadoop安装目录下修改 /etc/hadoop/hdfs-site.xml，在 <configuration> 和 </configuration> 之间添加下边代码，然后重启hdfs集群

<property>
  <name>dfs.permissions</name>
  <value>false</value>
  <description>
    If "true", enable permission checking in HDFS.
    If "false", permission checking is turned off,
    but all other behavior is unchanged.
    Switching from one parameter value to the other does not change the mode,
    owner or group of files or directories.
  </description>
</property>