0
点赞
收藏
分享

微信扫一扫

Mac安装Hadoop 3.1.1(伪分布模式)

艾米吖 2022-11-02 阅读 198


1、安装Java
​​​brew install java​​​即可
2、设置SSH
在系统里打开远程登录,位置在 System Preference -> Sharing 中,左边勾选 Remote Login,右边选择 All Users

然后

$ ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
$ chmod 0600 ~/.ssh/authorized_keys

3、配置Hadoop

  • 官网下载hadoop3.1.1
  • 设置hadoop-env.sh
  • ​$ /usr/libexec/java_home​​查看java安装位置
  • 打开hadoop-env.sh,添加
    ​​​$ /usr/libexec/java_home​
  • 设置core-site.xml

<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>

  • 设置hdfs-site.xml

<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>

  • 设置mapred-site.xml

<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>

  • yarn-site.xml

<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.env-whitelist</name>
<value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME</value>
</property>
</configuration>

4、格式化namenode
进入hadoop目录
​​​bin/hdfs namenode -format​​ 5、启东dfs、yarn

sbin/start-dfs.sh
sbin/start-yarn.sh

6、web ui
ResourceManager - ​​http://localhost:8088​​

NameNode - ​​http://localhost:9870​​

举报

相关推荐

0 条评论