0
点赞
收藏
分享

微信扫一扫

1003-Hive的安装


1、上传Hive安装包

2、解压Hive到app目录下


tar -zxvf hive-0.12.0.tar.gz  -C app/


3、Hive项目结构


[hadoop@cloud04 ~]$ cd app/hive-0.12.0/
 [hadoop@cloud04 hive-0.12.0]$ ll
 total 228
 drwxr-xr-x.  3 hadoop hadoop   4096 Oct  9  2013 bin
 drwxr-xr-x.  2 hadoop hadoop   4096 Apr 27 02:53 conf
 drwxr-xr-x.  6 hadoop hadoop   4096 Apr 27 02:53 docs
 drwxr-xr-x.  4 hadoop hadoop   4096 Oct  9  2013 examples
 drwxr-xr-x.  7 hadoop hadoop   4096 Oct  9  2013 hcatalog
 drwxr-xr-x.  4 hadoop hadoop   4096 Apr 27 02:53 lib
 -rw-rw-r--.  1 hadoop hadoop  23828 Oct  9  2013 LICENSE
 -rw-rw-r--.  1 hadoop hadoop   1559 Oct  9  2013 NOTICE
 -rw-rw-r--.  1 hadoop hadoop   3832 Oct  9  2013 README.txt
 -rw-rw-r--.  1 hadoop hadoop 166087 Oct  9  2013 RELEASE_NOTES.txt
 drwxr-xr-x.  3 hadoop hadoop   4096 Oct  9  2013 scripts
 drwxrwxr-x. 28 hadoop hadoop   4096 Apr 27 02:54 src








Hive配置文件介绍



•hive-site.xml      hive的配置文件



•hive-env.sh        hive的运行环境文件



•hive-default.xml.template  默认模板



•hive-env.sh.template     hive-env.sh默认配置



•hive-exec-log4j.properties.template   exec默认配置



• hive-log4j.properties.template log默认配置












4、设置环境变量



export JAVA_HOME=/home/hadoop/app/jdk1.7.0_65
export PATH=$PATH:$JAVA_HOME/bin

export HADOOP_HOME=/home/hadoop/app/hadoop-2.4.1
export HIVE_HOME=/home/hadoop/app/hive-0.12.0
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin



#使之生效



source /etc/profile






5、验证






#比较./hive 和 ./bin/hive 区别






#进入bin后,执行hive



[hadoop@cloud04 bin]$ ./hive



hive> create databases hive01;



hive> create table tlb01 (id int ,name string);



hive> show databases;
OK
default
hive01
Time taken: 3.45 seconds, Fetched: 2 row(s)
hive> use hive01;
OK
Time taken: 0.033 seconds
hive> show tables;
OK
tlb01
Time taken: 0.249 seconds, Fetched: 1 row(s)






[hadoop@cloud04 bin]$ ll
total 40
-rwxr-xr-x. 1 hadoop hadoop  881 Oct  9  2013 beeline
-rw-rw-r--. 1 hadoop hadoop  363 Apr 27 05:51 derby.log
drwxr-xr-x. 3 hadoop hadoop 4096 Apr 27 02:53 ext
-rwxr-xr-x. 1 hadoop hadoop 5691 Oct  9  2013 hive
-rwxr-xr-x. 1 hadoop hadoop 1900 Oct  9  2013 hive-config.sh
-rwxr-xr-x. 1 hadoop hadoop  885 Oct  9  2013 hiveserver2
drwxrwxr-x. 5 hadoop hadoop 4096 Apr 27 05:51 metastore_db
-rwxr-xr-x. 1 hadoop hadoop  832 Oct  9  2013 metatool
-rwxr-xr-x. 1 hadoop hadoop  884 Oct  9  2013 schematool












#在bin目录下执行命令   ./bin/hive



[hadoop@cloud04 bin]$ ./bin/hive



[hadoop@cloud04 bin]$ ll
total 40
-rwxr-xr-x. 1 hadoop hadoop  881 Oct  9  2013 beeline
-rw-rw-r--. 1 hadoop hadoop  363 Apr 27 05:51 derby.log
drwxr-xr-x. 3 hadoop hadoop 4096 Apr 27 02:53 ext
-rwxr-xr-x. 1 hadoop hadoop 5691 Oct  9  2013 hive
-rwxr-xr-x. 1 hadoop hadoop 1900 Oct  9  2013 hive-config.sh
-rwxr-xr-x. 1 hadoop hadoop  885 Oct  9  2013 hiveserver2
drwxrwxr-x. 5 hadoop hadoop 4096 Apr 27 05:51 metastore_db
-rwxr-xr-x. 1 hadoop hadoop  832 Oct  9  2013 metatool
-rwxr-xr-x. 1 hadoop hadoop  884 Oct  9  2013 schematool



[hadoop@cloud04 hive-0.12.0]$ ./bin/hive
hive> show databases;
OK
default
Time taken: 3.646 seconds, Fetched: 1 row(s)









发现没有刚才创建的数据库hive01






总结: 对比发现,元数据仅存在执行命令的目录,在不同的目录数据不能互相访问,要解决该问题,引入MySQL存储元数据。






6、 查看元数据在hdfs上存储位置



[hadoop@cloud01 app]$ hadoop fs -ls /user/
Found 1 items
drwxr-xr-x   - hadoop supergroup          0 2015-04-27 03:50 /user/hive
[hadoop@cloud01 app]$ hadoop fs -ls /user/hive
Found 1 items
drwxr-xr-x   - hadoop supergroup          0 2015-04-27 03:50 /user/hive/warehouse
[hadoop@cloud01 app]$ hadoop fs -ls /user/hive/warehouse
Found 1 items
drwxr-xr-x   - hadoop supergroup          0 2015-04-27 04:16 /user/hive/warehouse/hive01.db
[hadoop@cloud01 app]$ hadoop fs -ls /user/hive/warehouse/hive01.db/
Found 1 items
drwxr-xr-x   - hadoop supergroup          0 2015-04-27 04:16 /user/hive/warehouse/hive01.db/tlb01
[hadoop@cloud01 app]$ hadoop fs -ls /user/hive/warehouse/hive01.db/tlb01






其中:   /user/hive/warehouse/数据库名称/表名,例如: /user/hive/warehouse/hive01.db/tlb01   中hive01.db为数据库表名,tlb01位表名






7、常见错误



7.1 使用默认Derby存储元数据



错误1:



hive> create database hive02;


FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient




方案1:



 [hadoop@cloud04 bin]$./hive



 [hadoop@cloud04 bin]$ more derby.log 





java.sql.SQLException: Another instance of Derby may have already booted the database /home/hadoop/app/hive-0.12.0/bin/metastore_db.

     at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)

     at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)

     at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)

     at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)

     at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown Source)

     at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source)

     at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source)

     at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)

     at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)

     at java.sql.DriverManager.getConnection(DriverManager.java:571)

     at java.sql.DriverManager.getConnection(DriverManager.java:187)

     at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:254)

     at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:305)




[hadoop@cloud04 bin]$ ps -ef | grep hive
hadoop    4228  2373  0 04:26 pts/0    00:00:13 /home/hadoop/app/jdk1.7.0_65/bin/java -Xmx256m -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/home/hadoop/app/hadoop-2.4.1/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/home/hadoop/app/hadoop-2.4.1 -Dhadoop.id.str=hadoop -Dhadoop.root.logger=INFO,console -Djava.library.path=/home/hadoop/app/hadoop-2.4.1/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xmx512m -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /home/hadoop/app/hive-0.12.0/lib/hive-cli-0.12.0.jar org.apache.hadoop.hive.cli.CliDriver
hadoop    4862  4655  0 05:37 pts/2    00:00:00 grep hive
[hadoop@cloud04 bin]$ kill -9 4228




 



hive> show databases;
OK
default
hive01
Time taken: 4.016 seconds, Fetched: 2 row(s)






发现确实是因为这个原因,该问题解决。总之,无论遇到任何问题,不要着急,要通过日志可以快速定位并给予解决。




参考文章:



http://blog.chinaunix.net/uid-451-id-3143781.html

举报

相关推荐

0 条评论