1、首先安装hive,远程mysql
tar zxf apache-hive-0.13.1-bin.tar.gz -C /home/hadoop
默认没有mapred-site.xml文件,vi hive-site.xml
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://192.168.1.108:3306/hive?createDatabaseIfNotExist=true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>root</value>
</property>
<property>
<name>hive.metastore.local</name>
<value>false</value>
</property>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>hdfs://127.0.0.1:9000/hive/warehouse</value>
</property>
</configuration>
3、启动正常,运行命令都正常
4、与hbase 集成:hive-site.xml
<configuration>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>hdfs://127.0.0.1:9000/hive/warehousedir</value>
</property>
<property>
<name>hive.exec.scratchdir</name>
<value>hdfs://127.0.0.1:9000/hive/scratchdir</value>
</property>
<property>
<name>hive.querylog.location</name>
<value>/home/hadoop/hive-0.13.1-bin/logs</value>
</property>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://192.168.1.108:3306/hive_remote?createDatabaseIfNotExist=true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>root</value>
</property>
<property>
<name>hive.metastore.local</name>
<value>false</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>localhost</value>
</property>
<property>
<name>hive.aux.jars.path</name>
<value>file:////home/hadoop/hive-0.13.1-bin/lib/hive-hbase-handler-0.13.1.jar,file:////home/hadoop/hive-0.13.1-bin/lib/protobuf-java-2.5.0.jar,
file:////home/hadoop/hive-0.13.1-bin/lib/hbase-client-0.98.0-hadoop2.jar,file:////home/hadoop/hive-0.13.1-bin/lib/hbase-common-0.98.0-hadoop2.jar,file:///
/home/hadoop/hive-0.13.1-bin/lib/zookeeper-3.4.5.jar,file:////home/hadoop/hive-0.13.1-bin/lib/guava-11.0.2.jar,file:////home/hadoop/hive-0.13.1-bin/lib/hbase-server-0.98.4-hadoop2.jar,file:////home/hadoop/hive-0.13.1-bin/lib/hbase-protocol-0.98.4-hadoop2.jar,file:////home/hadoop/hive-0.13.1-bin/lib/hbase-hadoop-compat-0.98.4-hadoop2.jar,
file:////home/hadoop/hive-0.13.1-bin/lib/hbase-common-0.98.4-hadoop2.jar</value>
</property>
<property>
<name>hive.metastore.uris</name>
<value>thrift://127.0.0.1:9083</value>
</property>
</configuration>
hbase 配置文件 :
export HBASE_MANAGES_ZK=true
hbase-site.xml
<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://localhost:9000/hbase</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/home/hadoop/zookeeper</value>
</property>
<property>
<name>hbase.cluster.distributed </name>
<value>true</value>
</property>
</configuration>
5、启动hive ,报错
Logging initialized using configuration in jar:file:/home/hadoop/hive-0.13.1-bin/lib/hive-common-0.13.1.jar!/hive-log4j.properties
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: 拒绝连接
无法初始元数据,因为无法连接远程mysql
分享到:
相关推荐
Hadoop Hive与Hbase整合配置
Hive与Hbase的整合,集中两者的优势,使用HiveQL语言,同时具备了实时性
Hadoop2.4、Hbase0.98、Hive集群安装配置手册
jdk1.8.0_131、apache-zookeeper-3.8.0、hadoop-3.3.2、hbase-2.4.12 mysql5.7.38、mysql jdbc驱动mysql-connector-java-8.0.8-dmr-bin.jar、 apache-hive-3.1.3 2.本文软件均安装在自建的目录/export/server/下 ...
hive与hbase整合经验谈
配置,测试,导入数据详细操作,CREATE TABLE hive_hbase_table(key int, value string,name string) hadoop jar /usr/lib/hbase/hbase-0.90.4-cdh3u3.jar importtsv -Dimporttsv.columns=HBASE_ROW_KEY, catgyname...
浅谈Hive vs. HBase,大数据类
hive0.8.1和hbase0.92.0集成的hive-hbase-handler.Jar包,里面包含:hbase-0.92.0.jar、hbase-0.92.0-tests.jar、hive-hbase-handler-0.9.0-SNAPSHOT.jar。经测试没有问题。
此文档是本人在工作中用到的知识总结出来的整合过程,本人是菜鸟,希望得到大神们的建议。
hive0.10.0和hbase0.94.4集成的hive-hbase-handler.Jar包,经测试没有问题。
Hadoop2.2.0+Hbase0.98.1+Sqoop1.4.4+Hive0.13完全安装手册,经测试运行正常。
大数据工具篇之Hive与HBase整合完整教程
hive-hbase-handler-1.2.1.jar解决hive1.2.1,hbase不兼容情况!
Hive与hbase的结构,Hive与hbase整合后的结构图 , Hive与hbase整合的原理
HIVE建表时可以指定映射关系直接读取HBASE的数据,相当于有了一个HBASE向HIVE的通道。那HIVE向HBASE有通道吗?本文主要讲述了Hive库数据如何入到HBASE中。
HBase2.1.3整合Hive3.1.2,Hive官方的hive-hbase-handler-3.1.1.jar包不好用,自己编译后的,确认好用
hive和hbase整合的时候,如果出现不兼容的情况需要手动编译:hive-hbase-hander-1.2.2.jar把这个jar替换掉hive/lib里的那个jar包
hive安装
该文档保护了目前比较流行的大数据平台的原理过程梳理。Hadoop,Hive,Hbase,Spark,MapReduce,Storm