http://2hei.net/setup-multi-hbase-master-on-hadoop-cluster.html
Setup Multi Hbase master on Hadoop Cluster to avoid single point failure. When active master failed/down for some reason exceed timeout we expected, backup master will be active and take over the role of master, see the value of zookeeper.session.timeout.
$cat /usr/lib/hbase/conf/hbase-site.xml
<name>zookeeper.session.timeout</name>
<value>180000</value>
<description>ZooKeeper session timeout. HBase passes this to the zk quorum as suggested maximum time for a session.In milliseconds.</description>
</property>
- Test scenarios:
hadoop01.cluster.2hei.net (Hbase master)
hadoop02.cluster.2hei.net (Hbase backup master)
- Configuration Steps:
1. add backup master hostname to /usr/lib/hbase/conf/backup-masters
$cat /usr/lib/hbase/conf/backup-masters
hadoop02.cluster.2hei.net
2. start hbase (hbase not manage zookeeper):
#start zookeeper
/usr/lib/zookeeper/bin/zkServer.sh start#start master
sh /usr/lib/hbase/bin/hbase-daemon.sh start master#start backup master
sh /usr/lib/hbase/bin/hbase-daemon.sh start master –backup
- Test steps:
1. stop hbase master
sh /usr/lib/hbase/bin/hbase-daemon.sh stop master
2. check hbase master(hadoop01) logs:
2012-07-23 08:44:53,058 INFO org.apache.hadoop.hbase.master.AssignmentManager: No previous transition plan was found (or we are ignoring an existing plan) for -ROOT-,,0.70236052 so generated a random one; hri=-ROOT-,,0.70236052, src=, dest=hadoop07.cluster.2hei.net,60020,1338437695217; 5 (online=5, exclude=null) available servers
2012-07-23 08:44:53,058 INFO org.apache.hadoop.hbase.master.AssignmentManager: Assigning region -ROOT-,,0.70236052 to hadoop07.cluster.2hei.net,60020,1338437695217
2012-07-23 08:44:53,058 DEBUG org.apache.hadoop.hbase.master.ServerManager: New connection to hadoop07.cluster.2hei.net,60020,1338437695217
— stop master —
Mon July 23 08:45:47 UTC 2012 Killing master
3. check hbase backup master(hadoop02) logs:
2012-07-23 04:14:56,348 INFO org.apache.hadoop.hbase.master.metrics.MasterMetrics: Initialized
2012-07-23 04:14:56,356 DEBUG org.apache.hadoop.hbase.master.HMaster: HMaster started in backup mode. Stalling until master znode is written.
2012-07-23 04:14:56,389 INFO org.apache.hadoop.hbase.master.ActiveMasterManager: Another master is the active master, 192.168.0.100:60000; waiting to become the next active master
2012-07-23 05:34:15,361 WARN org.apache.hadoop.ipc.HBaseServer: Incorrect header or version mismatch from 192.168.0.201:44928 got version 47 expected version 3
— stop master —
2012-07-23 08:47:58,003 DEBUG org.apache.hadoop.hbase.master.LogCleaner: Add log cleaner in chain: org.apache.hadoop.hbase.master.TimeToLiveLogCleaner
2012-07-23 08:47:58,257 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2012-07-23 08:47:58,334 INFO org.apache.hadoop.http.HttpServer: Added global filtersafety (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
2012-07-23 08:47:58,344 INFO org.apache.hadoop.http.HttpServer: Port returned by webServer.getConnectors()[0].getLocalPort() before open() is -1. Opening the listener on 60010
2012-07-23 08:47:58,344 INFO org.apache.hadoop.http.HttpServer: listener.getLocalPort() returned 60010 webServer.getConnectors()[0].getLocalPort() returned 60010
2012-07-23 08:47:58,344 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 60010
2012-07-23 08:47:58,344 INFO org.mortbay.log: jetty-6.1.26
2012-07-23 08:47:58,879 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:60010
We can see hadoop02 is listening 60010 and to be the active hbase master now.
reference: http://wiki.apache.org/hadoop/Hbase/MultipleMasters
相关推荐
HBase 0.98.1-hadoop2 API
赠送jar包:hbase-hadoop2-compat-1.2.12.jar; 赠送原API文档:hbase-hadoop2-compat-1.2.12-javadoc.jar; 赠送源代码:hbase-hadoop2-compat-1.2.12-sources.jar; 赠送Maven依赖信息文件:hbase-hadoop2-compat-...
赠送jar包:hbase-hadoop-compat-1.1.3.jar; 赠送原API文档:hbase-hadoop-compat-1.1.3-javadoc.jar; 赠送源代码:hbase-hadoop-compat-1.1.3-sources.jar; 赠送Maven依赖信息文件:hbase-hadoop-compat-1.1.3....
hadoop-2.7.2-hbase-jar.tar.gz hadoop-2.7.2-hbase-jar.tar.gz hadoop-2.7.2-hbase-jar.tar.gz
赠送jar包:hbase-hadoop2-compat-1.1.3.jar; 赠送原API文档:hbase-hadoop2-compat-1.1.3-javadoc.jar; 赠送源代码:hbase-hadoop2-compat-1.1.3-sources.jar; 赠送Maven依赖信息文件:hbase-hadoop2-compat-...
赠送jar包:hbase-hadoop-compat-1.1.3.jar; 赠送原API文档:hbase-hadoop-compat-1.1.3-javadoc.jar; 赠送源代码:hbase-hadoop-compat-1.1.3-sources.jar; 赠送Maven依赖信息文件:hbase-hadoop-compat-1.1.3....
Hbase数据库在Hadoop上的部署详细过程
Hbase和Hadoop JMX监控实战
eclipse链接hbase所需jar包,hbase版本1.2.6,Hadoop版本2.7.1(转载的另一篇下载的,亲测可用)
nutch2.3+hbase0.94.14+hadoop1.2.1安装文档.txt )
赠送jar包:hbase-hadoop2-compat-1.1.3.jar; 赠送原API文档:hbase-hadoop2-compat-1.1.3-javadoc.jar; 赠送源代码:hbase-hadoop2-compat-1.1.3-sources.jar; 赠送Maven依赖信息文件:hbase-hadoop2-compat-...
在本地模式下,hbase只需要安装JDK...若使用HDFS文件系统模式,除了JDK,还需要安装hadoop(HDFS是hadoop的旗舰级文件系统,是hadoop项目的核心子项目,安装hadoop会带有hdfs),本版本hbase依赖安装的hadoop版本是2.4.1
赠送jar包:hbase-hadoop-compat-1.2.12.jar; 赠送原API文档:hbase-hadoop-compat-1.2.12-javadoc.jar; 赠送源代码:hbase-hadoop-compat-1.2.12-sources.jar; 赠送Maven依赖信息文件:hbase-hadoop-compat-...
介绍hbase和hadoop数据块损坏如何处理
该文档保护了目前比较流行的大数据平台的原理过程梳理。Hadoop,Hive,Hbase,Spark,MapReduce,Storm
赠送jar包:hbase-hadoop-compat-1.2.12.jar; 赠送原API文档:hbase-hadoop-compat-1.2.12-javadoc.jar; 赠送源代码:hbase-hadoop-compat-1.2.12-sources.jar; 赠送Maven依赖信息文件:hbase-hadoop-compat-...
就像Bigtable利用了Google文件系统(File System)所提供的分布式数据存储一样,HBase在Hadoop之上提供了类似于Bigtable的能力。HBase是Apache的Hadoop项目的子项目。HBase不同于一般的关系数据库,它是一个适合于非...
赠送jar包:hbase-hadoop-compat-1.4.3.jar; 赠送原API文档:hbase-hadoop-compat-1.4.3-javadoc.jar; 赠送源代码:hbase-hadoop-compat-1.4.3-sources.jar; 赠送Maven依赖信息文件:hbase-hadoop-compat-1.4.3....
赠送jar包:hbase-hadoop2-compat-1.2.12.jar; 赠送原API文档:hbase-hadoop2-compat-1.2.12-javadoc.jar; 赠送源代码:hbase-hadoop2-compat-1.2.12-sources.jar; 赠送Maven依赖信息文件:hbase-hadoop2-compat-...
本文将HBase-2.2.1安装在Hadoop-3.1.2上,关于Hadoop-3.1.2的安装,请参见《基于zookeeper-3.5.5安装hadoop-3.1.2》一文。安装环境为64位CentOS-Linux 7.2版本。 本文将在HBase官方提供的quickstart.html文件的指导...