Hardware resources
In order to deploy an HA cluster, you should prepare the following:
- NameNode machines - the machines on which you run the Active and Standby NameNodes should have equivalent hardware to each other, and equivalent hardware to what would be used in a non-HA cluster.
- JournalNode machines - the machines on which you run the JournalNodes. The JournalNode daemon is relatively lightweight, so these daemons may reasonably be collocated on machines with other Hadoop daemons, for example NameNodes, the JobTracker, or the YARN ResourceManager. Note: There must be at least 3 JournalNode daemons, since edit log modifications must be written to a majority of JNs. This will allow the system to tolerate the failure of a single machine. You may also run more than 3 JournalNodes, but in order to actually increase the number of failures the system can tolerate, you should run an odd number of JNs, (i.e. 3, 5, 7, etc.). Note that when running with N JournalNodes, the system can tolerate at most (N - 1) / 2 failures and continue to function normally.
Note that, in an HA cluster, the Standby NameNode also performs checkpoints of the namespace state, and thus it is not necessary to run a Secondary NameNode, CheckpointNode, or BackupNode in an HA cluster. In fact, to do so would be an error. This also allows one who is reconfiguring a non-HA-enabled HDFS cluster to be HA-enabled to reuse the hardware which they had previously dedicated to the Secondary NameNode.
在非HA的集群中,还是需要Secondary NameNode,但是在HA集群中,就不需要Secondary NameNode了,有了还是个错误
相关推荐
王家林的“云计算分布式大数据Hadoop实战高手之路---从零开始”的第九讲Hadoop图文训练课程:剖析NameNode和Secondary NameNode的工作机制和流程. 此教程来自于王家林免费发布的3本Hadoop教程:云计算分布式大数据...
hadoop快速入门,hadoop安装及hadoop配置
Hadoop大数据期末考试重点,选择、判断、简答
今天小编就为大家分享一篇关于Hadoop中namenode和secondarynamenode工作机制讲解,小编觉得内容挺不错的,现在分享给大家,具有很好的参考价值,需要的朋友一起跟随小编来看看吧
包括NameNode、 Secondary NameNode、DataNode、JobTracker、 TaskTrack start-dfs.sh 启动Hadoop HDFS守护进程NameNode、SecondaryNameNode和DataNode stop-dfs.sh 停⽌Hadoop HDFS守护进程NameNode、...
包括NameNode、 Secondary NameNode、DataNode、JobTracker、 TaskTrack stop-all.sh : 停止所有的Hadoop守护进程。包括NameNode、 Secondary NameNode、DataNode、JobTracker、 TaskTrack jps : 查看进程...
第4章 HDFS 2 4.1. 简介 2 4.1.1. 概述 2 4.1.2. 组成架构 4 4.1.3. HDFS文件块大小 5 4.2. HDFS的shell操作 6 4.2.1. 基本语法 7 ...4.6. NameNode&Secondary NameNode工作机制 36 4.7. 错误问题解决 37 4.
50090 dfs.namenode.secondary.http-address,如:172.25.39.166:50090 50091 dfs.namenode.secondary.https-address,如:172.25.39.166:50091 50020 dfs.datanode.ipc.address 50075 dfs.datanode.http.address ...
3-4节点,集群部署自己确定,比如集群涉及的多种角色namenode,secondary namenode, datanode,resourcemanager等的节点分配,并用表格形式给出。给出可访问的ip,用户名密码。(阿里云直接给公网ip,校园网需给公网和...
基于Hadoop部署实践对网站日志分析 1. 项目概述 本次要实践的数据日志来源于国内某技术学习论坛,该论坛由某培训机构主办,汇聚了众多技术学习者,每天都有人发帖、回帖。...Secondary NameNode.....
对于一个pdb文件,要统计其中的secondary structure的种类,可以用此程序
以记日志的形式将每一个操作写在磁盘的日志文件中,然后借助Secondary NameNode的checkpoint功能将fsImage和日志进行合并。 重点:记住checkpoint工作过程 5:如果服务器的磁盘坏了,如何挽救数据
目录 HDFS特点概况 HDFS核心组件的职责 HDFS数据流程 HDFS写数据流程 HDFS读数据流程 ...HDFS集群包括,NameNode和DataNode以及Secondary Namenode。 NameNode负责管理整个文件系统的元数据,以及每一个
centralaized secondary cotrol of microgrid
它分为两个部分:NameNode和DateNode,NameNode相当于一个领导,将文件系统的Meta-data存储在内存中,这些信息主要包括了文件信息、每一个文件对应的文件块的信息和每一个文件块在DataNode的信息等。它管理集群内的...
最详细的Ethernet secondary ISP Bootloader, 基于LPC23XX,有操作说明,Bootloader代码,还有上位机程序。
Secondary School Admission Test - Reading Comprehension题库,SSAT-RC
文章目录HDFS进阶总结1.01、HDFS知识点1.1、HDFS概述1.2、HDFS特点1.2.1、优点1.2.2、缺点1.3、HDFS角色1.3.1、Client1.3.2、NameNode1.3.3、DataNode1.3.4、Secondary NameNode1.4、HDFS结构1.5、NN与SNN工作机制...
SecondaryNameNode 内存需求和 NameNode 在⼀个数量级上,所以通常 secondary NameNode(运⾏在 单独的物理机器上)和 NameNode 运⾏在不同的机器上。 JobTracker 和 TaskTracker JobTracker 对应于 NameNode ...
辅助名称节点(Secondary NameNode) 是NameNode发生故障时的备用节点,主要功能是进行数据恢复,它的职责是合并NameNode的EditLog到FsImage文件中。 Secondary NameNode工作原理 大数据技术及应用教学课件第3章-...