`

(转)hadoop 根据SecondaryNameNode恢复Namenode

 
阅读更多

 制造namenode宕机的情况 
1) kill 掉namenode的进程

[hadoop@hadoop bin]$ kill -9 13481

2)删除dfs.name.dir所指向的文件夹,这里是/home/hadoop/hdfs/name.

 

current  image  in_use.lock  previous.checkpoint
[hadoop@hadoop name]$ rm -rf *

 

删除name目录下的所有内容,但是必须保证name这个目录是存在的
3)从secondarynamenode元数据namesecondary目录下复制到namenode name目录下

   我的secodarynamenode 元数据目录

/home/hadoop/tmp/dfs/namesecondary

   复制过程

[hadoop@hadoop name]$ cp -R /home/hadoop/tmp/dfs/namesecondary/* .

4)启动namenode

[hadoop@hadoop bin]$ ./hadoop-daemon.sh start namenode

 5)检查
使用hadoop fsck /user命令检查文件Block的完整性

<font size="2">[hadoop@hadoop bin]$ hadoop fsck /</font>
Warning: $HADOOP_HOME is deprecated.

FSCK started by hadoop from /192.168.0.101 for path / at Sun Dec 22 23:04:31 CST 2013
...................................
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310222119_0001/job.jar:  Under replicated blk_-8571652065964704775_1020. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310222119_0002/job.jar:  Under replicated blk_-5947701456602696019_1021. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310222119_0003/job.jar:  Under replicated blk_8214183112681524571_1022. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310222142_0001/job.jar:  Under replicated blk_4805420250921446015_1024. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310222142_0002/job.jar:  Under replicated blk_7913185784171356584_1027. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310222142_0004/job.jar:  Under replicated blk_-8411847042533891069_1035. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310222142_0005/job.jar:  Under replicated blk_2163772543235273521_1036. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310222142_0007/job.jar:  Under replicated blk_-3491660194168043022_1044. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242056_0002/job.jar:  Under replicated blk_5280511346594851641_1270. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242056_0003/job.jar:  Under replicated blk_5588149584508213931_1271. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242056_0004/job.jar:  Under replicated blk_-1846184614352398688_1272. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242056_0005/job.jar:  Under replicated blk_8253537375261552577_1273. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242100_0001/job.jar:  Under replicated blk_-6858089306760733073_1275. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242100_0002/job.jar:  Under replicated blk_-630176777256891004_1276. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242100_0003/job.jar:  Under replicated blk_3453389521553623867_1277. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242100_0004/job.jar:  Under replicated blk_-4262000880964323956_1278. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242105_0001/job.jar:  Under replicated blk_-5324801167724976561_1280. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242105_0002/job.jar:  Under replicated blk_3284342834321881345_1281. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242105_0004/job.jar:  Under replicated blk_5174401550469241860_1295. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242105_0009/job.jar:  Under replicated blk_6390129220783606015_1327. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242105_0010/job.jar:  Under replicated blk_8995477665353821346_1328. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201311292212_0007/job.jar:  Under replicated blk_-6447241034801532571_1699. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201312082210_0001/job.jar:  Under replicated blk_-187920261151639503_1741. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201312082210_0002/job.jar:  Under replicated blk_1912732980088631445_1742. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201312092348_0001/job.jar:  Under replicated blk_448639237400606735_1953. Target Replicas is 10 but found 1 replica(s).
.
/home/hadoop/tmp/mapred/system/jobtracker.info: CORRUPT block blk_-4973841422235657473

/home/hadoop/tmp/mapred/system/jobtracker.info: MISSING 1 blocks of total size 4 B.Status: CORRUPT
 Total size:        367257 B
 Total dirs:        83
 Total files:        60
 Total blocks (validated):        57 (avg. block size 6443 B)
  ********************************
  CORRUPT FILES:        1
  MISSING BLOCKS:        1
  MISSING SIZE:                4 B
  CORRUPT BLOCKS:         1
  ********************************
 Minimally replicated blocks:        56 (98.24561 %)
 Over-replicated blocks:        0 (0.0 %)
 Under-replicated blocks:        25 (43.85965 %)
 Mis-replicated blocks:                0 (0.0 %)
 Default replication factor:        1
 Average block replication:        0.98245615
 Corrupt blocks:                1
 Missing replicas:                225 (401.7857 %)
 Number of data-nodes:                1
 Number of racks:                1
FSCK ended at Sun Dec 22 23:04:31 CST 2013 in 89 milliseconds

恢复工作完成,检查hdfs的数据

 

原文链接:hadoop 根据SecondaryNameNode恢复Namenode
http://www.aboutyun.com/thread-6196-1-1.html
(出处: about云开发)

原作者:lzw

分享到:
评论

相关推荐

    大数据与云计算技术 Hadoop概论和快速入门 共40页.ppt

    SecondaryNamenode,NameNode的镜像备份节点 ==Map Reduce== JobTracker,hadoop的Map/Reduce调度器,负责与TackTracker通信分配计算任务并跟踪任务进度。 TaskTracker,启动和管理Map和Reduce子任务的节点。

    【大数据】Hadoop常用启动命令.pdf

    TaskTrack start-dfs.sh 启动Hadoop HDFS守护进程NameNode、SecondaryNameNode和DataNode stop-dfs.sh 停⽌Hadoop HDFS守护进程NameNode、SecondaryNameNode和DataNode hadoop-daemons.sh start namenode 单独启动...

    搭建hadoop伪分布式.docx

    DataNode,JobTracker,TaskTracker,SecondaryNameNode),请注意分布式运行中的这几个结点的区别:从分布式存储的角度来说,集群中的结点由一个NameNode和若干个DataNode组成,另有一个SecondaryNameNode作为NameNode的...

    Hadoop中namenode和secondarynamenode工作机制讲解

    今天小编就为大家分享一篇关于Hadoop中namenode和secondarynamenode工作机制讲解,小编觉得内容挺不错的,现在分享给大家,具有很好的参考价值,需要的朋友一起跟随小编来看看吧

    大数据hadoop,centos7的安装

    格式化文件系统(仅第一次执行即可,不要重复执行):hdfs/hadoop namenode -format 启动hdfs: sbin/start-dfs.sh 验证是否启动成功: jps DataNode SecondaryNameNode NameNode 浏览器访问方式: ...

    大数据技术课程设计.docx

    DataNode SecondaryNameNode DataNode YARN NodeManager ResourceManager NodeManager NodeManager 3. 配置集群 (1)核心配置文件 配置core-site.xml (2)HDFS配置文件 配置 hadoop-env.sh 配置 hadoop-site.xml...

    大数据运维技术第5章 Hadoop集群运行课件.pptx

    Hadoop运行状态;;5.1 Hadoop运行状态;Hadoop运行状态;Hadoop运行状态;Hadoop运行状态;Hadoop运行状态;5.2 配置Hadoop格式化;;5.3 查看JAVA进程;查看JAVA进程;查看JAVA进程;查看JAVA进程;5.4 查看HDFS的报告;查看HDFS...

    hadoop-formula

    哪些服务hadoop最终在给定主机上运行取决于盐粒定义的角色: hadoop_master将运行hadoop-namenode和hadoop-secondarynamenode服务hadoop_slave将运行hadoop-datanode服务roles: - hadoop_slave hadoop.mapred 安装...

    高可用性的HDFS:Hadoop分布式文件系统深度实践

    1.4.2 Hadoop的SecondaryNameNode方案 1.4.3 Hadoop的Checkpoint ode方案 1.4.4 Hadoop的BackupNode方案 1.4.5 DRDB方案 1.4.6 FaceBook的AvatarNode方案 1.5 方案优缺点比较 第2章 HDFS元数据解析 2.1 概述 2.2 ...

    hadoop安装

    [hadoop@a1 hadoop-0.20.2-cdh3u5]$ bin/hadoop namenode -format 开启 [hadoop@a1 hadoop-0.20.2-cdh3u5]$ bin/start-all.sh 在所有节点查看进程状态验证启动 [hadoop@a1 hadoop-0.20.2-cdh3u5]$ jps 8602 ...

    大数据技术与应用期末大作业

    DataNode SecondaryNameNode DataNode YARN NodeManager ResourceManager NodeManager NodeManager 3. 配置集群 (1)核心配置文件 配置core-site.xml (2)HDFS配置文件 配置 hadoop-env.sh 配置 hadoop-site.xml...

    工程实践III(大数据应用开发实践)大作业模板 - 副本.docx

    DataNode SecondaryNameNode DataNode YARN NodeManager ResourceManager NodeManager NodeManager 3. 配置集群 (1)核心配置文件 配置core-site.xml (2)HDFS配置文件 配置 hadoop-env.sh 配置 hadoop-site.xml...

    大数据技术大实验.docx

    DataNode SecondaryNameNode DataNode YARN NodeManager ResourceManager NodeManager NodeManager 3. 配置集群 (1)核心配置文件 配置core-site.xml (2)HDFS配置文件 配置 hadoop-env.sh 配置 hadoop-site.xml...

    Hadoop单机模式的配置与安装

    前言: 由于Hadoop属于java程序,所以,安装Hadoop之前需要先安装jdk。 对于hadoop0.20.2的单机模式安装就需要在一个节点(一台主机)上运行5个节点。...这样的一个类似于备份的节点叫做SecondaryNameNode。

    Hadoop平台搭建 1.搭建 步骤文档 2.搭建时要用到的组件安装包

    启动Hadoop集群:启动Hadoop集群的各个节点,包括NameNode、SecondaryNameNode、DataNode等。 用途: 数据存储和处理:Hadoop可以存储海量数据,并使用MapReduce等编程模型进行数据处理和分析。 数据挖掘和机器学习...

    hadoop快速入门

    hadoop快速入门,hadoop安装及hadoop配置

    hadoop的伪分布式模式配置与安装

    Hadoop版本:hadoop-0.20.2-cdh3u4.tar.gz 硬件环境: 三台主机:分别为 gdy192 192.168.61.192 gdy194 192.168.61.194 gdy195 192.168.61.195 本次部署模型为: gdy192上部署:NameNode和JobTracker gdy194上...

    hadoop-2.7.2.rar

    hadoop-2.7.2安装包,解压至/opt/...启动HDFS只需要敲入命令行:sbin/hadoop-daemon.sh start|stop namenode|datanode|secondarynamenode 启动YARN: sbin/yarn-daemon.sh start|stop resourcemanager|nodemanager

    hadoop-2.4.1安装软件包以及教程jdk.zip

    hdfs namenode -format (hadoop namenode -format) 3.4启动hadoop 先启动HDFS sbin/start-dfs.sh 再启动YARN sbin/start-yarn.sh 3.5验证是否启动成功 使用jps命令验证 27408 NameNode 28218 Jps ...

    大数据面试题-.docx

    a)它是 NameNode 的热备 b)它对内存没有要求 c)它的目的是帮助 NameNode 合并编辑日志,减少 NameNode 启动时间 d)SecondaryNameNode 应与 NameNode 部署到一个节点 多选题 8. 下列哪项可以作为集群的管理工具 ...

Global site tag (gtag.js) - Google Analytics