制造namenode宕机的情况
1) kill 掉namenode的进程
[hadoop@hadoop bin]$ kill -9 13481
2)删除dfs.name.dir所指向的文件夹,这里是/home/hadoop/hdfs/name.
current image in_use.lock previous.checkpoint [hadoop@hadoop name]$ rm -rf *
删除name目录下的所有内容,但是必须保证name这个目录是存在的
3)从secondarynamenode元数据namesecondary目录下复制到namenode name目录下
我的secodarynamenode 元数据目录
/home/hadoop/tmp/dfs/namesecondary
复制过程
[hadoop@hadoop name]$ cp -R /home/hadoop/tmp/dfs/namesecondary/* .
4)启动namenode
[hadoop@hadoop bin]$ ./hadoop-daemon.sh start namenode
5)检查
使用hadoop fsck /user命令检查文件Block的完整性
<font size="2">[hadoop@hadoop bin]$ hadoop fsck /</font> Warning: $HADOOP_HOME is deprecated. FSCK started by hadoop from /192.168.0.101 for path / at Sun Dec 22 23:04:31 CST 2013 ................................... /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310222119_0001/job.jar: Under replicated blk_-8571652065964704775_1020. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310222119_0002/job.jar: Under replicated blk_-5947701456602696019_1021. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310222119_0003/job.jar: Under replicated blk_8214183112681524571_1022. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310222142_0001/job.jar: Under replicated blk_4805420250921446015_1024. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310222142_0002/job.jar: Under replicated blk_7913185784171356584_1027. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310222142_0004/job.jar: Under replicated blk_-8411847042533891069_1035. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310222142_0005/job.jar: Under replicated blk_2163772543235273521_1036. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310222142_0007/job.jar: Under replicated blk_-3491660194168043022_1044. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242056_0002/job.jar: Under replicated blk_5280511346594851641_1270. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242056_0003/job.jar: Under replicated blk_5588149584508213931_1271. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242056_0004/job.jar: Under replicated blk_-1846184614352398688_1272. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242056_0005/job.jar: Under replicated blk_8253537375261552577_1273. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242100_0001/job.jar: Under replicated blk_-6858089306760733073_1275. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242100_0002/job.jar: Under replicated blk_-630176777256891004_1276. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242100_0003/job.jar: Under replicated blk_3453389521553623867_1277. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242100_0004/job.jar: Under replicated blk_-4262000880964323956_1278. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242105_0001/job.jar: Under replicated blk_-5324801167724976561_1280. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242105_0002/job.jar: Under replicated blk_3284342834321881345_1281. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242105_0004/job.jar: Under replicated blk_5174401550469241860_1295. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242105_0009/job.jar: Under replicated blk_6390129220783606015_1327. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201310242105_0010/job.jar: Under replicated blk_8995477665353821346_1328. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201311292212_0007/job.jar: Under replicated blk_-6447241034801532571_1699. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201312082210_0001/job.jar: Under replicated blk_-187920261151639503_1741. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201312082210_0002/job.jar: Under replicated blk_1912732980088631445_1742. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/staging/hadoop/.staging/job_201312092348_0001/job.jar: Under replicated blk_448639237400606735_1953. Target Replicas is 10 but found 1 replica(s). . /home/hadoop/tmp/mapred/system/jobtracker.info: CORRUPT block blk_-4973841422235657473 /home/hadoop/tmp/mapred/system/jobtracker.info: MISSING 1 blocks of total size 4 B.Status: CORRUPT Total size: 367257 B Total dirs: 83 Total files: 60 Total blocks (validated): 57 (avg. block size 6443 B) ******************************** CORRUPT FILES: 1 MISSING BLOCKS: 1 MISSING SIZE: 4 B CORRUPT BLOCKS: 1 ******************************** Minimally replicated blocks: 56 (98.24561 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 25 (43.85965 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 1 Average block replication: 0.98245615 Corrupt blocks: 1 Missing replicas: 225 (401.7857 %) Number of data-nodes: 1 Number of racks: 1 FSCK ended at Sun Dec 22 23:04:31 CST 2013 in 89 milliseconds
恢复工作完成,检查hdfs的数据
原文链接:hadoop 根据SecondaryNameNode恢复Namenode
http://www.aboutyun.com/thread-6196-1-1.html
(出处: about云开发)
原作者:lzw
相关推荐
SecondaryNamenode,NameNode的镜像备份节点 ==Map Reduce== JobTracker,hadoop的Map/Reduce调度器,负责与TackTracker通信分配计算任务并跟踪任务进度。 TaskTracker,启动和管理Map和Reduce子任务的节点。
TaskTrack start-dfs.sh 启动Hadoop HDFS守护进程NameNode、SecondaryNameNode和DataNode stop-dfs.sh 停⽌Hadoop HDFS守护进程NameNode、SecondaryNameNode和DataNode hadoop-daemons.sh start namenode 单独启动...
DataNode,JobTracker,TaskTracker,SecondaryNameNode),请注意分布式运行中的这几个结点的区别:从分布式存储的角度来说,集群中的结点由一个NameNode和若干个DataNode组成,另有一个SecondaryNameNode作为NameNode的...
今天小编就为大家分享一篇关于Hadoop中namenode和secondarynamenode工作机制讲解,小编觉得内容挺不错的,现在分享给大家,具有很好的参考价值,需要的朋友一起跟随小编来看看吧
格式化文件系统(仅第一次执行即可,不要重复执行):hdfs/hadoop namenode -format 启动hdfs: sbin/start-dfs.sh 验证是否启动成功: jps DataNode SecondaryNameNode NameNode 浏览器访问方式: ...
DataNode SecondaryNameNode DataNode YARN NodeManager ResourceManager NodeManager NodeManager 3. 配置集群 (1)核心配置文件 配置core-site.xml (2)HDFS配置文件 配置 hadoop-env.sh 配置 hadoop-site.xml...
Hadoop运行状态;;5.1 Hadoop运行状态;Hadoop运行状态;Hadoop运行状态;Hadoop运行状态;Hadoop运行状态;5.2 配置Hadoop格式化;;5.3 查看JAVA进程;查看JAVA进程;查看JAVA进程;查看JAVA进程;5.4 查看HDFS的报告;查看HDFS...
哪些服务hadoop最终在给定主机上运行取决于盐粒定义的角色: hadoop_master将运行hadoop-namenode和hadoop-secondarynamenode服务hadoop_slave将运行hadoop-datanode服务roles: - hadoop_slave hadoop.mapred 安装...
1.4.2 Hadoop的SecondaryNameNode方案 1.4.3 Hadoop的Checkpoint ode方案 1.4.4 Hadoop的BackupNode方案 1.4.5 DRDB方案 1.4.6 FaceBook的AvatarNode方案 1.5 方案优缺点比较 第2章 HDFS元数据解析 2.1 概述 2.2 ...
[hadoop@a1 hadoop-0.20.2-cdh3u5]$ bin/hadoop namenode -format 开启 [hadoop@a1 hadoop-0.20.2-cdh3u5]$ bin/start-all.sh 在所有节点查看进程状态验证启动 [hadoop@a1 hadoop-0.20.2-cdh3u5]$ jps 8602 ...
DataNode SecondaryNameNode DataNode YARN NodeManager ResourceManager NodeManager NodeManager 3. 配置集群 (1)核心配置文件 配置core-site.xml (2)HDFS配置文件 配置 hadoop-env.sh 配置 hadoop-site.xml...
DataNode SecondaryNameNode DataNode YARN NodeManager ResourceManager NodeManager NodeManager 3. 配置集群 (1)核心配置文件 配置core-site.xml (2)HDFS配置文件 配置 hadoop-env.sh 配置 hadoop-site.xml...
DataNode SecondaryNameNode DataNode YARN NodeManager ResourceManager NodeManager NodeManager 3. 配置集群 (1)核心配置文件 配置core-site.xml (2)HDFS配置文件 配置 hadoop-env.sh 配置 hadoop-site.xml...
前言: 由于Hadoop属于java程序,所以,安装Hadoop之前需要先安装jdk。 对于hadoop0.20.2的单机模式安装就需要在一个节点(一台主机)上运行5个节点。...这样的一个类似于备份的节点叫做SecondaryNameNode。
启动Hadoop集群:启动Hadoop集群的各个节点,包括NameNode、SecondaryNameNode、DataNode等。 用途: 数据存储和处理:Hadoop可以存储海量数据,并使用MapReduce等编程模型进行数据处理和分析。 数据挖掘和机器学习...
hadoop快速入门,hadoop安装及hadoop配置
Hadoop版本:hadoop-0.20.2-cdh3u4.tar.gz 硬件环境: 三台主机:分别为 gdy192 192.168.61.192 gdy194 192.168.61.194 gdy195 192.168.61.195 本次部署模型为: gdy192上部署:NameNode和JobTracker gdy194上...
hadoop-2.7.2安装包,解压至/opt/...启动HDFS只需要敲入命令行:sbin/hadoop-daemon.sh start|stop namenode|datanode|secondarynamenode 启动YARN: sbin/yarn-daemon.sh start|stop resourcemanager|nodemanager
hdfs namenode -format (hadoop namenode -format) 3.4启动hadoop 先启动HDFS sbin/start-dfs.sh 再启动YARN sbin/start-yarn.sh 3.5验证是否启动成功 使用jps命令验证 27408 NameNode 28218 Jps ...
a)它是 NameNode 的热备 b)它对内存没有要求 c)它的目的是帮助 NameNode 合并编辑日志,减少 NameNode 启动时间 d)SecondaryNameNode 应与 NameNode 部署到一个节点 多选题 8. 下列哪项可以作为集群的管理工具 ...