由于原来配置的hadoop data目录快要用满了,故准备修改配置文件增加数据目录,以便扩容,但由于疏忽,把core-site.xml, hdfs-site.xml配置文件dfs.datanode.data.dir 配置项增加了配置目录,但未创建实际目录,重启datanode服务时,报如下错误:
2014-11-18 08:51:39,128 WARN org.apache.hadoop.hdfs.server.common.Util: Path /data1/cdh/data/hadoop-hdfs/data should be specified as a
URI in configuration files. Please update hdfs configuration.
2014-11-18 08:51:39,129 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for block pool Block pool <registe
ring> (Datanode Uuid unassigned) service to namenode.hadoop/192.168.136.177:9000
org.apache.hadoop.util.DiskChecker$DiskErrorException: Too many failed volumes - current valid volumes: 1, volumes configured: 2, volum
es failed: 1, volume failures tolerated: 0
at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.<init>(FsDatasetImpl.java:194)
at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetFactory.newInstance(FsDatasetFactory.java:34)
at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetFactory.newInstance(FsDatasetFactory.java:30)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:936)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:895)
at org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:274)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:220)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:813)
at java.lang.Thread.run(Thread.java:745)
2014-11-18 08:51:39,129 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Ending block pool service for: Block pool <registering> (
Datanode Uuid unassigned) service to namenode.hadoop/10.219.136.170:9000
2014-11-18 08:51:39,230 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool ID needed, but service not yet registered with
NN
java.lang.Exception: trace
at org.apache.hadoop.hdfs.server.datanode.BPOfferService.getBlockPoolId(BPOfferService.java:143)
at org.apache.hadoop.hdfs.server.datanode.BlockPoolManager.remove(BlockPoolManager.java:91)
at org.apache.hadoop.hdfs.server.datanode.DataNode.shutdownBlockPool(DataNode.java:854)
at org.apache.hadoop.hdfs.server.datanode.BPOfferService.shutdownActor(BPOfferService.java:350)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.cleanUp(BPServiceActor.java:617)
at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:835)
at java.lang.Thread.run(Thread.java:745)
?error
2014-11-18 08:51:06,567 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2014-11-18 08:51:06,568 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50020: starting
2014-11-18 08:51:06,851 INFO org.apache.hadoop.hdfs.server.common.Storage: Data-node version: -55 and name-node layout version: -55
2014-11-18 08:51:06,954 INFO org.apache.hadoop.hdfs.server.common.Storage: Locking is disabled
2014-11-18 08:51:06,955 INFO org.apache.hadoop.hdfs.server.common.Storage: Restored 0 block files from trash.
2014-11-18 08:51:06,502 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Refresh request received for nameservices: null
2014-11-18 08:51:06,532 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting BPOfferServices for nameservices: <default>
2014-11-18 08:51:06,567 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2014-11-18 08:51:06,568 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50020: starting
2014-11-18 08:51:06,851 INFO org.apache.hadoop.hdfs.server.common.Storage: Data-node version: -55 and name-node layout version: -55
2014-11-18 08:51:06,954 INFO org.apache.hadoop.hdfs.server.common.Storage: Locking is disabled
2014-11-18 08:51:06,955 INFO org.apache.hadoop.hdfs.server.common.Storage: Restored 0 block files from trash.
at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.<init>(FsDatasetImpl.java:194)
at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetFactory.newInstance(FsDatasetFactory.java:34)
at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetFactory.newInstance(FsDatasetFactory.java:30)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:936)
at org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:895)
java.lang.Exception: trace
at org.apache.hadoop.hdfs.server.datanode.BPOfferService.getBlockPoolId(BPOfferService.java:143)
at org.apache.hadoop.hdfs.server.datanode.BlockPoolManager.remove(BlockPoolManager.java:91)
at org.apache.hadoop.hdfs.server.datanode.DataNode.shutdownBlockPool(DataNode.java:854)
java.lang.Exception: trace
at org.apache.hadoop.hdfs.server.datanode.BPOfferService.getBlockPoolId(BPOfferService.java:143)
at org.apache.hadoop.hdfs.server.datanode.DataNode.shutdownBlockPool(DataNode.java:856)
2014-11-18 08:51:09,077 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 0
2014-11-18 08:51:09,108 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down DataNode at slave1/192.168.36.20
解决方法:修改hdfs-site.xml文件中dfs.datanode.data.dir内容为如下配置方式
<property>
<name>dfs.datanode.data.dir</name>
<value>
file:///data/cdh/data/dfs/data,file:///data1/cdh/data/hadoop-hdfs/data
</value>
<final>true</final>
</property>
之前是后面添加的目录配置方式没有加上file://前缀
,重启就OK
分享到:
相关推荐
深度剖析Hadoop HDFS-高清-完整目录-2017年3月。。。。
大数据 Hadoop HDFS 详解
java整合spring和hadoop HDFS全部jar
snapshot为hadoop2.1.0时新增加的功能。 主要为防止用户误删数据,和数据备份的作用 快照的对象为HDFS目录,前提是该目录是可以允许设置快照的(SnapShotable)
Hadoop HDFS和MapReduce架构浅析.pdf 更多资源请点击:https://blog.csdn.net/weixin_44155966
赠送jar包:hadoop-hdfs-2.7.3.jar; 赠送原API文档:hadoop-hdfs-2.7.3-javadoc.jar; 赠送源代码:hadoop-hdfs-2.7.3-sources.jar; 赠送Maven依赖信息文件:hadoop-hdfs-2.7.3.pom; 包含翻译后的API文档:hadoop...
python 操作 Hadoop hdfs 读read 写write 下载download
1. 查看HDFS文件系统目录:使用dfs -ls命令查看HDFS文件系统目录,例如bin/hadoop dfs -ls / 2. 删除HDFS文件系统中的文件:使用dfs -rm命令删除HDFS文件系统中的文件,例如bin/hadoop dfs -rm /in/test1.txt 3. ...
《HDFS——Hadoop分布式文件系统深度实践》
Prometheus Hadoop HDFS FSImage导出器 | 将Hadoop HDFS统计信息导出到包括 总数/每个用户/每个组/每个配置的目录路径/每个路径集 目录数 文件数 文件大小和大小分布(可选) 块数 文件复制(总体/每个用户摘要)...
从hadoop hdfs中读取数据,进行groupby 显示统计结果count、avg、max,用文字和柱状图两种图形界面表示
赠送jar包:hadoop-hdfs-2.6.5.jar; 赠送原API文档:hadoop-hdfs-2.6.5-javadoc.jar; 赠送源代码:hadoop-hdfs-2.6.5-sources.jar; 赠送Maven依赖信息文件:hadoop-hdfs-2.6.5.pom; 包含翻译后的API文档:hadoop...
Hadoop(HDFS)
Hadoop hdfs.md
赠送jar包:hadoop-hdfs-client-2.9.1.jar 赠送原API文档:hadoop-hdfs-client-2.9.1-javadoc.jar 赠送源代码:hadoop-hdfs-client-2.9.1-sources.jar 包含翻译后的API文档:hadoop-hdfs-client-2.9.1-javadoc-...
Hadoop HDFS_Shell命令详解
hadoop+hdfs实验数据资源
window客户端连接hadoop hdfs,完美解决障碍
深入理解Hadoop HDFS高级教程.txt打包整理.zip
HadoopHDFS架构概述推荐系统框架图