- 浏览: 214350 次
- 性别:
- 来自: 北京
文章分类
- 全部博客 (114)
- hbase (3)
- akka (7)
- hdfs (6)
- mapreduce (1)
- hive (0)
- zookeeper (8)
- storm (0)
- geese (0)
- leaf (0)
- stormbase (0)
- scala (2)
- oozie (11)
- zeromq (1)
- netty (3)
- mongodb (0)
- sqoop (2)
- flume (3)
- mahout (1)
- redis (0)
- lucene (1)
- solr (1)
- ganglia (3)
- 分布式理论 (2)
- hadoop (42)
- others (14)
- mq (1)
- clojure (3)
- flume ng (1)
- linux (1)
- esper (0)
最新评论
-
javalogo:
[b][i][u]引用[list]
[*][*][flash= ...
什么是Flume -
leibnitz:
what are they meanings
Hadoop Ganglia Metric Item -
di1984HIT:
没用过啊。
akka 介绍-Actor 基础 -
di1984HIT:
写的不错。
Hadoop管理-集群维护 -
developerinit:
很好,基本上介绍了
什么是Flume
场景:
NN HA 设置成功,HA切换客户端出现异常,
错误分析
用户执行Shell脚本问题
日志:
客户端
2012-08-01 14:37:07,798 WARN ipc.Client (Client.java:run(787)) - Unexpected error reading responses on connection Thread[IPC Client (1333933549) connection to bigdata-3/172.16.206.206:9000 from peter,5,main]
java.lang.NullPointerException
at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:852)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:781)
2012-08-01 14:37:07,807 WARN retry.RetryInvocationHandler (RetryInvocationHandler.java:invoke(118)) - Exception while invoking complete of class ClientNamenodeProtocolTranslatorPB. Trying to fail over immediately.
2012-08-01 14:37:07,970 WARN retry.RetryInvocationHandler (RetryInvocationHandler.java:invoke(118)) - Exception while invoking complete of class ClientNamenodeProtocolTranslatorPB after 1 fail over attempts. Trying to fail over after sleeping for 713ms.
2012-08-01 14:37:08,686 WARN retry.RetryInvocationHandler (RetryInvocationHandler.java:invoke(118)) - Exception while invoking complete of class ClientNamenodeProtocolTranslatorPB after 2 fail over attempts. Trying to fail over after sleeping for 1596ms.
2012-08-01 14:37:10,286 WARN retry.RetryInvocationHandler (RetryInvocationHandler.java:invoke(118)) - Exception while invoking complete of class ClientNamenodeProtocolTranslatorPB after 3 fail over attempts. Trying to fail over after sleeping for 2974ms.
2012-08-01 14:37:13,262 WARN retry.RetryInvocationHandler (RetryInvocationHandler.java:invoke(118)) - Exception while invoking complete of class ClientNamenodeProtocolTranslatorPB after 4 fail over attempts. Trying to fail over after sleeping for 7861ms.
服务器端
2012-08-01 14:54:45,614 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user peter
2012-08-01 14:54:45,619 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.allocateBlock: /user/peter/FS/100wan/1413. BP-283690147-172.16.206.206-1343792626658 blk_-6816230619303558443_3866{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[172.16.206.209:50010|RBW], ReplicaUnderConstruction[172.16.206.206:50010|RBW]]}
2012-08-01 14:54:46,529 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* addStoredBlock: blockMap updated: 172.16.206.206:50010 is added to blk_-6816230619303558443_3866{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[172.16.206.209:50010|RBW], ReplicaUnderConstruction[172.16.206.206:50010|RBW]]} size 0
2012-08-01 14:54:46,529 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* addStoredBlock: blockMap updated: 172.16.206.209:50010 is added to blk_-6816230619303558443_3866{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[172.16.206.209:50010|RBW], ReplicaUnderConstruction[172.16.206.206:50010|RBW]]} size 0
2012-08-01 14:54:46,531 INFO org.apache.hadoop.hdfs.StateChange: DIR* NameSystem.completeFile: file /user/peter/FS/100wan/1413 is closed by DFSClient_NONMAPREDUCE_-1368488343_1
2012-08-01 14:54:46,540 WARN org.apache.hadoop.security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user peter
org.apache.hadoop.util.Shell$ExitCodeException: id: peter:无此用户
at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
at org.apache.hadoop.util.Shell.run(Shell.java:188)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1116)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.<init>(FSPermissionChecker.java:51)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:4259)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4236)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1579)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1514)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:408)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:200)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:42590)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:427)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:916)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1692)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1688)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1686)
NN HA 设置成功,HA切换客户端出现异常,
错误分析
用户执行Shell脚本问题
日志:
客户端
2012-08-01 14:37:07,798 WARN ipc.Client (Client.java:run(787)) - Unexpected error reading responses on connection Thread[IPC Client (1333933549) connection to bigdata-3/172.16.206.206:9000 from peter,5,main]
java.lang.NullPointerException
at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:852)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:781)
2012-08-01 14:37:07,807 WARN retry.RetryInvocationHandler (RetryInvocationHandler.java:invoke(118)) - Exception while invoking complete of class ClientNamenodeProtocolTranslatorPB. Trying to fail over immediately.
2012-08-01 14:37:07,970 WARN retry.RetryInvocationHandler (RetryInvocationHandler.java:invoke(118)) - Exception while invoking complete of class ClientNamenodeProtocolTranslatorPB after 1 fail over attempts. Trying to fail over after sleeping for 713ms.
2012-08-01 14:37:08,686 WARN retry.RetryInvocationHandler (RetryInvocationHandler.java:invoke(118)) - Exception while invoking complete of class ClientNamenodeProtocolTranslatorPB after 2 fail over attempts. Trying to fail over after sleeping for 1596ms.
2012-08-01 14:37:10,286 WARN retry.RetryInvocationHandler (RetryInvocationHandler.java:invoke(118)) - Exception while invoking complete of class ClientNamenodeProtocolTranslatorPB after 3 fail over attempts. Trying to fail over after sleeping for 2974ms.
2012-08-01 14:37:13,262 WARN retry.RetryInvocationHandler (RetryInvocationHandler.java:invoke(118)) - Exception while invoking complete of class ClientNamenodeProtocolTranslatorPB after 4 fail over attempts. Trying to fail over after sleeping for 7861ms.
服务器端
2012-08-01 14:54:45,614 WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user peter
2012-08-01 14:54:45,619 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.allocateBlock: /user/peter/FS/100wan/1413. BP-283690147-172.16.206.206-1343792626658 blk_-6816230619303558443_3866{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[172.16.206.209:50010|RBW], ReplicaUnderConstruction[172.16.206.206:50010|RBW]]}
2012-08-01 14:54:46,529 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* addStoredBlock: blockMap updated: 172.16.206.206:50010 is added to blk_-6816230619303558443_3866{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[172.16.206.209:50010|RBW], ReplicaUnderConstruction[172.16.206.206:50010|RBW]]} size 0
2012-08-01 14:54:46,529 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* addStoredBlock: blockMap updated: 172.16.206.209:50010 is added to blk_-6816230619303558443_3866{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[172.16.206.209:50010|RBW], ReplicaUnderConstruction[172.16.206.206:50010|RBW]]} size 0
2012-08-01 14:54:46,531 INFO org.apache.hadoop.hdfs.StateChange: DIR* NameSystem.completeFile: file /user/peter/FS/100wan/1413 is closed by DFSClient_NONMAPREDUCE_-1368488343_1
2012-08-01 14:54:46,540 WARN org.apache.hadoop.security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user peter
org.apache.hadoop.util.Shell$ExitCodeException: id: peter:无此用户
at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)
at org.apache.hadoop.util.Shell.run(Shell.java:188)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:467)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:450)
at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:86)
at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:55)
at org.apache.hadoop.security.Groups.getGroups(Groups.java:88)
at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1116)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.<init>(FSPermissionChecker.java:51)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:4259)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:4236)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1579)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1514)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:408)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:200)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:42590)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:427)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:916)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1692)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1688)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1686)
发表评论
-
Hadoop TestDFSIO
2013-04-21 21:02 2401@VM [bigdata@bigdata hadoo ... -
Hadoop NNBENCH
2013-04-21 20:46 1602@VM [bigdata@bigdata hadoop]$ ... -
Hadoop 安装手册
2013-04-08 15:47 1161Hadoop 安装手册 软件准备 ... -
What do real life hadoop workloads look like
2012-09-10 15:52 787http://www.cloudera.com/blog/20 ... -
CDH4 HA 切换时间
2012-09-05 15:15 4296blocksize:35M filesize 96M zk-s ... -
CDH4 HA 切换
2012-09-05 10:51 1334HA 切换问题 切换时间太长。。。 copy 0 ... ... -
Hadoop CDh4 Standby HA 启动过程
2012-08-02 11:40 2831根据日志: StandBy NN启动过程 1.获得Active ... -
Hadoop TextOutput
2012-07-29 21:08 871TextOutputFormat 分隔符参数: mapredu ... -
Hadoop SteamXMLRecordReader
2012-07-28 23:59 669StreamXmlRecordReader 设置属性 str ... -
Hadoop NLineInputFormat
2012-07-28 23:52 1600NLineInputFormat 重写了splits 设置 ... -
KeyValueTextInputFormat
2012-07-28 23:40 920key/value 分割符 mapreduce.input. ... -
Hadoop 控制split尺寸
2012-07-28 23:08 1294三个参数决定Map的Split尺寸 1.mapred.min ... -
Setting up Disks for Hadoop
2012-07-22 12:13 842Setting up Disks for Hadoop He ... -
Upgrade hadoop need think about it
2012-07-21 17:17 838Compatibility When movin ... -
Hadoop 0.23 config differ from 0.20.205
2012-07-21 17:14 897http://hadoop.apache.org/common ... -
Hadoop hdfs block 状态
2012-07-15 13:37 6901.In Service -
Hadoop 配置不当引起集群不稳
2012-07-05 15:35 984配置不当内容 资源配置不当:内存、文件句柄数量、磁盘空间 ... -
Hadoop管理-集群维护
2012-07-03 15:27 49551.检查HDFS状态 fsck命令 1)f ... -
Hadoop Ganglia Metric Item
2012-06-27 11:13 1990dfs.FSDirectory.files_delete ... -
Hadoop 参数
2012-06-27 10:05 986转发自:http://www.cnblogs.com/g ...
相关推荐
CDH HA部署
presto-hadoop-cdh4.zip,CDH4 Hadoop for Presto的阴影版本CDH4 Hadoop for Presto的阴影版本
cloudera公司的CDH4版本hadoop安装说明
ha 方式安装 cdh4,hbase,补充原文档的内容
4 HDFS启用HA高可用性(基于Quorum-based Storage) 16 5.CDH安装使用lzo 22 5.1 hadoop_lzo安装 22 5.2 配置MapReduce: 23 5.3相关服务重启 25 6.安装Storm 25 7.附录. 25 7.1 CDH安装部署问题记录 25
由于CSDN上传文件大小限制,大家可以下载《CDH6.3.2下载.txt》获取网盘地址进行下载,我打包了CDH6.3.2 搭建所需要的各种安装文件,包括: manifest.json cloudera-manager.repo RPM-GPG-KEY-cloudera cm6.3.1-...
cdh6.3.2 适配 Phoenix; cdh6.3.2 集成 Phoenix
CDH7及以上版本已经更名为CDP 本资源打包了CDH7.1.5 搭建所需要的各种安装文件,包括: cm7.2.4-redhat7.tar.gz manifest.json cloudera-manager.repo RPM-GPG-KEY-cloudera CDH-7.1.5-1.cdh7.1.5.p0.7431829-el7....
cdh7.1.7包括: CDH-7.1.7-1.cdh7.1.7.p0.15945976-el7.parcel CDH-7.1.7-1.cdh7.1.7.p0.15945976-el7.parcel.sha1 CDH-7.1.7-1.cdh7.1.7.p0.15945976-el7.parcel.sha256 manifest.json cm7.4.7包括: cloudera-...
CDH6.3.2完整安装包网盘下载,包含 CDH-6.3.2-1.cdh6.3.2.p0.1605554-bionic.parcel、CDH-6.3.2-1.cdh6.3.2.p0.1605554-bionic.parcel.sha1、CDH-6.3.2-1.cdh6.3.2.p0.1605554-bionic.parcel.sha256、CDH-6.3.2-1....
CDH5.12.0
presto.zip,presto-hive connector-cdh 4 presto分布式大数据sql查询引擎的官方主页
CDH-6.3.3-1.cdh6.3.3.p0.1796617-el7.parcel CDH-6.3.3-1.cdh6.3.3.p0.1796617-el7.parcel.sha1 CDH-6.3.3-1.cdh6.3.3.p0.1796617-el7.parcel.sha256 如遇技术问题可添加微信咨询:15854186970
CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha1 CDH-6.3.2-1.cdh6.3.2.p0.1605554-el7.parcel.sha256 cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm ...
01、hadoop-common-3.0.0-cdh6.3.1.jar 02、hive-exec-2.1.1-cdh6.3.1.jar 03、hive-jdbc-2.1.1-cdh6.3.1.jar 04、hive-jdbc-2.1.1-cdh6.3.1-standalone.jar 05、hive-metastore-2.1.1-cdh6.3.1.jar 06、hive-...
Cloudera发布的实时查询开源项目,...mpala采用与Hive相同的元数据、SQL语法、ODBC驱动程序和用户接口(Hue Beeswax),这样在使用CDH产品时,批处理和实时查询的平台是统一的。此文档详细解释了Impala的安装配置和使用。
CDH6.3.2完整安装包网盘下载,包含以下内容: cdh离线安装教程;enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm;cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm;cloudera-manager-agent-6.3.1-...
CDH安装包
CDH4的高可用性HA,即双NameNode,一个为active,一个为standby。 原理介绍及安装配置操作详细说明。