通过eclipse 远程提交mr(处理文件到hbase)到hadoop集群报错:
错误信息如下:
2015-04-19 11:31:46,800 WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-04-19 11:31:47,753 INFO org.apache.hadoop.conf.Configuration.deprecation - session.id is deprecated. Instead, use dfs.metrics.session-id
2015-04-19 11:31:47,753 INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics with processName=JobTracker, sessionId=
2015-04-19 11:31:47,821 INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
2015-04-19 11:31:47,824 INFO org.apache.hadoop.conf.Configuration.deprecation - mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
2015-04-19 11:31:47,824 INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.job.name is deprecated. Instead, use mapreduce.job.name
2015-04-19 11:31:47,824 INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
2015-04-19 11:31:47,824 INFO org.apache.hadoop.conf.Configuration.deprecation - mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
2015-04-19 11:31:47,824 INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
2015-04-19 11:31:47,929 INFO org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
2015-04-19 11:31:47,929 INFO org.apache.zookeeper.ZooKeeper - Client environment:host.name=lyq
2015-04-19 11:31:47,931 INFO org.apache.zookeeper.ZooKeeper - Client environment:java.version=1.6.0_45
2015-04-19 11:31:47,931 INFO org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Sun Microsystems Inc.
2015-04-19 11:31:47,931 INFO org.apache.zookeeper.ZooKeeper - Client environment:java.home=D:\Program Files\Java\jdk1.6.0_45\jre
2015-04-19 11:31:47,931 INFO org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=E:\Work\workspace\.metadata\.plugins\org.apache.hadoop.eclipse\hadoop-conf-401984830001496437;E:\Work\workspace\hadoop_study\bin;E:\Work\workspace\hadoop_study\lib\activation-1.1.jar;E:\Work\workspace\hadoop_study\lib\aopalliance-1.0.jar;E:\Work\workspace\hadoop_study\lib\asm-3.2.jar;E:\Work\workspace\hadoop_study\lib\avro-1.7.4.jar;E:\Work\workspace\hadoop_study\lib\commons-beanutils-1.7.0.jar;E:\Work\workspace\hadoop_study\lib\commons-beanutils-core-1.8.0.jar;E:\Work\workspace\hadoop_study\lib\commons-cli-1.2.jar;E:\Work\workspace\hadoop_study\lib\commons-codec-1.4.jar;E:\Work\workspace\hadoop_study\lib\commons-collections-3.2.1.jar;E:\Work\workspace\hadoop_study\lib\commons-compress-1.4.1.jar;E:\Work\workspace\hadoop_study\lib\commons-configuration-1.6.jar;E:\Work\workspace\hadoop_study\lib\commons-daemon-1.0.13.jar;E:\Work\workspace\hadoop_study\lib\commons-digester-1.8.jar;E:\Work\workspace\hadoop_study\lib\commons-el-1.0.jar;E:\Work\workspace\hadoop_study\lib\commons-httpclient-3.1.jar;E:\Work\workspace\hadoop_study\lib\commons-io-2.1.jar;E:\Work\workspace\hadoop_study\lib\commons-lang-2.5.jar;E:\Work\workspace\hadoop_study\lib\commons-logging-1.1.1.jar;E:\Work\workspace\hadoop_study\lib\commons-math-2.1.jar;E:\Work\workspace\hadoop_study\lib\commons-net-3.1.jar;E:\Work\workspace\hadoop_study\lib\guava-11.0.2.jar;E:\Work\workspace\hadoop_study\lib\guice-3.0.jar;E:\Work\workspace\hadoop_study\lib\guice-servlet-3.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-annotations-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-archives-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-auth-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-common-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-datajoin-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-distcp-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-extras-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-gridmix-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-hdfs-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-hdfs-nfs-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-mapreduce-client-app-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-mapreduce-client-common-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-mapreduce-client-core-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-mapreduce-client-hs-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-mapreduce-client-hs-plugins-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-mapreduce-client-jobclient-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-mapreduce-client-shuffle-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-mapreduce-examples-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-nfs-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-rumen-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-streaming-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-yarn-api-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-yarn-applications-distributedshell-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-yarn-applications-unmanaged-am-launcher-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-yarn-client-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-yarn-common-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-yarn-server-common-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-yarn-server-nodemanager-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-yarn-server-resourcemanager-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-yarn-server-web-proxy-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hadoop-yarn-site-2.2.0.jar;E:\Work\workspace\hadoop_study\lib\hamcrest-core-1.1.jar;E:\Work\workspace\hadoop_study\lib\jackson-core-asl-1.8.8.jar;E:\Work\workspace\hadoop_study\lib\jackson-jaxrs-1.8.8.jar;E:\Work\workspace\hadoop_study\lib\jackson-mapper-asl-1.8.8.jar;E:\Work\workspace\hadoop_study\lib\jackson-xc-1.8.8.jar;E:\Work\workspace\hadoop_study\lib\jasper-compiler-5.5.23.jar;E:\Work\workspace\hadoop_study\lib\jasper-runtime-5.5.23.jar;E:\Work\workspace\hadoop_study\lib\javax.inject-1.jar;E:\Work\workspace\hadoop_study\lib\jaxb-api-2.2.2.jar;E:\Work\workspace\hadoop_study\lib\jaxb-impl-2.2.3-1.jar;E:\Work\workspace\hadoop_study\lib\jersey-core-1.9.jar;E:\Work\workspace\hadoop_study\lib\jersey-guice-1.9.jar;E:\Work\workspace\hadoop_study\lib\jersey-json-1.9.jar;E:\Work\workspace\hadoop_study\lib\jersey-server-1.9.jar;E:\Work\workspace\hadoop_study\lib\jets3t-0.6.1.jar;E:\Work\workspace\hadoop_study\lib\jettison-1.1.jar;E:\Work\workspace\hadoop_study\lib\jetty-6.1.26.jar;E:\Work\workspace\hadoop_study\lib\jetty-util-6.1.26.jar;E:\Work\workspace\hadoop_study\lib\jsch-0.1.42.jar;E:\Work\workspace\hadoop_study\lib\jsp-api-2.1.jar;E:\Work\workspace\hadoop_study\lib\jsr305-1.3.9.jar;E:\Work\workspace\hadoop_study\lib\junit-4.10.jar;E:\Work\workspace\hadoop_study\lib\log4j-1.2.17.jar;E:\Work\workspace\hadoop_study\lib\mockito-all-1.8.5.jar;E:\Work\workspace\hadoop_study\lib\netty-3.6.2.Final.jar;E:\Work\workspace\hadoop_study\lib\paranamer-2.3.jar;E:\Work\workspace\hadoop_study\lib\protobuf-java-2.5.0.jar;E:\Work\workspace\hadoop_study\lib\servlet-api-2.5.jar;E:\Work\workspace\hadoop_study\lib\slf4j-api-1.7.5.jar;E:\Work\workspace\hadoop_study\lib\slf4j-log4j12-1.7.5.jar;E:\Work\workspace\hadoop_study\lib\snappy-java-1.0.4.1.jar;E:\Work\workspace\hadoop_study\lib\stax-api-1.0.1.jar;E:\Work\workspace\hadoop_study\lib\xmlenc-0.52.jar;E:\Work\workspace\hadoop_study\lib\xz-1.0.jar;E:\Work\workspace\hadoop_study\lib\zookeeper-3.4.5.jar;E:\Work\workspace\hadoop_study\lib\hbase-client-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-common-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-examples-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-hadoop-compat-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-hadoop2-compat-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-it-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-prefix-tree-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-protocol-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-server-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-shell-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-testing-util-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\hbase-thrift-0.96.2-hadoop2.jar;E:\Work\workspace\hadoop_study\lib\findbugs-annotations-1.3.9-1.jar;E:\Work\workspace\hadoop_study\lib\htrace-core-2.04.jar
2015-04-19 11:31:47,931 INFO org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=D:\Program Files\Java\jdk1.6.0_45\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;D:/Program Files/Java/jdk1.6.0_45/bin/../jre/bin/server;D:/Program Files/Java/jdk1.6.0_45/bin/../jre/bin;D:/Program Files/Java/jdk1.6.0_45/bin/../jre/lib/amd64;D:\Program Files\Java\jdk1.6.0_45\bin;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS Client\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;D:\App\hadoop-2.2.0\bin;D:\App\hadoop-2.2.0\sbin;D:\Program Files\SSH\;.;;D:\App\eclipse;;.
2015-04-19 11:31:47,931 INFO org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=C:\Users\liuyq\AppData\Local\Temp\
2015-04-19 11:31:47,931 INFO org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA>
2015-04-19 11:31:47,931 INFO org.apache.zookeeper.ZooKeeper - Client environment:os.name=Windows 7
2015-04-19 11:31:47,931 INFO org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64
2015-04-19 11:31:47,931 INFO org.apache.zookeeper.ZooKeeper - Client environment:os.version=6.1
2015-04-19 11:31:47,931 INFO org.apache.zookeeper.ZooKeeper - Client environment:user.name=liuyq
2015-04-19 11:31:47,931 INFO org.apache.zookeeper.ZooKeeper - Client environment:user.home=C:\Users\liuyq
2015-04-19 11:31:47,931 INFO org.apache.zookeeper.ZooKeeper - Client environment:user.dir=E:\Work\workspace\hadoop_study
2015-04-19 11:31:47,931 INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=localhost:2181 sessionTimeout=90000 watcher=hconnection-0x6a510e39, quorum=localhost:2181, baseZNode=/hbase
2015-04-19 11:31:47,956 INFO org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper - Process identifier=hconnection-0x6a510e39 connecting to ZooKeeper ensemble=localhost:2181
2015-04-19 11:31:48,111 INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (无法定位登录配置)
2015-04-19 11:31:48,114 ERROR org.apache.zookeeper.ClientCnxnSocketNIO - Unable to open socket to 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2181
2015-04-19 11:31:48,149 WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.SocketException: Address family not supported by protocol family: connect
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:266)
at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:276)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:958)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
2015-04-19 11:31:48,559 INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (无法定位登录配置)
2015-04-19 11:31:48,564 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper - Possibly transient ZooKeeper, quorum=localhost:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
2015-04-19 11:31:48,564 INFO org.apache.hadoop.hbase.util.RetryCounter - Sleeping 1000ms before retry #0...
2015-04-19 11:31:49,561 WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
2015-04-19 11:31:49,662 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper - Possibly transient ZooKeeper, quorum=localhost:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
2015-04-19 11:31:49,662 INFO org.apache.hadoop.hbase.util.RetryCounter - Sleeping 2000ms before retry #1...
2015-04-19 11:31:50,665 INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (无法定位登录配置)
2015-04-19 11:31:50,665 ERROR org.apache.zookeeper.ClientCnxnSocketNIO - Unable to open socket to 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2181
2015-04-19 11:31:50,665 WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.SocketException: Address family not supported by protocol family: connect
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:266)
at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:276)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:958)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
2015-04-19 11:31:50,768 INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (无法定位登录配置)
2015-04-19 11:31:51,773 WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
2015-04-19 11:31:51,874 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper - Possibly transient ZooKeeper, quorum=localhost:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
2015-04-19 11:31:51,874 INFO org.apache.hadoop.hbase.util.RetryCounter - Sleeping 4000ms before retry #2...
2015-04-19 11:31:52,874 INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (无法定位登录配置)
2015-04-19 11:31:52,874 ERROR org.apache.zookeeper.ClientCnxnSocketNIO - Unable to open socket to 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2181
2015-04-19 11:31:52,874 WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.SocketException: Address family not supported by protocol family: connect
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)
at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:266)
at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:276)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:958)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
原因:linux集群 etc/hosts 首行没有配置 127.0.0.1 localhost
解决方法:配置etc/hosts, 重启hadoop、hbase集群即可。
相关推荐
java 利用 sping-data-hadoop HbaseTemplate 操作hbase find get execute 等方法 可以直接运行
在eclipse中开发HBase程序的方法
hadoop2.73-eclipse开发hbase所需要的所有jar包,便于大家下载使用
eclipse构建HBase开发环境并运行实例对Hbase建表增删改查。本次HBase版本为hbase-0.90.5版本,Hadoop版本为Hadoop-0.20.2版本。
博客配套文件,详细演示了如何通过MR程序的方式bulkload数据到hbase,代码可直接用于生产环境。
Hbase笔记 —— 利用JavaAPI的方式操作Hbase数据库(往hbase的表中批量插入数据)
Java操作Hbase进行建表、删表以及对数据进行增删改查,条件查询
A.3实验三:熟悉常用的HBase操作 本实验对应第5章的内容。 A.3.1 实验目的 (1)理解HBase在Hadoop体系结构中的角色。(2)熟练使用HBase操作常用的 Shell命令。(3)熟悉HBase操作常用的 Java API。 A.3.2 实验平台 (1...
HBase基本操作 增删改查 java代码 要使用须导入对应的jar包
spring操作hbase demo
该资源主要用于在window10的Hadoop的bin文件下所需要的东西,主要用于使用window10中的eclipse访问当前的Linux的HBase,使用MapReduce的Job来实现复制HBase的表操作
java代码使用thrift2操作hbase示例,thrift2连接hbase添加数据,单条查找,删除数据,根据扫描器查找,修改数据等测试实例
在集群中创建java项目调用api来操作hbase,主要涉及对hbase的创建表格,删除表格,插入数据,删除数据,查询一条数据,查询所有数据等操作。 具体流程如下: 1.创建项目 2.获取jar包到项目的lib目录下(这边试用的事...
eclipse链接hbase所需jar包,hbase版本1.2.6,Hadoop版本2.7.1
java操作Hbase之从Hbase中读取数据写入hdfs中源码,附带全部所需jar包,欢迎下载学习。
NULL 博文链接:https://tangay.iteye.com/blog/724589
hadoop中Map-Reduce使用示例,输入(DBInputFormat),输出(DBOu-MR_HBase
代码文件,C#语言编写,通过Thrift2操作Hbase库,对数据进行简单的增、删、改、查处理·。
python3 使用 thrift 操作hbase 安装hbase-thirft后有一个Hbase报错 使用这个修改完成的Hbase类替换掉原来的Hbase类问题全部解决 主要是因为python版本兼容性带来的问题
java操作Hbase之Hbase专用过滤器PageFilter的使用源代码,附带全部所需源代码,欢迎下载学习。