由于测试环境的改变,使用sqoop从oracle往hbase导数时一直出现下列错误:
13/10/12 10:16:54 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/dw/hadoop-2.0.0-cdh4.3.0/lib/native
13/10/12 10:16:54 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
13/10/12 10:16:54 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
13/10/12 10:16:54 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
13/10/12 10:16:54 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
13/10/12 10:16:54 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-358.el6.x86_64
13/10/12 10:16:54 INFO zookeeper.ZooKeeper: Client environment:user.name=hadoop
13/10/12 10:16:54 INFO zookeeper.ZooKeeper: Client environment:user.home=/home/hadoop
13/10/12 10:16:54 INFO zookeeper.ZooKeeper: Client environment:user.dir=/home/hadoop
13/10/12 10:16:54 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection
13/10/12 10:16:54 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 7576@master1
13/10/12 10:16:54 INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
13/10/12 10:16:54 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
13/10/12 10:16:54 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
13/10/12 10:16:54 INFO util.RetryCounter: Sleeping 2000ms before retry #1...
13/10/12 10:16:55 INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
13/10/12 10:16:55 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
13/10/12 10:16:56 INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
13/10/12 10:16:56 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
13/10/12 10:16:56 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
13/10/12 10:16:56 INFO util.RetryCounter: Sleeping 4000ms before retry #2...
13/10/12 10:16:57 INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
13/10/12 10:16:57 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
13/10/12 10:16:58 INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
13/10/12 10:16:58 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
13/10/12 10:16:59 INFO zookeeper.ClientCnxn: Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
13/10/12 10:16:59 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
导致这个问题的原因是:Master认为Region servers的IP为127.0.0.1 即localhost;Region servers错误的告诉Master它们的IP为127.0.0.1。
解决办法:
在Region servers上修改/etc/hosts
127.0.0.1 fully.qualified.regionservername regionservername localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
修改为:
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
上述问题解决。
分享到:
相关推荐
Hadoop集群搭建必备安装包,包括zookeeper3.4.12+hbase1.4.4+sqoop1.4.7bin_hadoop-2.6.0+kafka2.10亲测可用。
flume+hbase+sqoop+zookeeper的整合包,linux版本,如果还需要别的关于hadoop工具的话可以留言,我再上传
分享一下Hadoop、HBase、Hive、Pig、Zookeeper相关资料。
Hadoop+Hive+Mysql+Zookeeper+Hbase+Sqoop详细安装手册
hadoop环境搭建,涉及zookeeper、hbase、hive、mysql、kafka等服务的搭建测试运用。
一、 HBase技术介绍 HBase简介 HBase – Hadoop Database,是一个高可靠性、高性能、面向列、可伸缩的分布式存储系统... Sqoop则为HBase提供了方便的RDBMS数据导入功能,使得传统数据库数据向HBase中迁移变的非常方便。
hadoop生态圈所用到的组件,hadoop、hbase、kafka、spark、sqoop、zookeeper。远程连接服务器的Xshell和远程传送文件到服务器的Xftp。
实战开发通过jmx获取HBase运行时数据,监控集群状态 5-1 HBase备份与恢复 5-2 HBase监控简介 5-3 Hadoop JMX监控实战 5-4 HBase JMX监控实战 第6章 Phoenix & Sqoop 对HBase业务应用过程中常用到的第三方开源工具...
Rhel7.3 cdh5.11.2 mysql5.6.38 jdk8 HDFS、YARN、Zookeeper、Oozie、Hive、Hue、Sqoop和Hbase
大数据集群搭建,内含hadoop hive sqoop HBASE Phoenix spark kafka flume zookeeper flink 安装
基于 Zookeeper 搭建 Hadoop 高可用集群 二、Hive 简介及核心概念 Linux 环境下 Hive 的安装部署 CLI 和 Beeline 命令行的基本使用 常用 DDL 操作 分区表和分桶表 视图和索引 常用 DML 操作 数据查询详解 三、Spark ...
伪分布式安装教程: Flume Hbase Spark Hive Kafka Sqoop zookeeper等分布式系统框架 备注:Hadoop安装教程当时忘记记录,后续也懒得弄,所以上传资料也暂无hadoop安装教程,尽请理解!!!!
Ambari已支持大多数Hadoop组件,包括HDFS、MapReduce、Hive、Pig、 Hbase、Zookeeper、Sqoop和Hcatalog等。但是这里的 Hadoop 是广义,指的是 Hadoop 整个生态圈(例如 Hive,Hbase,Sqoop,Zookeeper 等),而并...
1、对hadoop、zookeeper、hbase、hive、sqoop、flume、pig的理论体系有个系统掌握,对核心知识能够掌握;阅读市面上的各种图书和各种文章不再有困难; 2、能够在linux环境搭建hadoop、hbase等的伪分布和分布模式,...
hadoop-2.7.7.tar.gz+zookeeper-3.4.14.tar.gz.zip,配置hadoop+zookeeper必须,版本已适配。 hbase下载链接:https://download.csdn.net/download/tracelessle/11797862; sqoop + jdk + mysql-connector-java下载...
包括Hadoop、Hive、Spark、Storm、Flink、HBase、Kafka、Zookeeper、Flume、Sqoop等技术的学习 Hadoop 分布式文件存储系统 —— HDFS 分布式计算框架 —— MapReduce 集群资源管理器 —— YARN Hadoop 单机伪集群...
cloudera hadoop搭建手册,CDH包括hdfs mapreduce hbase hive oozie sqoop zookeeper pig 等
Hadoop, Hive, HBase and ZooKeeper must be installed and configured for Sqoop to work. vi ~/.bash_profile export ZOOKEEPER_HOME=/usr/local/Cellar/zookeeper/3.4.14 export HADOOP_HOME=/usr/loc
第3阶段 - HBase、Hive、Sqoop、Flume、ZooKeeper、Storm、Kafka、Redis、MongoDB