使用 sqoop-1.99.3-bin-hadoop200
配置如下
1.解压
tar -xvf sqoop--bin-hadoop.tar.gz
2.加入配置文件 server/conf/catalina.properties 中的common.loader 属性加入hadoop2的lib
因为hadoop_home 下的lib是空的 所以指向
/home/hadoop-2.3.0/share/hadoop/common/*.jar
/home/hadoop-2.3.0/share/hadoop/common/lib/*.jar
/home/hadoop-2.3.0/share/hadoop/yarn/*.jar
/home/hadoop-2.3.0/share/hadoop/hdfs/*.jar
/home/hadoop-2.3.0/share/hadoop/mapreduce/*.jar
启动后发现有问题, 查看 sqoop_home/server/log 报错了
Mar 19, 2014 8:08:55 AM org.apache.catalina.core.StandardContext listenerStart
SEVERE: Exception sending context initialized event to listener instance of class org.apache.sqoop.server.ServerInitializer
java.lang.RuntimeException: Failure in server initialization
at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:57)
Caused by: org.apache.sqoop.common.SqoopException: MAPREDUCE_0002:Failure on submission engine initialization - Invalid Hadoop configuration directory (not a directory or permission issues): /etc/hadoop/conf/
at org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.initialize(MapreduceSubmissionEngine.java:94)
at org.apache.sqoop.framework.JobManager.initialize(JobManager.java:215)
at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:53)
... 25 more
找到了 server/conf/sqoop.properties 里面的属性要改成
# Hadoop configuration directory
#org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/etc/hadoop/conf/
org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/home/hadoop-2.3.0/etc/hadoop
查看log 没有错误.进入./bin/sqoop.sh client
show version --all 查看有没错误。再次确认启动成功.
建立一个connection 根据提示输入
create connection --cid 1 (connection id为1)
Creating connection for connector with id 1
Please fill following values to create new connection object
Name: VPRCFJTBCOBI-02
Connection configuration
JDBC Driver Class: com.microsoft.sqlserver.jdbc.SQLServerDriver
JDBC Connection String: jdbc:sqlserver://VPRCFJTBCOBI-02\\BOE140:62064;DatabaseName=AdventureWorksDW2008R2;useUnicode=true;characterEncoding=UTF-8
Username: sa
Password: ********
JDBC Connection Properties:
There are currently 0 values in the map:
entry#
Security related configuration options
Max connections: 0
New connection was successfully created with validation status FINE and persistent id 2
查看下成果
show connector
+----+------------------------+---------+------------------------------------------------------+
| Id | Name | Version | Class |
+----+------------------------+---------+------------------------------------------------------+
| 1 | generic-jdbc-connector | 1.99.3 | org.apache.sqoop.connector.jdbc.GenericJdbcConnector |
show connection
+----+-----------------+-----------+---------+
| Id | Name | Connector | Enabled |
+----+-----------------+-----------+---------+
| 2 | VPRCFJTBCOBI-02 | 1 | true |
+----+-----------------+-----------+---------+
PS 建错了connection 就删了 delete connection --xid 1 也可以update connection --xid 1
相关推荐
sqoop-1.4.6-hadoop-2.6最小资源包已经经过严格测试,
sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz
Sqoop项目开始于2009年,最早是作为Hadoop的一个第三方模块存在,后来为了让使用者能够快速部署,也为了让开发人员能够更快速的迭代开发,Sqoop独立成为一个Apache项目。sqoop是什么? sqoop是个传输数据的工具。 ...
apache-atlas-2.3.0-hbase-hook.tar.gz Apache Atlas 框架是一组可扩展的核心基础治理服务,使企业能够有效且高效地满足 Hadoop 内的合规性要求,并允许与整个企业数据生态系统集成。这将通过使用规范和取证模型、...
sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.zip
sqoop-1.4.7.bin__hadoop-2.6.0.tar 目前能下载到的最新版本 为不喜欢去官网下载的小伙伴提供便利
sqoop-1.4.6.bin__hadoop-2.0.4-alpha.zip。。。。。。。。。。。。。。。。
Sqoop is a command-line interface application for transferring data between relational databases and Hadoop.
mv /usr/local/sqoop-1.4.6-cdh5.13.2/conf/sqoop-env.template.sh /usr/local/sqoop-1.4.6-cdh5.13.2/conf/sqoop-env.sh vi /usr/local/sqoop-1.4.6-cdh5.13.2/conf/sqoop-env.sh export HADOOP_COMMON_HOME=/usr/...
其中包含Sqoop将SqlServer文件导入HDFS文件的所有jar包
Sqoop是一个用于在Hadoop和关系数据库或大型机之间传输数据的...您可以使用Sqoop将关系数据库管理系统(RDBMS)中的数据导入Hadoop分布式文件系统(HDFS),转换Hadoop MapReduce中的数据,然后将数据导出回RDBMS 。
sqoop-1.4.6.bin__hadoop-2.0.4-alpha 导入导出工具,, 可以去官网下载, 也可以用这个
sqoop1.4.2版本,下载后请自行验证可用性
2、sqoop导入(RMDB-mysql、sybase到HDFS-hive) 网址:https://blog.csdn.net/chenwewi520feng/article/details/130572275 介绍sqoop从关系型数据库mysql、sybase同步到hdfs、hive中
sqoop-1.4.6.bin__hadoop-1.0.0.tar.gz
sqoop-1.4.6.bin__hadoop-0.23.tar.gz
java连接sqoop源码Hadoop-project-with-maven 设置了 Hadoop maven 依赖项的 Java WordCount 示例 这是一个练习,可帮助您安装和运行用 Java 编写的 hadoop 程序,首先在本地模式下的 IDE 中,然后在您将自己构建的 ...
sqlserver connector for apache hadoop
Sqoop(发音:skup)是一款开源的工具,主要用于在Hadoop(Hive)与传统的数据库(mysql、postgresql...)间进行数据的传递,可以将一个关系型数据库(例如 : MySQL ,Oracle ,Postgres等)中的数据导进到Hadoop的HDFS中,...