- 浏览: 2602215 次
- 来自: 深圳
文章分类
最新评论
-
Wangwei86609:
非常好的规则引擎框架,支持决策树和多线程运行规则https:/ ...
规则引擎 -
comet0515:
dist.lib是什么文件?提示找不到
Jenkins(Hudson)+Findbugs/PMD/Checkstyle的配置要点 -
comet0515:
不是maven工程能不能用checkstyle插件?job是一 ...
HUDSON/Jenkins -
comet0515:
不是maven工程能不能用checkstyle插件?job是一 ...
Jenkins(Hudson)+Findbugs/PMD/Checkstyle的配置要点 -
comet0515:
不是maven工程能不能用checkstyle插件?job是一 ...
Jenkins(Hudson)+Findbugs/PMD/Checkstyle的配备要点
开源云计算技术系列(四)(Cloudera体验篇)
Cloudera 的定位在于
Bringing Big Da<wbr>ta to the Enterprise with Hadoop</wbr>
Cloudera为了让Hadoop的配置标准化,可以帮助企业安装,配置,运行hadoop以达到大规模企业数据的处理和分析。
既然是给企业使用,Cloudera的软件配置不是采用最新的hadoop 0.20,而是采用了Hadoop 0.18.3-12.cloudera.CH0_3的版本进行封装,并且集成了facebook提供的hive,yahoo提供的pig等基于hadoop的sql实现接口,使得这些软件的安装,配置和使用的成本降低并且进行了标准化。当然除了集成和封装这些成熟的工具外,Cloudera一个比较有意思的工具是sqoop,目前这个工具没有独立提供,因此这也是这次我们全面体验Cloudera的一个出发点,就是体验一下sqoop的工具的便捷性。
Sqoop (”SQL-to-Hadoop”),a tool designed to easily imp<wbr>ort information from SQL databases into your Hadoop cluster.通过sqoop,可以很方便的从传统的RDBMS里面导入数据到hadoop的集群,比如从mysql和oracle里面导入数据,非常方便,从导出到导入一条命令搞定,而且可以进行表的筛选,比起目前比较成熟的通过文本文件或者管道中转来说,开发的效率提升和配置的简洁是这个工具的特色所在。</wbr>
Sqoop可以做到
- Imp<wbr>orts individual tables or entire databases to files in HDFS</wbr>
- Generates Java classes to allow you to interact with your imp<wbr>orted da<wbr>ta</wbr></wbr>
- Provides the ability to imp<wbr>ort from SQL databases straight into your</wbr>Hiveda<wbr>ta warehouse</wbr>
After setting up an imp<wbr>ort job in Sqoop, you can get started working with SQL database-backed da<wbr>ta from your Hadoop MapReduce cluster in minutes.</wbr></wbr>
这里我们先通过一个例子来立即体验一下sqoop,然后在给大家介绍完整的这套云计算环境的配置。
这个例子演示的是如果把客户表的数据拿到hadoop集群上进行分析,如何导出users表的数据并自动导入到hive,在通过hive进行ad-hoc的sql查询分析。这样可以体现出hadoop的强大数据处理能力,并且不影响生产库。
先建立测试USERS表:
mysql> CREATE TABLE USERS (
-> user_id INTEGER NOT NULL PRIMARY KEY,
-> first_name VARCHAR(32) NOT NULL,
-> last_name VARCHAR(32) NOT NULL,
-> join_date DATE NOT NULL,
-> zip INTEGER,
-> state CHAR(2),
-> email VARCHAR(128),
-> password_hash CHAR(64));
Query OK, 0 rows affected (0.00 sec)
插入一条测试数据
insert into USERS (user_id,first_name,last_name,join_date,zip,state,email,password_hash) values (1,’a',’b',’20080808′,330440,’ha’,'test@test.com’,'xxxx’);
Query OK, 1 row affected, 1 warning (0.00 sec)
mysql> select * from USERS;
+———+————+———–+————+——–+——-+—————+—————+
| user_id | first_name | last_name | join_date | zip | state | email | password_hash |
+———+————+———–+————+——–+——-+—————+—————+
| 1 | a | b | 2008-08-08 | 330440 | ha | test@test.com | xxxx |
+———+————+———–+————+——–+——-+—————+—————+
1 row in set (0.00 sec)
然后我们使用sqoop导入mysq库的USERS表到hive。
sqoop –connect jdbc:mysql://localhost/test –username root –password xxx –local –table USERS –hive-imp<wbr>ort<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:43:50 INFO sqoop.Sqoop: Beginning co<wbr>de generation<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:43:50 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM USERS AS t WHERE 1 = 1<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:43:50 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM USERS AS t WHERE 1 = 1<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:43:50 INFO orm.CompilationManager: HADOOP_HOME is /usr/lib/hadoop<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:43:50 INFO orm.CompilationManager: Found hadoop core jar at: /usr/lib/hadoop/hadoop-0.18.3-12.cloudera.CH0_3-core.jar<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:43:50 INFO orm.CompilationManager: Invoking javac with args: -sourcepath ./ -d /tmp/sqoop/compile/ -classpath /etc/hadoop/conf:/home/hadoop/jdk1.6/lib/tools.jar:/usr/lib/hadoop:/usr/lib/hadoop/hadoop-0.18.3-12.cloudera.CH0_3-core.jar:/usr/lib/hadoop/lib/commons-cli-2.0-SNAPSHOT.jar:/usr/lib/hadoop/lib/commons-codec-1.3.jar:/usr/lib/hadoop/lib/commons-httpclient-3.0.1.jar:/usr/lib/hadoop/lib/commons-logging-1.0.4.jar:/usr/lib/hadoop/lib/commons-logging-api-1.0.4.jar:/usr/lib/hadoop/lib/commons-net-1.4.1.jar:/usr/lib/hadoop/lib/hadoop-0.18.3-12.cloudera.CH0_3-fairscheduler.jar:/usr/lib/hadoop/lib/hadoop-0.18.3-12.cloudera.CH0_3-scribe-log4j.jar:/usr/lib/hadoop/lib/hsqldb.jar:/usr/lib/hadoop/lib/jets3t-0.6.1.jar:/usr/lib/hadoop/lib/jetty-5.1.4.jar:/usr/lib/hadoop/lib/junit-4.5.jar:/usr/lib/hadoop/lib/kfs-0.1.3.jar:/usr/lib/hadoop/lib/libfb303.jar:/usr/lib/hadoop/lib/libthrift.jar:/usr/lib/hadoop/lib/log4j-1.2.15.jar:/usr/lib/hadoop/lib/mysql-connector-java-5.0.8-bin.jar:/usr/lib/hadoop/lib/oro-2.0.8.jar:/usr/lib/hadoop/lib/servlet-api.jar:/usr/lib/hadoop/lib/slf4j-api-1.4.3.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.4.3.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/jetty-ext/commons-el.jar:/usr/lib/hadoop/lib/jetty-ext/jasper-compiler.jar:/usr/lib/hadoop/lib/jetty-ext/jasper-runtime.jar:/usr/lib/hadoop/lib/jetty-ext/jsp-api.jar:/usr/lib/hadoop/hadoop-0.18.3-12.cloudera.CH0_3-core.jar:/usr/lib/hadoop/contrib/sqoop/hadoop-0.18.3-12.cloudera.CH0_3-sqoop.jar ./USERS.java<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:43:51 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop/compile/USERS.jar<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:43:51 INFO manager.LocalMySQLManager: Beginning mysqldump fast path imp<wbr>ort<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:43:51 INFO manager.LocalMySQLManager: Performing imp<wbr>ort of table USERS from database test<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:43:52 INFO manager.LocalMySQLManager: Transfer loop complete.<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:43:52 INFO hive.HiveImp<wbr>ort: Loading uploaded da<wbr>ta into Hive<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:43:52 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM USERS AS t WHERE 1 = 1<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:43:52 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM USERS AS t WHERE 1 = 1<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:43:52 WARN hive.TableDefWriter: Column join_date had to be cast to a less precise type in Hive<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:43:53 INFO hive.HiveImp<wbr>ort: Hive history file=/tmp/root/hive_job_log_root_200906201843_1606494848.txt<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:44:00 INFO hive.HiveImp<wbr>ort: OK<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:44:00 INFO hive.HiveImp<wbr>ort: Time taken: 5.916 seconds<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:44:00 INFO hive.HiveImp<wbr>ort: Loading da<wbr>ta to table users<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:44:00 INFO hive.HiveImp<wbr>ort: OK<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:44:00 INFO hive.HiveImp<wbr>ort: Time taken: 0.344 seconds<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px"> 09/06/20 18:44:01 INFO hive.HiveImp<wbr>ort: Hive imp<wbr>ort complete.</wbr></wbr></wbr></wbr></wbr></wbr></wbr></wbr></wbr></wbr></wbr></wbr></wbr></wbr></wbr>
导入成功,我们在hive里面验证一下导入的正确性。
hive
Hive history file=/tmp/root/hive_job_log_root_200906201844_376630602.txt
hive> select * from USERS;
OK
1 ‘a’ ‘b’ ‘2008-08-08′ 330440 ‘ha’ ‘test@test.com’ ‘xxxx’
Time taken: 5.019 seconds
hive>
可以看到和mysql库的数据完全一致。
这样我们就完成了从mysql库到HDFS的导入。
并且提供了一个自动生成的USERS.java程序供MapReduce 的分析使用。
more USERS.java
// ORM class for USERS
// WARNING: This class is AUTO-GENERATED. Modify at your own risk.
imp<wbr>ort org.apache.hadoop.io.Text;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
imp<wbr>ort org.apache.hadoop.io.Writable;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
imp<wbr>ort org.apache.hadoop.mapred.lib.db.DBWritable;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
imp<wbr>ort org.apache.hadoop.sqoop.lib.JdbcWritableBridge;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
imp<wbr>ort java.sql.PreparedStatement;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
imp<wbr>ort java.sql.ResultSet;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
imp<wbr>ort java.sql.SQLException;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
imp<wbr>ort java.io.DataInput;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
imp<wbr>ort java.io.DataOutput;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
imp<wbr>ort java.io.IOException;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
imp<wbr>ort java.sql.Date;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
imp<wbr>ort java.sql.Time;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
imp<wbr>ort java.sql.Timestamp;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
public class USERS implements DBWritable, Writable {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
public static final int PROTOCOL_VERSION = 1;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
private Integer user_id;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
public Integer get_user_id() {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
return user_id;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
private String first_name;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
public String get_first_name() {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
return first_name;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
private String last_name;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
public String get_last_name() {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
return last_name;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
private java.sql.Date join_date;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
public java.sql.Date get_join_date() {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
return join_date;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
private Integer zip;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
public Integer get_zip() {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
return zip;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
private String state;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
public String get_state() {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
return state;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
private String email;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
public String get_email() {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
return email;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
private String password_hash;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
public String get_password_hash() {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
return password_hash;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
public void readFields(ResultSet __dbResults) throws SQLException {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.user_id = JdbcWritableBridge.readInteger(1, __dbResults);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.first_name = JdbcWritableBridge.readString(2, __dbResults);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.last_name = JdbcWritableBridge.readString(3, __dbResults);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.join_date = JdbcWritableBridge.readDate(4, __dbResults);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.zip = JdbcWritableBridge.readInteger(5, __dbResults);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.state = JdbcWritableBridge.readString(6, __dbResults);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.email = JdbcWritableBridge.readString(7, __dbResults);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.password_hash = JdbcWritableBridge.readString(8, __dbResults);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
public void write(PreparedStatement __dbStmt) throws SQLException {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
JdbcWritableBridge.writeInteger(user_id, 1, 4, __dbStmt);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
JdbcWritableBridge.writeString(first_name, 2, 12, __dbStmt);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
JdbcWritableBridge.writeString(last_name, 3, 12, __dbStmt);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
JdbcWritableBridge.writeDate(join_date, 4, 91, __dbStmt);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
JdbcWritableBridge.writeInteger(zip, 5, 4, __dbStmt);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
JdbcWritableBridge.writeString(state, 6, 1, __dbStmt);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
JdbcWritableBridge.writeString(email, 7, 12, __dbStmt);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
JdbcWritableBridge.writeString(password_hash, 8, 1, __dbStmt);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
public void readFields(DataInput __dataIn) throws IOException {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
if (__dataIn.readBoolean()) {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.user_id = null;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
} else {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.user_id = Integer.valueOf(__dataIn.readInt());<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
if (__dataIn.readBoolean()) {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.first_name = null;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
} else {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.first_name = Text.readString(__dataIn);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
if (__dataIn.readBoolean()) {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.last_name = null;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
} else {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.last_name = Text.readString(__dataIn);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
if (__dataIn.readBoolean()) {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.join_date = null;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
} else {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.join_date = new Date(__dataIn.readLong());<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
if (__dataIn.readBoolean()) {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.zip = null;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
} else {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.zip = Integer.valueOf(__dataIn.readInt());<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
if (__dataIn.readBoolean()) {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.state = null;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
} else {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.state = Text.readString(__dataIn);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
if (__dataIn.readBoolean()) {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.email = null;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
} else {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.email = Text.readString(__dataIn);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
if (__dataIn.readBoolean()) {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.password_hash = null;<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
} else {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
this.password_hash = Text.readString(__dataIn);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
public void write(DataOutput __dataOut) throws IOException {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
if (null == this.user_id) {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
__dataOut.writeBoolean(true);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
} else {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
__dataOut.writeBoolean(false);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
__dataOut.writeInt(this.user_id);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
if (null == this.first_name) {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
__dataOut.writeBoolean(true);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
} else {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
__dataOut.writeBoolean(false);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
Text.writeString(__dataOut, first_name);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
if (null == this.last_name) {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
__dataOut.writeBoolean(true);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
} else {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
__dataOut.writeBoolean(false);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
Text.writeString(__dataOut, last_name);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
if (null == this.join_date) {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
__dataOut.writeBoolean(true);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
} else {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
__dataOut.writeBoolean(false);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
__dataOut.writeLong(this.join_date.getTime());<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
if (null == this.zip) {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
__dataOut.writeBoolean(true);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
} else {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
__dataOut.writeBoolean(false);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
__dataOut.writeInt(this.zip);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
if (null == this.state) {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
__dataOut.writeBoolean(true);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
} else {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
__dataOut.writeBoolean(false);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
Text.writeString(__dataOut, state);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
if (null == this.email) {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
__dataOut.writeBoolean(true);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
} else {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
__dataOut.writeBoolean(false);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
Text.writeString(__dataOut, email);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
if (null == this.password_hash) {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
__dataOut.writeBoolean(true);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
} else {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
__dataOut.writeBoolean(false);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
Text.writeString(__dataOut, password_hash);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
public String toString() {<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
StringBuilder sb = new StringBuilder();<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
sb.append(”" + user_id);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
sb.append(”,”);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
sb.append(first_name);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
sb.append(”,”);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
sb.append(last_name);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
sb.append(”,”);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
sb.append(”" + join_date);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
sb.append(”,”);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
sb.append(”" + zip);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
sb.append(”,”);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
sb.append(state);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
sb.append(”,”);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
sb.append(email);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
sb.append(”,”);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
sb.append(password_hash);<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
return sb.toString();<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}<br style="padding-bottom:0px; margin:0px; padding-left:0px; padding-right:0px; padding-top:0px">
}</wbr></wbr></wbr></wbr></wbr></wbr></wbr></wbr></wbr></wbr></wbr></wbr></wbr>
可以看到,自动生成的程序可读性非常好,可以进行自定义的二次开发使用。
转自:http://rdc.taobao.com/blog/dw/archives/411
相关推荐
Cloudera
该文档来自CCTC 2016中国云计算技术大会。Cloudera大数据技术专家韩军发表的题为“Hadoop架构在云上的应用实践”的主题演讲,欢迎下载!
妳那伊抹微笑_云计算之ClouderaManager以及CHD5.1.0集群部署安装文档V1.0.docx 该文档与教程http://blog.csdn.net/u012185296/article/details/38084107 配套使用的,主要讲的是妳那伊抹微笑_云计算之Cloudera...
Cloudera 5.4.x Documentation系列官方文档。压缩包里面共有十个文档!分别是: cloudera-administration.pdf-配置管理文档 cloudera-datamgmt.pdf-数据管理文档 cloudera-impala.pdf-impala使用文档 cloudera-...
cloudera 5.12安装包。
Cloudera Hadoop 安装指南
cloudera impala的官方文档,带书签
cloudera 安装部署手册 和base、solr运维
Cloudera Manager API调用实例(JAVA版),已通过测试可用。Cloudera Manager API调用实例(JAVA版),已通过测试可用。
cloudera-manager-installer.bin
完整系列: cloudera-manager-agent-5.9.0-1.cm590.p0.249.el7.x86_64.rpm cloudera-manager-daemons-5.9.0-1.cm590.p0.249.el7.x86_64.rpm cloudera-manager-server-5.9.0-1.cm590.p0.249.el7.x86_64.rpm cloudera-...
Cloudera Manager API v32 Introduced in Cloudera Manager 6.2.0 Cloudera Product Documentation
cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm, cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm, cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm, cloudera-manager-server-db-2-6.3.1-...
cloudera连接自定义MySQL数据库,cloudera集成hadoop以及所有组件
cloudera-impala.pdf )cloudera-impala.pdf )cloudera-impala.pdf )cloudera-impala.pdf )cloudera-impala.pdf )cloudera-impala.pdf )
Cloudera Manager安装部署.pdf
资源名称:cloudera官方文档资源目录:【】cloudera-administration【】cloudera-datamgmt【】cloudera-impala【】cloudera-installation【】cloudera-introduction【】cloudera-operation【】Cloudera-Opera ...
Cloudera产品体系及服务介绍,包含了Cloudera所有产品的介绍与使用
Cloudera Administration Handbook
作为业界最领先的企业级数据平台软件,Cloudera企业版除了包含业界最流行的基于开源Hadoop及其生态组件构建的CDH核心,还包含了大量为支撑企业级业务的高级管理特性。 借助于Cloudera企业版的整体解决方案,企业可以...