- 浏览: 144208 次
- 性别:
- 来自: 北京
文章分类
最新评论
-
EclipseEye:
fair_jm 写道不错 蛮详细的 谢谢分享
SWT/JFace专题 --- SWT中Display和多线程 -
fair_jm:
不错 蛮详细的 谢谢分享
SWT/JFace专题 --- SWT中Display和多线程
@Public
@Stable
Objects that are read from/written to a database should implement DBWritable. DBWritable, is similar to Writable except that the write(PreparedStatement) method takes a PreparedStatement, and readFields(ResultSet) takes a ResultSet.
Implementations are responsible for writing the fields of the object to PreparedStatement, and reading the fields of the object from the ResultSet.
Example:
If we have the following table in the database :
CREATE TABLE MyTable (
counter INTEGER NOT NULL,
timestamp BIGINT NOT NULL,
);
then we can read/write the tuples from/to the table with :
public class MyWritable implements Writable, DBWritable {
// Some data
private int counter;
private long timestamp;
//Writable#write() implementation
public void write(DataOutput out) throws IOException {
out.writeInt(counter);
out.writeLong(timestamp);
}
//Writable#readFields() implementation
public void readFields(DataInput in) throws IOException {
counter = in.readInt();
timestamp = in.readLong();
}
public void write(PreparedStatement statement) throws SQLException {
statement.setInt(1, counter);
statement.setLong(2, timestamp);
}
public void readFields(ResultSet resultSet) throws SQLException {
counter = resultSet.getInt(1);
timestamp = resultSet.getLong(2);
}
}
---------------
@InterfaceAudience.Public
@InterfaceStability.Stable
public interface DBWritable {
/**
* Sets the fields of the object in the {@link PreparedStatement}.
* @param statement the statement that the fields are put into.
* @throws SQLException
*/
public void write(PreparedStatement statement) throws SQLException;
/**
* Reads the fields of the object from the {@link ResultSet}.
* @param resultSet the {@link ResultSet} to get the fields from.
* @throws SQLException
*/
public void readFields(ResultSet resultSet) throws SQLException ;
}
@Stable
Objects that are read from/written to a database should implement DBWritable. DBWritable, is similar to Writable except that the write(PreparedStatement) method takes a PreparedStatement, and readFields(ResultSet) takes a ResultSet.
Implementations are responsible for writing the fields of the object to PreparedStatement, and reading the fields of the object from the ResultSet.
Example:
If we have the following table in the database :
CREATE TABLE MyTable (
counter INTEGER NOT NULL,
timestamp BIGINT NOT NULL,
);
then we can read/write the tuples from/to the table with :
public class MyWritable implements Writable, DBWritable {
// Some data
private int counter;
private long timestamp;
//Writable#write() implementation
public void write(DataOutput out) throws IOException {
out.writeInt(counter);
out.writeLong(timestamp);
}
//Writable#readFields() implementation
public void readFields(DataInput in) throws IOException {
counter = in.readInt();
timestamp = in.readLong();
}
public void write(PreparedStatement statement) throws SQLException {
statement.setInt(1, counter);
statement.setLong(2, timestamp);
}
public void readFields(ResultSet resultSet) throws SQLException {
counter = resultSet.getInt(1);
timestamp = resultSet.getLong(2);
}
}
---------------
@InterfaceAudience.Public
@InterfaceStability.Stable
public interface DBWritable {
/**
* Sets the fields of the object in the {@link PreparedStatement}.
* @param statement the statement that the fields are put into.
* @throws SQLException
*/
public void write(PreparedStatement statement) throws SQLException;
/**
* Reads the fields of the object from the {@link ResultSet}.
* @param resultSet the {@link ResultSet} to get the fields from.
* @throws SQLException
*/
public void readFields(ResultSet resultSet) throws SQLException ;
}
发表评论
-
数据迁移相关(关系型数据库mysql,oracle和nosql数据库如hbase)
2015-04-01 15:15 720HBase数据迁移(1) http://www.importn ... -
zookeeper适用场景:如何竞选Master及代码实现
2015-04-01 14:53 767zookeeper适用场景:如何竞选Master及代码实现 h ... -
MR/hive 数据去重
2015-04-01 14:43 708海量数据去重的五大策略 http://www.ciotimes ... -
面试牛x题
2015-03-18 23:50 0hive、mr(各需三道) 1.分别使用Hadoop MapR ... -
使用shell并发上传文件到hdfs
2015-03-16 21:41 1226使用shell并发上传文件到hdfs http://mos19 ... -
hadoop集群监控工具Apache Ambari
2015-03-14 17:27 0Apache Ambari官网 http://ambari.a ... -
Hadoop MapReduce优化相关
2015-03-16 21:46 449[大牛翻译系列]Hadoop 翻译文章索引 http://ww ... -
数据倾斜问题 牛逼(1)数据倾斜之MapReduce&hive
2015-03-16 21:43 776数据倾斜总结 http://www.alidata.org/a ... -
MapReduce牛逼(4)WritableComparable接口
2015-03-12 08:57 575@Public @Stable A Writable whi ... -
MapReduce牛逼(3)(继承WritableComparable)实现自定义key键,实现二重排序
2015-03-12 08:57 621package sort; import jav ... -
MapReduce牛逼(2)MR简单实现 导入数据到hbase例子
2015-03-12 08:57 1245package cmd; /** * MapRe ... -
MapReduce牛逼(1)MR单词计数例子
2015-03-11 00:44 1181package cmd; import org. ... -
InputFormat牛逼(9)FileInputFormat实现类之SequenceFileInputFormat
2015-03-11 00:24 1379一、SequenceFileInputFormat及Seque ... -
InputFormat牛逼(8)FileInputFormat实现类之TextInputFormat
2015-03-11 00:19 555/** An {@link InputFormat} for ... -
InputFormat牛逼(6)org.apache.hadoop.mapreduce.lib.db.DBRecordReader<T>
2015-03-11 00:11 646@Public @Evolving A RecordRead ... -
InputFormat牛逼(5)org.apache.hadoop.mapreduce.lib.db.DBInputFormat<T>
2015-03-10 23:10 578@Public @Stable A InputFormat ... -
InputFormat牛逼(4)org.apache.hadoop.mapreduce.RecordReader<KEYIN, VALUEIN>
2015-03-10 22:50 342@Public @Stable The record rea ... -
InputFormat牛逼(3)org.apache.hadoop.mapreduce.InputFormat<K, V>
2015-03-10 22:46 627@Public @Stable InputFormat d ... -
InputFormat牛逼(2)org.apache.hadoop.mapreduce.InputSplit & DBInputSplit
2015-03-10 22:22 503@Public @Stable InputSplit rep ... -
如何把hadoop2 的job作业 提交到 yarn平台
2015-01-08 21:09 0aaa萨芬撒点
相关推荐
Hadoop 代码使用方式 job.setInputFormatClass(SmallFileCombineTextInputFormat.class);...org.apache.hadoop.mapreduce.sample.SmallFileWordCount -Dmapreduce.input.fileinputformat.split.maxsize=10
编译环境:centos 6.4 64bit、maven 3.3.9、jdk1.7.0_79、lzo-2.09;...解决:hive报错:Cannot create an instance of InputFormat class org.apache.hadoop ....... as specified in mapredwork!
有前三章的内容前言第一部分 基础篇第1章 阅读源代码前的准备1.1 准备源代码学习环境1.1.1 基础软件下载1.1.2 如何准备Windows环境1.1.3 如何准备Linux环境1.2 获取Hadoop源代码1.3 搭建Hadoop源代码阅读...
71.5.2 相同程序在MapReduce中的扩展 91.6 用Hadoop统计单词——运行第一个程序 111.7 Hadoop历史 151.8 小结 161.9 资源 16第2章 初识Hadoop 172.1 Hadoop的构造模块 172.1.1 NameNode 172.1.2 ...
71.5.2 相同程序在MapReduce中的扩展 91.6 用Hadoop统计单词——运行第一个程序 111.7 Hadoop历史 151.8 小结 161.9 资源 16第2章 初识Hadoop 172.1 Hadoop的构造模块 172.1.1 NameNode 172.1.2 DataNode 182.1.3 ...
自定义MapReduce的InputFormat,实现提取指定开始与结束限定符的内容。
结合Hadoop源码,详细讲解了MapReduce开发中的InputFormat,很详细。
解决openOffice jodconverter-2.2.1包不能将2007及以上office转为PDF和解决txt乱码问题
书籍目录: 第一部分 Hadoop——一种分布式编程框架 第1章 Hadoop简介 1.1 为什么写《Hadoop 实战》 1.2 什么是Hadoop 1.3 了解分布式系统和Hadoop 1.4 比较SQL 数据库和Hadoop 1.5 理解MapReduce 1.5.1 动手...
问题背景:框架默认的TextInputformat切片机制是对任务按文件规划切片,不管文件多小,都会是一个单独的切片,都会交给一个MapTask,这样如果有大量
ExcelRecordReaderMapReducehadoop mapreduce的MapReduce输入格式以读取Microsoft Excel电子表格执照Apache许可。用法1.下载并运行ant。 2.在您的环境中包括ExcelRecordReaderMapReduce-0.0.1-SNAPSHOT.jar 3.使用...
Hadoop——一种分布式编程框架第1 章 Hadoop简介1.1 为什么写《Hadoop 实战》1.2 什么是Hadoop1.3 了解分布式系统和Hadoop1.4 比较SQL 数据库和Hadoop1.5 理解MapReduce1.5.1 动手扩展一个简单程序1.5.2 相同程序在...
MapReduce自定义InputFormat和RecordReader实现 MapReduce自定义OutputFormat和RecordWriter实现 Pig自定义LoadFunc加载和解析Apache HTTP日志事件 Pig的自定义EvalFunc使用MaxMind GEO API将IP地址转换为位置 另外...
在您的环境中包含dist/lib/json-mapreduce-1.0.jar 使用MultiLineJsonInputFormat类作为您的 Mapper InputFormat 假设您有一些类似于这样的 JSON: {"menu": { "header": "SVG Viewer", "items": [ {"id": ...
自定义inputFormat&&outputFormat1
hadoop中的mapreduce框架、spark。 hadoop中的mapreduce框架: 对编程模型阶段1实现就是:map task 对编程模型阶段2的实现就是reduce task。 map task: 读数据:InputFormat–>TextInputFormat
维基百科-Hadoop 维基百科输入格式和一些有用的维基百科 Hadoop 工具。用法首先,您必须将WikiInputFormat设置为您的作业 InputFormat: job . setInputFormatClass( WikiInputFormat . class); 您的 Mappers 传入 ...
3.2 InputFormat 数据输入 3.2.1 Job 提交流程和切片源码详解 3.2.2 FileInputFormat 切片机制
Apache Hive 的 InputFormat,在查询 SequenceFiles 时将返回 (Text) 键和 (Text) 值。 我需要在不拆分内容的情况下完整解析大量文本文件。 HDFS 在处理大型连续文件时提供最佳吞吐量,因此我使用 Apache Mahout 将...