How to load data into Hive ------------------------ In order to load data into Hive, we need to tell Hive the format of the data through "CREATE TABLE" statement:
FileFormat: the data has to be in Text or SequenceFile. Format of the row: If the data is in delimited format, use MetadataTypedColumnsetSerDe If the data is in delimited format and has more than 1 levels of delimitor, use DynamicSerDe with TCTLSeparatedProtocol If the data is a serialized thrift object, use ThriftSerDe The steps to load the data: 1 Create a table:
CREATE TABLE t (foo STRING, bar STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE;
CREATE TABLE t2 (foo STRING, bar ARRAY<STRING>) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' COLLECTION ITEMS TERMINATED BY ',' STORED AS TEXTFILE;
CREATE TABLE t3 (foo STRING, bar MAP<STRING,STRING>) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' COLLECTION ITEMS TERMINATED BY ',' MAP KEYS TERMINATED BY ':' STORED AS TEXTFILE;
CREATE TABLE t4 (foo STRING, bar MAP<STRING,STRING>) ROW FORMAT SERIALIZER 'org.apache.hadoop.hive.serde2.MetadataTypedColumnsetSerDe' WITH SERDEPROPERTIES ('columns'='foo,bar','SERIALIZATION.FORMAT'='9');
(RegexDeserializer is not done yet) CREATE TABLE t5 (foo STRING, bar STRING) ROW FORMAT SERIALIZER 'org.apache.hadoop.hive.serde2.RegexDeserializer' WITH SERDEPROPERTIES ('regex'='([a-z]*) ([a-z])');
2 Load the data: LOAD DATA LOCAL INPATH '../examples/files/kv1.txt' OVERWRITE INTO TABLE t;
How to read data from Hive tables ------------------------ In order to read data from Hive tables, we need to know the same 2 things:
File Format Row Format Then we just need to directly open the HDFS file and read the data.
CREATE TABLE table1 (a STRING, b STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' ESCAPED BY '\\' STORED AS TEXTFILE;
ESCAPED BY 指定转义字符
|
|
分享到:
相关推荐
From deploying Hive on your hardware or virtual machine and setting up its initial configuration to learning how Hive interacts with Hadoop, MapReduce, Tez and other big data technologies, Practical ...
DataGrip连接Hive所需jar包
hive 驱动包 hive 链接 datagrip的驱动包
With this example-driven guide, you'll learn how to use the Hive infrastructure to provide data summarization, query, and analysis - particularly with HiveQL, the query language dialect of SQL....
Dive into the world of SQL on Hadoop and get the most out of your Hive data warehouses. This book is your go-to resource for using Hive: authors Scott Shaw, Ankur Gupta, David Kjerrumgaard, and ...
(Hive输出)ETLDesigner\common\system\karaf\system\pentaho\pentaho-big-data-kettle-plugins-hive\6.1.0.1-196\下的文件。 (Hive输出)pentaho-big-data-kettle-plugins-hive-6.1.0.1-196。
How to start use Apache Hive.
Author Elton Stoneman uses Hive Succinctly to introduce the core principles of Hive and guides readers through mapping Hadoop and HBase data in Hive, writing complex queries in HiveQL, and running ...
2.6.1 Inserting data into Hive Tables from queries 21 2.6.2 Writing data into filesystem from queries 21 2.7 Cli 22 2.7.1 Hive Command line Options 22 2.7.2 Hive interactive Shell Command 24 2.7.3 ...
2.6.1 Inserting data into Hive Tables from queries 21 2.6.2 Writing data into filesystem from queries 21 2.7 Cli 22 2.7.1 Hive Command line Options 22 2.7.2 Hive interactive Shell Command 24 2.7.3 ...
hive3.0以上连接Datagrip所需要的jar包
DataGrip连接Hive驱动1.1.0,所需Jar包
This book is a practical guide on using the Apache Hadoop projects including ...How to transfer data from MySQL database to Hive, HDFS and HBase with Sqoop How create a Hive table over Apache Solr
How to create and use Hive Functions, Listing of Built-In Functions that are supported in Hive
/qybpm/ods/ods_INCIDENTS
hive1.x版本连接DataGrip需要的jar包
In this book, we prepare you for your journey into big data by firstly introducing you to backgrounds in the big data domain along with the process of setting up and getting familiar with your Hive ...
从数据库(mysql)中读取数据写入HDFS并建hive表;在该转换中,需要配置2个DB链接(Mysql和HiveServer2),配置Hadoop集群(HDFS),配置子服务器(远程执行服务器)。需要实践的元数据文件:mysql-to-hive.ktr
How to use HBase, Hive, Pig, Mahout, and Nutch to get things done easily and efficiently How to use MapReduce to solve many types of analytics problems Solve complex problems such as classifications, ...