Pig: Using Hcat Example

最新推荐文章于 2026-01-05 20:44:47 发布

原创最新推荐文章于 2026-01-05 20:44:47 发布 · 243 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#大数据 #数据库

Pig 专栏收录该内容

11 篇文章

订阅专栏

本文详细介绍了如何使用Pig和HCat进行数据的加载和存储操作，包括默认数据库的使用、指定数据库和创建数据库、表的操作，以及如何使用HCatStorer添加新的分区。

HCatLoader

//use default database
A = LOAD 'tablename' USING org.apache.hcatalog.pig.HCatLoader();

//specify database 'exampledb' and you could create the database and table using
//Hsql statements in Pig script after 0.10.
A = LOAD 'exampledb.tablename' USING org.apache.hcatalog.pig.HCatLoader();

because hive metastore is the same with Hcatalog, so the obove commands could be written as follows:

A = LOAD 'exampledb.tablename' USING org.apache.hive.hcatalog.pig.HCatLoader();

HCatStorer

//use default database
STORE processed_data INTO 'tablename' USING org.apache.hcatalog.pig.HCatStorer();

//or using hive metastore class
STORE processed_data INTO 'tablename' USING org.apache.hive.hcatalog.pig.HCatStorer();

//To add one new partition to a partitioned table, specify the partition value 
//in the store function.
store z into 'web_data' using org.apache.hcatalog.pig.HCatStorer('datestamp=20110924');

When using pig with hcatalog, you should care about the type system.