You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by "Oleksiy Sayankin (JIRA)" <ji...@apache.org> on 2016/10/21 11:52:58 UTC
[jira] [Commented] (PARQUET-137) Add support for Pig datetimes
[ https://issues.apache.org/jira/browse/PARQUET-137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15594884#comment-15594884 ]
Oleksiy Sayankin commented on PARQUET-137:
------------------------------------------
Tested fix with Pig and Hive
*STEP 1: Create parquet data in Hive*
{code:sql}
CREATE TABLE IF NOT EXISTS `test` (id int);
CREATE External TABLE `pig` (
`campaignid` bigint,
`siteid` bigint,
`name` string,
`lastupdated` timestamp,
`created` timestamp,
`active` boolean
) STORED AS PARQUET LOCATION '/user/test/pig';
{code}
Insert data.
{code:sql}
INSERT OVERWRITE TABLE `test` VALUES (1),(2),(3),(4),(5),(6),(7),(8),(9),(10);
INSERT OVERWRITE TABLE `pig`
SELECT
1,
2,
'sample',
'2016-10-17 11:22:33.232323434',
'2016-10-17 11:22:33.232323434',
1
FROM `test`
LIMIT 10;
{code}
*STEP 2. Load the data using pig:*
{noformat}
REGISTER /usr/pig/pig-0.16/contrib/piggybank/java/piggybank.jar;
parqData = LOAD '/user/test/pig/000000_0' USING parquet.pig.ParquetLoader('campaignid:long,siteid:long,name:chararray,lastupdated:datetime,created:datetime,active:boolean');
DUMP parqData;
{noformat}
*EXPECTED RESULT:*
{noformat}
(1,2,sample,2016-10-17T20:22:33.232Z,2016-10-17T20:22:33.232Z,true)
(1,2,sample,2016-10-17T20:22:33.232Z,2016-10-17T20:22:33.232Z,true)
(1,2,sample,2016-10-17T20:22:33.232Z,2016-10-17T20:22:33.232Z,true)
(1,2,sample,2016-10-17T20:22:33.232Z,2016-10-17T20:22:33.232Z,true)
(1,2,sample,2016-10-17T20:22:33.232Z,2016-10-17T20:22:33.232Z,true)
(1,2,sample,2016-10-17T20:22:33.232Z,2016-10-17T20:22:33.232Z,true)
(1,2,sample,2016-10-17T20:22:33.232Z,2016-10-17T20:22:33.232Z,true)
(1,2,sample,2016-10-17T20:22:33.232Z,2016-10-17T20:22:33.232Z,true)
(1,2,sample,2016-10-17T20:22:33.232Z,2016-10-17T20:22:33.232Z,true)
(1,2,sample,2016-10-17T20:22:33.232Z,2016-10-17T20:22:33.232Z,true)
{noformat}
Worked as expected.
> Add support for Pig datetimes
> -----------------------------
>
> Key: PARQUET-137
> URL: https://issues.apache.org/jira/browse/PARQUET-137
> Project: Parquet
> Issue Type: Improvement
> Affects Versions: 1.6.0
> Reporter: Christian Rolf
> Priority: Minor
> Attachments: PARQUET-137.1.patch, parquet_datetime_support.patch
>
>
> There's currenly no support for conversion to/from Pig datetimes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)