You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@falcon.apache.org by "Satish Mittal (JIRA)" <ji...@apache.org> on 2014/03/18 07:32:44 UTC
[jira] [Comment Edited] (FALCON-357) HCatalog Feed replication:
Hive export job fails when table partition contains multiple dated columns
[ https://issues.apache.org/jira/browse/FALCON-357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13938199#comment-13938199 ]
Satish Mittal edited comment on FALCON-357 at 3/18/14 6:31 AM:
---------------------------------------------------------------
The HQL used by feed replication to perform partition export:
{code}
export table ${falconSourceDatabase}.${falconSourceTable} partition ${falconSourcePartition} to '${falconSourceStagingDir}';
{code}
The export fails because the value of falconSourcePartition is of the format:
falconSourcePartition=(minute='00' AND month='03' AND year='2014' AND hour='09' AND day='11')
Hive Export/Import DDL does NOT accept a partition value that consists of AND keywords. Instead it accepts a partition value string of format: (key1='value1',key2='value2',...).
Currently feed replication code uses oozie coord function dataInPartitionFilter() to create a partition value string. However this function is meant to create partition filters for Pig/Java/Hive. We need a new oozie coord function that returns a complete partition value string which can work with Hive export/import scenarios.
Created OOZIE-1741 for the same.
was (Author: satish.mittal):
The HQL used by feed replication to perform partition export:
export table ${falconSourceDatabase}.${falconSourceTable} partition ${falconSourcePartition} to '${falconSourceStagingDir}';
The export fails because the value of falconSourcePartition is of the format:
falconSourcePartition=(minute='00' AND month='03' AND year='2014' AND hour='09' AND day='11')
Hive Export/Import DDL does NOT accept a partition value that consists of AND keywords. Instead it accepts a partition value string of format: (key1='value1',key2='value2',...).
Currently feed replication code uses oozie coord function dataInPartitionFilter() to create a partition value string. However this function is meant to create partition filters for Pig/Java/Hive. We need a new oozie coord function that returns a complete partition value string which can work with Hive export/import scenarios.
Created OOZIE-1741 for the same.
> HCatalog Feed replication: Hive export job fails when table partition contains multiple dated columns
> -----------------------------------------------------------------------------------------------------
>
> Key: FALCON-357
> URL: https://issues.apache.org/jira/browse/FALCON-357
> Project: Falcon
> Issue Type: Bug
> Reporter: Satish Mittal
> Assignee: Satish Mittal
>
> Suppose a falcon feed is based on an HCatalog table partitioned by (year, month, day, hour, minute). The feed replication fails during hive export with the following error:
> Intercepting System.exit(40000)
> Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [40000]
> stderr logs:
> FAILED: ParseException line 1:51 mismatched input 'AND' expecting ) near ''00'' in export statement
--
This message was sent by Atlassian JIRA
(v6.2#6252)