You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org> on 2010/12/27 05:02:45 UTC
[jira] Commented: (PIG-1717) pig needs to call setPartitionFilter
if schema is null but getPartitionKeys is not
[ https://issues.apache.org/jira/browse/PIG-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12975196#action_12975196 ]
Dmitriy V. Ryaboy commented on PIG-1717:
----------------------------------------
Gerrit,
This is a bit tangential, but it seems like you could write a LoadMetadata to get this info for the HiveRC classes.
In fact I bet something like that already exists in Howl.
> pig needs to call setPartitionFilter if schema is null but getPartitionKeys is not
> ----------------------------------------------------------------------------------
>
> Key: PIG-1717
> URL: https://issues.apache.org/jira/browse/PIG-1717
> Project: Pig
> Issue Type: Improvement
> Components: impl
> Reporter: Gerrit Jansen van Vuuren
> Priority: Minor
>
> I'm writing a loader that works with hive style partitioning e.g. /logs/type1/daydate=2010-11-01
> The loader does not know the schema upfront and this is something that the user adds in the script using the AS clause.
> The problem is that this user defined schema is not available to the loader, so the loader cannot return any schema, the Loader does know what the partition keys are and pig needs in some way to know about these partition keys.
> Currently if the schema is null pig never calls the LoadMetaData:getPartitionKeys method or the setPartitionFilter method.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.