You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Jonathan Packer (JIRA)" <ji...@apache.org> on 2013/08/06 03:01:34 UTC
[jira] [Updated] (PIG-3359) Register Statements and Param
Substitution in Macros
[ https://issues.apache.org/jira/browse/PIG-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Packer updated PIG-3359:
---------------------------------
Attachment: PIG-3359-v7.diff
Updated the patch.
There's a slight complication with the handling of the warnings in that "val" passed to those functions in PreprocessorContext is the raw string, which might not be the same as the val put into param_val. I added a separate hashtable "param_source" to keep track of this original argument, and if it's seen again to skip it instead of logging the warning.
Also updated the docs.
> Register Statements and Param Substitution in Macros
> ----------------------------------------------------
>
> Key: PIG-3359
> URL: https://issues.apache.org/jira/browse/PIG-3359
> Project: Pig
> Issue Type: Bug
> Components: parser
> Reporter: Jonathan Packer
> Assignee: Jonathan Packer
> Attachments: PIG-3359_test.tar.gz, PIG-3359-v1.diff, PIG-3359-v2.diff, PIG-3359-v3.diff, PIG-3359-v3-test-failures.txt, PIG-3359-v4.diff, PIG-3359-v5.diff, PIG-3359-v6.diff, PIG-3359-v7.diff
>
>
> There are some gaps in the functionality of macros that I've made a patch to address. The goal is to provide everything you'd need to make reusable algorithms libraries.
> 1. You can't register udfs inside a macro
> 2. Paramater substitutions aren't done inside macros
> 3. Resources (including macros) should not be redundantly acquired if they are already present.
> Rohini's patch https://issues.apache.org/jira/browse/PIG-3204 should address problem 3 where Pig reparses everything every time it reads a line, but there still would be a problem if two separate files import the same macro / udf file.
> To get this working, I moved methods for registering jars/udfs and param substitution from PigServer to PigContext so they can be accessed in QueryParserDriver which processes macros (QPD was already passed a PigContext reference). Is that ok?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira