[jira] [Updated] (PIG-4897) Scope of param substitution for run/exec commands

["Koji Noguchi (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-4897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Koji Noguchi updated PIG-4897:
------------------------------
    Attachment: pig-4897-v05.patch

{quote}
* We would always want to deleteOnExit. So not sure if that needs to be a method argument
* Can we have one testcase with run -param output=xxx which will override the %default output value in the script?
{quote}
Attaching a new patch that does the above.  (pig-4897-v05.patch)
However, 

bq. Can we have one testcase for the macro case?
This turns out to be a major feedback.

After writing a test for this, realized that PigMacro has almost nothing to do with the changes I'm making here. 
General param saving and substitution happens from PigScriptParser.jj but Macro expansion is done through QueryParser.g.  I've seen it mentioned elsewhere but never hit me how this would affect my patch here.

I wish we didn't enable " global param substitutions in macros." in PIG-3359.  It's pretty confusing even without this jira.

For example, 

{code:title=example1.pig}
DEFINE mymacro (A) RETURNS void {
  store $A into '${output}';
}

A = load 'input.txt'
%declare output '/tmp/abc';
mymacro(A);
%declare output '/tmp/def';
{code}

would save the output to /tmp/def.  
Irrespective of where the macro is defined or called, it'll use the values defined after all the scripts are read.


> Scope of param substitution for run/exec commands
> -------------------------------------------------
>
>                 Key: PIG-4897
>                 URL: https://issues.apache.org/jira/browse/PIG-4897
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Koji Noguchi
>            Assignee: Koji Noguchi
>         Attachments: pig-4897-v01-notestyet.patch, pig-4897-v02.patch, pig-4897-v03.patch, pig-4897-v04.patch, pig-4897-v05.patch
>
>
> After PIG-3359, pig param substitution became global in that parameter declared in the pig script called from {{run}} or {{exec}} would live after that script finishes.  
> This created an interesting situation.
> {code:title=test1.pig}
> exec -param output=/tmp/deleteme111 test1_1.pig
> exec -param output=/tmp/deleteme222 test1_1.pig
> {code}
> {code:title=test1_1.pig}
> %default myout '$output.out';
> A = load 'input.txt' as (a0:int);
> store A into '$myout';
> {code}
> Running {{test1.pig}} would try to run two jobs that both tries to write to /tmp/deleteme111 and fail.  (Second param output=/tmp/deleteme222 is ignored.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)