You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@pig.apache.org by Markus Resch <ma...@adtech.de> on 2012/06/27 18:02:45 UTC

modulize pig scripts via 'run'; pass param containing special chars

Hey everyone, 

we're still using CDH3u3 pig (0.8.1). 
As out pig scripts are growing we like to split them to modules and call
them via run. the parameter substitution allows us to write very generic
scripts and modify them while calling. This worked very well until we
came to the point where we tried to pass a kind of complex stack of UDF
calls:

run -param timestamp=$timestamp -param
time_in_customers_timezone="(int)SUBSTRING(DATE_TIME(UnixToISO(((long)Our.TimeStamp)*1000), Timezone),11,13) AS Hour" our_script.pig

This line created the following error message:

ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during
parsing. Lexical error at line 32, column 63.  Encountered: "(" (40),
after : ""

We've desperately tried a lot of things (escaping, single/double quotes,
storing it in a local string ...)

Does anyone have a suggestion for us?

Thanks

Markus

Re: modulize pig scripts via 'run'; pass param containing special chars

Posted by Alan Gates <ga...@hortonworks.com>.

Does putting the parameters in a file using -param_file help?

Alan.

On Jun 27, 2012, at 9:02 AM, Markus Resch wrote:

> Hey everyone, 
> 
> we're still using CDH3u3 pig (0.8.1). 
> As out pig scripts are growing we like to split them to modules and call
> them via run. the parameter substitution allows us to write very generic
> scripts and modify them while calling. This worked very well until we
> came to the point where we tried to pass a kind of complex stack of UDF
> calls:
> 
> run -param timestamp=$timestamp -param
> time_in_customers_timezone="(int)SUBSTRING(DATE_TIME(UnixToISO(((long)Our.TimeStamp)*1000), Timezone),11,13) AS Hour" our_script.pig
> 
> This line created the following error message:
> 
> ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during
> parsing. Lexical error at line 32, column 63.  Encountered: "(" (40),
> after : ""
> 
> We've desperately tried a lot of things (escaping, single/double quotes,
> storing it in a local string ...)
> 
> Does anyone have a suggestion for us?
> 
> Thanks
> 
> Markus
> 
> 
> 
>