You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Christopher Maier <ch...@gm.com> on 2015/11/13 19:47:51 UTC

RE: Backslash at end of string literal

Hi,

I haven't received any response on this, is there a known answer available?

Thanks,
Kit

From: Christopher Maier
Sent: Friday, October 23, 2015 11:39 AM
To: 'user@pig.apache.org' <us...@pig.apache.org>
Cc: Arnaud Balat <ar...@gm.com>
Subject: Backslash at end of string literal

Hi,

Pig seems not to handle backslashes at the end of string literals properly. I've tried several variations and haven't gotten it to work. Backslashes inside of string literals still work fine. Can someone confirm this, or what the correct syntax would be for a string literal ending in a backslash?

SCRIPT

tgzk9r9@wrnidvmedg001:/data/commonScripts/util/Omniture/pigscripts> cat test2.pig
a = load 'test1.txt' using PigStorage(',') as (A:chararray,B:chararray,C:chararray);
dump a;
b = foreach a generate 'abcd\\\\abcd';
dump b;
c = foreach a generate 'abcd\\\\';
dump c;

ERROR

tgzk9r9@wrnidvmedg001:/data/commonScripts/util/Omniture/pigscripts> pig -f test2.pig
INFO  [JobControl] org.apache.hadoop.mapreduce.lib.input.FileInputFormat     - Total input paths to process : 1
INFO  [main] org.apache.hadoop.mapreduce.lib.input.FileInputFormat     - Total input paths to process : 1
(a,b,c)
INFO  [JobControl] org.apache.hadoop.mapreduce.lib.input.FileInputFormat     - Total input paths to process : 1
INFO  [main] org.apache.hadoop.mapreduce.lib.input.FileInputFormat     - Total input paths to process : 1
(abcd\\abcd)
ERROR [main]      org.apache.pig.tools.grunt.Grunt     - ERROR 1000: Error during parsing. Lexical error at line 7, column 0.  Encountered: <EOF> after : ""
Details at logfile: /data/commonScripts/util/Omniture/pigscripts/pig_1445618087248.log

STACK TRACE

tgzk9r9@wrnidvmedg001:/data/commonScripts/util/Omniture/pigscripts> cat /data/commonScripts/util/Omniture/pigscripts/pig_1445618087248.log
Pig Stack Trace
---------------
ERROR 1000: Error during parsing. Lexical error at line 7, column 0.  Encountered: <EOF> after : ""

org.apache.pig.tools.pigscript.parser.TokenMgrError: Lexical error at line 7, column 0.  Encountered: <EOF> after : ""
        at org.apache.pig.tools.pigscript.parser.PigScriptParserTokenManager.getNextToken(PigScriptParserTokenManager.java:3318)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.jj_ntk(PigScriptParser.java:1379)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:106)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
        at org.apache.pig.Main.run(Main.java:478)
        at org.apache.pig.Main.main(Main.java:156)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
        at java.lang.reflect.Method.invoke(Method.java:619)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
================================================================================

PIG VERSION

tgzk9r9@wrnidvmedg001:/data/commonScripts/util/Omniture/pigscripts> pig -version
Apache Pig version 0.12.0 (rexported)
compiled Oct 26 2014, 23:43:04

Thanks,
Kit



Nothing in this message is intended to constitute an electronic signature unless a specific statement to the contrary is included in this message.

Confidentiality Note: This message is intended only for the person or entity to which it is addressed. It may contain confidential and/or privileged material. Any review, transmission, dissemination or other use, or taking of any action in reliance upon this message by persons or entities other than the intended recipient is prohibited and may be unlawful. If you received this message in error, please contact the sender and delete it from your computer.

Re: Backslash at end of string literal

Posted by Arvind S <ar...@gmail.com>.
reading this as a string from the file does not seem to pose this issue
..so it looks like a syntax parsing issue.

*Cheers !!*
Arvind

On Mon, Nov 16, 2015 at 9:25 AM, Arvind S <ar...@gmail.com> wrote:

> this same behavior seen in pig 0.15 also .. could not get any workaround
> ..
>
> *Cheers !!*
> Arvind
>
> On Sat, Nov 14, 2015 at 12:17 AM, Christopher Maier <
> christopher.maier@gm.com> wrote:
>
>> Hi,
>>
>> I haven't received any response on this, is there a known answer
>> available?
>>
>> Thanks,
>> Kit
>>
>> From: Christopher Maier
>> Sent: Friday, October 23, 2015 11:39 AM
>> To: 'user@pig.apache.org' <us...@pig.apache.org>
>> Cc: Arnaud Balat <ar...@gm.com>
>> Subject: Backslash at end of string literal
>>
>> Hi,
>>
>> Pig seems not to handle backslashes at the end of string literals
>> properly. I've tried several variations and haven't gotten it to work.
>> Backslashes inside of string literals still work fine. Can someone confirm
>> this, or what the correct syntax would be for a string literal ending in a
>> backslash?
>>
>> SCRIPT
>>
>> tgzk9r9@wrnidvmedg001:/data/commonScripts/util/Omniture/pigscripts> cat
>> test2.pig
>> a = load 'test1.txt' using PigStorage(',') as
>> (A:chararray,B:chararray,C:chararray);
>> dump a;
>> b = foreach a generate 'abcd\\\\abcd';
>> dump b;
>> c = foreach a generate 'abcd\\\\';
>> dump c;
>>
>> ERROR
>>
>> tgzk9r9@wrnidvmedg001:/data/commonScripts/util/Omniture/pigscripts> pig
>> -f test2.pig
>> INFO  [JobControl] org.apache.hadoop.mapreduce.lib.input.FileInputFormat
>>    - Total input paths to process : 1
>> INFO  [main] org.apache.hadoop.mapreduce.lib.input.FileInputFormat     -
>> Total input paths to process : 1
>> (a,b,c)
>> INFO  [JobControl] org.apache.hadoop.mapreduce.lib.input.FileInputFormat
>>    - Total input paths to process : 1
>> INFO  [main] org.apache.hadoop.mapreduce.lib.input.FileInputFormat     -
>> Total input paths to process : 1
>> (abcd\\abcd)
>> ERROR [main]      org.apache.pig.tools.grunt.Grunt     - ERROR 1000:
>> Error during parsing. Lexical error at line 7, column 0.  Encountered:
>> <EOF> after : ""
>> Details at logfile:
>> /data/commonScripts/util/Omniture/pigscripts/pig_1445618087248.log
>>
>> STACK TRACE
>>
>> tgzk9r9@wrnidvmedg001:/data/commonScripts/util/Omniture/pigscripts> cat
>> /data/commonScripts/util/Omniture/pigscripts/pig_1445618087248.log
>> Pig Stack Trace
>> ---------------
>> ERROR 1000: Error during parsing. Lexical error at line 7, column 0.
>> Encountered: <EOF> after : ""
>>
>> org.apache.pig.tools.pigscript.parser.TokenMgrError: Lexical error at
>> line 7, column 0.  Encountered: <EOF> after : ""
>>         at
>> org.apache.pig.tools.pigscript.parser.PigScriptParserTokenManager.getNextToken(PigScriptParserTokenManager.java:3318)
>>         at
>> org.apache.pig.tools.pigscript.parser.PigScriptParser.jj_ntk(PigScriptParser.java:1379)
>>         at
>> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:106)
>>         at
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
>>         at
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
>>         at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
>>         at org.apache.pig.Main.run(Main.java:478)
>>         at org.apache.pig.Main.main(Main.java:156)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94)
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
>>         at java.lang.reflect.Method.invoke(Method.java:619)
>>         at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>>
>> ================================================================================
>>
>> PIG VERSION
>>
>> tgzk9r9@wrnidvmedg001:/data/commonScripts/util/Omniture/pigscripts> pig
>> -version
>> Apache Pig version 0.12.0 (rexported)
>> compiled Oct 26 2014, 23:43:04
>>
>> Thanks,
>> Kit
>>
>>
>>
>> Nothing in this message is intended to constitute an electronic signature
>> unless a specific statement to the contrary is included in this message.
>>
>> Confidentiality Note: This message is intended only for the person or
>> entity to which it is addressed. It may contain confidential and/or
>> privileged material. Any review, transmission, dissemination or other use,
>> or taking of any action in reliance upon this message by persons or
>> entities other than the intended recipient is prohibited and may be
>> unlawful. If you received this message in error, please contact the sender
>> and delete it from your computer.
>>
>
>

Re: Backslash at end of string literal

Posted by Arvind S <ar...@gmail.com>.
this same behavior seen in pig 0.15 also .. could not get any workaround ..

*Cheers !!*
Arvind

On Sat, Nov 14, 2015 at 12:17 AM, Christopher Maier <
christopher.maier@gm.com> wrote:

> Hi,
>
> I haven't received any response on this, is there a known answer available?
>
> Thanks,
> Kit
>
> From: Christopher Maier
> Sent: Friday, October 23, 2015 11:39 AM
> To: 'user@pig.apache.org' <us...@pig.apache.org>
> Cc: Arnaud Balat <ar...@gm.com>
> Subject: Backslash at end of string literal
>
> Hi,
>
> Pig seems not to handle backslashes at the end of string literals
> properly. I've tried several variations and haven't gotten it to work.
> Backslashes inside of string literals still work fine. Can someone confirm
> this, or what the correct syntax would be for a string literal ending in a
> backslash?
>
> SCRIPT
>
> tgzk9r9@wrnidvmedg001:/data/commonScripts/util/Omniture/pigscripts> cat
> test2.pig
> a = load 'test1.txt' using PigStorage(',') as
> (A:chararray,B:chararray,C:chararray);
> dump a;
> b = foreach a generate 'abcd\\\\abcd';
> dump b;
> c = foreach a generate 'abcd\\\\';
> dump c;
>
> ERROR
>
> tgzk9r9@wrnidvmedg001:/data/commonScripts/util/Omniture/pigscripts> pig
> -f test2.pig
> INFO  [JobControl] org.apache.hadoop.mapreduce.lib.input.FileInputFormat
>    - Total input paths to process : 1
> INFO  [main] org.apache.hadoop.mapreduce.lib.input.FileInputFormat     -
> Total input paths to process : 1
> (a,b,c)
> INFO  [JobControl] org.apache.hadoop.mapreduce.lib.input.FileInputFormat
>    - Total input paths to process : 1
> INFO  [main] org.apache.hadoop.mapreduce.lib.input.FileInputFormat     -
> Total input paths to process : 1
> (abcd\\abcd)
> ERROR [main]      org.apache.pig.tools.grunt.Grunt     - ERROR 1000: Error
> during parsing. Lexical error at line 7, column 0.  Encountered: <EOF>
> after : ""
> Details at logfile:
> /data/commonScripts/util/Omniture/pigscripts/pig_1445618087248.log
>
> STACK TRACE
>
> tgzk9r9@wrnidvmedg001:/data/commonScripts/util/Omniture/pigscripts> cat
> /data/commonScripts/util/Omniture/pigscripts/pig_1445618087248.log
> Pig Stack Trace
> ---------------
> ERROR 1000: Error during parsing. Lexical error at line 7, column 0.
> Encountered: <EOF> after : ""
>
> org.apache.pig.tools.pigscript.parser.TokenMgrError: Lexical error at line
> 7, column 0.  Encountered: <EOF> after : ""
>         at
> org.apache.pig.tools.pigscript.parser.PigScriptParserTokenManager.getNextToken(PigScriptParserTokenManager.java:3318)
>         at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.jj_ntk(PigScriptParser.java:1379)
>         at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:106)
>         at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
>         at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
>         at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
>         at org.apache.pig.Main.run(Main.java:478)
>         at org.apache.pig.Main.main(Main.java:156)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
>         at java.lang.reflect.Method.invoke(Method.java:619)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>
> ================================================================================
>
> PIG VERSION
>
> tgzk9r9@wrnidvmedg001:/data/commonScripts/util/Omniture/pigscripts> pig
> -version
> Apache Pig version 0.12.0 (rexported)
> compiled Oct 26 2014, 23:43:04
>
> Thanks,
> Kit
>
>
>
> Nothing in this message is intended to constitute an electronic signature
> unless a specific statement to the contrary is included in this message.
>
> Confidentiality Note: This message is intended only for the person or
> entity to which it is addressed. It may contain confidential and/or
> privileged material. Any review, transmission, dissemination or other use,
> or taking of any action in reliance upon this message by persons or
> entities other than the intended recipient is prohibited and may be
> unlawful. If you received this message in error, please contact the sender
> and delete it from your computer.
>