You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Xu Zhang (JIRA)" <ji...@apache.org> on 2008/04/04 00:07:25 UTC
[jira] Created: (PIG-184) Parsing issue on the stream statement?
Parsing issue on the stream statement?
--------------------------------------
Key: PIG-184
URL: https://issues.apache.org/jira/browse/PIG-184
Project: Pig
Issue Type: Bug
Reporter: Xu Zhang
Assignee: Arun C Murthy
When I ran the following Pig script on the command line (pig -c cluster -latest p_11), Pig returns immediately with doing any processing. When I entered the script line by line into GRUNT, after I entered the second line, I had to type one extra ";" in addition to the original one in order for Pig to accept the line. In the end, Pig gave me the correct results.
Here is the script, p_11:
{code}
A = load '/user/pig/tests/data/singlefile/studenttab10k';
B = stream A through `python ./python/MySimpleStreamApp.py --upper` as (name, age, gpa);
store B into 'results_11';
{code}
Here is the python streaming script, MySimpleStreamApp.py:
{code}
#!/usr/bin/python
import sys
import optparse
def main():
p = optparse.OptionParser()
p.add_option('--upper', '-u', action="store_true")
options, arguments = p.parse_args()
line = sys.stdin.readline()
while line:
if options.upper == True:
line = line.upper()
sys.stdout.write(line)
line = sys.stdin.readline()
if __name__ == '__main__':
main()
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-184) Parsing issue on the stream statement?
Posted by "Xu Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12586595#action_12586595 ]
Xu Zhang commented on PIG-184:
------------------------------
It works now with the latest Pig stuff. Thanks Arun.
> Parsing issue on the stream statement?
> --------------------------------------
>
> Key: PIG-184
> URL: https://issues.apache.org/jira/browse/PIG-184
> Project: Pig
> Issue Type: Bug
> Reporter: Xu Zhang
> Assignee: Arun C Murthy
> Attachments: PIG-184_0_20080407.patch
>
>
> When I ran the following Pig script on the command line (pig -c cluster -latest p_11), Pig returns immediately without doing any processing. When I entered the script line by line into GRUNT, after I entered the second line, I had to type one extra ";" in addition to the original one in order for Pig to accept the line. In the end, Pig gave me the correct results.
> Here is the script, p_11:
> {code}
> A = load '/user/pig/tests/data/singlefile/studenttab10k';
> B = stream A through `python ./python/MySimpleStreamApp.py --upper` as (name, age, gpa);
> store B into 'results_11';
> {code}
> Here is the python streaming script, MySimpleStreamApp.py:
> {code}
> #!/usr/bin/python
> import sys
> import optparse
> def main():
> p = optparse.OptionParser()
> p.add_option('--upper', '-u', action="store_true")
> options, arguments = p.parse_args()
> line = sys.stdin.readline()
> while line:
> if options.upper == True:
> line = line.upper()
> sys.stdout.write(line)
> line = sys.stdin.readline()
> if __name__ == '__main__':
> main()
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-184) Parsing issue on the stream statement?
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-184:
-------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
patch committed. thanks arun for contributing!
> Parsing issue on the stream statement?
> --------------------------------------
>
> Key: PIG-184
> URL: https://issues.apache.org/jira/browse/PIG-184
> Project: Pig
> Issue Type: Bug
> Reporter: Xu Zhang
> Assignee: Arun C Murthy
> Attachments: PIG-184_0_20080407.patch
>
>
> When I ran the following Pig script on the command line (pig -c cluster -latest p_11), Pig returns immediately without doing any processing. When I entered the script line by line into GRUNT, after I entered the second line, I had to type one extra ";" in addition to the original one in order for Pig to accept the line. In the end, Pig gave me the correct results.
> Here is the script, p_11:
> {code}
> A = load '/user/pig/tests/data/singlefile/studenttab10k';
> B = stream A through `python ./python/MySimpleStreamApp.py --upper` as (name, age, gpa);
> store B into 'results_11';
> {code}
> Here is the python streaming script, MySimpleStreamApp.py:
> {code}
> #!/usr/bin/python
> import sys
> import optparse
> def main():
> p = optparse.OptionParser()
> p.add_option('--upper', '-u', action="store_true")
> options, arguments = p.parse_args()
> line = sys.stdin.readline()
> while line:
> if options.upper == True:
> line = line.upper()
> sys.stdout.write(line)
> line = sys.stdin.readline()
> if __name__ == '__main__':
> main()
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-184) Parsing issue on the stream statement?
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585461#action_12585461 ]
Arun C Murthy commented on PIG-184:
-----------------------------------
This is a very interesting bug:
A = stream IP through `--upper`;
also exhibits this problem.
Looks like anything with "--" causes the parser to freeze up, maybe it needs a LOOKAHEAD - I'll take a closer look in the morning.
> Parsing issue on the stream statement?
> --------------------------------------
>
> Key: PIG-184
> URL: https://issues.apache.org/jira/browse/PIG-184
> Project: Pig
> Issue Type: Bug
> Reporter: Xu Zhang
> Assignee: Arun C Murthy
>
> When I ran the following Pig script on the command line (pig -c cluster -latest p_11), Pig returns immediately without doing any processing. When I entered the script line by line into GRUNT, after I entered the second line, I had to type one extra ";" in addition to the original one in order for Pig to accept the line. In the end, Pig gave me the correct results.
> Here is the script, p_11:
> {code}
> A = load '/user/pig/tests/data/singlefile/studenttab10k';
> B = stream A through `python ./python/MySimpleStreamApp.py --upper` as (name, age, gpa);
> store B into 'results_11';
> {code}
> Here is the python streaming script, MySimpleStreamApp.py:
> {code}
> #!/usr/bin/python
> import sys
> import optparse
> def main():
> p = optparse.OptionParser()
> p.add_option('--upper', '-u', action="store_true")
> options, arguments = p.parse_args()
> line = sys.stdin.readline()
> while line:
> if options.upper == True:
> line = line.upper()
> sys.stdout.write(line)
> line = sys.stdin.readline()
> if __name__ == '__main__':
> main()
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-184) Parsing issue on the stream statement?
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated PIG-184:
------------------------------
Attachment: PIG-184_0_20080407.patch
Fixes the grunt parser to correctly handle -- inside backtick-quoted strings ... thanks for noticing this Xu, it's a very interesting corner case! *smile*
> Parsing issue on the stream statement?
> --------------------------------------
>
> Key: PIG-184
> URL: https://issues.apache.org/jira/browse/PIG-184
> Project: Pig
> Issue Type: Bug
> Reporter: Xu Zhang
> Assignee: Arun C Murthy
> Attachments: PIG-184_0_20080407.patch
>
>
> When I ran the following Pig script on the command line (pig -c cluster -latest p_11), Pig returns immediately without doing any processing. When I entered the script line by line into GRUNT, after I entered the second line, I had to type one extra ";" in addition to the original one in order for Pig to accept the line. In the end, Pig gave me the correct results.
> Here is the script, p_11:
> {code}
> A = load '/user/pig/tests/data/singlefile/studenttab10k';
> B = stream A through `python ./python/MySimpleStreamApp.py --upper` as (name, age, gpa);
> store B into 'results_11';
> {code}
> Here is the python streaming script, MySimpleStreamApp.py:
> {code}
> #!/usr/bin/python
> import sys
> import optparse
> def main():
> p = optparse.OptionParser()
> p.add_option('--upper', '-u', action="store_true")
> options, arguments = p.parse_args()
> line = sys.stdin.readline()
> while line:
> if options.upper == True:
> line = line.upper()
> sys.stdout.write(line)
> line = sys.stdin.readline()
> if __name__ == '__main__':
> main()
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-184) Parsing issue on the stream statement?
Posted by "Xu Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xu Zhang updated PIG-184:
-------------------------
Description:
When I ran the following Pig script on the command line (pig -c cluster -latest p_11), Pig returns immediately without doing any processing. When I entered the script line by line into GRUNT, after I entered the second line, I had to type one extra ";" in addition to the original one in order for Pig to accept the line. In the end, Pig gave me the correct results.
Here is the script, p_11:
{code}
A = load '/user/pig/tests/data/singlefile/studenttab10k';
B = stream A through `python ./python/MySimpleStreamApp.py --upper` as (name, age, gpa);
store B into 'results_11';
{code}
Here is the python streaming script, MySimpleStreamApp.py:
{code}
#!/usr/bin/python
import sys
import optparse
def main():
p = optparse.OptionParser()
p.add_option('--upper', '-u', action="store_true")
options, arguments = p.parse_args()
line = sys.stdin.readline()
while line:
if options.upper == True:
line = line.upper()
sys.stdout.write(line)
line = sys.stdin.readline()
if __name__ == '__main__':
main()
{code}
was:
When I ran the following Pig script on the command line (pig -c cluster -latest p_11), Pig returns immediately with doing any processing. When I entered the script line by line into GRUNT, after I entered the second line, I had to type one extra ";" in addition to the original one in order for Pig to accept the line. In the end, Pig gave me the correct results.
Here is the script, p_11:
{code}
A = load '/user/pig/tests/data/singlefile/studenttab10k';
B = stream A through `python ./python/MySimpleStreamApp.py --upper` as (name, age, gpa);
store B into 'results_11';
{code}
Here is the python streaming script, MySimpleStreamApp.py:
{code}
#!/usr/bin/python
import sys
import optparse
def main():
p = optparse.OptionParser()
p.add_option('--upper', '-u', action="store_true")
options, arguments = p.parse_args()
line = sys.stdin.readline()
while line:
if options.upper == True:
line = line.upper()
sys.stdout.write(line)
line = sys.stdin.readline()
if __name__ == '__main__':
main()
{code}
> Parsing issue on the stream statement?
> --------------------------------------
>
> Key: PIG-184
> URL: https://issues.apache.org/jira/browse/PIG-184
> Project: Pig
> Issue Type: Bug
> Reporter: Xu Zhang
> Assignee: Arun C Murthy
>
> When I ran the following Pig script on the command line (pig -c cluster -latest p_11), Pig returns immediately without doing any processing. When I entered the script line by line into GRUNT, after I entered the second line, I had to type one extra ";" in addition to the original one in order for Pig to accept the line. In the end, Pig gave me the correct results.
> Here is the script, p_11:
> {code}
> A = load '/user/pig/tests/data/singlefile/studenttab10k';
> B = stream A through `python ./python/MySimpleStreamApp.py --upper` as (name, age, gpa);
> store B into 'results_11';
> {code}
> Here is the python streaming script, MySimpleStreamApp.py:
> {code}
> #!/usr/bin/python
> import sys
> import optparse
> def main():
> p = optparse.OptionParser()
> p.add_option('--upper', '-u', action="store_true")
> options, arguments = p.parse_args()
> line = sys.stdin.readline()
> while line:
> if options.upper == True:
> line = line.upper()
> sys.stdout.write(line)
> line = sys.stdin.readline()
> if __name__ == '__main__':
> main()
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-184) Parsing issue on the stream statement?
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated PIG-184:
------------------------------
Status: Patch Available (was: Open)
> Parsing issue on the stream statement?
> --------------------------------------
>
> Key: PIG-184
> URL: https://issues.apache.org/jira/browse/PIG-184
> Project: Pig
> Issue Type: Bug
> Reporter: Xu Zhang
> Assignee: Arun C Murthy
> Attachments: PIG-184_0_20080407.patch
>
>
> When I ran the following Pig script on the command line (pig -c cluster -latest p_11), Pig returns immediately without doing any processing. When I entered the script line by line into GRUNT, after I entered the second line, I had to type one extra ";" in addition to the original one in order for Pig to accept the line. In the end, Pig gave me the correct results.
> Here is the script, p_11:
> {code}
> A = load '/user/pig/tests/data/singlefile/studenttab10k';
> B = stream A through `python ./python/MySimpleStreamApp.py --upper` as (name, age, gpa);
> store B into 'results_11';
> {code}
> Here is the python streaming script, MySimpleStreamApp.py:
> {code}
> #!/usr/bin/python
> import sys
> import optparse
> def main():
> p = optparse.OptionParser()
> p.add_option('--upper', '-u', action="store_true")
> options, arguments = p.parse_args()
> line = sys.stdin.readline()
> while line:
> if options.upper == True:
> line = line.upper()
> sys.stdout.write(line)
> line = sys.stdin.readline()
> if __name__ == '__main__':
> main()
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.