You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Doug Daniels (JIRA)" <ji...@apache.org> on 2011/09/09 21:36:10 UTC
[jira] [Created] (PIG-2273) Pig.compileFromFile in embedded python
fails when pig script starts with a comment
Pig.compileFromFile in embedded python fails when pig script starts with a comment
----------------------------------------------------------------------------------
Key: PIG-2273
URL: https://issues.apache.org/jira/browse/PIG-2273
Project: Pig
Issue Type: Bug
Affects Versions: 0.9.0
Reporter: Doug Daniels
When using embedded Pig inside python, Pig.compileFromFile fails when the referenced pig file starts with a comment.
When you start the referenced pig file with a comment, you get no plan back and hence cannot successfully execute the job. When you start the same script without a comment, you get a plan and can execute the script successfully.
Strangely, this issue does not appear when using Pig.compile to compile directly from a string.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2273) Pig.compileFromFile in embedded
python fails when pig script starts with a comment
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104095#comment-13104095 ]
Alan Gates commented on PIG-2273:
---------------------------------
Unit tests pass, test-patch returns:
{code}
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
[exec]
[exec]
{code}
> Pig.compileFromFile in embedded python fails when pig script starts with a comment
> ----------------------------------------------------------------------------------
>
> Key: PIG-2273
> URL: https://issues.apache.org/jira/browse/PIG-2273
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Doug Daniels
> Assignee: Doug Daniels
> Attachments: PIG-2273.patch, PIG-2273.tar.gz, e2etest.patch
>
>
> When using embedded Pig inside python, Pig.compileFromFile fails when the referenced pig file starts with a comment.
> When you start the referenced pig file with a comment, you get no plan back and hence cannot successfully execute the job. When you start the same script without a comment, you get a plan and can execute the script successfully.
> Strangely, this issue does not appear when using Pig.compile to compile directly from a string.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2273) Pig.compileFromFile in embedded python
fails when pig script starts with a comment
Posted by "Doug Daniels (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Daniels updated PIG-2273:
------------------------------
Attachment: (was: PIG-2273.patch)
> Pig.compileFromFile in embedded python fails when pig script starts with a comment
> ----------------------------------------------------------------------------------
>
> Key: PIG-2273
> URL: https://issues.apache.org/jira/browse/PIG-2273
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Doug Daniels
> Attachments: PIG-2273.patch, PIG-2273.tar.gz
>
>
> When using embedded Pig inside python, Pig.compileFromFile fails when the referenced pig file starts with a comment.
> When you start the referenced pig file with a comment, you get no plan back and hence cannot successfully execute the job. When you start the same script without a comment, you get a plan and can execute the script successfully.
> Strangely, this issue does not appear when using Pig.compile to compile directly from a string.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2273) Pig.compileFromFile in embedded python
fails when pig script starts with a comment
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates updated PIG-2273:
----------------------------
Resolution: Fixed
Fix Version/s: 0.10
Status: Resolved (was: Patch Available)
Patch committed to trunk. Thanks Doug.
> Pig.compileFromFile in embedded python fails when pig script starts with a comment
> ----------------------------------------------------------------------------------
>
> Key: PIG-2273
> URL: https://issues.apache.org/jira/browse/PIG-2273
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Doug Daniels
> Assignee: Doug Daniels
> Fix For: 0.10
>
> Attachments: PIG-2273.patch, PIG-2273.tar.gz, e2etest.patch
>
>
> When using embedded Pig inside python, Pig.compileFromFile fails when the referenced pig file starts with a comment.
> When you start the referenced pig file with a comment, you get no plan back and hence cannot successfully execute the job. When you start the same script without a comment, you get a plan and can execute the script successfully.
> Strangely, this issue does not appear when using Pig.compile to compile directly from a string.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (PIG-2273) Pig.compileFromFile in embedded python
fails when pig script starts with a comment
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates reassigned PIG-2273:
-------------------------------
Assignee: Doug Daniels
> Pig.compileFromFile in embedded python fails when pig script starts with a comment
> ----------------------------------------------------------------------------------
>
> Key: PIG-2273
> URL: https://issues.apache.org/jira/browse/PIG-2273
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Doug Daniels
> Assignee: Doug Daniels
> Attachments: PIG-2273.patch, PIG-2273.tar.gz
>
>
> When using embedded Pig inside python, Pig.compileFromFile fails when the referenced pig file starts with a comment.
> When you start the referenced pig file with a comment, you get no plan back and hence cannot successfully execute the job. When you start the same script without a comment, you get a plan and can execute the script successfully.
> Strangely, this issue does not appear when using Pig.compile to compile directly from a string.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2273) Pig.compileFromFile in embedded python
fails when pig script starts with a comment
Posted by "Doug Daniels (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Daniels updated PIG-2273:
------------------------------
Status: Patch Available (was: Open)
> Pig.compileFromFile in embedded python fails when pig script starts with a comment
> ----------------------------------------------------------------------------------
>
> Key: PIG-2273
> URL: https://issues.apache.org/jira/browse/PIG-2273
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Doug Daniels
> Attachments: PIG-2273.patch, PIG-2273.tar.gz
>
>
> When using embedded Pig inside python, Pig.compileFromFile fails when the referenced pig file starts with a comment.
> When you start the referenced pig file with a comment, you get no plan back and hence cannot successfully execute the job. When you start the same script without a comment, you get a plan and can execute the script successfully.
> Strangely, this issue does not appear when using Pig.compile to compile directly from a string.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2273) Pig.compileFromFile in embedded python
fails when pig script starts with a comment
Posted by "Doug Daniels (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Daniels updated PIG-2273:
------------------------------
Attachment: PIG-2273.tar.gz
I've attached two scripts to show the issue.
Running 'pig -x local withComment.py' fails to produce a plan or run the script. Running 'pig -x local withoutComment.py' produces a plan and runs the script.
> Pig.compileFromFile in embedded python fails when pig script starts with a comment
> ----------------------------------------------------------------------------------
>
> Key: PIG-2273
> URL: https://issues.apache.org/jira/browse/PIG-2273
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Doug Daniels
> Attachments: PIG-2273.tar.gz
>
>
> When using embedded Pig inside python, Pig.compileFromFile fails when the referenced pig file starts with a comment.
> When you start the referenced pig file with a comment, you get no plan back and hence cannot successfully execute the job. When you start the same script without a comment, you get a plan and can execute the script successfully.
> Strangely, this issue does not appear when using Pig.compile to compile directly from a string.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2273) Pig.compileFromFile in embedded python
fails when pig script starts with a comment
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates updated PIG-2273:
----------------------------
Attachment: e2etest.patch
This patch adds an e2e test that tests the same functionality.
> Pig.compileFromFile in embedded python fails when pig script starts with a comment
> ----------------------------------------------------------------------------------
>
> Key: PIG-2273
> URL: https://issues.apache.org/jira/browse/PIG-2273
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Doug Daniels
> Assignee: Doug Daniels
> Attachments: PIG-2273.patch, PIG-2273.tar.gz, e2etest.patch
>
>
> When using embedded Pig inside python, Pig.compileFromFile fails when the referenced pig file starts with a comment.
> When you start the referenced pig file with a comment, you get no plan back and hence cannot successfully execute the job. When you start the same script without a comment, you get a plan and can execute the script successfully.
> Strangely, this issue does not appear when using Pig.compile to compile directly from a string.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2273) Pig.compileFromFile in embedded
python fails when pig script starts with a comment
Posted by "Doug Daniels (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101513#comment-13101513 ]
Doug Daniels commented on PIG-2273:
-----------------------------------
It seems now like any comments in the script, not just ones at the beginning of the file, are hitting this bug.
> Pig.compileFromFile in embedded python fails when pig script starts with a comment
> ----------------------------------------------------------------------------------
>
> Key: PIG-2273
> URL: https://issues.apache.org/jira/browse/PIG-2273
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Doug Daniels
> Attachments: PIG-2273.tar.gz
>
>
> When using embedded Pig inside python, Pig.compileFromFile fails when the referenced pig file starts with a comment.
> When you start the referenced pig file with a comment, you get no plan back and hence cannot successfully execute the job. When you start the same script without a comment, you get a plan and can execute the script successfully.
> Strangely, this issue does not appear when using Pig.compile to compile directly from a string.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2273) Pig.compileFromFile in embedded
python fails when pig script starts with a comment
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103761#comment-13103761 ]
Alan Gates commented on PIG-2273:
---------------------------------
Doug, when generating patches from git please use "git diff --no-prefix" This way we can use the patch tool to apply the patches.
Patch looks good. If you can regenerate it, I'll run it through the tests.
> Pig.compileFromFile in embedded python fails when pig script starts with a comment
> ----------------------------------------------------------------------------------
>
> Key: PIG-2273
> URL: https://issues.apache.org/jira/browse/PIG-2273
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Doug Daniels
> Attachments: PIG-2273.patch, PIG-2273.tar.gz
>
>
> When using embedded Pig inside python, Pig.compileFromFile fails when the referenced pig file starts with a comment.
> When you start the referenced pig file with a comment, you get no plan back and hence cannot successfully execute the job. When you start the same script without a comment, you get a plan and can execute the script successfully.
> Strangely, this issue does not appear when using Pig.compile to compile directly from a string.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2273) Pig.compileFromFile in embedded python
fails when pig script starts with a comment
Posted by "Doug Daniels (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Daniels updated PIG-2273:
------------------------------
Attachment: PIG-2273.patch
Newlines were being removed from the pig script when it was parsed with compileFromFile. This caused line-style comments to extend through the whole file, rather than just the line they comment.
Attached a patch with test update.
Also: I can't see why the getScriptFromFile method is using a LineNumberReader and iterating through each line when it just returns the String output of what it gets and doesn't use the line numbers. Might be simpler just to read the file directly.
> Pig.compileFromFile in embedded python fails when pig script starts with a comment
> ----------------------------------------------------------------------------------
>
> Key: PIG-2273
> URL: https://issues.apache.org/jira/browse/PIG-2273
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Doug Daniels
> Attachments: PIG-2273.patch, PIG-2273.tar.gz
>
>
> When using embedded Pig inside python, Pig.compileFromFile fails when the referenced pig file starts with a comment.
> When you start the referenced pig file with a comment, you get no plan back and hence cannot successfully execute the job. When you start the same script without a comment, you get a plan and can execute the script successfully.
> Strangely, this issue does not appear when using Pig.compile to compile directly from a string.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2273) Pig.compileFromFile in embedded python
fails when pig script starts with a comment
Posted by "Doug Daniels (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Daniels updated PIG-2273:
------------------------------
Attachment: PIG-2273.patch
Here's an updated patch with git diff --no-prefix.
> Pig.compileFromFile in embedded python fails when pig script starts with a comment
> ----------------------------------------------------------------------------------
>
> Key: PIG-2273
> URL: https://issues.apache.org/jira/browse/PIG-2273
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.9.0
> Reporter: Doug Daniels
> Attachments: PIG-2273.patch, PIG-2273.tar.gz
>
>
> When using embedded Pig inside python, Pig.compileFromFile fails when the referenced pig file starts with a comment.
> When you start the referenced pig file with a comment, you get no plan back and hence cannot successfully execute the job. When you start the same script without a comment, you get a plan and can execute the script successfully.
> Strangely, this issue does not appear when using Pig.compile to compile directly from a string.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira