You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "John Gordon (JIRA)" <ji...@apache.org> on 2012/07/07 02:25:34 UTC
[jira] [Created] (PIG-2795) Fix test cases that generate pig
scripts with "load " + pathStr to encode "\" in the path
John Gordon created PIG-2795:
--------------------------------
Summary: Fix test cases that generate pig scripts with "load " + pathStr to encode "\" in the path
Key: PIG-2795
URL: https://issues.apache.org/jira/browse/PIG-2795
Project: Pig
Issue Type: Sub-task
Reporter: John Gordon
Pig reads "\" as an escape character in pig scripts -- but Windows paths natively contain \. When we formulate pig queries that should be read through the parser with file paths, we should call encodeEscape first to handle paths generated on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2795) Fix test cases that generate pig
scripts with "load " + pathStr to encode "\" in the path
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477118#comment-13477118 ]
Alan Gates commented on PIG-2795:
---------------------------------
John,
After applying this patch to we expect the tests in this patch to pass on Windows? I see the same fail rate for the tests before and after applying it.
> Fix test cases that generate pig scripts with "load " + pathStr to encode "\" in the path
> -----------------------------------------------------------------------------------------
>
> Key: PIG-2795
> URL: https://issues.apache.org/jira/browse/PIG-2795
> Project: Pig
> Issue Type: Sub-task
> Affects Versions: 0.10.0
> Reporter: John Gordon
> Assignee: John Gordon
> Fix For: 0.11
>
> Attachments: 0006-Test-Changes-For-Path-Separator-on-Windows.patch, trunkEncodeEscape.patch
>
>
> Pig reads "\" as an escape character in pig scripts -- but Windows paths natively contain \. When we formulate pig queries that should be read through the parser with file paths, we should call encodeEscape first to handle paths generated on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2795) Fix test cases that generate pig
scripts with "load " + pathStr to encode "\" in the path
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-2795:
----------------------------
Fix Version/s: (was: 0.10.0)
0.11
Assignee: John Gordon
> Fix test cases that generate pig scripts with "load " + pathStr to encode "\" in the path
> -----------------------------------------------------------------------------------------
>
> Key: PIG-2795
> URL: https://issues.apache.org/jira/browse/PIG-2795
> Project: Pig
> Issue Type: Sub-task
> Affects Versions: 0.10.0
> Reporter: John Gordon
> Assignee: John Gordon
> Fix For: 0.11
>
> Attachments: 0006-Test-Changes-For-Path-Separator-on-Windows.patch, trunkEncodeEscape.patch
>
>
> Pig reads "\" as an escape character in pig scripts -- but Windows paths natively contain \. When we formulate pig queries that should be read through the parser with file paths, we should call encodeEscape first to handle paths generated on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2795) Fix test cases that generate pig
scripts with "load " + pathStr to encode "\" in the path
Posted by "John Gordon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Gordon updated PIG-2795:
-----------------------------
Attachment: 0006-Test-Changes-For-Path-Separator-on-Windows.patch
Updated test changes for windows path separator "\"
> Fix test cases that generate pig scripts with "load " + pathStr to encode "\" in the path
> -----------------------------------------------------------------------------------------
>
> Key: PIG-2795
> URL: https://issues.apache.org/jira/browse/PIG-2795
> Project: Pig
> Issue Type: Sub-task
> Reporter: John Gordon
> Attachments: 0006-Test-Changes-For-Path-Separator-on-Windows.patch, trunkEncodeEscape.patch
>
>
> Pig reads "\" as an escape character in pig scripts -- but Windows paths natively contain \. When we formulate pig queries that should be read through the parser with file paths, we should call encodeEscape first to handle paths generated on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2795) Fix test cases that generate pig
scripts with "load " + pathStr to encode "\" in the path
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-2795:
----------------------------
Resolution: Fixed
Fix Version/s: 0.12
0.10.1
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
+1.
Patch committed to 0.10/0.11/trunk.
Thanks John!
> Fix test cases that generate pig scripts with "load " + pathStr to encode "\" in the path
> -----------------------------------------------------------------------------------------
>
> Key: PIG-2795
> URL: https://issues.apache.org/jira/browse/PIG-2795
> Project: Pig
> Issue Type: Sub-task
> Affects Versions: 0.10.0
> Reporter: John Gordon
> Assignee: John Gordon
> Fix For: 0.11, 0.10.1, 0.12
>
> Attachments: 0006-Test-Changes-For-Path-Separator-on-Windows.patch, trunkEncodeEscape.patch
>
>
> Pig reads "\" as an escape character in pig scripts -- but Windows paths natively contain \. When we formulate pig queries that should be read through the parser with file paths, we should call encodeEscape first to handle paths generated on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2795) Fix test cases that generate pig
scripts with "load " + pathStr to encode "\" in the path
Posted by "John Gordon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13480951#comment-13480951 ]
John Gordon commented on PIG-2795:
----------------------------------
Are you running under cygwin or without cygwin? With Cygwin, you might get cygpaths, which don't have \. Given that pig uses \ as an escape character, a standard Windows path -- e.g. "C:\newdir\text.txt" would look like this C:<newline>ewdir<tab>ext.txt -- where newline and tab are visual placeholders for whitespace characters.
> Fix test cases that generate pig scripts with "load " + pathStr to encode "\" in the path
> -----------------------------------------------------------------------------------------
>
> Key: PIG-2795
> URL: https://issues.apache.org/jira/browse/PIG-2795
> Project: Pig
> Issue Type: Sub-task
> Affects Versions: 0.10.0
> Reporter: John Gordon
> Assignee: John Gordon
> Fix For: 0.11
>
> Attachments: 0006-Test-Changes-For-Path-Separator-on-Windows.patch, trunkEncodeEscape.patch
>
>
> Pig reads "\" as an escape character in pig scripts -- but Windows paths natively contain \. When we formulate pig queries that should be read through the parser with file paths, we should call encodeEscape first to handle paths generated on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2795) Fix test cases that generate pig
scripts with "load " + pathStr to encode "\" in the path
Posted by "John Gordon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Gordon updated PIG-2795:
-----------------------------
Attachment: trunkEncodeEscape.patch
Updates to tests to encode pig escape characters that naturally occur in paths on Windows. Doesn't affect Linux pass rate -- improves Windows pass rate significantly without Cygwin.
> Fix test cases that generate pig scripts with "load " + pathStr to encode "\" in the path
> -----------------------------------------------------------------------------------------
>
> Key: PIG-2795
> URL: https://issues.apache.org/jira/browse/PIG-2795
> Project: Pig
> Issue Type: Sub-task
> Reporter: John Gordon
> Attachments: trunkEncodeEscape.patch
>
>
> Pig reads "\" as an escape character in pig scripts -- but Windows paths natively contain \. When we formulate pig queries that should be read through the parser with file paths, we should call encodeEscape first to handle paths generated on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2795) Fix test cases that generate pig
scripts with "load " + pathStr to encode "\" in the path
Posted by "John Gordon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Gordon updated PIG-2795:
-----------------------------
Fix Version/s: 0.10.0
Affects Version/s: 0.10.0
Status: Patch Available (was: Open)
> Fix test cases that generate pig scripts with "load " + pathStr to encode "\" in the path
> -----------------------------------------------------------------------------------------
>
> Key: PIG-2795
> URL: https://issues.apache.org/jira/browse/PIG-2795
> Project: Pig
> Issue Type: Sub-task
> Affects Versions: 0.10.0
> Reporter: John Gordon
> Fix For: 0.10.0
>
> Attachments: 0006-Test-Changes-For-Path-Separator-on-Windows.patch, trunkEncodeEscape.patch
>
>
> Pig reads "\" as an escape character in pig scripts -- but Windows paths natively contain \. When we formulate pig queries that should be read through the parser with file paths, we should call encodeEscape first to handle paths generated on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2795) Fix test cases that generate pig
scripts with "load " + pathStr to encode "\" in the path
Posted by "John Gordon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13480929#comment-13480929 ]
John Gordon commented on PIG-2795:
----------------------------------
If you run pig against any hadoop build other than branch-1-win, no. If you build against branch-1-win core, so that unit tests are on a Windows-compatible hadoop core, this should improve the pass rates of all of these tests. If you look at your root causes for failures, you should notice most of the failures changing from paths with escape sequences in them to HDFS issues with otherwise valid path strings.
> Fix test cases that generate pig scripts with "load " + pathStr to encode "\" in the path
> -----------------------------------------------------------------------------------------
>
> Key: PIG-2795
> URL: https://issues.apache.org/jira/browse/PIG-2795
> Project: Pig
> Issue Type: Sub-task
> Affects Versions: 0.10.0
> Reporter: John Gordon
> Assignee: John Gordon
> Fix For: 0.11
>
> Attachments: 0006-Test-Changes-For-Path-Separator-on-Windows.patch, trunkEncodeEscape.patch
>
>
> Pig reads "\" as an escape character in pig scripts -- but Windows paths natively contain \. When we formulate pig queries that should be read through the parser with file paths, we should call encodeEscape first to handle paths generated on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira