[jira] Updated: (PIG-501) Make branches/types work under cygwin

["Daniel Dai (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Dai updated PIG-501:
---------------------------

    Attachment: javacc.jar
                PIG_cygwin.Patch

We've already make all unit tests pass under cygwin for trunk (See [PIG-243|https://issues.apache.org/jira/browse/PIG-243]). We need to do the same for branches/types. Here is the problem I noticed and fixed for branches/types under cygwin.

1. We can not compile under cygwin. javacc fails in branches/types, error message "Invalid Escape character". We need to update javacc to the latest snapshot. Javacc 4.1 (latest) release do not solve the problem. This issue is described in https://javacc.dev.java.net/issues/show_bug.cgi?id=135. We do not have it before because only in branches/types we will let javacc generate output to directory "xxx\util", which contains "\u", triggering the problem. Currently I do not notice any problem with this latest javacc snapshot. One difference I notice is it sometimes generates a slightly different message upon error, and to me this message become more informative.


2. The tailing '\r' in input txt file cause some problem as before. I modify PigStorage.java and TextLoader.java to deal with it. However in branches/types, PigStorage is no longer line based and this make the situation much harder. In order to deal with it, we need to be able to read ahead, if we see a '\n' ahead, we can discard current '\r'. But in reality we do not look ahead and changes will be relatively big. So here I simply remove the tailing '\r' for each field under cygwin. There is a semantic diference but I think the chance that tailing '\r' carrying meaningful information is low. Correct me if I am wrong.

3. Most testing errors in cygwin is caused by wrong path name just like before. cygwin only takes the input/output file name in windows form, such as c:\ \ xxxx, and if we need to use a complete scheme, use "file:/c:\ \ xxxx", notice there is an extra "/" between "file:" and file path, which do not exists in unix. Many test cases do not notice that and fail in cygwin. Also we need to use "\ \" not "\" as seperator as described in [PIG-243|https://issues.apache.org/jira/browse/PIG-243].



> Make branches/types work under cygwin
> -------------------------------------
>
>                 Key: PIG-501
>                 URL: https://issues.apache.org/jira/browse/PIG-501
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: types_branch
>         Environment: cygwin
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>             Fix For: types_branch
>
>         Attachments: javacc.jar, PIG_cygwin.Patch
>
>
> We've already make all unit tests pass under cygwin for trunk (See [PIG-243|https://issues.apache.org/jira/browse/PIG-243]). We need to do the same for branches/types. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.