You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Zhenxiao Luo (JIRA)" <ji...@apache.org> on 2012/07/26 02:45:33 UTC

[jira] [Created] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Zhenxiao Luo created HIVE-3301:
----------------------------------

             Summary: Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
                 Key: HIVE-3301
                 URL: https://issues.apache.org/jira/browse/HIVE-3301
             Project: Hive
          Issue Type: Bug
            Reporter: Zhenxiao Luo
            Assignee: Zhenxiao Luo


When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:

quote is printed as: "&quot;", instead of "

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhenxiao Luo updated HIVE-3301:
-------------------------------

    Attachment: HIVE-3301.2.patch.txt
    
> Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3301
>                 URL: https://issues.apache.org/jira/browse/HIVE-3301
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Zhenxiao Luo
>         Attachments: HIVE-3301.1.patch.txt, HIVE-3301.2.patch.txt
>
>
> When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:
> quote is printed as: '&quot;', instead of "
> Seems not able to state the bug clearly in html:
> quote is printed as 'address sign' + 'quot' + semicolon
> not the expected 'quote sign'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422795#comment-13422795 ] 

Zhenxiao Luo commented on HIVE-3301:
------------------------------------

@Edward: oh yes. As I know, HIVE-3301, HIVE-3275, HIVE-3273, HIVE-3242, HIVE-3240, HIVE-3257, HIVE-3249 and HIVE-2804 are all hadoop 23 bugs. I am fixing these one by one. Thanks for your advice. I will try to put them into a larger shim layer.

I just found an Error Code retrieval inconsistency between hadoop20 and hadoop23. Will file another one soon.

Thanks,
Zhenxiao
                
> Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3301
>                 URL: https://issues.apache.org/jira/browse/HIVE-3301
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Zhenxiao Luo
>         Attachments: HIVE-3301.1.patch.txt
>
>
> When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:
> quote is printed as: '&quot;', instead of "
> Seems not able to state the bug clearly in html:
> quote is printed as 'address sign' + 'quot' + semicolon
> not the expected 'quote sign'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ashutosh Chauhan updated HIVE-3301:
-----------------------------------

       Resolution: Fixed
    Fix Version/s: 0.10.0
           Status: Resolved  (was: Patch Available)

Committed to trunk. Thanks, Zhenxiao!
                
> Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3301
>                 URL: https://issues.apache.org/jira/browse/HIVE-3301
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Zhenxiao Luo
>             Fix For: 0.10.0
>
>         Attachments: HIVE-3301.1.patch.txt, HIVE-3301.2.patch.txt, HIVE-3301.3.patch.txt
>
>
> When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:
> quote is printed as: '&quot;', instead of "
> Seems not able to state the bug clearly in html:
> quote is printed as 'address sign' + 'quot' + semicolon
> not the expected 'quote sign'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhenxiao Luo updated HIVE-3301:
-------------------------------

    Status: Patch Available  (was: Open)
    
> Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3301
>                 URL: https://issues.apache.org/jira/browse/HIVE-3301
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Zhenxiao Luo
>         Attachments: HIVE-3301.1.patch.txt
>
>
> When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:
> quote is printed as: '&quot;', instead of "
> Seems not able to state the bug clearly in html:
> quote is printed as 'address sign' + 'quot' + semicolon
> not the expected 'quote sign'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422800#comment-13422800 ] 

Edward Capriolo commented on HIVE-3301:
---------------------------------------

I am not saying we need to shim layer all the fixes, but having a reasonably exhaustive list of the problems linked together in jira would make me more confident that we are taking the right plan of action. 
                
> Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3301
>                 URL: https://issues.apache.org/jira/browse/HIVE-3301
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Zhenxiao Luo
>         Attachments: HIVE-3301.1.patch.txt
>
>
> When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:
> quote is printed as: '&quot;', instead of "
> Seems not able to state the bug clearly in html:
> quote is printed as 'address sign' + 'quot' + semicolon
> not the expected 'quote sign'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhenxiao Luo updated HIVE-3301:
-------------------------------

    Status: Open  (was: Patch Available)

patch has prefix in it
                
> Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3301
>                 URL: https://issues.apache.org/jira/browse/HIVE-3301
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Zhenxiao Luo
>         Attachments: HIVE-3301.1.patch.txt, HIVE-3301.2.patch.txt, HIVE-3301.3.patch.txt
>
>
> When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:
> quote is printed as: '&quot;', instead of "
> Seems not able to state the bug clearly in html:
> quote is printed as 'address sign' + 'quot' + semicolon
> not the expected 'quote sign'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422832#comment-13422832 ] 

Zhenxiao Luo commented on HIVE-3301:
------------------------------------

@ashutosh: Thanks a lot for the comments.

I made updates and resubmitted review request at:
https://reviews.facebook.net/D4353
                
> Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3301
>                 URL: https://issues.apache.org/jira/browse/HIVE-3301
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Zhenxiao Luo
>         Attachments: HIVE-3301.1.patch.txt, HIVE-3301.2.patch.txt
>
>
> When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:
> quote is printed as: '&quot;', instead of "
> Seems not able to state the bug clearly in html:
> quote is printed as 'address sign' + 'quot' + semicolon
> not the expected 'quote sign'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422789#comment-13422789 ] 

Edward Capriolo commented on HIVE-3301:
---------------------------------------

You know these hadoop 23 jiras are like death of a thousand paper cuts, if I had known we were going to face so many issues i would have proposed making a larger shim layer. Can we come up with a definitive list of all the 23 problems?
                
> Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3301
>                 URL: https://issues.apache.org/jira/browse/HIVE-3301
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Zhenxiao Luo
>
> When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:
> quote is printed as: '&quot;', instead of "
> Seems not able to state the bug clearly in html:
> quote is printed as 'address sign' + 'quot' + semicolon
> not the expected 'quote sign'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422831#comment-13422831 ] 

Zhenxiao Luo commented on HIVE-3301:
------------------------------------

@Edward: related HIVE tickets are linked. I will add more whenever any new bugs filed. Do we need a separate upper level JIRA to trace all the hadoop23 integration bugs?
                
> Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3301
>                 URL: https://issues.apache.org/jira/browse/HIVE-3301
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Zhenxiao Luo
>         Attachments: HIVE-3301.1.patch.txt, HIVE-3301.2.patch.txt
>
>
> When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:
> quote is printed as: '&quot;', instead of "
> Seems not able to state the bug clearly in html:
> quote is printed as 'address sign' + 'quot' + semicolon
> not the expected 'quote sign'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhenxiao Luo updated HIVE-3301:
-------------------------------

    Description: 
When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:

quote is printed as: '&quot;', instead of "

  was:
When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:

quote is printed as: "&quot;", instead of "

    
> Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3301
>                 URL: https://issues.apache.org/jira/browse/HIVE-3301
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Zhenxiao Luo
>
> When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:
> quote is printed as: '&quot;', instead of "

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423778#comment-13423778 ] 

Hudson commented on HIVE-3301:
------------------------------

Integrated in Hive-trunk-h0.21 #1570 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1570/])
    HIVE-3301 : Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23 (Zhenxiao Luo via Ashutosh Chauhan) (Revision 1366233)

     Result = SUCCESS
hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1366233
Files : 
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/errors/TaskLogProcessor.java
* /hive/trunk/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java
* /hive/trunk/shims/src/common-secure/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java
* /hive/trunk/shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java

                
> Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3301
>                 URL: https://issues.apache.org/jira/browse/HIVE-3301
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Zhenxiao Luo
>             Fix For: 0.10.0
>
>         Attachments: HIVE-3301.1.patch.txt, HIVE-3301.2.patch.txt, HIVE-3301.3.patch.txt
>
>
> When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:
> quote is printed as: '&quot;', instead of "
> Seems not able to state the bug clearly in html:
> quote is printed as 'address sign' + 'quot' + semicolon
> not the expected 'quote sign'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhenxiao Luo updated HIVE-3301:
-------------------------------

    Status: Patch Available  (was: Open)
    
> Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3301
>                 URL: https://issues.apache.org/jira/browse/HIVE-3301
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Zhenxiao Luo
>         Attachments: HIVE-3301.1.patch.txt, HIVE-3301.2.patch.txt, HIVE-3301.3.patch.txt
>
>
> When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:
> quote is printed as: '&quot;', instead of "
> Seems not able to state the bug clearly in html:
> quote is printed as 'address sign' + 'quot' + semicolon
> not the expected 'quote sign'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhenxiao Luo updated HIVE-3301:
-------------------------------

    Attachment: HIVE-3301.1.patch.txt
    
> Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3301
>                 URL: https://issues.apache.org/jira/browse/HIVE-3301
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Zhenxiao Luo
>         Attachments: HIVE-3301.1.patch.txt
>
>
> When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:
> quote is printed as: '&quot;', instead of "
> Seems not able to state the bug clearly in html:
> quote is printed as 'address sign' + 'quot' + semicolon
> not the expected 'quote sign'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422787#comment-13422787 ] 

Zhenxiao Luo commented on HIVE-3301:
------------------------------------

The problem is:

In hadoop23, TaskLogServlet.java is using a new utility HtmlQuoting.java to print Task Log.

In TaskLogServlet.java, printTaskLog() function:

result = taskLogReader.read(b);
        if (result > 0) {
          if (plainText) {
            out.write(b, 0, result);
          } else {
            HtmlQuoting.quoteHtmlChars(out, b, 0, result);
          }
        } else {
          break;
        }


While, in hadoop20,  TaskLogServlet.java is using its own utility(there is no such HtmlQuoting.java at all) to print Task Log:

In TaskLogServlet.java, printTaskLog fucntion:

result = taskLogReader.read(b);
        if (result > 0) {
          if (plainText) {
            out.write(b, 0, result);
          } else {
            quotedWrite(out, b, 0, result);
          }
        } else {
          break;
        }


And in Hive, TaskLogProcessor.java is generating stack trace by reading the raw taskAttemptLog.

In ql/src/java/org/apache/hadoop/hive/ql/exec/errors/TaskLogProcessor.java, getStackTraces() fuction:


List<String> stackTrace = null;

        // Patterns that match the middle/end of stack traces
        Pattern stackTracePattern = Pattern.compile("^\tat .*", Pattern.CASE_INSENSITIVE);
        Pattern endStackTracePattern =
            Pattern.compile("^\t... [0-9]+ more.*", Pattern.CASE_INSENSITIVE);

        while ((inputLine = in.readLine()) != null) {

          if (stackTracePattern.matcher(inputLine).matches() ||
              endStackTracePattern.matcher(inputLine).matches()) {


To have Hive working for both hadoop20 and hadoop23, we should use different mechanisms when hive TaskLogProcessor is parsing TaskAttemptLog.

My plan is creating a shim, which have different implementations for hadoop20 and hadoop23.

In hadoop23, HtmlQuoting.unquoteHtmlChars() is used to parse the TaskAttemptLog.
                
> Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3301
>                 URL: https://issues.apache.org/jira/browse/HIVE-3301
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Zhenxiao Luo
>
> When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:
> quote is printed as: '&quot;', instead of "
> Seems not able to state the bug clearly in html:
> quote is printed as 'address sign' + 'quot' + semicolon
> not the expected 'quote sign'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhenxiao Luo updated HIVE-3301:
-------------------------------

    Description: 
When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:

quote is printed as: '&quot;', instead of "

Seems not able to state the bug clearly in html:

quote is printed as 'address sign' + 'quot' + semicolon
not the expected 'quote sign'

  was:
When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:

quote is printed as: '&quot;', instead of "

    
> Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3301
>                 URL: https://issues.apache.org/jira/browse/HIVE-3301
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Zhenxiao Luo
>
> When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:
> quote is printed as: '&quot;', instead of "
> Seems not able to state the bug clearly in html:
> quote is printed as 'address sign' + 'quot' + semicolon
> not the expected 'quote sign'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423167#comment-13423167 ] 

Ashutosh Chauhan commented on HIVE-3301:
----------------------------------------

+1 Running tests.
                
> Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3301
>                 URL: https://issues.apache.org/jira/browse/HIVE-3301
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Zhenxiao Luo
>         Attachments: HIVE-3301.1.patch.txt, HIVE-3301.2.patch.txt, HIVE-3301.3.patch.txt
>
>
> When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:
> quote is printed as: '&quot;', instead of "
> Seems not able to state the bug clearly in html:
> quote is printed as 'address sign' + 'quot' + semicolon
> not the expected 'quote sign'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ashutosh Chauhan updated HIVE-3301:
-----------------------------------

    Status: Open  (was: Patch Available)

Zhenxiao I left comments on Phabricator.
                
> Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3301
>                 URL: https://issues.apache.org/jira/browse/HIVE-3301
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Zhenxiao Luo
>         Attachments: HIVE-3301.1.patch.txt
>
>
> When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:
> quote is printed as: '&quot;', instead of "
> Seems not able to state the bug clearly in html:
> quote is printed as 'address sign' + 'quot' + semicolon
> not the expected 'quote sign'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhenxiao Luo updated HIVE-3301:
-------------------------------

    Attachment: HIVE-3301.3.patch.txt

updated patch without prefix
could apply cleanly
                
> Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3301
>                 URL: https://issues.apache.org/jira/browse/HIVE-3301
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Zhenxiao Luo
>         Attachments: HIVE-3301.1.patch.txt, HIVE-3301.2.patch.txt, HIVE-3301.3.patch.txt
>
>
> When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:
> quote is printed as: '&quot;', instead of "
> Seems not able to state the bug clearly in html:
> quote is printed as 'address sign' + 'quot' + semicolon
> not the expected 'quote sign'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/HIVE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhenxiao Luo updated HIVE-3301:
-------------------------------

    Status: Patch Available  (was: Open)
    
> Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3301
>                 URL: https://issues.apache.org/jira/browse/HIVE-3301
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Zhenxiao Luo
>         Attachments: HIVE-3301.1.patch.txt, HIVE-3301.2.patch.txt
>
>
> When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:
> quote is printed as: '&quot;', instead of "
> Seems not able to state the bug clearly in html:
> quote is printed as 'address sign' + 'quot' + semicolon
> not the expected 'quote sign'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (HIVE-3301) Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23

Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/HIVE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422790#comment-13422790 ] 

Zhenxiao Luo commented on HIVE-3301:
------------------------------------

review request submitted at:
https://reviews.facebook.net/D4353
                
> Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
> ------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3301
>                 URL: https://issues.apache.org/jira/browse/HIVE-3301
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Zhenxiao Luo
>
> When running hive on hadoop0.23, mapreduce_stack_trace.q is failing due to quote printing bug:
> quote is printed as: '&quot;', instead of "
> Seems not able to state the bug clearly in html:
> quote is printed as 'address sign' + 'quot' + semicolon
> not the expected 'quote sign'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira