You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Namit Jain (JIRA)" <ji...@apache.org> on 2009/11/25 06:48:39 UTC
[jira] Created: (HIVE-953) script_broken_pipe3.q broken
script_broken_pipe3.q broken
----------------------------
Key: HIVE-953
URL: https://issues.apache.org/jira/browse/HIVE-953
Project: Hadoop Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.5.0
Reporter: Namit Jain
Assignee: Paul Yang
The negative test script_broken_pipe3.q is broken if we allow partial consumption.
For now, I have disabled partial consumption. Can you take a look ?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-953) script_broken_pipe3.q broken
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain resolved HIVE-953.
-----------------------------
Resolution: Fixed
Fix Version/s: 0.5.0
Hadoop Flags: [Reviewed]
Committed. Thanks Paul
> script_broken_pipe3.q broken
> ----------------------------
>
> Key: HIVE-953
> URL: https://issues.apache.org/jira/browse/HIVE-953
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Reporter: Namit Jain
> Assignee: Paul Yang
> Fix For: 0.5.0
>
> Attachments: HIVE-953.1.patch, HIVE-953.2.patch, script_broken_pipe3_output.txt
>
>
> The negative test script_broken_pipe3.q is broken if we allow partial consumption.
> For now, I have disabled partial consumption. Can you take a look ?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-953) script_broken_pipe3.q broken
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Yang updated HIVE-953:
---------------------------
Attachment: (was: HIVE-953.1.patch)
> script_broken_pipe3.q broken
> ----------------------------
>
> Key: HIVE-953
> URL: https://issues.apache.org/jira/browse/HIVE-953
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Reporter: Namit Jain
> Assignee: Paul Yang
> Attachments: HIVE-953.1.patch, script_broken_pipe3_output.txt
>
>
> The negative test script_broken_pipe3.q is broken if we allow partial consumption.
> For now, I have disabled partial consumption. Can you take a look ?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-953) script_broken_pipe3.q broken
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Yang updated HIVE-953:
---------------------------
Attachment: (was: script_broken_pipe3_output.txt)
> script_broken_pipe3.q broken
> ----------------------------
>
> Key: HIVE-953
> URL: https://issues.apache.org/jira/browse/HIVE-953
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Reporter: Namit Jain
> Assignee: Paul Yang
> Attachments: HIVE-953.1.patch, script_broken_pipe3_output.txt
>
>
> The negative test script_broken_pipe3.q is broken if we allow partial consumption.
> For now, I have disabled partial consumption. Can you take a look ?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-953) script_broken_pipe3.q broken
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Yang updated HIVE-953:
---------------------------
Attachment: HIVE-953.2.patch
Sorry for the confusion - there were changes to the open source repository that weren't propagated to the FB repository.
Anyway, the problem is that there was a change to ScriptOperator that broke the test. The change always reported the exit value of the script as 0, meaning that it was always successful. Since the test was expecting the script to fail with an error code, the negative test broke. Removing 'set hive.exec.script.allow.partial.consumption = true' allowed the negative test to pass as disallowing partial consumption would result in a broken pipe exception and masked the fact that the error code was incorrect.
In the 2nd patch, I fixed this and included the modifications for the other two tests as well.
> script_broken_pipe3.q broken
> ----------------------------
>
> Key: HIVE-953
> URL: https://issues.apache.org/jira/browse/HIVE-953
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Reporter: Namit Jain
> Assignee: Paul Yang
> Attachments: HIVE-953.1.patch, HIVE-953.2.patch, script_broken_pipe3_output.txt
>
>
> The negative test script_broken_pipe3.q is broken if we allow partial consumption.
> For now, I have disabled partial consumption. Can you take a look ?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-953) script_broken_pipe3.q broken
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782674#action_12782674 ]
Paul Yang commented on HIVE-953:
--------------------------------
Could I take a look at your log files? When I say 'failed', I don't necessarily see diffs - I see something like the following:
{code}
[junit] Exception: Client Execution failed with error code = 0
[junit] junit.framework.AssertionFailedError: Client Execution failed with error code = 0
[junit] at junit.framework.Assert.fail(Assert.java:47)
[junit] at org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_script_broken_pipe2(TestNegativeCliDriver.java:51)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at junit.framework.TestCase.runTest(TestCase.java:154)
[junit] at junit.framework.TestCase.runBare(TestCase.java:127)
[junit] at junit.framework.TestResult$1.protect(TestResult.java:106)
[junit] at junit.framework.TestResult.runProtected(TestResult.java:124)
[junit] at junit.framework.TestResult.run(TestResult.java:109)
[junit] at junit.framework.TestCase.run(TestCase.java:118)
[junit] at junit.framework.TestSuite.runTest(TestSuite.java:208)
[junit] at junit.framework.TestSuite.run(TestSuite.java:203)
[junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
[junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
[junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
[junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 12.676 sec
[junit] Test org.apache.hadoop.hive.cli.TestNegativeCliDriver FAILED
{code}
> script_broken_pipe3.q broken
> ----------------------------
>
> Key: HIVE-953
> URL: https://issues.apache.org/jira/browse/HIVE-953
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Reporter: Namit Jain
> Assignee: Paul Yang
>
> The negative test script_broken_pipe3.q is broken if we allow partial consumption.
> For now, I have disabled partial consumption. Can you take a look ?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-953) script_broken_pipe3.q broken
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Yang updated HIVE-953:
---------------------------
Attachment: HIVE-953.1.patch
script_broken_pipe3_output.txt
> script_broken_pipe3.q broken
> ----------------------------
>
> Key: HIVE-953
> URL: https://issues.apache.org/jira/browse/HIVE-953
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Reporter: Namit Jain
> Assignee: Paul Yang
> Attachments: HIVE-953.1.patch, script_broken_pipe3_output.txt
>
>
> The negative test script_broken_pipe3.q is broken if we allow partial consumption.
> For now, I have disabled partial consumption. Can you take a look ?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-953) script_broken_pipe3.q broken
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782627#action_12782627 ]
Namit Jain commented on HIVE-953:
---------------------------------
They are run as part of negative tests - for me 1/2 are broken as expected - I mean, they dont lead to diffs
3 as it stood before I changed it yesterday was leading to diffs
> script_broken_pipe3.q broken
> ----------------------------
>
> Key: HIVE-953
> URL: https://issues.apache.org/jira/browse/HIVE-953
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Reporter: Namit Jain
> Assignee: Paul Yang
>
> The negative test script_broken_pipe3.q is broken if we allow partial consumption.
> For now, I have disabled partial consumption. Can you take a look ?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-953) script_broken_pipe3.q broken
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Yang updated HIVE-953:
---------------------------
Attachment: HIVE-953.1.patch
script_broken_pipe3_output.txt
With respect to pipe3, I still was not able to produce an error. I've attached the output of the test run on my checkout. Can you try running that test again?
In any case, I've fixed the tests for pipe1 and pipe2 to run correctly with partial consumption enabled in the attached patch.
> script_broken_pipe3.q broken
> ----------------------------
>
> Key: HIVE-953
> URL: https://issues.apache.org/jira/browse/HIVE-953
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Reporter: Namit Jain
> Assignee: Paul Yang
> Attachments: HIVE-953.1.patch, script_broken_pipe3_output.txt
>
>
> The negative test script_broken_pipe3.q is broken if we allow partial consumption.
> For now, I have disabled partial consumption. Can you take a look ?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-953) script_broken_pipe3.q broken
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782581#action_12782581 ]
Paul Yang commented on HIVE-953:
--------------------------------
Strange, that test is working fine on my end. Inside script_broken_pipe3.q, partial consumption is enabled with a set command, so it should work regardless of what the setting for partial consumption was beforehand.
However, script_broken_pipe1.q and script_broken_pipe2.q, were written with having partial consumption disabled as per defaults and lack a set command. So those tests should fail if partial consumption is enabled by default - can you check if that is the case?
> script_broken_pipe3.q broken
> ----------------------------
>
> Key: HIVE-953
> URL: https://issues.apache.org/jira/browse/HIVE-953
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Reporter: Namit Jain
> Assignee: Paul Yang
>
> The negative test script_broken_pipe3.q is broken if we allow partial consumption.
> For now, I have disabled partial consumption. Can you take a look ?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-953) script_broken_pipe3.q broken
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783981#action_12783981 ]
Namit Jain commented on HIVE-953:
---------------------------------
+1
will merge if the tests pass
> script_broken_pipe3.q broken
> ----------------------------
>
> Key: HIVE-953
> URL: https://issues.apache.org/jira/browse/HIVE-953
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Reporter: Namit Jain
> Assignee: Paul Yang
> Attachments: HIVE-953.1.patch, HIVE-953.2.patch, script_broken_pipe3_output.txt
>
>
> The negative test script_broken_pipe3.q is broken if we allow partial consumption.
> For now, I have disabled partial consumption. Can you take a look ?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-953) script_broken_pipe3.q broken
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782689#action_12782689 ]
Namit Jain commented on HIVE-953:
---------------------------------
I dont have them anymore - but run with
test.silent=false
ant test ..... -Dtest.silent=false
It will print more info.
> script_broken_pipe3.q broken
> ----------------------------
>
> Key: HIVE-953
> URL: https://issues.apache.org/jira/browse/HIVE-953
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Reporter: Namit Jain
> Assignee: Paul Yang
>
> The negative test script_broken_pipe3.q is broken if we allow partial consumption.
> For now, I have disabled partial consumption. Can you take a look ?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-953) script_broken_pipe3.q broken
Posted by "Paul Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782587#action_12782587 ]
Paul Yang commented on HIVE-953:
--------------------------------
Just to clarify - on my end, script_broken_pipe1.q and script_broken_pipe2.q are failing with partial consumption enabled by default. I wanted to see if that's the case with your setup.
> script_broken_pipe3.q broken
> ----------------------------
>
> Key: HIVE-953
> URL: https://issues.apache.org/jira/browse/HIVE-953
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.5.0
> Reporter: Namit Jain
> Assignee: Paul Yang
>
> The negative test script_broken_pipe3.q is broken if we allow partial consumption.
> For now, I have disabled partial consumption. Can you take a look ?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.