You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Zhenxiao Luo (Created) (JIRA)" <ji...@apache.org> on 2012/02/08 01:27:03 UTC
[jira] [Created] (HIVE-2782) New BINARY type produces unexpected
results with supported UDFS when using MapReduce2
New BINARY type produces unexpected results with supported UDFS when using MapReduce2
-------------------------------------------------------------------------------------
Key: HIVE-2782
URL: https://issues.apache.org/jira/browse/HIVE-2782
Project: Hive
Issue Type: Bug
Reporter: Zhenxiao Luo
Assignee: Zhenxiao Luo
When using MapReduce2 for Hive
ba_table_udfs is failing with unexpected output:
[junit] Begin query: ba_table_udfs.q
[junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
[junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
[junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/ba_table_udfs.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/ba_table_udfs.q.out
[junit] 20,26c20,26
[junit] < 2 10val_10 1
[junit] < 3 164val_164 1
[junit] < 3 150val_150 1
[junit] < 2 18val_18 1
[junit] < 3 177val_177 1
[junit] < 2 12val_12 1
[junit] < 2 11val_11 1
[junit] —
[junit] > 3 120val_120 1
[junit] > 3 192val_192 1
[junit] > 3 119val_119 1
[junit] > 3 187val_187 1
[junit] > 3 176val_176 1
[junit] > 3 199val_199 1
[junit] > 3 118val_118 1
[junit] Exception: Client execution results failed with error code = 1
[junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
[junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1
[junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
[junit] at junit.framework.Assert.fail(Assert.java:50)
[junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs(TestCliDriver.java:129)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit] at java.lang.reflect.Method.invoke(Method.java:616)
[junit] at junit.framework.TestCase.runTest(TestCase.java:168)
[junit] at junit.framework.TestCase.runBare(TestCase.java:134)
[junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
[junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
[junit] at junit.framework.TestResult.run(TestResult.java:113)
[junit] at junit.framework.TestCase.run(TestCase.java:124)
[junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
[junit] at junit.framework.TestSuite.run(TestSuite.java:238)
[junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
[junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
[junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
[junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.)
[junit] Cleaning up TestCliDriver
[junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 10.751 sec
[junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
[for] /home/cloudera/Code/hive/ql/build.xml: The following error occurred while executing this line:
[for] /home/cloudera/Code/hive/build.xml:328: The following error occurred while executing this line:
[for] /home/cloudera/Code/hive/build-common.xml:453: Tests failed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2782) New BINARY type produces unexpected
results with supported UDFS when using MapReduce2
Posted by "Carl Steinbach (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203886#comment-13203886 ]
Carl Steinbach commented on HIVE-2782:
--------------------------------------
@Ashutosh: here's a slightly reformatted version of the original query from ba_table_udfs.q:
{noformat}
SELECT
LENGTH(CAST(src.key AS BINARY)) AS len,
CONCAT(CAST(src.key AS BINARY), CAST(src.value AS BINARY)),
SUBSTR(src.value, 5, 1) AS sub
FROM src
ORDER BY sub
LIMIT 10;
{noformat}
I agree that the orignal query has an order by clause, but this ordering is being applied to the fifth character of the src.value field, which takes on the values 1-9. This is not a strong enough condition to guarantee deterministic ordering of the result set. In the modified version of this query that appears in the patch I have instead applied the ordering to the src.value field. I also cast the SUBSTR input as BINARY, which I assume was an oversight in the original patch.
In this patch I have replaced your original query with this new one:
> New BINARY type produces unexpected results with supported UDFS when using MapReduce2
> -------------------------------------------------------------------------------------
>
> Key: HIVE-2782
> URL: https://issues.apache.org/jira/browse/HIVE-2782
> Project: Hive
> Issue Type: Bug
> Reporter: Zhenxiao Luo
> Assignee: Carl Steinbach
> Attachments: HIVE-2782.D1653.1.patch, HIVE-2782.D1653.1.patch
>
>
> When using MapReduce2 for Hive
> ba_table_udfs is failing with unexpected output:
> [junit] Begin query: ba_table_udfs.q
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
> [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/ba_table_udfs.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/ba_table_udfs.q.out
> [junit] 20,26c20,26
> [junit] < 2 10val_10 1
> [junit] < 3 164val_164 1
> [junit] < 3 150val_150 1
> [junit] < 2 18val_18 1
> [junit] < 3 177val_177 1
> [junit] < 2 12val_12 1
> [junit] < 2 11val_11 1
> [junit] —
> [junit] > 3 120val_120 1
> [junit] > 3 192val_192 1
> [junit] > 3 119val_119 1
> [junit] > 3 187val_187 1
> [junit] > 3 176val_176 1
> [junit] > 3 199val_199 1
> [junit] > 3 118val_118 1
> [junit] Exception: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] at junit.framework.Assert.fail(Assert.java:50)
> [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs(TestCliDriver.java:129)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at junit.framework.TestCase.runTest(TestCase.java:168)
> [junit] at junit.framework.TestCase.runBare(TestCase.java:134)
> [junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
> [junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
> [junit] at junit.framework.TestResult.run(TestResult.java:113)
> [junit] at junit.framework.TestCase.run(TestCase.java:124)
> [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
> [junit] at junit.framework.TestSuite.run(TestSuite.java:238)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.)
> [junit] Cleaning up TestCliDriver
> [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 10.751 sec
> [junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
> [for] /home/cloudera/Code/hive/ql/build.xml: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build.xml:328: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build-common.xml:453: Tests failed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2782) New BINARY type produces unexpected
results with supported UDFS when using MapReduce2
Posted by "Ashutosh Chauhan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203844#comment-13203844 ]
Ashutosh Chauhan commented on HIVE-2782:
----------------------------------------
You modified the original query in non-trivial way. I am not sure if its an ordering issue. Original query had the order-by and its output also was indeed ordered. Further, if you look at the output diff posted by Zhenxiao, its clear that rows are not out of order, but are rather different. This requires further analysis as to why this query is giving wrong results with MR2.
> New BINARY type produces unexpected results with supported UDFS when using MapReduce2
> -------------------------------------------------------------------------------------
>
> Key: HIVE-2782
> URL: https://issues.apache.org/jira/browse/HIVE-2782
> Project: Hive
> Issue Type: Bug
> Reporter: Zhenxiao Luo
> Assignee: Carl Steinbach
> Attachments: HIVE-2782.D1653.1.patch, HIVE-2782.D1653.1.patch
>
>
> When using MapReduce2 for Hive
> ba_table_udfs is failing with unexpected output:
> [junit] Begin query: ba_table_udfs.q
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
> [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/ba_table_udfs.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/ba_table_udfs.q.out
> [junit] 20,26c20,26
> [junit] < 2 10val_10 1
> [junit] < 3 164val_164 1
> [junit] < 3 150val_150 1
> [junit] < 2 18val_18 1
> [junit] < 3 177val_177 1
> [junit] < 2 12val_12 1
> [junit] < 2 11val_11 1
> [junit] —
> [junit] > 3 120val_120 1
> [junit] > 3 192val_192 1
> [junit] > 3 119val_119 1
> [junit] > 3 187val_187 1
> [junit] > 3 176val_176 1
> [junit] > 3 199val_199 1
> [junit] > 3 118val_118 1
> [junit] Exception: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] at junit.framework.Assert.fail(Assert.java:50)
> [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs(TestCliDriver.java:129)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at junit.framework.TestCase.runTest(TestCase.java:168)
> [junit] at junit.framework.TestCase.runBare(TestCase.java:134)
> [junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
> [junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
> [junit] at junit.framework.TestResult.run(TestResult.java:113)
> [junit] at junit.framework.TestCase.run(TestCase.java:124)
> [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
> [junit] at junit.framework.TestSuite.run(TestSuite.java:238)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.)
> [junit] Cleaning up TestCliDriver
> [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 10.751 sec
> [junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
> [for] /home/cloudera/Code/hive/ql/build.xml: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build.xml:328: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build-common.xml:453: Tests failed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2782) New BINARY type produces unexpected
results with supported UDFS when using MapReduce2
Posted by "Carl Steinbach (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2782:
---------------------------------
Attachment: HIVE-2782.D1653.1.patch
> New BINARY type produces unexpected results with supported UDFS when using MapReduce2
> -------------------------------------------------------------------------------------
>
> Key: HIVE-2782
> URL: https://issues.apache.org/jira/browse/HIVE-2782
> Project: Hive
> Issue Type: Bug
> Reporter: Zhenxiao Luo
> Assignee: Zhenxiao Luo
> Attachments: HIVE-2782.D1653.1.patch, HIVE-2782.D1653.1.patch
>
>
> When using MapReduce2 for Hive
> ba_table_udfs is failing with unexpected output:
> [junit] Begin query: ba_table_udfs.q
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
> [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/ba_table_udfs.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/ba_table_udfs.q.out
> [junit] 20,26c20,26
> [junit] < 2 10val_10 1
> [junit] < 3 164val_164 1
> [junit] < 3 150val_150 1
> [junit] < 2 18val_18 1
> [junit] < 3 177val_177 1
> [junit] < 2 12val_12 1
> [junit] < 2 11val_11 1
> [junit] —
> [junit] > 3 120val_120 1
> [junit] > 3 192val_192 1
> [junit] > 3 119val_119 1
> [junit] > 3 187val_187 1
> [junit] > 3 176val_176 1
> [junit] > 3 199val_199 1
> [junit] > 3 118val_118 1
> [junit] Exception: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] at junit.framework.Assert.fail(Assert.java:50)
> [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs(TestCliDriver.java:129)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at junit.framework.TestCase.runTest(TestCase.java:168)
> [junit] at junit.framework.TestCase.runBare(TestCase.java:134)
> [junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
> [junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
> [junit] at junit.framework.TestResult.run(TestResult.java:113)
> [junit] at junit.framework.TestCase.run(TestCase.java:124)
> [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
> [junit] at junit.framework.TestSuite.run(TestSuite.java:238)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.)
> [junit] Cleaning up TestCliDriver
> [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 10.751 sec
> [junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
> [for] /home/cloudera/Code/hive/ql/build.xml: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build.xml:328: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build-common.xml:453: Tests failed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2782) New BINARY type produces unexpected
results with supported UDFS when using MapReduce2
Posted by "Carl Steinbach (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204092#comment-13204092 ]
Carl Steinbach commented on HIVE-2782:
--------------------------------------
@Zhenxiao: Can you please explain how this is an improvement on the version I provided in my patch?
> New BINARY type produces unexpected results with supported UDFS when using MapReduce2
> -------------------------------------------------------------------------------------
>
> Key: HIVE-2782
> URL: https://issues.apache.org/jira/browse/HIVE-2782
> Project: Hive
> Issue Type: Bug
> Reporter: Zhenxiao Luo
> Assignee: Carl Steinbach
> Attachments: HIVE-2782.D1653.1.patch, HIVE-2782.D1653.1.patch
>
>
> When using MapReduce2 for Hive
> ba_table_udfs is failing with unexpected output:
> [junit] Begin query: ba_table_udfs.q
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
> [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/ba_table_udfs.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/ba_table_udfs.q.out
> [junit] 20,26c20,26
> [junit] < 2 10val_10 1
> [junit] < 3 164val_164 1
> [junit] < 3 150val_150 1
> [junit] < 2 18val_18 1
> [junit] < 3 177val_177 1
> [junit] < 2 12val_12 1
> [junit] < 2 11val_11 1
> [junit] —
> [junit] > 3 120val_120 1
> [junit] > 3 192val_192 1
> [junit] > 3 119val_119 1
> [junit] > 3 187val_187 1
> [junit] > 3 176val_176 1
> [junit] > 3 199val_199 1
> [junit] > 3 118val_118 1
> [junit] Exception: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] at junit.framework.Assert.fail(Assert.java:50)
> [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs(TestCliDriver.java:129)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at junit.framework.TestCase.runTest(TestCase.java:168)
> [junit] at junit.framework.TestCase.runBare(TestCase.java:134)
> [junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
> [junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
> [junit] at junit.framework.TestResult.run(TestResult.java:113)
> [junit] at junit.framework.TestCase.run(TestCase.java:124)
> [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
> [junit] at junit.framework.TestSuite.run(TestSuite.java:238)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.)
> [junit] Cleaning up TestCliDriver
> [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 10.751 sec
> [junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
> [for] /home/cloudera/Code/hive/ql/build.xml: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build.xml:328: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build-common.xml:453: Tests failed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2782) New BINARY type produces unexpected
results with supported UDFS when using MapReduce2
Posted by "Zhenxiao Luo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204081#comment-13204081 ]
Zhenxiao Luo commented on HIVE-2782:
------------------------------------
I think Carl is correct. Order by the fifth character of the src.value field does not guarantee deterministic ordering.
How about:
SELECT
key,
value,
LENGTH(CAST(src.key AS BINARY)) as key_length,
LENGTH(CAST(src.value AS BINARY)) as value_length,
CONCAT(CAST(src.key AS BINARY), CAST(src.value AS BINARY)) as key_value_concat,
SUBSTR(CAST(src.value AS BINARY), 1, 4) as value_substr
FROM src
ORDER BY key ASC, value ASC, key_length ASC, value_length ASC, key_value_concat ASC, value_substr ASC
LIMIT 100;
This should guarantee the ordering.
> New BINARY type produces unexpected results with supported UDFS when using MapReduce2
> -------------------------------------------------------------------------------------
>
> Key: HIVE-2782
> URL: https://issues.apache.org/jira/browse/HIVE-2782
> Project: Hive
> Issue Type: Bug
> Reporter: Zhenxiao Luo
> Assignee: Carl Steinbach
> Attachments: HIVE-2782.D1653.1.patch, HIVE-2782.D1653.1.patch
>
>
> When using MapReduce2 for Hive
> ba_table_udfs is failing with unexpected output:
> [junit] Begin query: ba_table_udfs.q
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
> [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/ba_table_udfs.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/ba_table_udfs.q.out
> [junit] 20,26c20,26
> [junit] < 2 10val_10 1
> [junit] < 3 164val_164 1
> [junit] < 3 150val_150 1
> [junit] < 2 18val_18 1
> [junit] < 3 177val_177 1
> [junit] < 2 12val_12 1
> [junit] < 2 11val_11 1
> [junit] —
> [junit] > 3 120val_120 1
> [junit] > 3 192val_192 1
> [junit] > 3 119val_119 1
> [junit] > 3 187val_187 1
> [junit] > 3 176val_176 1
> [junit] > 3 199val_199 1
> [junit] > 3 118val_118 1
> [junit] Exception: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] at junit.framework.Assert.fail(Assert.java:50)
> [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs(TestCliDriver.java:129)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at junit.framework.TestCase.runTest(TestCase.java:168)
> [junit] at junit.framework.TestCase.runBare(TestCase.java:134)
> [junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
> [junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
> [junit] at junit.framework.TestResult.run(TestResult.java:113)
> [junit] at junit.framework.TestCase.run(TestCase.java:124)
> [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
> [junit] at junit.framework.TestSuite.run(TestSuite.java:238)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.)
> [junit] Cleaning up TestCliDriver
> [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 10.751 sec
> [junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
> [for] /home/cloudera/Code/hive/ql/build.xml: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build.xml:328: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build-common.xml:453: Tests failed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2782) New BINARY type produces unexpected
results with supported UDFS when using MapReduce2
Posted by "Zhenxiao Luo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204109#comment-13204109 ]
Zhenxiao Luo commented on HIVE-2782:
------------------------------------
@carl: No problem with the original patch.
@Ashutosh:
My understanding is,
The original query:
select
length (cast(src.key as binary)) as len,
concat(cast(src.key as binary), cast(src.value as binary)),
substr(src.value, 5,1) as sub
from src
order by sub
limit 10;
Has the order by on sub, which is the fifth character of src.value.
In the result set, we could see sub is ordered(only showing sub == 1)
While, for the len and concat with the same sub, their orders are not deterministic.
Eg.
2 10val_10 1
3 120val_120 1
both has sub == 1. but the order of these two tuples is not deterministic.
since we only show the top 10 results(limit 10), and there seems are more than 7 tuples with sub == 1, so what shows in the result set is non-deterministic.
Carl solves the problem by providing order into the query.
Correct me if I am wrong.
Thanks
> New BINARY type produces unexpected results with supported UDFS when using MapReduce2
> -------------------------------------------------------------------------------------
>
> Key: HIVE-2782
> URL: https://issues.apache.org/jira/browse/HIVE-2782
> Project: Hive
> Issue Type: Bug
> Reporter: Zhenxiao Luo
> Assignee: Carl Steinbach
> Attachments: HIVE-2782.D1653.1.patch, HIVE-2782.D1653.1.patch
>
>
> When using MapReduce2 for Hive
> ba_table_udfs is failing with unexpected output:
> [junit] Begin query: ba_table_udfs.q
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
> [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/ba_table_udfs.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/ba_table_udfs.q.out
> [junit] 20,26c20,26
> [junit] < 2 10val_10 1
> [junit] < 3 164val_164 1
> [junit] < 3 150val_150 1
> [junit] < 2 18val_18 1
> [junit] < 3 177val_177 1
> [junit] < 2 12val_12 1
> [junit] < 2 11val_11 1
> [junit] —
> [junit] > 3 120val_120 1
> [junit] > 3 192val_192 1
> [junit] > 3 119val_119 1
> [junit] > 3 187val_187 1
> [junit] > 3 176val_176 1
> [junit] > 3 199val_199 1
> [junit] > 3 118val_118 1
> [junit] Exception: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] at junit.framework.Assert.fail(Assert.java:50)
> [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs(TestCliDriver.java:129)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at junit.framework.TestCase.runTest(TestCase.java:168)
> [junit] at junit.framework.TestCase.runBare(TestCase.java:134)
> [junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
> [junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
> [junit] at junit.framework.TestResult.run(TestResult.java:113)
> [junit] at junit.framework.TestCase.run(TestCase.java:124)
> [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
> [junit] at junit.framework.TestSuite.run(TestSuite.java:238)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.)
> [junit] Cleaning up TestCliDriver
> [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 10.751 sec
> [junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
> [for] /home/cloudera/Code/hive/ql/build.xml: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build.xml:328: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build-common.xml:453: Tests failed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2782) New BINARY type produces unexpected
results with supported UDFS when using MapReduce2
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208493#comment-13208493 ]
Hudson commented on HIVE-2782:
------------------------------
Integrated in Hive-trunk-h0.21 #1259 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1259/])
HIVE-2782 [jira] New BINARY type produces unexpected results with supported UDFS
when using MapReduce2
Summary:
HIVE-2782. Make ba_table_udfs.q deterministic
Test Plan: EMPTY
Reviewers: JIRA, ashutoshc
Reviewed By: ashutoshc
CC: ashutoshc
Differential Revision: https://reviews.facebook.net/D1653 (Revision 1244314)
Result = FAILURE
cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1244314
Files :
* /hive/trunk/ql/src/test/queries/clientpositive/ba_table_udfs.q
* /hive/trunk/ql/src/test/results/clientpositive/ba_table_udfs.q.out
> New BINARY type produces unexpected results with supported UDFS when using MapReduce2
> -------------------------------------------------------------------------------------
>
> Key: HIVE-2782
> URL: https://issues.apache.org/jira/browse/HIVE-2782
> Project: Hive
> Issue Type: Bug
> Reporter: Zhenxiao Luo
> Assignee: Carl Steinbach
> Fix For: 0.9.0
>
> Attachments: HIVE-2782.D1653.1.patch, HIVE-2782.D1653.1.patch
>
>
> When using MapReduce2 for Hive
> ba_table_udfs is failing with unexpected output:
> [junit] Begin query: ba_table_udfs.q
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
> [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/ba_table_udfs.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/ba_table_udfs.q.out
> [junit] 20,26c20,26
> [junit] < 2 10val_10 1
> [junit] < 3 164val_164 1
> [junit] < 3 150val_150 1
> [junit] < 2 18val_18 1
> [junit] < 3 177val_177 1
> [junit] < 2 12val_12 1
> [junit] < 2 11val_11 1
> [junit] —
> [junit] > 3 120val_120 1
> [junit] > 3 192val_192 1
> [junit] > 3 119val_119 1
> [junit] > 3 187val_187 1
> [junit] > 3 176val_176 1
> [junit] > 3 199val_199 1
> [junit] > 3 118val_118 1
> [junit] Exception: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] at junit.framework.Assert.fail(Assert.java:50)
> [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs(TestCliDriver.java:129)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at junit.framework.TestCase.runTest(TestCase.java:168)
> [junit] at junit.framework.TestCase.runBare(TestCase.java:134)
> [junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
> [junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
> [junit] at junit.framework.TestResult.run(TestResult.java:113)
> [junit] at junit.framework.TestCase.run(TestCase.java:124)
> [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
> [junit] at junit.framework.TestSuite.run(TestSuite.java:238)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.)
> [junit] Cleaning up TestCliDriver
> [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 10.751 sec
> [junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
> [for] /home/cloudera/Code/hive/ql/build.xml: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build.xml:328: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build-common.xml:453: Tests failed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2782) New BINARY type produces unexpected
results with supported UDFS when using MapReduce2
Posted by "Ashutosh Chauhan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207908#comment-13207908 ]
Ashutosh Chauhan commented on HIVE-2782:
----------------------------------------
@Carl,
Unfortunately I don't have time to look at this one right now.
> New BINARY type produces unexpected results with supported UDFS when using MapReduce2
> -------------------------------------------------------------------------------------
>
> Key: HIVE-2782
> URL: https://issues.apache.org/jira/browse/HIVE-2782
> Project: Hive
> Issue Type: Bug
> Reporter: Zhenxiao Luo
> Assignee: Carl Steinbach
> Attachments: HIVE-2782.D1653.1.patch, HIVE-2782.D1653.1.patch
>
>
> When using MapReduce2 for Hive
> ba_table_udfs is failing with unexpected output:
> [junit] Begin query: ba_table_udfs.q
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
> [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/ba_table_udfs.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/ba_table_udfs.q.out
> [junit] 20,26c20,26
> [junit] < 2 10val_10 1
> [junit] < 3 164val_164 1
> [junit] < 3 150val_150 1
> [junit] < 2 18val_18 1
> [junit] < 3 177val_177 1
> [junit] < 2 12val_12 1
> [junit] < 2 11val_11 1
> [junit] —
> [junit] > 3 120val_120 1
> [junit] > 3 192val_192 1
> [junit] > 3 119val_119 1
> [junit] > 3 187val_187 1
> [junit] > 3 176val_176 1
> [junit] > 3 199val_199 1
> [junit] > 3 118val_118 1
> [junit] Exception: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] at junit.framework.Assert.fail(Assert.java:50)
> [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs(TestCliDriver.java:129)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at junit.framework.TestCase.runTest(TestCase.java:168)
> [junit] at junit.framework.TestCase.runBare(TestCase.java:134)
> [junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
> [junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
> [junit] at junit.framework.TestResult.run(TestResult.java:113)
> [junit] at junit.framework.TestCase.run(TestCase.java:124)
> [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
> [junit] at junit.framework.TestSuite.run(TestSuite.java:238)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.)
> [junit] Cleaning up TestCliDriver
> [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 10.751 sec
> [junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
> [for] /home/cloudera/Code/hive/ql/build.xml: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build.xml:328: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build-common.xml:453: Tests failed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2782) New BINARY type produces unexpected
results with supported UDFS when using MapReduce2
Posted by "Carl Steinbach (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204202#comment-13204202 ]
Carl Steinbach commented on HIVE-2782:
--------------------------------------
@Ashutosh: I also found what I think is a bug in the way SUBSTR handles BINARY inputs. Can you please take a look?
https://issues.apache.org/jira/browse/HIVE-2792
> New BINARY type produces unexpected results with supported UDFS when using MapReduce2
> -------------------------------------------------------------------------------------
>
> Key: HIVE-2782
> URL: https://issues.apache.org/jira/browse/HIVE-2782
> Project: Hive
> Issue Type: Bug
> Reporter: Zhenxiao Luo
> Assignee: Carl Steinbach
> Attachments: HIVE-2782.D1653.1.patch, HIVE-2782.D1653.1.patch
>
>
> When using MapReduce2 for Hive
> ba_table_udfs is failing with unexpected output:
> [junit] Begin query: ba_table_udfs.q
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
> [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/ba_table_udfs.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/ba_table_udfs.q.out
> [junit] 20,26c20,26
> [junit] < 2 10val_10 1
> [junit] < 3 164val_164 1
> [junit] < 3 150val_150 1
> [junit] < 2 18val_18 1
> [junit] < 3 177val_177 1
> [junit] < 2 12val_12 1
> [junit] < 2 11val_11 1
> [junit] —
> [junit] > 3 120val_120 1
> [junit] > 3 192val_192 1
> [junit] > 3 119val_119 1
> [junit] > 3 187val_187 1
> [junit] > 3 176val_176 1
> [junit] > 3 199val_199 1
> [junit] > 3 118val_118 1
> [junit] Exception: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] at junit.framework.Assert.fail(Assert.java:50)
> [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs(TestCliDriver.java:129)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at junit.framework.TestCase.runTest(TestCase.java:168)
> [junit] at junit.framework.TestCase.runBare(TestCase.java:134)
> [junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
> [junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
> [junit] at junit.framework.TestResult.run(TestResult.java:113)
> [junit] at junit.framework.TestCase.run(TestCase.java:124)
> [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
> [junit] at junit.framework.TestSuite.run(TestSuite.java:238)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.)
> [junit] Cleaning up TestCliDriver
> [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 10.751 sec
> [junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
> [for] /home/cloudera/Code/hive/ql/build.xml: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build.xml:328: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build-common.xml:453: Tests failed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2782) New BINARY type produces unexpected
results with supported UDFS when using MapReduce2
Posted by "Carl Steinbach (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2782:
---------------------------------
Resolution: Fixed
Fix Version/s: 0.9.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to trunk.
> New BINARY type produces unexpected results with supported UDFS when using MapReduce2
> -------------------------------------------------------------------------------------
>
> Key: HIVE-2782
> URL: https://issues.apache.org/jira/browse/HIVE-2782
> Project: Hive
> Issue Type: Bug
> Reporter: Zhenxiao Luo
> Assignee: Carl Steinbach
> Fix For: 0.9.0
>
> Attachments: HIVE-2782.D1653.1.patch, HIVE-2782.D1653.1.patch
>
>
> When using MapReduce2 for Hive
> ba_table_udfs is failing with unexpected output:
> [junit] Begin query: ba_table_udfs.q
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
> [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/ba_table_udfs.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/ba_table_udfs.q.out
> [junit] 20,26c20,26
> [junit] < 2 10val_10 1
> [junit] < 3 164val_164 1
> [junit] < 3 150val_150 1
> [junit] < 2 18val_18 1
> [junit] < 3 177val_177 1
> [junit] < 2 12val_12 1
> [junit] < 2 11val_11 1
> [junit] —
> [junit] > 3 120val_120 1
> [junit] > 3 192val_192 1
> [junit] > 3 119val_119 1
> [junit] > 3 187val_187 1
> [junit] > 3 176val_176 1
> [junit] > 3 199val_199 1
> [junit] > 3 118val_118 1
> [junit] Exception: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] at junit.framework.Assert.fail(Assert.java:50)
> [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs(TestCliDriver.java:129)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at junit.framework.TestCase.runTest(TestCase.java:168)
> [junit] at junit.framework.TestCase.runBare(TestCase.java:134)
> [junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
> [junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
> [junit] at junit.framework.TestResult.run(TestResult.java:113)
> [junit] at junit.framework.TestCase.run(TestCase.java:124)
> [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
> [junit] at junit.framework.TestSuite.run(TestSuite.java:238)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.)
> [junit] Cleaning up TestCliDriver
> [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 10.751 sec
> [junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
> [for] /home/cloudera/Code/hive/ql/build.xml: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build.xml:328: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build-common.xml:453: Tests failed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2782) New BINARY type produces unexpected
results with supported UDFS when using MapReduce2
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207915#comment-13207915 ]
Phabricator commented on HIVE-2782:
-----------------------------------
ashutoshc has accepted the revision "HIVE-2782 [jira] New BINARY type produces unexpected results with supported UDFS when using MapReduce2".
+1 Looks good.
REVISION DETAIL
https://reviews.facebook.net/D1653
> New BINARY type produces unexpected results with supported UDFS when using MapReduce2
> -------------------------------------------------------------------------------------
>
> Key: HIVE-2782
> URL: https://issues.apache.org/jira/browse/HIVE-2782
> Project: Hive
> Issue Type: Bug
> Reporter: Zhenxiao Luo
> Assignee: Carl Steinbach
> Attachments: HIVE-2782.D1653.1.patch, HIVE-2782.D1653.1.patch
>
>
> When using MapReduce2 for Hive
> ba_table_udfs is failing with unexpected output:
> [junit] Begin query: ba_table_udfs.q
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
> [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/ba_table_udfs.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/ba_table_udfs.q.out
> [junit] 20,26c20,26
> [junit] < 2 10val_10 1
> [junit] < 3 164val_164 1
> [junit] < 3 150val_150 1
> [junit] < 2 18val_18 1
> [junit] < 3 177val_177 1
> [junit] < 2 12val_12 1
> [junit] < 2 11val_11 1
> [junit] —
> [junit] > 3 120val_120 1
> [junit] > 3 192val_192 1
> [junit] > 3 119val_119 1
> [junit] > 3 187val_187 1
> [junit] > 3 176val_176 1
> [junit] > 3 199val_199 1
> [junit] > 3 118val_118 1
> [junit] Exception: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] at junit.framework.Assert.fail(Assert.java:50)
> [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs(TestCliDriver.java:129)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at junit.framework.TestCase.runTest(TestCase.java:168)
> [junit] at junit.framework.TestCase.runBare(TestCase.java:134)
> [junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
> [junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
> [junit] at junit.framework.TestResult.run(TestResult.java:113)
> [junit] at junit.framework.TestCase.run(TestCase.java:124)
> [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
> [junit] at junit.framework.TestSuite.run(TestSuite.java:238)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.)
> [junit] Cleaning up TestCliDriver
> [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 10.751 sec
> [junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
> [for] /home/cloudera/Code/hive/ql/build.xml: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build.xml:328: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build-common.xml:453: Tests failed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2782) New BINARY type produces unexpected
results with supported UDFS when using MapReduce2
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2782:
------------------------------
Attachment: HIVE-2782.D1653.1.patch
cwsteinbach requested code review of "HIVE-2782 [jira] New BINARY type produces unexpected results with supported UDFS when using MapReduce2".
Reviewers: JIRA
HIVE-2782. Make ba_table_udfs.q deterministic
When using MapReduce2 for Hive
ba_table_udfs is failing with unexpected output:
[junit] Begin query: ba_table_udfs.q
[junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
[junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
[junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/ba_table_udfs.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/ba_table_udfs.q.out
[junit] 20,26c20,26
[junit] < 2 10val_10 1
[junit] < 3 164val_164 1
[junit] < 3 150val_150 1
[junit] < 2 18val_18 1
[junit] < 3 177val_177 1
[junit] < 2 12val_12 1
[junit] < 2 11val_11 1
[junit] —
[junit] > 3 120val_120 1
[junit] > 3 192val_192 1
[junit] > 3 119val_119 1
[junit] > 3 187val_187 1
[junit] > 3 176val_176 1
[junit] > 3 199val_199 1
[junit] > 3 118val_118 1
[junit] Exception: Client execution results failed with error code = 1
[junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
[junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1
[junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
[junit] at junit.framework.Assert.fail(Assert.java:50)
[junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs(TestCliDriver.java:129)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit] at java.lang.reflect.Method.invoke(Method.java:616)
[junit] at junit.framework.TestCase.runTest(TestCase.java:168)
[junit] at junit.framework.TestCase.runBare(TestCase.java:134)
[junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
[junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
[junit] at junit.framework.TestResult.run(TestResult.java:113)
[junit] at junit.framework.TestCase.run(TestCase.java:124)
[junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
[junit] at junit.framework.TestSuite.run(TestSuite.java:238)
[junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
[junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
[junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
[junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.)
[junit] Cleaning up TestCliDriver
[junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 10.751 sec
[junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
[for] /home/cloudera/Code/hive/ql/build.xml: The following error occurred while executing this line:
[for] /home/cloudera/Code/hive/build.xml:328: The following error occurred while executing this line:
[for] /home/cloudera/Code/hive/build-common.xml:453: Tests failed!
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D1653
AFFECTED FILES
ql/src/test/queries/clientpositive/ba_table_udfs.q
ql/src/test/results/clientpositive/ba_table_udfs.q.out
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/3531/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> New BINARY type produces unexpected results with supported UDFS when using MapReduce2
> -------------------------------------------------------------------------------------
>
> Key: HIVE-2782
> URL: https://issues.apache.org/jira/browse/HIVE-2782
> Project: Hive
> Issue Type: Bug
> Reporter: Zhenxiao Luo
> Assignee: Zhenxiao Luo
> Attachments: HIVE-2782.D1653.1.patch
>
>
> When using MapReduce2 for Hive
> ba_table_udfs is failing with unexpected output:
> [junit] Begin query: ba_table_udfs.q
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
> [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/ba_table_udfs.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/ba_table_udfs.q.out
> [junit] 20,26c20,26
> [junit] < 2 10val_10 1
> [junit] < 3 164val_164 1
> [junit] < 3 150val_150 1
> [junit] < 2 18val_18 1
> [junit] < 3 177val_177 1
> [junit] < 2 12val_12 1
> [junit] < 2 11val_11 1
> [junit] —
> [junit] > 3 120val_120 1
> [junit] > 3 192val_192 1
> [junit] > 3 119val_119 1
> [junit] > 3 187val_187 1
> [junit] > 3 176val_176 1
> [junit] > 3 199val_199 1
> [junit] > 3 118val_118 1
> [junit] Exception: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] at junit.framework.Assert.fail(Assert.java:50)
> [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs(TestCliDriver.java:129)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at junit.framework.TestCase.runTest(TestCase.java:168)
> [junit] at junit.framework.TestCase.runBare(TestCase.java:134)
> [junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
> [junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
> [junit] at junit.framework.TestResult.run(TestResult.java:113)
> [junit] at junit.framework.TestCase.run(TestCase.java:124)
> [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
> [junit] at junit.framework.TestSuite.run(TestSuite.java:238)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.)
> [junit] Cleaning up TestCliDriver
> [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 10.751 sec
> [junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
> [for] /home/cloudera/Code/hive/ql/build.xml: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build.xml:328: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build-common.xml:453: Tests failed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2782) New BINARY type produces unexpected
results with supported UDFS when using MapReduce2
Posted by "Carl Steinbach (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2782:
---------------------------------
Assignee: Carl Steinbach (was: Zhenxiao Luo)
Status: Patch Available (was: Open)
> New BINARY type produces unexpected results with supported UDFS when using MapReduce2
> -------------------------------------------------------------------------------------
>
> Key: HIVE-2782
> URL: https://issues.apache.org/jira/browse/HIVE-2782
> Project: Hive
> Issue Type: Bug
> Reporter: Zhenxiao Luo
> Assignee: Carl Steinbach
> Attachments: HIVE-2782.D1653.1.patch, HIVE-2782.D1653.1.patch
>
>
> When using MapReduce2 for Hive
> ba_table_udfs is failing with unexpected output:
> [junit] Begin query: ba_table_udfs.q
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
> [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/ba_table_udfs.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/ba_table_udfs.q.out
> [junit] 20,26c20,26
> [junit] < 2 10val_10 1
> [junit] < 3 164val_164 1
> [junit] < 3 150val_150 1
> [junit] < 2 18val_18 1
> [junit] < 3 177val_177 1
> [junit] < 2 12val_12 1
> [junit] < 2 11val_11 1
> [junit] —
> [junit] > 3 120val_120 1
> [junit] > 3 192val_192 1
> [junit] > 3 119val_119 1
> [junit] > 3 187val_187 1
> [junit] > 3 176val_176 1
> [junit] > 3 199val_199 1
> [junit] > 3 118val_118 1
> [junit] Exception: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] at junit.framework.Assert.fail(Assert.java:50)
> [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs(TestCliDriver.java:129)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at junit.framework.TestCase.runTest(TestCase.java:168)
> [junit] at junit.framework.TestCase.runBare(TestCase.java:134)
> [junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
> [junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
> [junit] at junit.framework.TestResult.run(TestResult.java:113)
> [junit] at junit.framework.TestCase.run(TestCase.java:124)
> [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
> [junit] at junit.framework.TestSuite.run(TestSuite.java:238)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.)
> [junit] Cleaning up TestCliDriver
> [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 10.751 sec
> [junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
> [for] /home/cloudera/Code/hive/ql/build.xml: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build.xml:328: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build-common.xml:453: Tests failed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2782) New BINARY type produces unexpected
results with supported UDFS when using MapReduce2
Posted by "Ashutosh Chauhan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204094#comment-13204094 ]
Ashutosh Chauhan commented on HIVE-2782:
----------------------------------------
{noformat}
[junit] < 2 10val_10 1
[junit] < 3 164val_164 1
[junit] < 3 150val_150 1
[junit] < 2 18val_18 1
[junit] < 3 177val_177 1
[junit] < 2 12val_12 1
[junit] < 2 11val_11 1
[junit] —
[junit] > 3 120val_120 1
[junit] > 3 192val_192 1
[junit] > 3 119val_119 1
[junit] > 3 187val_187 1
[junit] > 3 176val_176 1
[junit] > 3 199val_199 1
[junit] > 3 118val_118 1
{noformat}
If I am reading this correctly, this diff is not showing ordering issues of rows, rather few of the rows are different then whats expected, which then is a correctness issue. Isn't it ?
> New BINARY type produces unexpected results with supported UDFS when using MapReduce2
> -------------------------------------------------------------------------------------
>
> Key: HIVE-2782
> URL: https://issues.apache.org/jira/browse/HIVE-2782
> Project: Hive
> Issue Type: Bug
> Reporter: Zhenxiao Luo
> Assignee: Carl Steinbach
> Attachments: HIVE-2782.D1653.1.patch, HIVE-2782.D1653.1.patch
>
>
> When using MapReduce2 for Hive
> ba_table_udfs is failing with unexpected output:
> [junit] Begin query: ba_table_udfs.q
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
> [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/ba_table_udfs.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/ba_table_udfs.q.out
> [junit] 20,26c20,26
> [junit] < 2 10val_10 1
> [junit] < 3 164val_164 1
> [junit] < 3 150val_150 1
> [junit] < 2 18val_18 1
> [junit] < 3 177val_177 1
> [junit] < 2 12val_12 1
> [junit] < 2 11val_11 1
> [junit] —
> [junit] > 3 120val_120 1
> [junit] > 3 192val_192 1
> [junit] > 3 119val_119 1
> [junit] > 3 187val_187 1
> [junit] > 3 176val_176 1
> [junit] > 3 199val_199 1
> [junit] > 3 118val_118 1
> [junit] Exception: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] at junit.framework.Assert.fail(Assert.java:50)
> [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs(TestCliDriver.java:129)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at junit.framework.TestCase.runTest(TestCase.java:168)
> [junit] at junit.framework.TestCase.runBare(TestCase.java:134)
> [junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
> [junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
> [junit] at junit.framework.TestResult.run(TestResult.java:113)
> [junit] at junit.framework.TestCase.run(TestCase.java:124)
> [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
> [junit] at junit.framework.TestSuite.run(TestSuite.java:238)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.)
> [junit] Cleaning up TestCliDriver
> [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 10.751 sec
> [junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
> [for] /home/cloudera/Code/hive/ql/build.xml: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build.xml:328: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build-common.xml:453: Tests failed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2782) New BINARY type produces unexpected
results with supported UDFS when using MapReduce2
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209172#comment-13209172 ]
Phabricator commented on HIVE-2782:
-----------------------------------
cwsteinbach has committed the revision "HIVE-2782 [jira] New BINARY type produces unexpected results with supported UDFS when using MapReduce2".
Change committed by cws.
REVISION DETAIL
https://reviews.facebook.net/D1653
COMMIT
https://reviews.facebook.net/rHIVE1244314
> New BINARY type produces unexpected results with supported UDFS when using MapReduce2
> -------------------------------------------------------------------------------------
>
> Key: HIVE-2782
> URL: https://issues.apache.org/jira/browse/HIVE-2782
> Project: Hive
> Issue Type: Bug
> Reporter: Zhenxiao Luo
> Assignee: Carl Steinbach
> Fix For: 0.9.0
>
> Attachments: HIVE-2782.D1653.1.patch, HIVE-2782.D1653.1.patch
>
>
> When using MapReduce2 for Hive
> ba_table_udfs is failing with unexpected output:
> [junit] Begin query: ba_table_udfs.q
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
> [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/ba_table_udfs.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/ba_table_udfs.q.out
> [junit] 20,26c20,26
> [junit] < 2 10val_10 1
> [junit] < 3 164val_164 1
> [junit] < 3 150val_150 1
> [junit] < 2 18val_18 1
> [junit] < 3 177val_177 1
> [junit] < 2 12val_12 1
> [junit] < 2 11val_11 1
> [junit] —
> [junit] > 3 120val_120 1
> [junit] > 3 192val_192 1
> [junit] > 3 119val_119 1
> [junit] > 3 187val_187 1
> [junit] > 3 176val_176 1
> [junit] > 3 199val_199 1
> [junit] > 3 118val_118 1
> [junit] Exception: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] at junit.framework.Assert.fail(Assert.java:50)
> [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs(TestCliDriver.java:129)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at junit.framework.TestCase.runTest(TestCase.java:168)
> [junit] at junit.framework.TestCase.runBare(TestCase.java:134)
> [junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
> [junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
> [junit] at junit.framework.TestResult.run(TestResult.java:113)
> [junit] at junit.framework.TestCase.run(TestCase.java:124)
> [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
> [junit] at junit.framework.TestSuite.run(TestSuite.java:238)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.)
> [junit] Cleaning up TestCliDriver
> [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 10.751 sec
> [junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
> [for] /home/cloudera/Code/hive/ql/build.xml: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build.xml:328: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build-common.xml:453: Tests failed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2782) New BINARY type produces unexpected
results with supported UDFS when using MapReduce2
Posted by "Ashutosh Chauhan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207911#comment-13207911 ]
Ashutosh Chauhan commented on HIVE-2782:
----------------------------------------
Oh.. previous comment was for HIVE-2792 I assumed you meant HIVE-2792 in that I am looking at HIVE-2782 right now.
> New BINARY type produces unexpected results with supported UDFS when using MapReduce2
> -------------------------------------------------------------------------------------
>
> Key: HIVE-2782
> URL: https://issues.apache.org/jira/browse/HIVE-2782
> Project: Hive
> Issue Type: Bug
> Reporter: Zhenxiao Luo
> Assignee: Carl Steinbach
> Attachments: HIVE-2782.D1653.1.patch, HIVE-2782.D1653.1.patch
>
>
> When using MapReduce2 for Hive
> ba_table_udfs is failing with unexpected output:
> [junit] Begin query: ba_table_udfs.q
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
> [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/ba_table_udfs.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/ba_table_udfs.q.out
> [junit] 20,26c20,26
> [junit] < 2 10val_10 1
> [junit] < 3 164val_164 1
> [junit] < 3 150val_150 1
> [junit] < 2 18val_18 1
> [junit] < 3 177val_177 1
> [junit] < 2 12val_12 1
> [junit] < 2 11val_11 1
> [junit] —
> [junit] > 3 120val_120 1
> [junit] > 3 192val_192 1
> [junit] > 3 119val_119 1
> [junit] > 3 187val_187 1
> [junit] > 3 176val_176 1
> [junit] > 3 199val_199 1
> [junit] > 3 118val_118 1
> [junit] Exception: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] at junit.framework.Assert.fail(Assert.java:50)
> [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs(TestCliDriver.java:129)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at junit.framework.TestCase.runTest(TestCase.java:168)
> [junit] at junit.framework.TestCase.runBare(TestCase.java:134)
> [junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
> [junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
> [junit] at junit.framework.TestResult.run(TestResult.java:113)
> [junit] at junit.framework.TestCase.run(TestCase.java:124)
> [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
> [junit] at junit.framework.TestSuite.run(TestSuite.java:238)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.)
> [junit] Cleaning up TestCliDriver
> [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 10.751 sec
> [junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
> [for] /home/cloudera/Code/hive/ql/build.xml: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build.xml:328: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build-common.xml:453: Tests failed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2782) New BINARY type produces unexpected
results with supported UDFS when using MapReduce2
Posted by "Carl Steinbach (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207475#comment-13207475 ]
Carl Steinbach commented on HIVE-2782:
--------------------------------------
@Ashutosh: Do you have time to take a look at this? Thanks.
> New BINARY type produces unexpected results with supported UDFS when using MapReduce2
> -------------------------------------------------------------------------------------
>
> Key: HIVE-2782
> URL: https://issues.apache.org/jira/browse/HIVE-2782
> Project: Hive
> Issue Type: Bug
> Reporter: Zhenxiao Luo
> Assignee: Carl Steinbach
> Attachments: HIVE-2782.D1653.1.patch, HIVE-2782.D1653.1.patch
>
>
> When using MapReduce2 for Hive
> ba_table_udfs is failing with unexpected output:
> [junit] Begin query: ba_table_udfs.q
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
> [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime -I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun -I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.] [0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/ba_table_udfs.q.out /home/cloudera/Code/hive/ql/src/test/results/clientpositive/ba_table_udfs.q.out
> [junit] 20,26c20,26
> [junit] < 2 10val_10 1
> [junit] < 3 164val_164 1
> [junit] < 3 150val_150 1
> [junit] < 2 18val_18 1
> [junit] < 3 177val_177 1
> [junit] < 2 12val_12 1
> [junit] < 2 11val_11 1
> [junit] —
> [junit] > 3 120val_120 1
> [junit] > 3 192val_192 1
> [junit] > 3 119val_119 1
> [junit] > 3 187val_187 1
> [junit] > 3 176val_176 1
> [junit] > 3 199val_199 1
> [junit] > 3 118val_118 1
> [junit] Exception: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] junit.framework.AssertionFailedError: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.
> [junit] at junit.framework.Assert.fail(Assert.java:50)
> [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs(TestCliDriver.java:129)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at junit.framework.TestCase.runTest(TestCase.java:168)
> [junit] at junit.framework.TestCase.runBare(TestCase.java:134)
> [junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
> [junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
> [junit] at junit.framework.TestResult.run(TestResult.java:113)
> [junit] at junit.framework.TestCase.run(TestCase.java:124)
> [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
> [junit] at junit.framework.TestSuite.run(TestSuite.java:238)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more logs.)
> [junit] Cleaning up TestCliDriver
> [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 10.751 sec
> [junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
> [for] /home/cloudera/Code/hive/ql/build.xml: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build.xml:328: The following error occurred while executing this line:
> [for] /home/cloudera/Code/hive/build-common.xml:453: Tests failed!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira