You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Zhenxiao Luo (JIRA)" <ji...@apache.org> on 2012/07/06 22:46:34 UTC
[jira] [Created] (HIVE-3240) Fix non-deterministic results in
newline.q and timestamp_lazy.q
Zhenxiao Luo created HIVE-3240:
----------------------------------
Summary: Fix non-deterministic results in newline.q and timestamp_lazy.q
Key: HIVE-3240
URL: https://issues.apache.org/jira/browse/HIVE-3240
Project: Hive
Issue Type: Bug
Reporter: Zhenxiao Luo
Assignee: Zhenxiao Luo
newline.q and timestamp_lazy.q have non-deterministic results, which make them fail on MapReduce2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3240) Fix non-deterministic results in
newline.q and timestamp_lazy.q
Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhenxiao Luo updated HIVE-3240:
-------------------------------
Status: Patch Available (was: Open)
> Fix non-deterministic results in newline.q and timestamp_lazy.q
> ---------------------------------------------------------------
>
> Key: HIVE-3240
> URL: https://issues.apache.org/jira/browse/HIVE-3240
> Project: Hive
> Issue Type: Bug
> Components: Tests
> Reporter: Zhenxiao Luo
> Assignee: Zhenxiao Luo
> Attachments: HIVE-3240.1.patch.txt
>
>
> newline.q and timestamp_lazy.q have non-deterministic results, which make them fail on MapReduce2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3240) Fix non-deterministic results in
newline.q and timestamp_lazy.q
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-3240:
---------------------------------
Resolution: Fixed
Fix Version/s: 0.10.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks Zhenxiao!
> Fix non-deterministic results in newline.q and timestamp_lazy.q
> ---------------------------------------------------------------
>
> Key: HIVE-3240
> URL: https://issues.apache.org/jira/browse/HIVE-3240
> Project: Hive
> Issue Type: Bug
> Components: Tests
> Reporter: Zhenxiao Luo
> Assignee: Zhenxiao Luo
> Fix For: 0.10.0
>
> Attachments: HIVE-3240.1.patch.txt, HIVE-3240.2.patch.txt, Windows_alter1.q.out.patch.txt
>
>
> newline.q and timestamp_lazy.q have non-deterministic results, which make them fail on MapReduce2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3240) Fix non-deterministic results in
newline.q and timestamp_lazy.q
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-3240:
---------------------------------
Status: Open (was: Patch Available)
@Zhenxiao: please submit a review request.
> Fix non-deterministic results in newline.q and timestamp_lazy.q
> ---------------------------------------------------------------
>
> Key: HIVE-3240
> URL: https://issues.apache.org/jira/browse/HIVE-3240
> Project: Hive
> Issue Type: Bug
> Components: Tests
> Reporter: Zhenxiao Luo
> Assignee: Zhenxiao Luo
> Attachments: HIVE-3240.1.patch.txt
>
>
> newline.q and timestamp_lazy.q have non-deterministic results, which make them fail on MapReduce2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3240) Fix non-deterministic results in
newline.q and timestamp_lazy.q
Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhenxiao Luo updated HIVE-3240:
-------------------------------
Status: Patch Available (was: Open)
> Fix non-deterministic results in newline.q and timestamp_lazy.q
> ---------------------------------------------------------------
>
> Key: HIVE-3240
> URL: https://issues.apache.org/jira/browse/HIVE-3240
> Project: Hive
> Issue Type: Bug
> Reporter: Zhenxiao Luo
> Assignee: Zhenxiao Luo
> Attachments: HIVE-3240.1.patch.txt
>
>
> newline.q and timestamp_lazy.q have non-deterministic results, which make them fail on MapReduce2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3240) Fix non-deterministic results in
newline.q and timestamp_lazy.q
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-3240:
---------------------------------
Component/s: Tests
> Fix non-deterministic results in newline.q and timestamp_lazy.q
> ---------------------------------------------------------------
>
> Key: HIVE-3240
> URL: https://issues.apache.org/jira/browse/HIVE-3240
> Project: Hive
> Issue Type: Bug
> Components: Tests
> Reporter: Zhenxiao Luo
> Assignee: Zhenxiao Luo
> Attachments: HIVE-3240.1.patch.txt
>
>
> newline.q and timestamp_lazy.q have non-deterministic results, which make them fail on MapReduce2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3240) Fix non-deterministic results in
newline.q and timestamp_lazy.q
Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhenxiao Luo updated HIVE-3240:
-------------------------------
Status: Patch Available (was: Open)
> Fix non-deterministic results in newline.q and timestamp_lazy.q
> ---------------------------------------------------------------
>
> Key: HIVE-3240
> URL: https://issues.apache.org/jira/browse/HIVE-3240
> Project: Hive
> Issue Type: Bug
> Components: Tests
> Reporter: Zhenxiao Luo
> Assignee: Zhenxiao Luo
> Attachments: HIVE-3240.1.patch.txt, HIVE-3240.2.patch.txt, Windows_alter1.q.out.patch.txt
>
>
> newline.q and timestamp_lazy.q have non-deterministic results, which make them fail on MapReduce2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3240) Fix non-deterministic results in
newline.q and timestamp_lazy.q
Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13429616#comment-13429616 ]
Zhenxiao Luo commented on HIVE-3240:
------------------------------------
Patch rebased, Review request submitted at:
https://reviews.facebook.net/D4539
> Fix non-deterministic results in newline.q and timestamp_lazy.q
> ---------------------------------------------------------------
>
> Key: HIVE-3240
> URL: https://issues.apache.org/jira/browse/HIVE-3240
> Project: Hive
> Issue Type: Bug
> Components: Tests
> Reporter: Zhenxiao Luo
> Assignee: Zhenxiao Luo
> Attachments: HIVE-3240.1.patch.txt, HIVE-3240.2.patch.txt, Windows_alter1.q.out.patch.txt
>
>
> newline.q and timestamp_lazy.q have non-deterministic results, which make them fail on MapReduce2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3240) Fix non-deterministic results in
newline.q and timestamp_lazy.q
Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhenxiao Luo updated HIVE-3240:
-------------------------------
Attachment: HIVE-3240.2.patch.txt
> Fix non-deterministic results in newline.q and timestamp_lazy.q
> ---------------------------------------------------------------
>
> Key: HIVE-3240
> URL: https://issues.apache.org/jira/browse/HIVE-3240
> Project: Hive
> Issue Type: Bug
> Components: Tests
> Reporter: Zhenxiao Luo
> Assignee: Zhenxiao Luo
> Attachments: HIVE-3240.1.patch.txt, HIVE-3240.2.patch.txt, Windows_alter1.q.out.patch.txt
>
>
> newline.q and timestamp_lazy.q have non-deterministic results, which make them fail on MapReduce2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3240) Fix non-deterministic results in
newline.q and timestamp_lazy.q
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440165#comment-13440165 ]
Carl Steinbach commented on HIVE-3240:
--------------------------------------
+1. Will commit if tests pass.
> Fix non-deterministic results in newline.q and timestamp_lazy.q
> ---------------------------------------------------------------
>
> Key: HIVE-3240
> URL: https://issues.apache.org/jira/browse/HIVE-3240
> Project: Hive
> Issue Type: Bug
> Components: Tests
> Reporter: Zhenxiao Luo
> Assignee: Zhenxiao Luo
> Attachments: HIVE-3240.1.patch.txt, HIVE-3240.2.patch.txt, Windows_alter1.q.out.patch.txt
>
>
> newline.q and timestamp_lazy.q have non-deterministic results, which make them fail on MapReduce2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3240) Fix non-deterministic results in
newline.q and timestamp_lazy.q
Posted by "Kanna Karanam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kanna Karanam updated HIVE-3240:
--------------------------------
Attachment: Windows_alter1.q.out.patch.txt
@Edward – I do agree with you. Unit test shouldn’t restrict us on what to test and what not. Apart from the differences in new line characters, “ToString()” on some objects are introducing extra lines/spaces at the end on Windows compare to Unix and it is causing some tests fail. Here is the diff for “alter1.q” on windows. (Note: I trimmed the CR characters to minimize the patch)
Coming to this Jira – timestamp_lazy.q is passing with no issues. I couldn’t run newline/q test yet on windows because of some Path/Perl related issue.
> Fix non-deterministic results in newline.q and timestamp_lazy.q
> ---------------------------------------------------------------
>
> Key: HIVE-3240
> URL: https://issues.apache.org/jira/browse/HIVE-3240
> Project: Hive
> Issue Type: Bug
> Components: Tests
> Reporter: Zhenxiao Luo
> Assignee: Zhenxiao Luo
> Attachments: HIVE-3240.1.patch.txt, Windows_alter1.q.out.patch.txt
>
>
> newline.q and timestamp_lazy.q have non-deterministic results, which make them fail on MapReduce2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3240) Fix non-deterministic results in
newline.q and timestamp_lazy.q
Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408484#comment-13408484 ]
Zhenxiao Luo commented on HIVE-3240:
------------------------------------
review request submitted at:
https://reviews.facebook.net/D4041
> Fix non-deterministic results in newline.q and timestamp_lazy.q
> ---------------------------------------------------------------
>
> Key: HIVE-3240
> URL: https://issues.apache.org/jira/browse/HIVE-3240
> Project: Hive
> Issue Type: Bug
> Components: Tests
> Reporter: Zhenxiao Luo
> Assignee: Zhenxiao Luo
> Attachments: HIVE-3240.1.patch.txt
>
>
> newline.q and timestamp_lazy.q have non-deterministic results, which make them fail on MapReduce2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3240) Fix non-deterministic results in
newline.q and timestamp_lazy.q
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409142#comment-13409142 ]
Edward Capriolo commented on HIVE-3240:
---------------------------------------
Kanna,
Do you mind showing me the output that windows produces? I do not love the idea of juggling around the unit tests to make everyone "happy". There is just something fundamentally wrong with a unit test telling you what you can and can not test, or changing existing tests to make tools happy. In the case of mapreduce2 we may be able to write a partitioner and secondary sort (cluster by) to make this happen.
> Fix non-deterministic results in newline.q and timestamp_lazy.q
> ---------------------------------------------------------------
>
> Key: HIVE-3240
> URL: https://issues.apache.org/jira/browse/HIVE-3240
> Project: Hive
> Issue Type: Bug
> Components: Tests
> Reporter: Zhenxiao Luo
> Assignee: Zhenxiao Luo
> Attachments: HIVE-3240.1.patch.txt
>
>
> newline.q and timestamp_lazy.q have non-deterministic results, which make them fail on MapReduce2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3240) Fix non-deterministic results in
newline.q and timestamp_lazy.q
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409827#comment-13409827 ]
Carl Steinbach commented on HIVE-3240:
--------------------------------------
+1. Will commit if tests pass.
> Fix non-deterministic results in newline.q and timestamp_lazy.q
> ---------------------------------------------------------------
>
> Key: HIVE-3240
> URL: https://issues.apache.org/jira/browse/HIVE-3240
> Project: Hive
> Issue Type: Bug
> Components: Tests
> Reporter: Zhenxiao Luo
> Assignee: Zhenxiao Luo
> Attachments: HIVE-3240.1.patch.txt, Windows_alter1.q.out.patch.txt
>
>
> newline.q and timestamp_lazy.q have non-deterministic results, which make them fail on MapReduce2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3240) Fix non-deterministic results in
newline.q and timestamp_lazy.q
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-3240:
---------------------------------
Status: Open (was: Patch Available)
@Zhenxiao: This needs a rebase after HIVE-3327. Please update.
> Fix non-deterministic results in newline.q and timestamp_lazy.q
> ---------------------------------------------------------------
>
> Key: HIVE-3240
> URL: https://issues.apache.org/jira/browse/HIVE-3240
> Project: Hive
> Issue Type: Bug
> Components: Tests
> Reporter: Zhenxiao Luo
> Assignee: Zhenxiao Luo
> Attachments: HIVE-3240.1.patch.txt, Windows_alter1.q.out.patch.txt
>
>
> newline.q and timestamp_lazy.q have non-deterministic results, which make them fail on MapReduce2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3240) Fix non-deterministic results in
newline.q and timestamp_lazy.q
Posted by "Kanna Karanam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409179#comment-13409179 ]
Kanna Karanam commented on HIVE-3240:
-------------------------------------
Corrected: ToString()” on some objects are introducing differences in white lines/spaces on Windows compare to Unix and it is causing some tests fail
> Fix non-deterministic results in newline.q and timestamp_lazy.q
> ---------------------------------------------------------------
>
> Key: HIVE-3240
> URL: https://issues.apache.org/jira/browse/HIVE-3240
> Project: Hive
> Issue Type: Bug
> Components: Tests
> Reporter: Zhenxiao Luo
> Assignee: Zhenxiao Luo
> Attachments: HIVE-3240.1.patch.txt, Windows_alter1.q.out.patch.txt
>
>
> newline.q and timestamp_lazy.q have non-deterministic results, which make them fail on MapReduce2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3240) Fix non-deterministic results in
newline.q and timestamp_lazy.q
Posted by "Kanna Karanam (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409137#comment-13409137 ]
Kanna Karanam commented on HIVE-3240:
-------------------------------------
Thanks Zhenxiao. These are good changes. Instead of selecting all rows and then displaying them in order, can we just go with displaying the count of records in the table so that it works on Unix as well as on Windows (Windows has a problem with extra white spacing.. in this case they are new line records)
> Fix non-deterministic results in newline.q and timestamp_lazy.q
> ---------------------------------------------------------------
>
> Key: HIVE-3240
> URL: https://issues.apache.org/jira/browse/HIVE-3240
> Project: Hive
> Issue Type: Bug
> Components: Tests
> Reporter: Zhenxiao Luo
> Assignee: Zhenxiao Luo
> Attachments: HIVE-3240.1.patch.txt
>
>
> newline.q and timestamp_lazy.q have non-deterministic results, which make them fail on MapReduce2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3240) Fix non-deterministic results in
newline.q and timestamp_lazy.q
Posted by "Zhenxiao Luo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhenxiao Luo updated HIVE-3240:
-------------------------------
Attachment: HIVE-3240.1.patch.txt
> Fix non-deterministic results in newline.q and timestamp_lazy.q
> ---------------------------------------------------------------
>
> Key: HIVE-3240
> URL: https://issues.apache.org/jira/browse/HIVE-3240
> Project: Hive
> Issue Type: Bug
> Reporter: Zhenxiao Luo
> Assignee: Zhenxiao Luo
> Attachments: HIVE-3240.1.patch.txt
>
>
> newline.q and timestamp_lazy.q have non-deterministic results, which make them fail on MapReduce2
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira