You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Kevin Wilfong (JIRA)" <ji...@apache.org> on 2012/11/28 01:16:58 UTC
[jira] [Created] (HIVE-3750) JDBCStatsPublisher fails when ID
length exceeds length of ID column
Kevin Wilfong created HIVE-3750:
-----------------------------------
Summary: JDBCStatsPublisher fails when ID length exceeds length of ID column
Key: HIVE-3750
URL: https://issues.apache.org/jira/browse/HIVE-3750
Project: Hive
Issue Type: Bug
Components: Statistics
Affects Versions: 0.11
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
When the length of the ID field passed to JDBCStatsPublisher exceeds the length of the column in the table (currently 255 characters) stats collection fails. This causes the entire query to fail when hive.stats.reliable is set to true.
One way to prevent this would be to calculate a deterministic, very low collision hash of the ID prefix used for aggregation and use that when the length of the ID is too long.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3750) JDBCStatsPublisher fails when ID
length exceeds length of ID column
Posted by "Kevin Wilfong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506830#comment-13506830 ]
Kevin Wilfong commented on HIVE-3750:
-------------------------------------
https://reviews.facebook.net/D7017
> JDBCStatsPublisher fails when ID length exceeds length of ID column
> -------------------------------------------------------------------
>
> Key: HIVE-3750
> URL: https://issues.apache.org/jira/browse/HIVE-3750
> Project: Hive
> Issue Type: Bug
> Components: Statistics
> Affects Versions: 0.11
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-3750.1.patch.txt
>
>
> When the length of the ID field passed to JDBCStatsPublisher exceeds the length of the column in the table (currently 255 characters) stats collection fails. This causes the entire query to fail when hive.stats.reliable is set to true.
> One way to prevent this would be to calculate a deterministic, very low collision hash of the ID prefix used for aggregation and use that when the length of the ID is too long.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3750) JDBCStatsPublisher fails when ID
length exceeds length of ID column
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510876#comment-13510876 ]
Hudson commented on HIVE-3750:
------------------------------
Integrated in Hive-trunk-h0.21 #1834 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1834/])
HIVE-3771 HIVE-3750 broke TestParse
(Kevin Wilfong via namit) (Revision 1417294)
Result = ABORTED
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1417294
Files :
* /hive/trunk/ql/src/test/results/compiler/plan/case_sensitivity.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/groupby1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input2.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input3.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input5.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input6.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input7.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input9.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input_testsequencefile.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml
> JDBCStatsPublisher fails when ID length exceeds length of ID column
> -------------------------------------------------------------------
>
> Key: HIVE-3750
> URL: https://issues.apache.org/jira/browse/HIVE-3750
> Project: Hive
> Issue Type: Bug
> Components: Statistics
> Affects Versions: 0.11
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-3750.1.patch.txt
>
>
> When the length of the ID field passed to JDBCStatsPublisher exceeds the length of the column in the table (currently 255 characters) stats collection fails. This causes the entire query to fail when hive.stats.reliable is set to true.
> One way to prevent this would be to calculate a deterministic, very low collision hash of the ID prefix used for aggregation and use that when the length of the ID is too long.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3750) JDBCStatsPublisher fails when ID
length exceeds length of ID column
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511204#comment-13511204 ]
Hudson commented on HIVE-3750:
------------------------------
Integrated in Hive-trunk-h0.21 #1835 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1835/])
HIVE-3771 HIVE-3750 broke TestParse
(Kevin Wilfong via namit) (Revision 1417294)
Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1417294
Files :
* /hive/trunk/ql/src/test/results/compiler/plan/case_sensitivity.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/groupby1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input2.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input3.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input5.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input6.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input7.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input9.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input_testsequencefile.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join2.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join3.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample2.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample3.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample4.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample5.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample6.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/sample7.q.xml
> JDBCStatsPublisher fails when ID length exceeds length of ID column
> -------------------------------------------------------------------
>
> Key: HIVE-3750
> URL: https://issues.apache.org/jira/browse/HIVE-3750
> Project: Hive
> Issue Type: Bug
> Components: Statistics
> Affects Versions: 0.11
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-3750.1.patch.txt
>
>
> When the length of the ID field passed to JDBCStatsPublisher exceeds the length of the column in the table (currently 255 characters) stats collection fails. This causes the entire query to fail when hive.stats.reliable is set to true.
> One way to prevent this would be to calculate a deterministic, very low collision hash of the ID prefix used for aggregation and use that when the length of the ID is too long.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3750) JDBCStatsPublisher fails when ID
length exceeds length of ID column
Posted by "Kevin Wilfong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510146#comment-13510146 ]
Kevin Wilfong commented on HIVE-3750:
-------------------------------------
Sorry about that.
I filed HIVE-3771 to fix it and attached a diff. I only saw failures in TestParse.
> JDBCStatsPublisher fails when ID length exceeds length of ID column
> -------------------------------------------------------------------
>
> Key: HIVE-3750
> URL: https://issues.apache.org/jira/browse/HIVE-3750
> Project: Hive
> Issue Type: Bug
> Components: Statistics
> Affects Versions: 0.11
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-3750.1.patch.txt
>
>
> When the length of the ID field passed to JDBCStatsPublisher exceeds the length of the column in the table (currently 255 characters) stats collection fails. This causes the entire query to fail when hive.stats.reliable is set to true.
> One way to prevent this would be to calculate a deterministic, very low collision hash of the ID prefix used for aggregation and use that when the length of the ID is too long.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3750) JDBCStatsPublisher fails when ID
length exceeds length of ID column
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-3750:
-----------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed. Thanks Kevin
> JDBCStatsPublisher fails when ID length exceeds length of ID column
> -------------------------------------------------------------------
>
> Key: HIVE-3750
> URL: https://issues.apache.org/jira/browse/HIVE-3750
> Project: Hive
> Issue Type: Bug
> Components: Statistics
> Affects Versions: 0.11
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-3750.1.patch.txt
>
>
> When the length of the ID field passed to JDBCStatsPublisher exceeds the length of the column in the table (currently 255 characters) stats collection fails. This causes the entire query to fail when hive.stats.reliable is set to true.
> One way to prevent this would be to calculate a deterministic, very low collision hash of the ID prefix used for aggregation and use that when the length of the ID is too long.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3750) JDBCStatsPublisher fails when ID
length exceeds length of ID column
Posted by "Kevin Wilfong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Wilfong updated HIVE-3750:
--------------------------------
Attachment: HIVE-3750.1.patch.txt
> JDBCStatsPublisher fails when ID length exceeds length of ID column
> -------------------------------------------------------------------
>
> Key: HIVE-3750
> URL: https://issues.apache.org/jira/browse/HIVE-3750
> Project: Hive
> Issue Type: Bug
> Components: Statistics
> Affects Versions: 0.11
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-3750.1.patch.txt
>
>
> When the length of the ID field passed to JDBCStatsPublisher exceeds the length of the column in the table (currently 255 characters) stats collection fails. This causes the entire query to fail when hive.stats.reliable is set to true.
> One way to prevent this would be to calculate a deterministic, very low collision hash of the ID prefix used for aggregation and use that when the length of the ID is too long.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HIVE-3750) JDBCStatsPublisher fails when ID
length exceeds length of ID column
Posted by "Ashutosh Chauhan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan reopened HIVE-3750:
------------------------------------
This is making quite a few test case in TestParse and elsewhere to fail, because golden files were not updated corresponding to maxStatsKeyPrefixLength config which is now showing up in the outputs.
Please run all tests and update golden file (assuming they need to be updated).
> JDBCStatsPublisher fails when ID length exceeds length of ID column
> -------------------------------------------------------------------
>
> Key: HIVE-3750
> URL: https://issues.apache.org/jira/browse/HIVE-3750
> Project: Hive
> Issue Type: Bug
> Components: Statistics
> Affects Versions: 0.11
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-3750.1.patch.txt
>
>
> When the length of the ID field passed to JDBCStatsPublisher exceeds the length of the column in the table (currently 255 characters) stats collection fails. This causes the entire query to fail when hive.stats.reliable is set to true.
> One way to prevent this would be to calculate a deterministic, very low collision hash of the ID prefix used for aggregation and use that when the length of the ID is too long.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3750) JDBCStatsPublisher fails when ID
length exceeds length of ID column
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508583#comment-13508583 ]
Namit Jain commented on HIVE-3750:
----------------------------------
+1
> JDBCStatsPublisher fails when ID length exceeds length of ID column
> -------------------------------------------------------------------
>
> Key: HIVE-3750
> URL: https://issues.apache.org/jira/browse/HIVE-3750
> Project: Hive
> Issue Type: Bug
> Components: Statistics
> Affects Versions: 0.11
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-3750.1.patch.txt
>
>
> When the length of the ID field passed to JDBCStatsPublisher exceeds the length of the column in the table (currently 255 characters) stats collection fails. This causes the entire query to fail when hive.stats.reliable is set to true.
> One way to prevent this would be to calculate a deterministic, very low collision hash of the ID prefix used for aggregation and use that when the length of the ID is too long.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3750) JDBCStatsPublisher fails when ID
length exceeds length of ID column
Posted by "Kevin Wilfong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-3750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Wilfong updated HIVE-3750:
--------------------------------
Status: Patch Available (was: Open)
> JDBCStatsPublisher fails when ID length exceeds length of ID column
> -------------------------------------------------------------------
>
> Key: HIVE-3750
> URL: https://issues.apache.org/jira/browse/HIVE-3750
> Project: Hive
> Issue Type: Bug
> Components: Statistics
> Affects Versions: 0.11
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-3750.1.patch.txt
>
>
> When the length of the ID field passed to JDBCStatsPublisher exceeds the length of the column in the table (currently 255 characters) stats collection fails. This causes the entire query to fail when hive.stats.reliable is set to true.
> One way to prevent this would be to calculate a deterministic, very low collision hash of the ID prefix used for aggregation and use that when the length of the ID is too long.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira