You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Sergey Shelukhin <se...@hortonworks.com> on 2013/12/18 03:05:42 UTC
Review Request 16339: HIVE-6052 metastore JDO filter pushdown for integers
may produce unexpected results with non-normalized integer columns
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16339/
-----------------------------------------------------------
Review request for hive and Ashutosh Chauhan.
Repository: hive-git
Description
-------
see JIRA
Diffs
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java fa3e048
metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java a98d9d1
metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 04d399f
metastore/src/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java 93e9942
Diff: https://reviews.apache.org/r/16339/diff/
Testing
-------
Thanks,
Sergey Shelukhin
Re: Review Request 16339: HIVE-6052 metastore JDO filter pushdown for
integers may produce unexpected results with non-normalized integer columns
Posted by Sergey Shelukhin <se...@hortonworks.com>.
> On Dec. 19, 2013, 11:43 a.m., Ashutosh Chauhan wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 345
> > <https://reviews.apache.org/r/16339/diff/2/?file=399690#file399690line345>
> >
> > This should be on by default. After HIVE-5297 storing column values in non-canonical forms should not happen.
As far as I see, the jira doesn't do normalization, just verification. On trunk the issue still happens
> On Dec. 19, 2013, 11:43 a.m., Ashutosh Chauhan wrote:
> > ql/src/test/queries/clientpositive/alter_partition_coltype.q, line 1
> > <https://reviews.apache.org/r/16339/diff/2/?file=399694#file399694line1>
> >
> > This should work even with config set to true. No?
yeah, forgot to remove
- Sergey
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16339/#review30694
-----------------------------------------------------------
On Dec. 19, 2013, 2:17 a.m., Sergey Shelukhin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16339/
> -----------------------------------------------------------
>
> (Updated Dec. 19, 2013, 2:17 a.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> see JIRA
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java fa3e048
> metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java a98d9d1
> metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 04d399f
> metastore/src/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java 93e9942
> ql/src/test/queries/clientpositive/alter_partition_coltype.q 5479afb
> ql/src/test/queries/clientpositive/annotate_stats_part.q 83510e3
> ql/src/test/queries/clientpositive/dynamic_partition_skip_default.q 397a220
> ql/src/test/results/clientpositive/alter_partition_coltype.q.out 27b1fbc
> ql/src/test/results/clientpositive/annotate_stats_part.q.out 87fb980
> ql/src/test/results/clientpositive/dynamic_partition_skip_default.q.out baee525
>
> Diff: https://reviews.apache.org/r/16339/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sergey Shelukhin
>
>
Re: Review Request 16339: HIVE-6052 metastore JDO filter pushdown for
integers may produce unexpected results with non-normalized integer columns
Posted by Ashutosh Chauhan <ha...@apache.org>.
> On Dec. 19, 2013, 11:43 a.m., Ashutosh Chauhan wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 345
> > <https://reviews.apache.org/r/16339/diff/2/?file=399690#file399690line345>
> >
> > This should be on by default. After HIVE-5297 storing column values in non-canonical forms should not happen.
>
> Sergey Shelukhin wrote:
> As far as I see, the jira doesn't do normalization, just verification. On trunk the issue still happens
So will it result in incorrect result or exception? If this is exception, I will suggest to have config on by default. If it silently results in incorrect results, than off by default is better.
- Ashutosh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16339/#review30694
-----------------------------------------------------------
On Dec. 19, 2013, 2:17 a.m., Sergey Shelukhin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16339/
> -----------------------------------------------------------
>
> (Updated Dec. 19, 2013, 2:17 a.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> see JIRA
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java fa3e048
> metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java a98d9d1
> metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 04d399f
> metastore/src/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java 93e9942
> ql/src/test/queries/clientpositive/alter_partition_coltype.q 5479afb
> ql/src/test/queries/clientpositive/annotate_stats_part.q 83510e3
> ql/src/test/queries/clientpositive/dynamic_partition_skip_default.q 397a220
> ql/src/test/results/clientpositive/alter_partition_coltype.q.out 27b1fbc
> ql/src/test/results/clientpositive/annotate_stats_part.q.out 87fb980
> ql/src/test/results/clientpositive/dynamic_partition_skip_default.q.out baee525
>
> Diff: https://reviews.apache.org/r/16339/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sergey Shelukhin
>
>
Re: Review Request 16339: HIVE-6052 metastore JDO filter pushdown for
integers may produce unexpected results with non-normalized integer columns
Posted by Sergey Shelukhin <se...@hortonworks.com>.
> On Dec. 19, 2013, 11:43 a.m., Ashutosh Chauhan wrote:
> > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 345
> > <https://reviews.apache.org/r/16339/diff/2/?file=399690#file399690line345>
> >
> > This should be on by default. After HIVE-5297 storing column values in non-canonical forms should not happen.
>
> Sergey Shelukhin wrote:
> As far as I see, the jira doesn't do normalization, just verification. On trunk the issue still happens
>
> Ashutosh Chauhan wrote:
> So will it result in incorrect result or exception? If this is exception, I will suggest to have config on by default. If it silently results in incorrect results, than off by default is better.
incorrect result, as in the original bugs. Also, even verification doesn't cover all cases of adding partitions
- Sergey
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16339/#review30694
-----------------------------------------------------------
On Dec. 19, 2013, 2:17 a.m., Sergey Shelukhin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16339/
> -----------------------------------------------------------
>
> (Updated Dec. 19, 2013, 2:17 a.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> see JIRA
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java fa3e048
> metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java a98d9d1
> metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 04d399f
> metastore/src/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java 93e9942
> ql/src/test/queries/clientpositive/alter_partition_coltype.q 5479afb
> ql/src/test/queries/clientpositive/annotate_stats_part.q 83510e3
> ql/src/test/queries/clientpositive/dynamic_partition_skip_default.q 397a220
> ql/src/test/results/clientpositive/alter_partition_coltype.q.out 27b1fbc
> ql/src/test/results/clientpositive/annotate_stats_part.q.out 87fb980
> ql/src/test/results/clientpositive/dynamic_partition_skip_default.q.out baee525
>
> Diff: https://reviews.apache.org/r/16339/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sergey Shelukhin
>
>
Re: Review Request 16339: HIVE-6052 metastore JDO filter pushdown for
integers may produce unexpected results with non-normalized integer columns
Posted by Ashutosh Chauhan <ha...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16339/#review30694
-----------------------------------------------------------
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
<https://reviews.apache.org/r/16339/#comment58798>
This should be on by default. After HIVE-5297 storing column values in non-canonical forms should not happen.
ql/src/test/queries/clientpositive/alter_partition_coltype.q
<https://reviews.apache.org/r/16339/#comment58799>
This should work even with config set to true. No?
- Ashutosh Chauhan
On Dec. 19, 2013, 2:17 a.m., Sergey Shelukhin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16339/
> -----------------------------------------------------------
>
> (Updated Dec. 19, 2013, 2:17 a.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> see JIRA
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/conf/HiveConf.java fa3e048
> metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java a98d9d1
> metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 04d399f
> metastore/src/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java 93e9942
> ql/src/test/queries/clientpositive/alter_partition_coltype.q 5479afb
> ql/src/test/queries/clientpositive/annotate_stats_part.q 83510e3
> ql/src/test/queries/clientpositive/dynamic_partition_skip_default.q 397a220
> ql/src/test/results/clientpositive/alter_partition_coltype.q.out 27b1fbc
> ql/src/test/results/clientpositive/annotate_stats_part.q.out 87fb980
> ql/src/test/results/clientpositive/dynamic_partition_skip_default.q.out baee525
>
> Diff: https://reviews.apache.org/r/16339/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sergey Shelukhin
>
>
Re: Review Request 16339: HIVE-6052 metastore JDO filter pushdown for
integers may produce unexpected results with non-normalized integer columns
Posted by Sergey Shelukhin <se...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16339/
-----------------------------------------------------------
(Updated Dec. 19, 2013, 2:17 a.m.)
Review request for hive and Ashutosh Chauhan.
Repository: hive-git
Description
-------
see JIRA
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java fa3e048
metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java a98d9d1
metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 04d399f
metastore/src/java/org/apache/hadoop/hive/metastore/parser/ExpressionTree.java 93e9942
ql/src/test/queries/clientpositive/alter_partition_coltype.q 5479afb
ql/src/test/queries/clientpositive/annotate_stats_part.q 83510e3
ql/src/test/queries/clientpositive/dynamic_partition_skip_default.q 397a220
ql/src/test/results/clientpositive/alter_partition_coltype.q.out 27b1fbc
ql/src/test/results/clientpositive/annotate_stats_part.q.out 87fb980
ql/src/test/results/clientpositive/dynamic_partition_skip_default.q.out baee525
Diff: https://reviews.apache.org/r/16339/diff/
Testing
-------
Thanks,
Sergey Shelukhin