You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Xuefu Zhang <xz...@cloudera.com> on 2013/11/22 22:42:46 UTC
Review Request 15804: HIVE-5866: Hive divide operator generates wrong
results in certain cases
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15804/
-----------------------------------------------------------
Review request for hive.
Bugs: HIVE-5866
https://issues.apache.org/jira/browse/HIVE-5866
Repository: hive-git
Description
-------
Fixed the problem. Added a unit test. Corrected the output of a few q tests.
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseNumeric.java a1015e9
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDivide.java 0b902e9
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMinus.java 538c07e
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMod.java 472e1dd
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMultiply.java 2e8d364
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPPlus.java 35f639e
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFPosMod.java 6b18303
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFOPDivide.java 581c1a8
ql/src/test/results/clientpositive/decimal_precision.q.out 2ee3578
ql/src/test/results/clientpositive/decimal_udf.q.out ed5bc65
ql/src/test/results/clientpositive/ql_rewrite_gbtoidx.q.out 83787ee
ql/src/test/results/clientpositive/vectorization_5.q.out 54aad90
ql/src/test/results/clientpositive/vectorization_short_regress.q.out c9296e1
Diff: https://reviews.apache.org/r/15804/diff/
Testing
-------
Thanks,
Xuefu Zhang
Re: Review Request 15804: HIVE-5866: Hive divide operator generates wrong
results in certain cases
Posted by Xuefu Zhang <xz...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15804/#review29332
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDivide.java
<https://reviews.apache.org/r/15804/#comment56518>
I don't think I get you. Could you give an example about how implementation here is different from sql server's? The decimal part of a decimal number is as important as the integer part in applications where decimal type is required. Otherwise, double might be better. Thus, a decimal number of a certain decimal type needs to comply with the type's precision/scale. I don't think we should store number 456.78 to a type decimal(6,4), about which we already concluded the discussion.
- Xuefu Zhang
On Nov. 22, 2013, 9:42 p.m., Xuefu Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15804/
> -----------------------------------------------------------
>
> (Updated Nov. 22, 2013, 9:42 p.m.)
>
>
> Review request for hive.
>
>
> Bugs: HIVE-5866
> https://issues.apache.org/jira/browse/HIVE-5866
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Fixed the problem. Added a unit test. Corrected the output of a few q tests.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseNumeric.java a1015e9
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDivide.java 0b902e9
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMinus.java 538c07e
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMod.java 472e1dd
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMultiply.java 2e8d364
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPPlus.java 35f639e
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFPosMod.java 6b18303
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFOPDivide.java 581c1a8
> ql/src/test/results/clientpositive/decimal_precision.q.out 2ee3578
> ql/src/test/results/clientpositive/decimal_udf.q.out ed5bc65
> ql/src/test/results/clientpositive/ql_rewrite_gbtoidx.q.out 83787ee
> ql/src/test/results/clientpositive/vectorization_5.q.out 54aad90
> ql/src/test/results/clientpositive/vectorization_short_regress.q.out c9296e1
>
> Diff: https://reviews.apache.org/r/15804/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Xuefu Zhang
>
>
Re: Review Request 15804: HIVE-5866: Hive divide operator generates wrong
results in certain cases
Posted by Prasad Mujumdar <pr...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15804/#review29643
-----------------------------------------------------------
Ship it!
LGTM.
I guess this will cause the some of the decimal results to change over 0.12, but this is the right way to compute it.
- Prasad Mujumdar
On Nov. 24, 2013, 4:13 p.m., Xuefu Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15804/
> -----------------------------------------------------------
>
> (Updated Nov. 24, 2013, 4:13 p.m.)
>
>
> Review request for hive.
>
>
> Bugs: HIVE-5866
> https://issues.apache.org/jira/browse/HIVE-5866
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Fixed the problem. Added a unit test. Corrected the output of a few q tests.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseNumeric.java a1015e9
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDivide.java 0b902e9
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMinus.java 538c07e
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMod.java 472e1dd
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMultiply.java 2e8d364
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPPlus.java 35f639e
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFPosMod.java 6b18303
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFOPDivide.java 581c1a8
> ql/src/test/queries/clientpositive/decimal_precision.q d46d3ce
> ql/src/test/queries/clientpositive/decimal_udf.q d8fd835
> ql/src/test/results/clientpositive/decimal_precision.q.out 2ee3578
> ql/src/test/results/clientpositive/decimal_udf.q.out ed5bc65
> ql/src/test/results/clientpositive/ql_rewrite_gbtoidx.q.out 83787ee
> ql/src/test/results/clientpositive/vectorization_5.q.out 54aad90
> ql/src/test/results/clientpositive/vectorization_short_regress.q.out c9296e1
>
> Diff: https://reviews.apache.org/r/15804/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Xuefu Zhang
>
>
Re: Review Request 15804: HIVE-5866: Hive divide operator generates wrong
results in certain cases
Posted by Xuefu Zhang <xz...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15804/
-----------------------------------------------------------
(Updated Nov. 24, 2013, 4:13 p.m.)
Review request for hive.
Bugs: HIVE-5866
https://issues.apache.org/jira/browse/HIVE-5866
Repository: hive-git
Description
-------
Fixed the problem. Added a unit test. Corrected the output of a few q tests.
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseNumeric.java a1015e9
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDivide.java 0b902e9
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMinus.java 538c07e
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMod.java 472e1dd
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMultiply.java 2e8d364
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPPlus.java 35f639e
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFPosMod.java 6b18303
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFOPDivide.java 581c1a8
ql/src/test/queries/clientpositive/decimal_precision.q d46d3ce
ql/src/test/queries/clientpositive/decimal_udf.q d8fd835
ql/src/test/results/clientpositive/decimal_precision.q.out 2ee3578
ql/src/test/results/clientpositive/decimal_udf.q.out ed5bc65
ql/src/test/results/clientpositive/ql_rewrite_gbtoidx.q.out 83787ee
ql/src/test/results/clientpositive/vectorization_5.q.out 54aad90
ql/src/test/results/clientpositive/vectorization_short_regress.q.out c9296e1
Diff: https://reviews.apache.org/r/15804/diff/
Testing
-------
Thanks,
Xuefu Zhang
Re: Review Request 15804: HIVE-5866: Hive divide operator generates wrong
results in certain cases
Posted by Xuefu Zhang <xz...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15804/
-----------------------------------------------------------
(Updated Nov. 24, 2013, 7:27 a.m.)
Review request for hive.
Bugs: HIVE-5866
https://issues.apache.org/jira/browse/HIVE-5866
Repository: hive-git
Description
-------
Fixed the problem. Added a unit test. Corrected the output of a few q tests.
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseNumeric.java a1015e9
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDivide.java 0b902e9
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMinus.java 538c07e
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMod.java 472e1dd
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMultiply.java 2e8d364
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPPlus.java 35f639e
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFPosMod.java 6b18303
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFOPDivide.java 581c1a8
ql/src/test/queries/clientpositive/decimal_precision.q d46d3ce
ql/src/test/queries/clientpositive/decimal_udf.q d8fd835
ql/src/test/results/clientpositive/decimal_precision.q.out 2ee3578
ql/src/test/results/clientpositive/decimal_udf.q.out ed5bc65
ql/src/test/results/clientpositive/ql_rewrite_gbtoidx.q.out 83787ee
ql/src/test/results/clientpositive/vectorization_5.q.out 54aad90
ql/src/test/results/clientpositive/vectorization_short_regress.q.out c9296e1
Diff: https://reviews.apache.org/r/15804/diff/
Testing
-------
Thanks,
Xuefu Zhang
Re: Review Request 15804: HIVE-5866: Hive divide operator generates wrong
results in certain cases
Posted by Jason Dere <jd...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15804/#review29330
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDivide.java
<https://reviews.apache.org/r/15804/#comment56517>
So I think the issue here is that when (integer_prec + scale) > max_precision, we prioritize keeping the scale at the expense of the integer portion of the result type. Looks like the SQL Server precision/scale rules mention that it does not let the scale eat into the integer portion of the result type - it goes the other way and will reduce the scale to allow the total precision to fit within max_precision. This might be a better rule to follow than prioritizing the scale value, at least for the purposes of determining the return type.
- Jason Dere
On Nov. 22, 2013, 9:42 p.m., Xuefu Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15804/
> -----------------------------------------------------------
>
> (Updated Nov. 22, 2013, 9:42 p.m.)
>
>
> Review request for hive.
>
>
> Bugs: HIVE-5866
> https://issues.apache.org/jira/browse/HIVE-5866
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Fixed the problem. Added a unit test. Corrected the output of a few q tests.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseNumeric.java a1015e9
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDivide.java 0b902e9
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMinus.java 538c07e
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMod.java 472e1dd
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMultiply.java 2e8d364
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPPlus.java 35f639e
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFPosMod.java 6b18303
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFOPDivide.java 581c1a8
> ql/src/test/results/clientpositive/decimal_precision.q.out 2ee3578
> ql/src/test/results/clientpositive/decimal_udf.q.out ed5bc65
> ql/src/test/results/clientpositive/ql_rewrite_gbtoidx.q.out 83787ee
> ql/src/test/results/clientpositive/vectorization_5.q.out 54aad90
> ql/src/test/results/clientpositive/vectorization_short_regress.q.out c9296e1
>
> Diff: https://reviews.apache.org/r/15804/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Xuefu Zhang
>
>