You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Krisztian Kasa <kk...@hortonworks.com> on 2019/10/02 05:28:50 UTC
Review Request 71571: HIVE-22240
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71571/
-----------------------------------------------------------
Review request for hive, Jesús Camacho Rodríguez, Zoltan Haindrich, and Vineet Garg.
Bugs: HIVE-22240
https://issues.apache.org/jira/browse/HIVE-22240
Repository: hive-git
Description
-------
Function percentile_cont fails when array parameter passed
==========================================================
The method `GenericUDAFPercentileCont.iterate` expect a `HiveDecimalWritable` parameter. However array parameters are converted to `ArrayList<HiveDecimalWritable>` before they passed. This leads to a `ClassCastException`.
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileCont.java e7e4fda6ea
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileDisc.java d7c295cb11
ql/src/test/queries/clientpositive/udaf_percentile_cont.q 7a1ac5c968
ql/src/test/queries/clientpositive/udaf_percentile_disc.q 6d93b3418a
ql/src/test/results/clientpositive/udaf_percentile_cont.q.out a2dae4a06e
ql/src/test/results/clientpositive/udaf_percentile_disc.q.out 3ef0cf9874
Diff: https://reviews.apache.org/r/71571/diff/1/
Testing
-------
Existing q tests are extended with calling percentile_cont/disc with array parameters
Thanks,
Krisztian Kasa
Re: Review Request 71571: HIVE-22240
Posted by Jesús Camacho Rodríguez <jc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71571/#review218204
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileCont.java
Line 208 (original), 213 (patched)
<https://reviews.apache.org/r/71571/#comment305775>
With the previous comment, I meant we can preallocate the results here array/list here and populate it, then we can just call `set` on each of them from `calculatePercentile`?
- Jesús Camacho Rodríguez
On Oct. 14, 2019, 9:15 a.m., Krisztian Kasa wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71571/
> -----------------------------------------------------------
>
> (Updated Oct. 14, 2019, 9:15 a.m.)
>
>
> Review request for hive, Jesús Camacho Rodríguez, Zoltan Haindrich, and Vineet Garg.
>
>
> Bugs: HIVE-22240
> https://issues.apache.org/jira/browse/HIVE-22240
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Function percentile_cont fails when array parameter passed
> ==========================================================
> The method `GenericUDAFPercentileCont.iterate` expect a `HiveDecimalWritable` parameter. However array parameters are converted to `ArrayList<HiveDecimalWritable>` before they passed. This leads to a `ClassCastException`.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileCont.java 00e17f92da
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileDisc.java 21580f79cb
> ql/src/test/queries/clientpositive/udaf_percentile_cont.q 7a1ac5c968
> ql/src/test/queries/clientpositive/udaf_percentile_disc.q 6d93b3418a
> ql/src/test/results/clientpositive/udaf_percentile_cont.q.out cddd908d73
> ql/src/test/results/clientpositive/udaf_percentile_disc.q.out 83875ec514
>
>
> Diff: https://reviews.apache.org/r/71571/diff/3/
>
>
> Testing
> -------
>
> Existing q tests are extended with calling percentile_cont/disc with array parameters
>
>
> Thanks,
>
> Krisztian Kasa
>
>
Re: Review Request 71571: HIVE-22240
Posted by Krisztian Kasa <kk...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71571/
-----------------------------------------------------------
(Updated Oct. 15, 2019, 5:09 a.m.)
Review request for hive, Jesús Camacho Rodríguez, Zoltan Haindrich, and Vineet Garg.
Bugs: HIVE-22240
https://issues.apache.org/jira/browse/HIVE-22240
Repository: hive-git
Description
-------
Function percentile_cont fails when array parameter passed
==========================================================
The method `GenericUDAFPercentileCont.iterate` expect a `HiveDecimalWritable` parameter. However array parameters are converted to `ArrayList<HiveDecimalWritable>` before they passed. This leads to a `ClassCastException`.
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileCont.java 00e17f92da
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileDisc.java 21580f79cb
ql/src/test/queries/clientpositive/udaf_percentile_cont.q 7a1ac5c968
ql/src/test/queries/clientpositive/udaf_percentile_disc.q 6d93b3418a
ql/src/test/results/clientpositive/udaf_percentile_cont.q.out cddd908d73
ql/src/test/results/clientpositive/udaf_percentile_disc.q.out 83875ec514
Diff: https://reviews.apache.org/r/71571/diff/4/
Changes: https://reviews.apache.org/r/71571/diff/3-4/
Testing
-------
Existing q tests are extended with calling percentile_cont/disc with array parameters
Thanks,
Krisztian Kasa
Re: Review Request 71571: HIVE-22240
Posted by Krisztian Kasa <kk...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71571/
-----------------------------------------------------------
(Updated Oct. 14, 2019, 9:15 a.m.)
Review request for hive, Jesús Camacho Rodríguez, Zoltan Haindrich, and Vineet Garg.
Bugs: HIVE-22240
https://issues.apache.org/jira/browse/HIVE-22240
Repository: hive-git
Description
-------
Function percentile_cont fails when array parameter passed
==========================================================
The method `GenericUDAFPercentileCont.iterate` expect a `HiveDecimalWritable` parameter. However array parameters are converted to `ArrayList<HiveDecimalWritable>` before they passed. This leads to a `ClassCastException`.
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileCont.java 00e17f92da
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileDisc.java 21580f79cb
ql/src/test/queries/clientpositive/udaf_percentile_cont.q 7a1ac5c968
ql/src/test/queries/clientpositive/udaf_percentile_disc.q 6d93b3418a
ql/src/test/results/clientpositive/udaf_percentile_cont.q.out cddd908d73
ql/src/test/results/clientpositive/udaf_percentile_disc.q.out 83875ec514
Diff: https://reviews.apache.org/r/71571/diff/3/
Changes: https://reviews.apache.org/r/71571/diff/2-3/
Testing
-------
Existing q tests are extended with calling percentile_cont/disc with array parameters
Thanks,
Krisztian Kasa
Re: Review Request 71571: HIVE-22240
Posted by Jesús Camacho Rodríguez <jc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71571/#review218195
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileCont.java
Lines 343 (patched)
<https://reviews.apache.org/r/71571/#comment305751>
New logic is creating new objects for each partial result instead of reusing a single object array as it was done before. Any reason for this change? It may result in GC spike.
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileCont.java
Line 385 (original), 371 (patched)
<https://reviews.apache.org/r/71571/#comment305752>
We shoul preallocate the `results` array when we initialize the udf instead of creating a new one per call. Convert results can just act on the array based on a boolean set at initialization time.
- Jesús Camacho Rodríguez
On Oct. 2, 2019, 7:41 a.m., Krisztian Kasa wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71571/
> -----------------------------------------------------------
>
> (Updated Oct. 2, 2019, 7:41 a.m.)
>
>
> Review request for hive, Jesús Camacho Rodríguez, Zoltan Haindrich, and Vineet Garg.
>
>
> Bugs: HIVE-22240
> https://issues.apache.org/jira/browse/HIVE-22240
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Function percentile_cont fails when array parameter passed
> ==========================================================
> The method `GenericUDAFPercentileCont.iterate` expect a `HiveDecimalWritable` parameter. However array parameters are converted to `ArrayList<HiveDecimalWritable>` before they passed. This leads to a `ClassCastException`.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileCont.java e7e4fda6ea
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileDisc.java d7c295cb11
> ql/src/test/queries/clientpositive/udaf_percentile_cont.q 7a1ac5c968
> ql/src/test/queries/clientpositive/udaf_percentile_disc.q 6d93b3418a
> ql/src/test/results/clientpositive/udaf_percentile_cont.q.out a2dae4a06e
> ql/src/test/results/clientpositive/udaf_percentile_disc.q.out 3ef0cf9874
>
>
> Diff: https://reviews.apache.org/r/71571/diff/2/
>
>
> Testing
> -------
>
> Existing q tests are extended with calling percentile_cont/disc with array parameters
>
>
> Thanks,
>
> Krisztian Kasa
>
>
Re: Review Request 71571: HIVE-22240
Posted by Krisztian Kasa <kk...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71571/
-----------------------------------------------------------
(Updated Oct. 2, 2019, 7:41 a.m.)
Review request for hive, Jesús Camacho Rodríguez, Zoltan Haindrich, and Vineet Garg.
Bugs: HIVE-22240
https://issues.apache.org/jira/browse/HIVE-22240
Repository: hive-git
Description
-------
Function percentile_cont fails when array parameter passed
==========================================================
The method `GenericUDAFPercentileCont.iterate` expect a `HiveDecimalWritable` parameter. However array parameters are converted to `ArrayList<HiveDecimalWritable>` before they passed. This leads to a `ClassCastException`.
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileCont.java e7e4fda6ea
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFPercentileDisc.java d7c295cb11
ql/src/test/queries/clientpositive/udaf_percentile_cont.q 7a1ac5c968
ql/src/test/queries/clientpositive/udaf_percentile_disc.q 6d93b3418a
ql/src/test/results/clientpositive/udaf_percentile_cont.q.out a2dae4a06e
ql/src/test/results/clientpositive/udaf_percentile_disc.q.out 3ef0cf9874
Diff: https://reviews.apache.org/r/71571/diff/2/
Changes: https://reviews.apache.org/r/71571/diff/1-2/
Testing
-------
Existing q tests are extended with calling percentile_cont/disc with array parameters
Thanks,
Krisztian Kasa