You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hivemall.apache.org by myui <gi...@git.apache.org> on 2017/09/12 10:32:17 UTC
[GitHub] incubator-hivemall pull request #114: [HIVEMALL-138-2] refactored to_ordered...
GitHub user myui opened a pull request:
https://github.com/apache/incubator-hivemall/pull/114
[HIVEMALL-138-2] refactored to_ordered_map & to_ordered_list UDAF
## What changes were proposed in this pull request?
This PR applies refactoring to #108
## What type of PR is it?
[Feature | Refactoring]
## What is the Jira issue?
https://issues.apache.org/jira/browse/HIVEMALL-138
## How was this patch tested?
unit tests, manual tests
## How to use this feature?
see https://github.com/apache/incubator-hivemall/pull/108
## Checklist
- [x] Did you apply source code formatter, i.e., `mvn formatter:format`, for your commit?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/myui/incubator-hivemall HIVEMALL-138-2
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-hivemall/pull/114.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #114
----
commit 9cd3c59aebb67cc6b58cdd611b96fcf42f297cde
Author: Takuya Kitazawa <k....@gmail.com>
Date: 2017-09-11T06:38:05Z
Close #108: [HIVEMALL-138] to_ordered_map & to_ordered_list as a UDAF variant of each_top_k
commit 44528a83b065aa50255c751fb37041d1adbeb558
Author: Makoto Yui <my...@apache.org>
Date: 2017-09-12T10:22:59Z
Fixed to_ordered_map description
commit 21a87814895c9c63479a049387dd53b34cc5c56e
Author: Makoto Yui <my...@apache.org>
Date: 2017-09-12T10:24:25Z
Added utility classes and methods
commit 69730f65d76b54890b141ee13567eb681c0374ae
Author: Makoto Yui <my...@apache.org>
Date: 2017-09-12T10:26:49Z
Refactored to_ordered_list and to_ordered_map UDAF
----
---
[GitHub] incubator-hivemall issue #114: [HIVEMALL-138-2] refactored to_ordered_map & ...
Posted by myui <gi...@git.apache.org>.
Github user myui commented on the issue:
https://github.com/apache/incubator-hivemall/pull/114
@takuti applied some refactoring.
---
[GitHub] incubator-hivemall issue #114: [HIVEMALL-138-2] refactored to_ordered_map & ...
Posted by myui <gi...@git.apache.org>.
Github user myui commented on the issue:
https://github.com/apache/incubator-hivemall/pull/114
oops... I forget that default constructor is mandatory in UDAF.
```
Caused by: java.lang.RuntimeException: java.lang.NoSuchMethodException: hivemall.tools.map.UDAFToOrderedMap$DescendingMapEvaluator.<init>()
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:134)
at org.apache.hadoop.hive.ql.plan.AggregationDesc.getGenericUDAFEvaluator(AggregationDesc.java:92)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:350)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:366)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:556)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:508)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:556)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:508)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:501)
at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:317)
... 15 more
Caused by: java.lang.NoSuchMethodException: hivemall.tools.map.UDAFToOrderedMap$DescendingMapEvaluator.<init>()
at java.lang.Class.getConstructor0(Class.java:3082)
at java.lang.Class.getDeclaredConstructor(Class.java:2178)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
... 26 more
```
---
[GitHub] incubator-hivemall issue #114: [HIVEMALL-138-2] refactored to_ordered_map & ...
Posted by myui <gi...@git.apache.org>.
Github user myui commented on the issue:
https://github.com/apache/incubator-hivemall/pull/114
@takuti reverted some changes by me and merged.
---
[GitHub] incubator-hivemall issue #114: [HIVEMALL-138-2] refactored to_ordered_map & ...
Posted by myui <gi...@git.apache.org>.
Github user myui commented on the issue:
https://github.com/apache/incubator-hivemall/pull/114
Ah... It was Hive v2.3.0 bug and not related to estimate().
https://issues.apache.org/jira/browse/HIVE-17406
---
[GitHub] incubator-hivemall issue #114: [HIVEMALL-138-2] refactored to_ordered_map & ...
Posted by coveralls <gi...@git.apache.org>.
Github user coveralls commented on the issue:
https://github.com/apache/incubator-hivemall/pull/114
[](https://coveralls.io/builds/13251185)
Coverage increased (+0.3%) to 40.536% when pulling **df39bd5c8254db064ea2f54e7fce6d5e3f363961 on myui:HIVEMALL-138-2** into **3804789168dab5c5d43aac1fd4000e07688c6a06 on apache:master**.
---
[GitHub] incubator-hivemall pull request #114: [HIVEMALL-138-2] refactored to_ordered...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-hivemall/pull/114
---
[GitHub] incubator-hivemall issue #114: [HIVEMALL-138-2] refactored to_ordered_map & ...
Posted by myui <gi...@git.apache.org>.
Github user myui commented on the issue:
https://github.com/apache/incubator-hivemall/pull/114
```
FAILED: IllegalArgumentException Size requested for unknown type: hivemall.tools.list.UDAFToOrderedList$UDAFToOrderedListEvaluator$AbstractQueueHandler
```
failed in Hive v2.3.0 because UDAF estimate() is lacking.
---