You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2015/06/03 16:07:39 UTC
[jira] [Commented] (FLINK-2135) Java plan translation fails with
ClassCastException (probably in first())
[ https://issues.apache.org/jira/browse/FLINK-2135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14570875#comment-14570875 ]
ASF GitHub Bot commented on FLINK-2135:
---------------------------------------
GitHub user rmetzger opened a pull request:
https://github.com/apache/flink/pull/769
[FLINK-2135] Fix faulty cast to GroupReduceFunction
There seem to be two variants of that tuple unwrapping thing, one combinable, the other one non-combinable.
These unwrapping things use a helper class called `WrappingFunction`, which expects the type of the user function.
In this case, we need a combinable group reduce function. It seems that only RichGroupReduceFunction is combinable AND a GroupReduceFunction at the same time.
But users can also implement non-rich combinable group reduce functions (by implementing both interfaces).
The `PlanUnwrappingSortedReduceGroupOperator` was just casting the user function to a `RichGroupReduceFunction`, which will fail if the UDF is not a Rich one.
To avoid making the code uglier than it already is, I've decided to cast the wrapped function to a `GroupCombineFunction` for calling the combiner. The `TupleUnwrappingGroupCombinableGroupReducer` is only called for Combinable UDFs anyways.
Lets see if travis gives me green light for this fix.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rmetzger/flink flink2135
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/769.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #769
----
commit f4e66bd84ab7f0c47505336217f1ed3e86d4069c
Author: Robert Metzger <rm...@apache.org>
Date: 2015-06-03T13:52:12Z
[FLINK-2135] Fix faulty cast to GroupReduceFunction
----
> Java plan translation fails with ClassCastException (probably in first())
> -------------------------------------------------------------------------
>
> Key: FLINK-2135
> URL: https://issues.apache.org/jira/browse/FLINK-2135
> Project: Flink
> Issue Type: Bug
> Components: Java API
> Affects Versions: 0.9
> Reporter: Robert Metzger
> Assignee: Robert Metzger
>
> A user reported the following error
> {code}
> Exception in thread "main" java.lang.ClassCastException: org.apache.flink.api.java.functions.FirstReducer cannot be cast to org.apache.flink.api.common.functions.RichGroupReduceFunction
> at org.apache.flink.api.java.operators.translation.PlanUnwrappingSortedReduceGroupOperator.<init>(PlanUnwrappingSortedReduceGroupOperator.java:40)
> at org.apache.flink.api.java.operators.GroupReduceOperator.translateSelectorFunctionSortedReducer(GroupReduceOperator.java:278)
> at org.apache.flink.api.java.operators.GroupReduceOperator.translateToDataFlow(GroupReduceOperator.java:177)
> at org.apache.flink.api.java.operators.GroupReduceOperator.translateToDataFlow(GroupReduceOperator.java:50)
> at org.apache.flink.api.java.operators.OperatorTranslation.translateSingleInputOperator(OperatorTranslation.java:124)
> at org.apache.flink.api.java.operators.OperatorTranslation.translate(OperatorTranslation.java:86)
> at org.apache.flink.api.java.operators.OperatorTranslation.translateSingleInputOperator(OperatorTranslation.java:122)
> at org.apache.flink.api.java.operators.OperatorTranslation.translate(OperatorTranslation.java:86)
> at org.apache.flink.api.java.operators.OperatorTranslation.translateSingleInputOperator(OperatorTranslation.java:122)
> at org.apache.flink.api.java.operators.OperatorTranslation.translate(OperatorTranslation.java:86)
> at org.apache.flink.api.java.operators.OperatorTranslation.translate(OperatorTranslation.java:61)
> at org.apache.flink.api.java.operators.OperatorTranslation.translateToPlan(OperatorTranslation.java:49)
> at org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:925)
> at org.apache.flink.api.java.ExecutionEnvironment.createProgramPlan(ExecutionEnvironment.java:893)
> at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:50)
> at org.apache.flink.api.java.ExecutionEnvironment.execute(ExecutionEnvironment.java:789)
> at org.apache.flink.api.java.DataSet.collect(DataSet.java:411)
> at org.apache.flink.api.java.DataSet.print(DataSet.java:1346)
> at com.dataartisans.GroupReduceBug.main(GroupReduceBug.java:43)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
> {code}
> It is reproducible with the following code
> {code}
> ExecutionEnvironment ee = ExecutionEnvironment.getExecutionEnvironment();
> DataSet<String> b = ee.fromElements("a", "b");
> GroupReduceOperator<String, String> a = b.groupBy(new KeySelector<String, Long>() {
> @Override
> public Long getKey(String value) throws Exception {
> return 1L;
> }
> }).sortGroup(new KeySelector<String, Double>() {
> @Override
> public Double getKey(String value) throws Exception {
> return 1.0;
> }
> }, Order.DESCENDING).first(10);
> a.print();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)