You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Sebastian <ss...@googlemail.com> on 2015/02/10 10:05:12 UTC
Flink cannot compile my join code...
Hi,
I'm trying to write a simple join in flink-scala but for some reasons
flink fails to compile my code. I've tried several reformulations but
can't get rid of the error. Can you tell me how to fix this piece of
code? I'm using flink 0.8.0.
Sebastian
---------------------------------------------------------------------
Error:(24, 53) not enough arguments for method apply: (implicit
evidence$1:
org.apache.flink.api.common.typeinfo.TypeInformation[(String, Long)],
implicit evidence$2: scala.reflect.ClassTag[(String,
Long)])org.apache.flink.api.scala.DataSet[(String, Long)] in class
JoinDataSet.
Unspecified value parameters evidence$1, evidence$2.
topTrackers.join(domains).where(0).equalTo(1) { (topTracker,
domain) => domain.annotation -> topTracker._2 }
^
Error:(24, 53) could not find implicit value for evidence parameter of
type org.apache.flink.api.common.typeinfo.TypeInformation[(String, Long)]
topTrackers.join(domains).where(0).equalTo(1) { (topTracker,
domain) => domain.annotation -> topTracker._2 }
^
---------------------------------------------------------------------
def computeDistribution(trackingGraphFile: String,
domainIndexFile: String) = {
implicit val env = ExecutionEnvironment.getExecutionEnvironment
val edges = GraphUtils.readEdges(trackingGraphFile)
val domains = GraphUtils.readVertices(domainIndexFile)
val trackersWithNumDomainsTracked =
FlinkUtils.countByKey(edges, { edge: Edge => edge.src })
val topTrackers =
trackersWithNumDomainsTracked.filter { _._2 >= 10000 }
val topTrackerDomains =
topTrackers.join(domains).where(0).equalTo(1) {
(topTracker, domain) => domain.annotation -> topTracker._2 }
topTrackerDomains.print()
env.execute()
}
Re: Flink cannot compile my join code...
Posted by Sebastian <ss...@googlemail.com>.
Ah, I found an old jar in my classpath, thanks for the hint!
On 10.02.2015 12:12, Stephan Ewen wrote:
> This sounds very much like a version mixup. Can you check that you
> reference the same version for core, clients, runtime, ...?
>
> The exception is from the 0.8 RPC service. Are you using 0.8 everywhere?
>
> On Tue, Feb 10, 2015 at 12:09 PM, Sebastian <ssc.open@googlemail.com
> <ma...@googlemail.com>> wrote:
>
> I'm running it from within my IDE...
>
> On 10.02.2015 11:17, Robert Metzger wrote:
>
> Hi,
> the NoSuchMethodError exception indicates that there is a mixup
> of Flink
> versions.
> Are you running the job from within your IDE or are you
> submitting it to
> a cluster? Maybe the cluster has a different Flink version?
>
> On Tue, Feb 10, 2015 at 10:51 AM, Sebastian
> <ssc.open@googlemail.com <ma...@googlemail.com>
> <mailto:ssc.open@googlemail.__com
> <ma...@googlemail.com>>> wrote:
>
> Yes, the import was missing thank you for the hint!
>
> Now I'm getting the following error:
>
> java.io.IOException: java.lang.NoSuchMethodError:
>
> org.apache.flink.util.____ClassUtils.____isPrimitiveOrBoxedOrString(____Ljava/lang/Class;)Z
> at
> org.apache.flink.runtime.ipc.____RPC$Server.call(RPC.java:428)
> at
>
> org.apache.flink.runtime.ipc.____Server$Handler.run(Server.____java:949)
>
>
>
> On 10.02.2015 10:19, Aljoscha Krettek wrote:
>
> import org.apache.flink.api.scala._
>
>
>
Re: Flink cannot compile my join code...
Posted by Stephan Ewen <se...@apache.org>.
This sounds very much like a version mixup. Can you check that you
reference the same version for core, clients, runtime, ...?
The exception is from the 0.8 RPC service. Are you using 0.8 everywhere?
On Tue, Feb 10, 2015 at 12:09 PM, Sebastian <ss...@googlemail.com> wrote:
> I'm running it from within my IDE...
>
> On 10.02.2015 11:17, Robert Metzger wrote:
>
>> Hi,
>> the NoSuchMethodError exception indicates that there is a mixup of Flink
>> versions.
>> Are you running the job from within your IDE or are you submitting it to
>> a cluster? Maybe the cluster has a different Flink version?
>>
>> On Tue, Feb 10, 2015 at 10:51 AM, Sebastian <ssc.open@googlemail.com
>> <ma...@googlemail.com>> wrote:
>>
>> Yes, the import was missing thank you for the hint!
>>
>> Now I'm getting the following error:
>>
>> java.io.IOException: java.lang.NoSuchMethodError:
>> org.apache.flink.util.__ClassUtils.__isPrimitiveOrBoxedOrString(__
>> Ljava/lang/Class;)Z
>> at org.apache.flink.runtime.ipc._
>> _RPC$Server.call(RPC.java:428)
>> at
>> org.apache.flink.runtime.ipc.__Server$Handler.run(Server.__java:949)
>>
>>
>>
>> On 10.02.2015 10:19, Aljoscha Krettek wrote:
>>
>> import org.apache.flink.api.scala._
>>
>>
>>
Re: Flink cannot compile my join code...
Posted by Sebastian <ss...@googlemail.com>.
I'm running it from within my IDE...
On 10.02.2015 11:17, Robert Metzger wrote:
> Hi,
> the NoSuchMethodError exception indicates that there is a mixup of Flink
> versions.
> Are you running the job from within your IDE or are you submitting it to
> a cluster? Maybe the cluster has a different Flink version?
>
> On Tue, Feb 10, 2015 at 10:51 AM, Sebastian <ssc.open@googlemail.com
> <ma...@googlemail.com>> wrote:
>
> Yes, the import was missing thank you for the hint!
>
> Now I'm getting the following error:
>
> java.io.IOException: java.lang.NoSuchMethodError:
> org.apache.flink.util.__ClassUtils.__isPrimitiveOrBoxedOrString(__Ljava/lang/Class;)Z
> at org.apache.flink.runtime.ipc.__RPC$Server.call(RPC.java:428)
> at
> org.apache.flink.runtime.ipc.__Server$Handler.run(Server.__java:949)
>
>
>
> On 10.02.2015 10:19, Aljoscha Krettek wrote:
>
> import org.apache.flink.api.scala._
>
>
Re: Flink cannot compile my join code...
Posted by Robert Metzger <rm...@apache.org>.
Hi,
the NoSuchMethodError exception indicates that there is a mixup of Flink
versions.
Are you running the job from within your IDE or are you submitting it to a
cluster? Maybe the cluster has a different Flink version?
On Tue, Feb 10, 2015 at 10:51 AM, Sebastian <ss...@googlemail.com> wrote:
> Yes, the import was missing thank you for the hint!
>
> Now I'm getting the following error:
>
> java.io.IOException: java.lang.NoSuchMethodError: org.apache.flink.util.
> ClassUtils.isPrimitiveOrBoxedOrString(Ljava/lang/Class;)Z
> at org.apache.flink.runtime.ipc.RPC$Server.call(RPC.java:428)
> at org.apache.flink.runtime.ipc.Server$Handler.run(Server.
> java:949)
>
>
>
> On 10.02.2015 10:19, Aljoscha Krettek wrote:
>
>> import org.apache.flink.api.scala._
>>
>
Re: Flink cannot compile my join code...
Posted by Sebastian <ss...@googlemail.com>.
Yes, the import was missing thank you for the hint!
Now I'm getting the following error:
java.io.IOException: java.lang.NoSuchMethodError:
org.apache.flink.util.ClassUtils.isPrimitiveOrBoxedOrString(Ljava/lang/Class;)Z
at org.apache.flink.runtime.ipc.RPC$Server.call(RPC.java:428)
at org.apache.flink.runtime.ipc.Server$Handler.run(Server.java:949)
On 10.02.2015 10:19, Aljoscha Krettek wrote:
> import org.apache.flink.api.scala._
Re: Flink cannot compile my join code...
Posted by Aljoscha Krettek <al...@apache.org>.
Hi,
do you have this import?
import org.apache.flink.api.scala._
You could also just import
org.apache.flink.api.scala.createTypeInformation but most of the other
stuff in the package is required anyways.
Cheers,
Aljoscha
On Tue, Feb 10, 2015 at 10:05 AM, Sebastian <ss...@googlemail.com> wrote:
> Hi,
>
> I'm trying to write a simple join in flink-scala but for some reasons flink
> fails to compile my code. I've tried several reformulations but can't get
> rid of the error. Can you tell me how to fix this piece of code? I'm using
> flink 0.8.0.
>
> Sebastian
>
> ---------------------------------------------------------------------
>
> Error:(24, 53) not enough arguments for method apply: (implicit evidence$1:
> org.apache.flink.api.common.typeinfo.TypeInformation[(String, Long)],
> implicit evidence$2: scala.reflect.ClassTag[(String,
> Long)])org.apache.flink.api.scala.DataSet[(String, Long)] in class
> JoinDataSet.
> Unspecified value parameters evidence$1, evidence$2.
> topTrackers.join(domains).where(0).equalTo(1) { (topTracker, domain)
> => domain.annotation -> topTracker._2 }
> ^
> Error:(24, 53) could not find implicit value for evidence parameter of type
> org.apache.flink.api.common.typeinfo.TypeInformation[(String, Long)]
> topTrackers.join(domains).where(0).equalTo(1) { (topTracker, domain)
> => domain.annotation -> topTracker._2 }
> ^
> ---------------------------------------------------------------------
>
> def computeDistribution(trackingGraphFile: String,
> domainIndexFile: String) = {
>
> implicit val env = ExecutionEnvironment.getExecutionEnvironment
>
> val edges = GraphUtils.readEdges(trackingGraphFile)
> val domains = GraphUtils.readVertices(domainIndexFile)
>
> val trackersWithNumDomainsTracked =
> FlinkUtils.countByKey(edges, { edge: Edge => edge.src })
>
> val topTrackers =
> trackersWithNumDomainsTracked.filter { _._2 >= 10000 }
>
> val topTrackerDomains =
> topTrackers.join(domains).where(0).equalTo(1) {
> (topTracker, domain) => domain.annotation -> topTracker._2 }
>
> topTrackerDomains.print()
>
> env.execute()
> }
>