You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Vinay Patil <vi...@gmail.com> on 2016/06/10 16:35:26 UTC
[Issue] No Class Definition Found Error
Hi Guys,
I have deployed my application on a cluster, however when I try to run the
application it throws *NoClassDefFoundError for KeyedDeserializationSchema*,
all the dependencies are provided correctly since I have run it on a
different standalone node.
Please Help
Regards,
Vinay Patil
Re: [Issue] No Class Definition Found Error
Posted by Vinay Patil <vi...@gmail.com>.
Yes correct. Thank you for this suggestion.
This is the first time I am working on distributed application, so having
such basic questions
Now I am getting object is not serializable exception. I have generated the
serialUId for that class.
On Jun 12, 2016 11:28 AM, "Aljoscha Krettek" <al...@apache.org> wrote:
> Hi,
> the static field will be null after the code has been shipped to the
> cluster. You can use a RichMapFunction instead, there you can define an
> open() method to initialize such a field. Please see this part of the doc
> for rich functions:
>
> https://ci.apache.org/projects/flink/flink-docs-master/apis/common/index.html#rich-functions
>
> Cheers,
> Aljoscha
>
> On Sat, 11 Jun 2016 at 21:07 Vinay Patil <vi...@gmail.com> wrote:
>
> > Issue resolved.
> >
> > Created the uber jar (fat jar) as shown in the flink-quickstart guide.
> > My bad, I should have checked that first.
> >
> > However now I am facing NullPointerException , consider the following
> > example (this is the dummy code , since I cannot share the actual code):
> >
> > public class App implements MapFunction<String, List<String>> {
> > private static TempClass tempClass;
> >
> > public App() {
> > tempClass = new TempClass();
> > }
> >
> > @Override
> > public List<String> map(String dummyString) throws Exception {
> > *tempClass.doSomething(dummyString); // getting NPE
> here*
> > }
> > }
> >
> > I am not getting NPE when I run it on a single machine, however when I
> run
> > on a cluster , it gives me NPE.
> > Instantiating tempClass inside the map function will fix the issue ,
> > however my reviewer suggested not to initialize it inside the operator as
> > it will get initialized always, so did it in the constructor.
> >
> >
> > Also, when I looked at the flink examples, you have used static classes
> and
> > accessed the functions inside operators, so which is the better approach
> > instantiating inside the operators or use static class ?
> >
> >
> >
> > Regards,
> > Vinay Patil
> >
> > *+91-800-728-4749*
> >
> > On Sat, Jun 11, 2016 at 9:32 AM, Vinay Patil <vi...@gmail.com>
> > wrote:
> >
> > > Hi Robert,
> > >
> > > Yes we are using maven for building the jar, I have deployed both jar
> > with
> > > dependencies and without dependencies.
> > >
> > > I actually cannot share the pom since it is on the client machine.
> > > But all the dependencies required are there, I have attached a sample
> pom
> > > file which is similar to the pom we are using
> > >
> > > Regards,
> > > Vinay Patil
> > >
> > > *+91-800-728-4749*
> > >
> > > On Sat, Jun 11, 2016 at 1:36 AM, Robert Metzger <rm...@apache.org>
> > > wrote:
> > >
> > >> Are you using Maven for building your job jar?
> > >> If yes, can you post your pom file on the mailing list?
> > >>
> > >> On Fri, Jun 10, 2016 at 7:16 PM, THORMAN, ROBERT D <rt...@att.com>
> > >> wrote:
> > >>
> > >> > How did you “provide” the dependencies? Did you use the –C <URL>
> > >> > parameter when you submitted your job?
> > >> >
> > >> > On 6/10/16, 11:35 AM, "Vinay Patil" <vi...@gmail.com>
> wrote:
> > >> >
> > >> > >Hi Guys,
> > >> > >
> > >> > >I have deployed my application on a cluster, however when I try to
> > run
> > >> the
> > >> > >application it throws *NoClassDefFoundError for
> > >> > KeyedDeserializationSchema*,
> > >> > >all the dependencies are provided correctly since I have run it on
> a
> > >> > >different standalone node.
> > >> > >
> > >> > >Please Help
> > >> > >
> > >> > >Regards,
> > >> > >Vinay Patil
> > >> >
> > >> >
> > >>
> > >
> > >
> >
>
Re: [Issue] No Class Definition Found Error
Posted by Aljoscha Krettek <al...@apache.org>.
Hi,
the static field will be null after the code has been shipped to the
cluster. You can use a RichMapFunction instead, there you can define an
open() method to initialize such a field. Please see this part of the doc
for rich functions:
https://ci.apache.org/projects/flink/flink-docs-master/apis/common/index.html#rich-functions
Cheers,
Aljoscha
On Sat, 11 Jun 2016 at 21:07 Vinay Patil <vi...@gmail.com> wrote:
> Issue resolved.
>
> Created the uber jar (fat jar) as shown in the flink-quickstart guide.
> My bad, I should have checked that first.
>
> However now I am facing NullPointerException , consider the following
> example (this is the dummy code , since I cannot share the actual code):
>
> public class App implements MapFunction<String, List<String>> {
> private static TempClass tempClass;
>
> public App() {
> tempClass = new TempClass();
> }
>
> @Override
> public List<String> map(String dummyString) throws Exception {
> *tempClass.doSomething(dummyString); // getting NPE here*
> }
> }
>
> I am not getting NPE when I run it on a single machine, however when I run
> on a cluster , it gives me NPE.
> Instantiating tempClass inside the map function will fix the issue ,
> however my reviewer suggested not to initialize it inside the operator as
> it will get initialized always, so did it in the constructor.
>
>
> Also, when I looked at the flink examples, you have used static classes and
> accessed the functions inside operators, so which is the better approach
> instantiating inside the operators or use static class ?
>
>
>
> Regards,
> Vinay Patil
>
> *+91-800-728-4749*
>
> On Sat, Jun 11, 2016 at 9:32 AM, Vinay Patil <vi...@gmail.com>
> wrote:
>
> > Hi Robert,
> >
> > Yes we are using maven for building the jar, I have deployed both jar
> with
> > dependencies and without dependencies.
> >
> > I actually cannot share the pom since it is on the client machine.
> > But all the dependencies required are there, I have attached a sample pom
> > file which is similar to the pom we are using
> >
> > Regards,
> > Vinay Patil
> >
> > *+91-800-728-4749*
> >
> > On Sat, Jun 11, 2016 at 1:36 AM, Robert Metzger <rm...@apache.org>
> > wrote:
> >
> >> Are you using Maven for building your job jar?
> >> If yes, can you post your pom file on the mailing list?
> >>
> >> On Fri, Jun 10, 2016 at 7:16 PM, THORMAN, ROBERT D <rt...@att.com>
> >> wrote:
> >>
> >> > How did you “provide” the dependencies? Did you use the –C <URL>
> >> > parameter when you submitted your job?
> >> >
> >> > On 6/10/16, 11:35 AM, "Vinay Patil" <vi...@gmail.com> wrote:
> >> >
> >> > >Hi Guys,
> >> > >
> >> > >I have deployed my application on a cluster, however when I try to
> run
> >> the
> >> > >application it throws *NoClassDefFoundError for
> >> > KeyedDeserializationSchema*,
> >> > >all the dependencies are provided correctly since I have run it on a
> >> > >different standalone node.
> >> > >
> >> > >Please Help
> >> > >
> >> > >Regards,
> >> > >Vinay Patil
> >> >
> >> >
> >>
> >
> >
>
Re: [Issue] No Class Definition Found Error
Posted by Vinay Patil <vi...@gmail.com>.
Issue resolved.
Created the uber jar (fat jar) as shown in the flink-quickstart guide.
My bad, I should have checked that first.
However now I am facing NullPointerException , consider the following
example (this is the dummy code , since I cannot share the actual code):
public class App implements MapFunction<String, List<String>> {
private static TempClass tempClass;
public App() {
tempClass = new TempClass();
}
@Override
public List<String> map(String dummyString) throws Exception {
*tempClass.doSomething(dummyString); // getting NPE here*
}
}
I am not getting NPE when I run it on a single machine, however when I run
on a cluster , it gives me NPE.
Instantiating tempClass inside the map function will fix the issue ,
however my reviewer suggested not to initialize it inside the operator as
it will get initialized always, so did it in the constructor.
Also, when I looked at the flink examples, you have used static classes and
accessed the functions inside operators, so which is the better approach
instantiating inside the operators or use static class ?
Regards,
Vinay Patil
*+91-800-728-4749*
On Sat, Jun 11, 2016 at 9:32 AM, Vinay Patil <vi...@gmail.com>
wrote:
> Hi Robert,
>
> Yes we are using maven for building the jar, I have deployed both jar with
> dependencies and without dependencies.
>
> I actually cannot share the pom since it is on the client machine.
> But all the dependencies required are there, I have attached a sample pom
> file which is similar to the pom we are using
>
> Regards,
> Vinay Patil
>
> *+91-800-728-4749*
>
> On Sat, Jun 11, 2016 at 1:36 AM, Robert Metzger <rm...@apache.org>
> wrote:
>
>> Are you using Maven for building your job jar?
>> If yes, can you post your pom file on the mailing list?
>>
>> On Fri, Jun 10, 2016 at 7:16 PM, THORMAN, ROBERT D <rt...@att.com>
>> wrote:
>>
>> > How did you “provide” the dependencies? Did you use the –C <URL>
>> > parameter when you submitted your job?
>> >
>> > On 6/10/16, 11:35 AM, "Vinay Patil" <vi...@gmail.com> wrote:
>> >
>> > >Hi Guys,
>> > >
>> > >I have deployed my application on a cluster, however when I try to run
>> the
>> > >application it throws *NoClassDefFoundError for
>> > KeyedDeserializationSchema*,
>> > >all the dependencies are provided correctly since I have run it on a
>> > >different standalone node.
>> > >
>> > >Please Help
>> > >
>> > >Regards,
>> > >Vinay Patil
>> >
>> >
>>
>
>
Re: [Issue] No Class Definition Found Error
Posted by Vinay Patil <vi...@gmail.com>.
Hi Robert,
Yes we are using maven for building the jar, I have deployed both jar with
dependencies and without dependencies.
I actually cannot share the pom since it is on the client machine.
But all the dependencies required are there, I have attached a sample pom
file which is similar to the pom we are using
Regards,
Vinay Patil
*+91-800-728-4749*
On Sat, Jun 11, 2016 at 1:36 AM, Robert Metzger <rm...@apache.org> wrote:
> Are you using Maven for building your job jar?
> If yes, can you post your pom file on the mailing list?
>
> On Fri, Jun 10, 2016 at 7:16 PM, THORMAN, ROBERT D <rt...@att.com> wrote:
>
> > How did you “provide” the dependencies? Did you use the –C <URL>
> > parameter when you submitted your job?
> >
> > On 6/10/16, 11:35 AM, "Vinay Patil" <vi...@gmail.com> wrote:
> >
> > >Hi Guys,
> > >
> > >I have deployed my application on a cluster, however when I try to run
> the
> > >application it throws *NoClassDefFoundError for
> > KeyedDeserializationSchema*,
> > >all the dependencies are provided correctly since I have run it on a
> > >different standalone node.
> > >
> > >Please Help
> > >
> > >Regards,
> > >Vinay Patil
> >
> >
>
Re: [Issue] No Class Definition Found Error
Posted by Robert Metzger <rm...@apache.org>.
Are you using Maven for building your job jar?
If yes, can you post your pom file on the mailing list?
On Fri, Jun 10, 2016 at 7:16 PM, THORMAN, ROBERT D <rt...@att.com> wrote:
> How did you “provide” the dependencies? Did you use the –C <URL>
> parameter when you submitted your job?
>
> On 6/10/16, 11:35 AM, "Vinay Patil" <vi...@gmail.com> wrote:
>
> >Hi Guys,
> >
> >I have deployed my application on a cluster, however when I try to run the
> >application it throws *NoClassDefFoundError for
> KeyedDeserializationSchema*,
> >all the dependencies are provided correctly since I have run it on a
> >different standalone node.
> >
> >Please Help
> >
> >Regards,
> >Vinay Patil
>
>
Re: [Issue] No Class Definition Found Error
Posted by "THORMAN, ROBERT D" <rt...@att.com>.
How did you “provide” the dependencies? Did you use the –C <URL> parameter when you submitted your job?
On 6/10/16, 11:35 AM, "Vinay Patil" <vi...@gmail.com> wrote:
>Hi Guys,
>
>I have deployed my application on a cluster, however when I try to run the
>application it throws *NoClassDefFoundError for KeyedDeserializationSchema*,
>all the dependencies are provided correctly since I have run it on a
>different standalone node.
>
>Please Help
>
>Regards,
>Vinay Patil