You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Paolo Cristofanelli <cr...@gmail.com> on 2017/07/26 14:14:59 UTC

Class not found when deserializing

Hi,

I am trying to write and read in a Kafka topic a user-defined class (that
implements serializable, and all the fields are serializable).
Everything works fine when I am executing the program in the IDE or with
the mvn exec command.
When I try to execute the program in standalone mode I get the
ClassNotFoundException.

More specifically I get the exception only during the deserialization parts
:

@Override
>
> public TheGlobalModel deserialize(byte[] message) throws IOException {
>
> outlierDetection.network.TheGlobalModel model;
>
>
>>    model = (outlierDetection.network.TheGlobalModel)
>> SerializationUtils.deserialize(message);
>
>
>
> return model;
>
> }
>
>
It seems that the problem lies in the deserialize method. If I remove it
and simply return "new TheGlobalModel()" the exception is not thrown. I
don´t understand why in this case the program seems to be aware of the
existence of the class, I guess the problem is in the deserialize function.

I only know this method for sending a serializable class through Kafka, I
would be glad to hear other ways.

Thanks in advance for your time.
Cheers
Paolo

Re: Class not found when deserializing

Posted by Aljoscha Krettek <al...@apache.org>.
Hi,

I think the problem here is that the SerializationUtils don't use the correct ClassLoader when deserialising. When running a Flink job on a cluster the user code (from the user-code jar) is only available in the user-code ClassLoader. If you can pass a ClassLoader to your deserialize method you can try passing the result of Thread.currentThread().getContextClassLoader().

Best,
Aljoscha

> On 27. Jul 2017, at 09:52, Fabian Hueske <fh...@gmail.com> wrote:
> 
> Hi Paolo,
> 
> do you get the ClassNotFoundException for TheGlobalModel or for another class?
> Did you maybe forget to include SerializationUtils in the classpath?
> 
> Best, Fabian
> 
> 2017-07-26 16:14 GMT+02:00 Paolo Cristofanelli <cr...@gmail.com>:
> Hi, 
> 
> I am trying to write and read in a Kafka topic a user-defined class (that implements serializable, and all the fields are serializable). Everything works fine when I am executing the program in the IDE or with the mvn exec command.
> When I try to execute the program in standalone mode I get the ClassNotFoundException.
> 
> More specifically I get the exception only during the deserialization parts :
> 
> 		@Override
> 		public TheGlobalModel deserialize(byte[] message) throws IOException {
> 			
> 			outlierDetection.network.TheGlobalModel model;
> 
> 		    model = (outlierDetection.network.TheGlobalModel) SerializationUtils.deserialize(message);
> 		    
> 			return model;
> 			
> 		}
> 
> It seems that the problem lies in the deserialize method. If I remove it and simply return "new TheGlobalModel()" the exception is not thrown. I don´t understand why in this case the program seems to be aware of the existence of the class, I guess the problem is in the deserialize function.
> 
> I only know this method for sending a serializable class through Kafka, I would be glad to hear other ways. 
> 
> Thanks in advance for your time.
> Cheers 
> Paolo
>

Re: Class not found when deserializing

Posted by Fabian Hueske <fh...@gmail.com>.
Hi Paolo,

do you get the ClassNotFoundException for TheGlobalModel or for another
class?
Did you maybe forget to include SerializationUtils in the classpath?

Best, Fabian

2017-07-26 16:14 GMT+02:00 Paolo Cristofanelli <
cristofanelli.paolo@gmail.com>:

> Hi,
>
> I am trying to write and read in a Kafka topic a user-defined class (that
> implements serializable, and all the fields are serializable).
> Everything works fine when I am executing the program in the IDE or with
> the mvn exec command.
> When I try to execute the program in standalone mode I get the
> ClassNotFoundException.
>
> More specifically I get the exception only during the deserialization
> parts :
>
> @Override
>>
>> public TheGlobalModel deserialize(byte[] message) throws IOException {
>>
>> outlierDetection.network.TheGlobalModel model;
>>
>>
>>>    model = (outlierDetection.network.TheGlobalModel) SerializationUtils.
>>> deserialize(message);
>>
>>
>>
>> return model;
>>
>> }
>>
>>
> It seems that the problem lies in the deserialize method. If I remove it
> and simply return "new TheGlobalModel()" the exception is not thrown. I
> don´t understand why in this case the program seems to be aware of the
> existence of the class, I guess the problem is in the deserialize function.
>
> I only know this method for sending a serializable class through Kafka, I
> would be glad to hear other ways.
>
> Thanks in advance for your time.
> Cheers
> Paolo
>