KafkaDeserializationSchema.open() is not called after task state change

[Gil Amsalem <gi...@booking.com>]

Hi,

I have a class that implements KafkaDeserializationSchema and overrides the
open() method.
When deploying my job, everything seems to work as expected.
But, when my task hit an exception and switch to CANCELED -> CREATED ->
DEPLOYING, it seems that the open method is not called, and I am ending up
with uninitialized values.

Is that expected? Is it a bug?

-- 
Gil Amsalem

Re: [External] Re: KafkaDeserializationSchema.open() is not called after task state change

[Caizhi Weng <ts...@gmail.com>]

In this case there seems to be a bug. Please open a ticket in JIRA for that
and please describe the situation more thoroughly in the ticket. If
possible please also provide your user code and the exception message.
Thanks.

Gil Amsalem <gi...@booking.com> 于2021年8月9日周一 下午5:20写道：

> Hi Caizhi,
> Thanks for your reply.
> I already have some traces in the open() methods, and I do not see that it
> is being called.
> We are using Flink version 1.11.2.
> Should I open a bug for that?
>
>
> On Mon, Aug 9, 2021 at 5:24 AM Caizhi Weng <ts...@gmail.com> wrote:
>
>> Hi!
>>
>> This does not sound like an expected behavior. However there might be a
>> lot of reasons causing some values to be uninitialized (for example, once
>> I've met a bug that a thread is created and runs in the open method before
>> some values are initialized). You can always add some log at the beginning
>> of the open method and see if it is really not called. If it is the case
>> then it should be a bug and the developers should look into it.
>>
>> Gil Amsalem <gi...@booking.com> 于2021年8月8日周日 下午7:54写道：
>>
>>> Hi,
>>>
>>> I have a class that implements KafkaDeserializationSchema and overrides
>>> the open() method.
>>> When deploying my job, everything seems to work as expected.
>>> But, when my task hit an exception and switch to CANCELED -> CREATED ->
>>> DEPLOYING, it seems that the open method is not called, and I am ending up
>>> with uninitialized values.
>>>
>>> Is that expected? Is it a bug?
>>>
>>> --
>>> Gil Amsalem
>>>
>>>
>
> --
> Gil Amsalem
> Machine Learning Engineer
> [image: Booking.com] <https://www.booking.com/>
> Making it easier for everyone
> to experience the world.
>

Re: [External] Re: KafkaDeserializationSchema.open() is not called after task state change

[Gil Amsalem <gi...@booking.com>]

Hi Caizhi,
Thanks for your reply.
I already have some traces in the open() methods, and I do not see that it
is being called.
We are using Flink version 1.11.2.
Should I open a bug for that?


On Mon, Aug 9, 2021 at 5:24 AM Caizhi Weng <ts...@gmail.com> wrote:

> Hi!
>
> This does not sound like an expected behavior. However there might be a
> lot of reasons causing some values to be uninitialized (for example, once
> I've met a bug that a thread is created and runs in the open method before
> some values are initialized). You can always add some log at the beginning
> of the open method and see if it is really not called. If it is the case
> then it should be a bug and the developers should look into it.
>
> Gil Amsalem <gi...@booking.com> 于2021年8月8日周日 下午7:54写道：
>
>> Hi,
>>
>> I have a class that implements KafkaDeserializationSchema and overrides
>> the open() method.
>> When deploying my job, everything seems to work as expected.
>> But, when my task hit an exception and switch to CANCELED -> CREATED ->
>> DEPLOYING, it seems that the open method is not called, and I am ending up
>> with uninitialized values.
>>
>> Is that expected? Is it a bug?
>>
>> --
>> Gil Amsalem
>>
>>

-- 
Gil Amsalem
Machine Learning Engineer
[image: Booking.com] <https://www.booking.com/>
Making it easier for everyone
to experience the world.

Re: KafkaDeserializationSchema.open() is not called after task state change

[Caizhi Weng <ts...@gmail.com>]

Hi!

This does not sound like an expected behavior. However there might be a lot
of reasons causing some values to be uninitialized (for example, once I've
met a bug that a thread is created and runs in the open method before some
values are initialized). You can always add some log at the beginning of
the open method and see if it is really not called. If it is the case then
it should be a bug and the developers should look into it.

Gil Amsalem <gi...@booking.com> 于2021年8月8日周日 下午7:54写道：

> Hi,
>
> I have a class that implements KafkaDeserializationSchema and overrides
> the open() method.
> When deploying my job, everything seems to work as expected.
> But, when my task hit an exception and switch to CANCELED -> CREATED ->
> DEPLOYING, it seems that the open method is not called, and I am ending up
> with uninitialized values.
>
> Is that expected? Is it a bug?
>
> --
> Gil Amsalem
>
>