Re: FlinkKafkaConsumer - Save watermark assigner in the state

[Konstantin Knauf <ko...@ververica.com>]

Hi Juan,

can you elaborate a bit on why you want to put the WatermarkAssigner itself
into state? It is generally unusual to store a UDF in Managed State.

Cheers,

Konstantin




On Fri, Jul 5, 2019 at 5:07 PM Juan Gentile <j....@criteo.com> wrote:

> Hello,
>
>
>
> We are currently facing an issue where we need to store the instance of
> the watermark and timestamp assigner in the state while consumer from Kafka.
>
> For that purpose we took a look at FlinkKafkaConsumerBase and noticed that
> since the methods (snapshotState and initializeState from the
> CheckpointedFunction) are final we can’t override them.
>
> An alternative could be to wrap the class but we don’t like that solution
> either as it would make it harder to migrate in the future if there are
> changes there.
>
> We would like to know if it’d be possible to store the ‘assigner’ in the
> state of the FlinkKafkaConsumerBase. We think it’s not a big change and we
> are willing to do it if you agree.
>
>
>
> Thank you,
>
> Juan G.
>


-- 

Konstantin Knauf | Solutions Architect

+49 160 91394525


Planned Absences: 10.08.2019 - 31.08.2019, 05.09. - 06.09.2010


--

Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany

--

Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen