You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Hilmi Yildirim <hi...@neofonie.de> on 2015/06/17 09:36:11 UTC
Flink Streaming State Management
Hi,
does Flink Streaming support state management? For example, I have a
state which will be used inside the streaming operations but the state
can be updated.
For example:
stream.map( use state for operation).updateState(update state).
Best Regards,
Hilmi
--
--
Hilmi Yildirim
Software Developer R&D
http://www.neofonie.de
Besuchen Sie den Neo Tech Blog für Anwender:
http://blog.neofonie.de/
Folgen Sie uns:
https://plus.google.com/+neofonie
http://www.linkedin.com/company/neofonie-gmbh
https://www.xing.com/companies/neofoniegmbh
Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
Handelsregister Berlin-Charlottenburg: HRB 67460
Geschäftsführung: Thomas Kitlitschko
Re: Flink Streaming State Management
Posted by Hilmi Yildirim <hi...@neofonie.de>.
Hi Gyula,
thank you.
Best Regards,
Hilmi
Am 17.06.2015 um 09:44 schrieb Gyula Fóra:
> Hey Hilmi,
>
> Flink currently supports user defined state through the Checkpointed
> interface. Using this interface the user can define what state should
> the system be aware of when doing snapshots for fault tolerance. The
> state returned in the snapshotState method will be checkpointed (and
> restored upon failure).
>
> Currently there is no explicit update function so the user need to
> implement his state updates as part of the operator logic (inside the
> map function for instance). After the 0.9.0 we will change the state
> interfaces to have explicit update functions, you can preview some of
> the functionality in the PR Matthias referenced, or here is a document
> for a more detailed discussion of this feature:
> https://docs.google.com/document/d/1nTn4Tpafsnt-TCT6L1vlHtGGgRevU90yRsUQEmkRMjk/edit?usp=sharing
>
> Cheers,
> Gyula
>
> Hilmi Yildirim <hilmi.yildirim@neofonie.de
> <ma...@neofonie.de>> ezt írta (időpont: 2015. jún.
> 17., Sze, 9:36):
>
> Hi,
> does Flink Streaming support state management? For example, I have a
> state which will be used inside the streaming operations but the state
> can be updated.
>
> For example:
> stream.map( use state for operation).updateState(update state).
>
>
> Best Regards,
> Hilmi
>
>
>
>
> --
> --
> Hilmi Yildirim
> Software Developer R&D
>
>
> http://www.neofonie.de
>
> Besuchen Sie den Neo Tech Blog für Anwender:
> http://blog.neofonie.de/
>
> Folgen Sie uns:
> https://plus.google.com/+neofonie
> http://www.linkedin.com/company/neofonie-gmbh
> https://www.xing.com/companies/neofoniegmbh
>
> Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
> Handelsregister Berlin-Charlottenburg: HRB 67460
> Geschäftsführung: Thomas Kitlitschko
>
Re: Flink Streaming State Management
Posted by Gyula Fóra <gy...@gmail.com>.
Hey Hilmi,
Flink currently supports user defined state through the Checkpointed
interface. Using this interface the user can define what state should the
system be aware of when doing snapshots for fault tolerance. The state
returned in the snapshotState method will be checkpointed (and restored
upon failure).
Currently there is no explicit update function so the user need to
implement his state updates as part of the operator logic (inside the map
function for instance). After the 0.9.0 we will change the state interfaces
to have explicit update functions, you can preview some of the
functionality in the PR Matthias referenced, or here is a document for a
more detailed discussion of this feature:
https://docs.google.com/document/d/1nTn4Tpafsnt-TCT6L1vlHtGGgRevU90yRsUQEmkRMjk/edit?usp=sharing
Cheers,
Gyula
Hilmi Yildirim <hi...@neofonie.de> ezt írta (időpont: 2015. jún.
17., Sze, 9:36):
> Hi,
> does Flink Streaming support state management? For example, I have a
> state which will be used inside the streaming operations but the state
> can be updated.
>
> For example:
> stream.map( use state for operation).updateState(update state).
>
>
> Best Regards,
> Hilmi
>
>
>
>
> --
> --
> Hilmi Yildirim
> Software Developer R&D
>
>
> http://www.neofonie.de
>
> Besuchen Sie den Neo Tech Blog für Anwender:
> http://blog.neofonie.de/
>
> Folgen Sie uns:
> https://plus.google.com/+neofonie
> http://www.linkedin.com/company/neofonie-gmbh
> https://www.xing.com/companies/neofoniegmbh
>
> Neofonie GmbH | Robert-Koch-Platz 4 | 10115 Berlin
> Handelsregister Berlin-Charlottenburg: HRB 67460
> Geschäftsführung: Thomas Kitlitschko
>
>
Re: Flink Streaming State Management
Posted by Robert Metzger <rm...@apache.org>.
Hey Hilmi,
here is a great example of how to use the Checkpointed interface:
https://github.com/StephanEwen/flink-demos/blob/master/streaming-state-machine/src/main/scala/com/dataartisans/flink/example/eventpattern/StreamingDemo.scala#L82
On Wed, Jun 17, 2015 at 12:44 AM, Hilmi Yildirim <hilmi.yildirim@neofonie.de
> wrote:
> Hi Matthias,
> great! Thank you.
>
> Best Regards,
> Hilmi
>
>
> Am 17.06.2015 um 09:38 schrieb Matthias J. Sax:
>
>> Hi Hilmi,
>>
>> currently, this is not supported. However, state management is already
>> work in progress and should be available soon. See
>> https://github.com/apache/flink/pull/747
>>
>> -Matthias
>>
>> On 06/17/2015 09:36 AM, Hilmi Yildirim wrote:
>>
>>> Hi,
>>> does Flink Streaming support state management? For example, I have a
>>> state which will be used inside the streaming operations but the state
>>> can be updated.
>>>
>>> For example:
>>> stream.map( use state for operation).updateState(update state).
>>>
>>>
>>> Best Regards,
>>> Hilmi
>>>
>>>
>>>
>>>
>>>
Re: Flink Streaming State Management
Posted by Hilmi Yildirim <hi...@neofonie.de>.
Hi Matthias,
great! Thank you.
Best Regards,
Hilmi
Am 17.06.2015 um 09:38 schrieb Matthias J. Sax:
> Hi Hilmi,
>
> currently, this is not supported. However, state management is already
> work in progress and should be available soon. See
> https://github.com/apache/flink/pull/747
>
> -Matthias
>
> On 06/17/2015 09:36 AM, Hilmi Yildirim wrote:
>> Hi,
>> does Flink Streaming support state management? For example, I have a
>> state which will be used inside the streaming operations but the state
>> can be updated.
>>
>> For example:
>> stream.map( use state for operation).updateState(update state).
>>
>>
>> Best Regards,
>> Hilmi
>>
>>
>>
>>
Re: Flink Streaming State Management
Posted by "Matthias J. Sax" <mj...@informatik.hu-berlin.de>.
Hi Hilmi,
currently, this is not supported. However, state management is already
work in progress and should be available soon. See
https://github.com/apache/flink/pull/747
-Matthias
On 06/17/2015 09:36 AM, Hilmi Yildirim wrote:
> Hi,
> does Flink Streaming support state management? For example, I have a
> state which will be used inside the streaming operations but the state
> can be updated.
>
> For example:
> stream.map( use state for operation).updateState(update state).
>
>
> Best Regards,
> Hilmi
>
>
>
>