You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Boris Lublinsky <bo...@lightbend.com> on 2018/01/16 00:54:31 UTC
MapState Initializion
Any response to this?
List State is created using
checkpointedState = context.getOperatorStateStore.getListState (checkPointDescriptor)
But there is no similar APIs for MapState
Boris Lublinsky
FDP Architect
boris.lublinsky@lightbend.com
https://www.lightbend.com/
> Begin forwarded message:
>
> From: Boris Lublinsky <bo...@lightbend.com>
> Subject: Re: Questions about managed operator state
> Date: January 14, 2018 at 7:39:00 PM CST
> To: Fabian Hueske <fh...@gmail.com>
> Cc: user <us...@flink.apache.org>
>
> Thanks Fabian,
> After I switched to var it compiles, but its not initialized.
> @transient private var currentModels : MapState[String, Model] = _
> Assignes null to MapState.
> Do I create an empty hashMap there?
>
> Boris Lublinsky
> FDP Architect
> boris.lublinsky@lightbend.com <ma...@lightbend.com>
> https://www.lightbend.com/
>
>> On Jan 14, 2018, at 11:09 AM, Fabian Hueske <fhueske@gmail.com <ma...@gmail.com>> wrote:
>>
>> Hi Boris,
>>
>> the CheckpointedRestoring interface was removed in Flink 1.4.0 (and deprecated in an earlier version). Unfortunately, the docs have not been updated accordingly. I'll open a JIRA to fix this.
>> The replacements for CheckpointedRestoring are the CheckpointedFunction or ListCheckpointed interfaces (see [1]).
>>
>> I think the compile error is caused because you define newModels as val and not as var.
>>
>> Best, Fabian
>>
>> [1] https://github.com/apache/flink/blob/release-1.3/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/checkpoint/CheckpointedRestoring.java <https://github.com/apache/flink/blob/release-1.3/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/checkpoint/CheckpointedRestoring.java>
>>
>> 2018-01-14 2:39 GMT+01:00 Boris Lublinsky <boris.lublinsky@lightbend.com <ma...@lightbend.com>>:
>> Documentation https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/state/state.html#using-managed-operator-state <https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/state/state.html#using-managed-operator-state>
>> Refers to CheckpointedRestoring interface.
>> Which jar defines this interface - can’t find it
>>
>> Also documentation refers to MapState<UK, UV>. Do you have any example of creation of Map State.
>> For value State in Scala, I can do just do
>> private var sum: ValueState[(Long, Long)] = _
>> But it does not work for MapState -
>> Error:(44, 53) unbound placeholder parameter
>> private val newModels : MapState[String, Model] = _
>>
>>
>> Boris Lublinsky
>> FDP Architect
>> boris.lublinsky@lightbend.com <ma...@lightbend.com>
>> https://www.lightbend.com/ <https://www.lightbend.com/>
>>
>
> Begin forwarded message:
>
> From: Boris Lublinsky <bo...@lightbend.com>
> Subject: Questions about managed operator state
> Date: January 13, 2018 at 7:39:09 PM CST
> To: user <us...@flink.apache.org>
>
> Documentation https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/state/state.html#using-managed-operator-state <https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/state/state.html#using-managed-operator-state>
> Refers to CheckpointedRestoring interface.
> Which jar defines this interface - can’t find it
>
> Also documentation refers to MapState<UK, UV>. Do you have any example of creation of Map State.
> For value State in Scala, I can do just do
> private var sum: ValueState[(Long, Long)] = _
> But it does not work for MapState -
> Error:(44, 53) unbound placeholder parameter
> private val newModels : MapState[String, Model] = _
>
>
> Boris Lublinsky
> FDP Architect
> boris.lublinsky@lightbend.com <ma...@lightbend.com>
> https://www.lightbend.com/
>
Re: MapState Initializion
Posted by Fabian Hueske <fh...@gmail.com>.
Alternatively, you can also create a keyed MapState as
context.getKeyedStateStore().getMapState() in
CheckpointedFunction.initializeState().
2018-01-16 9:58 GMT+01:00 Fabian Hueske <fh...@gmail.com>:
> Sorry for the late response.
>
> MapState is currently only support as keyed state but not as operator
> state.
> If you want to create a keyed MapState the object should be created using
> a MapStateDescriptor in the open() method via the RuntimeContext.
>
>
>
> 2018-01-16 1:54 GMT+01:00 Boris Lublinsky <bo...@lightbend.com>:
>
>> Any response to this?
>>
>> List State is created using
>>
>> checkpointedState = context.getOperatorStateStore.getListState (checkPointDescriptor)
>>
>> But there is no similar APIs for MapState
>>
>> Boris Lublinsky
>> FDP Architect
>> boris.lublinsky@lightbend.com
>> https://www.lightbend.com/
>>
>> Begin forwarded message:
>>
>> *From: *Boris Lublinsky <bo...@lightbend.com>
>> *Subject: **Re: Questions about managed operator state*
>> *Date: *January 14, 2018 at 7:39:00 PM CST
>> *To: *Fabian Hueske <fh...@gmail.com>
>> *Cc: *user <us...@flink.apache.org>
>>
>> Thanks Fabian,
>> After I switched to var it compiles, but its not initialized.
>>
>> @transient private var currentModels : MapState[String, Model] = _
>>
>> Assignes null to MapState.
>> Do I create an empty hashMap there?
>>
>> Boris Lublinsky
>> FDP Architect
>> boris.lublinsky@lightbend.com
>> https://www.lightbend.com/
>>
>> On Jan 14, 2018, at 11:09 AM, Fabian Hueske <fh...@gmail.com> wrote:
>>
>> Hi Boris,
>>
>> the CheckpointedRestoring interface was removed in Flink 1.4.0 (and
>> deprecated in an earlier version). Unfortunately, the docs have not been
>> updated accordingly. I'll open a JIRA to fix this.
>> The replacements for CheckpointedRestoring are the CheckpointedFunction
>> or ListCheckpointed interfaces (see [1]).
>>
>> I think the compile error is caused because you define newModels as val
>> and not as var.
>>
>> Best, Fabian
>>
>> [1] https://github.com/apache/flink/blob/release-1.3/flink-strea
>> ming-java/src/main/java/org/apache/flink/streaming/api/
>> checkpoint/CheckpointedRestoring.java
>>
>> 2018-01-14 2:39 GMT+01:00 Boris Lublinsky <bo...@lightbend.com>
>> :
>>
>>> Documentation https://ci.apache.org/projects/flink/flink-doc
>>> s-release-1.4/dev/stream/state/state.html#using-managed-operator-state
>>> Refers to CheckpointedRestoring interface.
>>> *Which jar defines this interface - can’t find it*
>>>
>>> *Also documentation refers to *MapState<UK, UV>. Do you have any
>>> example of creation of Map State.
>>> For value State in Scala, I can do just do
>>>
>>> private var sum: ValueState[(Long, Long)] = _
>>>
>>> But it does not work for MapState -
>>> Error:(44, 53) unbound placeholder parameter
>>> private val newModels : MapState[String, Model] = _
>>>
>>>
>>> Boris Lublinsky
>>> FDP Architect
>>> boris.lublinsky@lightbend.com
>>> https://www.lightbend.com/
>>>
>>>
>>
>> Begin forwarded message:
>>
>> *From: *Boris Lublinsky <bo...@lightbend.com>
>> *Subject: **Questions about managed operator state*
>> *Date: *January 13, 2018 at 7:39:09 PM CST
>> *To: *user <us...@flink.apache.org>
>>
>> Documentation https://ci.apache.org/projects/flink/flink-
>> docs-release-1.4/dev/stream/state/state.html#using-managed-operator-state
>> Refers to CheckpointedRestoring interface.
>> *Which jar defines this interface - can’t find it*
>>
>> *Also documentation refers to *MapState<UK, UV>. Do you have any example
>> of creation of Map State.
>> For value State in Scala, I can do just do
>>
>> private var sum: ValueState[(Long, Long)] = _
>>
>> But it does not work for MapState -
>> Error:(44, 53) unbound placeholder parameter
>> private val newModels : MapState[String, Model] = _
>>
>>
>> Boris Lublinsky
>> FDP Architect
>> boris.lublinsky@lightbend.com
>> https://www.lightbend.com/
>>
>>
>>
>
Re: MapState Initializion
Posted by Fabian Hueske <fh...@gmail.com>.
Sorry for the late response.
MapState is currently only support as keyed state but not as operator state.
If you want to create a keyed MapState the object should be created using a
MapStateDescriptor in the open() method via the RuntimeContext.
2018-01-16 1:54 GMT+01:00 Boris Lublinsky <bo...@lightbend.com>:
> Any response to this?
>
> List State is created using
>
> checkpointedState = context.getOperatorStateStore.getListState (checkPointDescriptor)
>
> But there is no similar APIs for MapState
>
> Boris Lublinsky
> FDP Architect
> boris.lublinsky@lightbend.com
> https://www.lightbend.com/
>
> Begin forwarded message:
>
> *From: *Boris Lublinsky <bo...@lightbend.com>
> *Subject: **Re: Questions about managed operator state*
> *Date: *January 14, 2018 at 7:39:00 PM CST
> *To: *Fabian Hueske <fh...@gmail.com>
> *Cc: *user <us...@flink.apache.org>
>
> Thanks Fabian,
> After I switched to var it compiles, but its not initialized.
>
> @transient private var currentModels : MapState[String, Model] = _
>
> Assignes null to MapState.
> Do I create an empty hashMap there?
>
> Boris Lublinsky
> FDP Architect
> boris.lublinsky@lightbend.com
> https://www.lightbend.com/
>
> On Jan 14, 2018, at 11:09 AM, Fabian Hueske <fh...@gmail.com> wrote:
>
> Hi Boris,
>
> the CheckpointedRestoring interface was removed in Flink 1.4.0 (and
> deprecated in an earlier version). Unfortunately, the docs have not been
> updated accordingly. I'll open a JIRA to fix this.
> The replacements for CheckpointedRestoring are the CheckpointedFunction
> or ListCheckpointed interfaces (see [1]).
>
> I think the compile error is caused because you define newModels as val
> and not as var.
>
> Best, Fabian
>
> [1] https://github.com/apache/flink/blob/release-1.3/flink-
> streaming-java/src/main/java/org/apache/flink/streaming/api/checkpoint/
> CheckpointedRestoring.java
>
> 2018-01-14 2:39 GMT+01:00 Boris Lublinsky <bo...@lightbend.com>:
>
>> Documentation https://ci.apache.org/projects/flink/flink-
>> docs-release-1.4/dev/stream/state/state.html#using-managed-operator-state
>> Refers to CheckpointedRestoring interface.
>> *Which jar defines this interface - can’t find it*
>>
>> *Also documentation refers to *MapState<UK, UV>. Do you have any example
>> of creation of Map State.
>> For value State in Scala, I can do just do
>>
>> private var sum: ValueState[(Long, Long)] = _
>>
>> But it does not work for MapState -
>> Error:(44, 53) unbound placeholder parameter
>> private val newModels : MapState[String, Model] = _
>>
>>
>> Boris Lublinsky
>> FDP Architect
>> boris.lublinsky@lightbend.com
>> https://www.lightbend.com/
>>
>>
>
> Begin forwarded message:
>
> *From: *Boris Lublinsky <bo...@lightbend.com>
> *Subject: **Questions about managed operator state*
> *Date: *January 13, 2018 at 7:39:09 PM CST
> *To: *user <us...@flink.apache.org>
>
> Documentation https://ci.apache.org/projects/flink/
> flink-docs-release-1.4/dev/stream/state/state.html#using-
> managed-operator-state
> Refers to CheckpointedRestoring interface.
> *Which jar defines this interface - can’t find it*
>
> *Also documentation refers to *MapState<UK, UV>. Do you have any example
> of creation of Map State.
> For value State in Scala, I can do just do
>
> private var sum: ValueState[(Long, Long)] = _
>
> But it does not work for MapState -
> Error:(44, 53) unbound placeholder parameter
> private val newModels : MapState[String, Model] = _
>
>
> Boris Lublinsky
> FDP Architect
> boris.lublinsky@lightbend.com
> https://www.lightbend.com/
>
>
>