You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Abdullah bin Omar <ab...@gmail.com> on 2021/04/30 14:52:36 UTC
"myuid" in snapshot.readingstate
Hi,
when we readstate of of savepooint, we use, "myuid" as a argument of the
function. For example,
DataSet<KeyedState> keyedState = savepoint.readKeyedState("my-uid",
new ReaderFunction());
*Question 1:*
In [1] (line no 79), we get the "uid" with datastream. Then in [2] (line no
45), *how can we use the "uid" that we have got from in [1]?* Because in
[2], there is no declaration of using the "uid" from [1].
*Question 2:*
*what does it mean by "uid" in the datastream of [1]?* is it something a
unique user identification for each stream or for each state of the
datastream?
[1]
https://github.com/ververica/flink-training/blob/master/rides-and-fares/src/solution/java/org/apache/flink/training/solutions/ridesandfares/RidesAndFaresSolution.java
[2]
https://github.com/ververica/flink-training/blob/master/state-processor/src/main/java/com/ververica/flink/training/exercises/ReadRidesAndFaresSnapshot.java
Thank you
Re: "myuid" in snapshot.readingstate
Posted by Abu Bakar Siddiqur Rahman Rocky <ba...@gmail.com>.
I am asking on behalf of my colleagues the previous question. Though we
have a couple of questions that were in another email
On Fri, Apr 30, 2021 at 10:18 AM Abu Bakar Siddiqur Rahman Rocky <
bakar121107@gmail.com> wrote:
> Hi David,
>
> A quick question more
>
> I am trying to *import* org.apache.flink.training.exercises.common.sources.TaxiFareGenerator;
> However, it can not resolve.
>
> What is dependency (in pom.xml) for the org.apache.flink.training?
>
>
> Thank you
>
> On Fri, Apr 30, 2021 at 10:12 AM David Anderson <da...@apache.org>
> wrote:
>
>> You can read about assigning unique IDs to stateful operators in the docs
>> [1][2]. What the uid() method does is to establish a stable and unique
>> identifier for a stateful operator. Then as you evolve your application,
>> this helps ensure that future versions of your job will be able to restore
>> savepoints taken by earlier versions and find the state they need, despite
>> changes to the topology.
>>
>> The two uids specified in those two independent jobs (the streaming
>> RidesAndFaresSolution job and the batch ReadRidesAndFaresSnapshot job) must
>> match -- in other words, the strings must be the same. But there's nothing
>> in that example that ensures this will be the case.
>>
>> Regards,
>> David
>>
>> [1]
>> https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/production_ready.html#set-uuids-for-all-operators
>> [2]
>> https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/state/savepoints.html#assigning-operator-ids
>>
>> On Fri, Apr 30, 2021 at 4:52 PM Abdullah bin Omar <
>> abdullahbinomar29@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> when we readstate of of savepooint, we use, "myuid" as a argument of the
>>> function. For example,
>>>
>>> DataSet<KeyedState> keyedState = savepoint.readKeyedState("my-uid", new ReaderFunction());
>>>
>>>
>>> *Question 1:*
>>>
>>> In [1] (line no 79), we get the "uid" with datastream. Then in [2] (line
>>> no 45), *how can we use the "uid" that we have got from in [1]?*
>>> Because in [2], there is no declaration of using the "uid" from [1].
>>>
>>> *Question 2:*
>>>
>>> *what does it mean by "uid" in the datastream of [1]?* is it something
>>> a unique user identification for each stream or for each state of the
>>> datastream?
>>>
>>>
>>> [1]
>>> https://github.com/ververica/flink-training/blob/master/rides-and-fares/src/solution/java/org/apache/flink/training/solutions/ridesandfares/RidesAndFaresSolution.java
>>> [2]
>>> https://github.com/ververica/flink-training/blob/master/state-processor/src/main/java/com/ververica/flink/training/exercises/ReadRidesAndFaresSnapshot.java
>>>
>>> Thank you
>>>
>>>
>>>
>
> --
> Regards,
> Abu Bakar Siddiqur Rahman
>
>
--
Regards,
Abu Bakar Siddiqur Rahman
Re: "myuid" in snapshot.readingstate
Posted by Abu Bakar Siddiqur Rahman Rocky <ba...@gmail.com>.
Thank you, David.
On Fri, Apr 30, 2021 at 10:31 AM David Anderson <da...@apache.org>
wrote:
> What is dependency (in pom.xml) for the org.apache.flink.training?
>
>
> We don't publish artifacts for this repository.
>
> David
>
> On Fri, Apr 30, 2021 at 5:19 PM Abu Bakar Siddiqur Rahman Rocky <
> bakar121107@gmail.com> wrote:
>
>> Hi David,
>>
>> A quick question more
>>
>> I am trying to *import* org.apache.flink.training.exercises.common.sources.TaxiFareGenerator;
>> However, it can not resolve.
>>
>> What is dependency (in pom.xml) for the org.apache.flink.training?
>>
>>
>> Thank you
>>
>> On Fri, Apr 30, 2021 at 10:12 AM David Anderson <da...@apache.org>
>> wrote:
>>
>>> You can read about assigning unique IDs to stateful operators in the
>>> docs [1][2]. What the uid() method does is to establish a stable and unique
>>> identifier for a stateful operator. Then as you evolve your application,
>>> this helps ensure that future versions of your job will be able to restore
>>> savepoints taken by earlier versions and find the state they need, despite
>>> changes to the topology.
>>>
>>> The two uids specified in those two independent jobs (the streaming
>>> RidesAndFaresSolution job and the batch ReadRidesAndFaresSnapshot job) must
>>> match -- in other words, the strings must be the same. But there's nothing
>>> in that example that ensures this will be the case.
>>>
>>> Regards,
>>> David
>>>
>>> [1]
>>> https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/production_ready.html#set-uuids-for-all-operators
>>> [2]
>>> https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/state/savepoints.html#assigning-operator-ids
>>>
>>> On Fri, Apr 30, 2021 at 4:52 PM Abdullah bin Omar <
>>> abdullahbinomar29@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> when we readstate of of savepooint, we use, "myuid" as a argument of
>>>> the function. For example,
>>>>
>>>> DataSet<KeyedState> keyedState = savepoint.readKeyedState("my-uid", new ReaderFunction());
>>>>
>>>>
>>>> *Question 1:*
>>>>
>>>> In [1] (line no 79), we get the "uid" with datastream. Then in [2]
>>>> (line no 45), *how can we use the "uid" that we have got from in [1]?*
>>>> Because in [2], there is no declaration of using the "uid" from [1].
>>>>
>>>> *Question 2:*
>>>>
>>>> *what does it mean by "uid" in the datastream of [1]?* is it something
>>>> a unique user identification for each stream or for each state of the
>>>> datastream?
>>>>
>>>>
>>>> [1]
>>>> https://github.com/ververica/flink-training/blob/master/rides-and-fares/src/solution/java/org/apache/flink/training/solutions/ridesandfares/RidesAndFaresSolution.java
>>>> [2]
>>>> https://github.com/ververica/flink-training/blob/master/state-processor/src/main/java/com/ververica/flink/training/exercises/ReadRidesAndFaresSnapshot.java
>>>>
>>>> Thank you
>>>>
>>>>
>>>>
>>
>> --
>> Regards,
>> Abu Bakar Siddiqur Rahman
>>
>>
--
Regards,
Abu Bakar Siddiqur Rahman
Re: "myuid" in snapshot.readingstate
Posted by David Anderson <da...@apache.org>.
>
> What is dependency (in pom.xml) for the org.apache.flink.training?
We don't publish artifacts for this repository.
David
On Fri, Apr 30, 2021 at 5:19 PM Abu Bakar Siddiqur Rahman Rocky <
bakar121107@gmail.com> wrote:
> Hi David,
>
> A quick question more
>
> I am trying to *import* org.apache.flink.training.exercises.common.sources.TaxiFareGenerator;
> However, it can not resolve.
>
> What is dependency (in pom.xml) for the org.apache.flink.training?
>
>
> Thank you
>
> On Fri, Apr 30, 2021 at 10:12 AM David Anderson <da...@apache.org>
> wrote:
>
>> You can read about assigning unique IDs to stateful operators in the docs
>> [1][2]. What the uid() method does is to establish a stable and unique
>> identifier for a stateful operator. Then as you evolve your application,
>> this helps ensure that future versions of your job will be able to restore
>> savepoints taken by earlier versions and find the state they need, despite
>> changes to the topology.
>>
>> The two uids specified in those two independent jobs (the streaming
>> RidesAndFaresSolution job and the batch ReadRidesAndFaresSnapshot job) must
>> match -- in other words, the strings must be the same. But there's nothing
>> in that example that ensures this will be the case.
>>
>> Regards,
>> David
>>
>> [1]
>> https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/production_ready.html#set-uuids-for-all-operators
>> [2]
>> https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/state/savepoints.html#assigning-operator-ids
>>
>> On Fri, Apr 30, 2021 at 4:52 PM Abdullah bin Omar <
>> abdullahbinomar29@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> when we readstate of of savepooint, we use, "myuid" as a argument of the
>>> function. For example,
>>>
>>> DataSet<KeyedState> keyedState = savepoint.readKeyedState("my-uid", new ReaderFunction());
>>>
>>>
>>> *Question 1:*
>>>
>>> In [1] (line no 79), we get the "uid" with datastream. Then in [2] (line
>>> no 45), *how can we use the "uid" that we have got from in [1]?*
>>> Because in [2], there is no declaration of using the "uid" from [1].
>>>
>>> *Question 2:*
>>>
>>> *what does it mean by "uid" in the datastream of [1]?* is it something
>>> a unique user identification for each stream or for each state of the
>>> datastream?
>>>
>>>
>>> [1]
>>> https://github.com/ververica/flink-training/blob/master/rides-and-fares/src/solution/java/org/apache/flink/training/solutions/ridesandfares/RidesAndFaresSolution.java
>>> [2]
>>> https://github.com/ververica/flink-training/blob/master/state-processor/src/main/java/com/ververica/flink/training/exercises/ReadRidesAndFaresSnapshot.java
>>>
>>> Thank you
>>>
>>>
>>>
>
> --
> Regards,
> Abu Bakar Siddiqur Rahman
>
>
Re: "myuid" in snapshot.readingstate
Posted by Abu Bakar Siddiqur Rahman Rocky <ba...@gmail.com>.
Hi David,
A quick question more
I am trying to *import*
org.apache.flink.training.exercises.common.sources.TaxiFareGenerator;
However, it can not resolve.
What is dependency (in pom.xml) for the org.apache.flink.training?
Thank you
On Fri, Apr 30, 2021 at 10:12 AM David Anderson <da...@apache.org>
wrote:
> You can read about assigning unique IDs to stateful operators in the docs
> [1][2]. What the uid() method does is to establish a stable and unique
> identifier for a stateful operator. Then as you evolve your application,
> this helps ensure that future versions of your job will be able to restore
> savepoints taken by earlier versions and find the state they need, despite
> changes to the topology.
>
> The two uids specified in those two independent jobs (the streaming
> RidesAndFaresSolution job and the batch ReadRidesAndFaresSnapshot job) must
> match -- in other words, the strings must be the same. But there's nothing
> in that example that ensures this will be the case.
>
> Regards,
> David
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/production_ready.html#set-uuids-for-all-operators
> [2]
> https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/state/savepoints.html#assigning-operator-ids
>
> On Fri, Apr 30, 2021 at 4:52 PM Abdullah bin Omar <
> abdullahbinomar29@gmail.com> wrote:
>
>> Hi,
>>
>> when we readstate of of savepooint, we use, "myuid" as a argument of the
>> function. For example,
>>
>> DataSet<KeyedState> keyedState = savepoint.readKeyedState("my-uid", new ReaderFunction());
>>
>>
>> *Question 1:*
>>
>> In [1] (line no 79), we get the "uid" with datastream. Then in [2] (line
>> no 45), *how can we use the "uid" that we have got from in [1]?* Because
>> in [2], there is no declaration of using the "uid" from [1].
>>
>> *Question 2:*
>>
>> *what does it mean by "uid" in the datastream of [1]?* is it something a
>> unique user identification for each stream or for each state of the
>> datastream?
>>
>>
>> [1]
>> https://github.com/ververica/flink-training/blob/master/rides-and-fares/src/solution/java/org/apache/flink/training/solutions/ridesandfares/RidesAndFaresSolution.java
>> [2]
>> https://github.com/ververica/flink-training/blob/master/state-processor/src/main/java/com/ververica/flink/training/exercises/ReadRidesAndFaresSnapshot.java
>>
>> Thank you
>>
>>
>>
--
Regards,
Abu Bakar Siddiqur Rahman
Re: "myuid" in snapshot.readingstate
Posted by David Anderson <da...@apache.org>.
You can read about assigning unique IDs to stateful operators in the docs
[1][2]. What the uid() method does is to establish a stable and unique
identifier for a stateful operator. Then as you evolve your application,
this helps ensure that future versions of your job will be able to restore
savepoints taken by earlier versions and find the state they need, despite
changes to the topology.
The two uids specified in those two independent jobs (the streaming
RidesAndFaresSolution job and the batch ReadRidesAndFaresSnapshot job) must
match -- in other words, the strings must be the same. But there's nothing
in that example that ensures this will be the case.
Regards,
David
[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/production_ready.html#set-uuids-for-all-operators
[2]
https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/state/savepoints.html#assigning-operator-ids
On Fri, Apr 30, 2021 at 4:52 PM Abdullah bin Omar <
abdullahbinomar29@gmail.com> wrote:
> Hi,
>
> when we readstate of of savepooint, we use, "myuid" as a argument of the
> function. For example,
>
> DataSet<KeyedState> keyedState = savepoint.readKeyedState("my-uid", new ReaderFunction());
>
>
> *Question 1:*
>
> In [1] (line no 79), we get the "uid" with datastream. Then in [2] (line
> no 45), *how can we use the "uid" that we have got from in [1]?* Because
> in [2], there is no declaration of using the "uid" from [1].
>
> *Question 2:*
>
> *what does it mean by "uid" in the datastream of [1]?* is it something a
> unique user identification for each stream or for each state of the
> datastream?
>
>
> [1]
> https://github.com/ververica/flink-training/blob/master/rides-and-fares/src/solution/java/org/apache/flink/training/solutions/ridesandfares/RidesAndFaresSolution.java
> [2]
> https://github.com/ververica/flink-training/blob/master/state-processor/src/main/java/com/ververica/flink/training/exercises/ReadRidesAndFaresSnapshot.java
>
> Thank you
>
>
>