You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/03/07 02:11:27 UTC
[GitHub] [flink] rkhachatryan opened a new pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
rkhachatryan opened a new pull request #18989:
URL: https://github.com/apache/flink/pull/18989
## What is the purpose of the change
Currently, all tasks perform materialization rougly at the same time.
This creates a spike in state deletion requests from JM to DFS.
That can delay new checkpoints because of how JM IO tasks are scheduled:
- every deletion is a separate task in the IO thread pool queue
- the queue is FIFO (unbounded)
- the default number of threads in the pool equals number of cores
- so the new checkpoint has to wait for an available thread to initialize its location
Note, that while the checkpoint is waiting, it's already "triggered" on JM, but not broadcasted to any TM.
This PR introduces a random delay in materialization.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? no
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
}, {
"hash" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32656",
"triggerID" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32656)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pnowojski commented on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
pnowojski commented on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060843905
I believe that would contradict the idea of hashing or in other words, it would mean somewhere bad hashing function has been used. I would be surprised if the built in hash functions were susceptible to issues like that. Even if so, finding a hash functions without those issues should be quite easy.
Note scheme like this is being widely used in our code base (keyBy partitioning)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 73d6764f05ac68f1f665f9783de182d8e3255495 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rkhachatryan commented on a change in pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
rkhachatryan commented on a change in pull request #18989:
URL: https://github.com/apache/flink/pull/18989#discussion_r820618527
##########
File path: flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/PeriodicMaterializationManager.java
##########
@@ -104,7 +105,7 @@ public void start() {
LOG.info("Task {} starts periodic materialization", subtaskName);
- scheduleNextMaterialization();
+ scheduleNextMaterialization(new Random().nextInt((int) periodicMaterializeDelay));
Review comment:
Yes, the order is deterministic, but I don't see how this can help.
The hash code can be arbitrarily large, so it needs to be transformed into range `[0..delay)`. For that, the number of operators is needed.
Known is the number of operators in each chain. But it can be the same in different chains, so the delay would be the same.
The number of chains, as well as the total number of operators is unknown on TM AFAIK (at least without involving JM).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
}, {
"hash" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
* 1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rkhachatryan commented on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
rkhachatryan commented on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060809002
I think with this approach only a small fraction of hash code will be used
(because of %) if the interval is small enough.
And depending on the hash function and task naming, this part can be same
or not.
Regards,
Roman
On Mon, Mar 7, 2022, 15:56 Piotr Nowojski ***@***.***> wrote:
> ***@***.**** commented on this pull request.
> ------------------------------
>
> In
> flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/PeriodicMaterializationManager.java
> <https://github.com/apache/flink/pull/18989#discussion_r820784463>:
>
> > @@ -104,7 +105,7 @@ public void start() {
>
> LOG.info("Task {} starts periodic materialization", subtaskName);
>
> - scheduleNextMaterialization();
> + scheduleNextMaterialization(new Random().nextInt((int) periodicMaterializeDelay));
>
> Anyway, can not we use here:
>
> Objects.hash(
> env.getTaskInfo().getTaskName(),
> env.getTaskInfo().getIndexOfThisSubtask(),
> incrementalId++) % periodicMaterializeDelay
>
> ?
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/flink/pull/18989#discussion_r820784463>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AA6BX6S7C7DKWT4DJPYMYEDU6YKI7ANCNFSM5QB62PZA>
> .
> Triage notifications on the go with GitHub Mobile for iOS
> <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
> or Android
> <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
>
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
}, {
"hash" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32656",
"triggerID" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"triggerType" : "PUSH"
}, {
"hash" : "54b33720f0d51945cf58fabf4c3c6945e178c1ed",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32811",
"triggerID" : "54b33720f0d51945cf58fabf4c3c6945e178c1ed",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 54b33720f0d51945cf58fabf4c3c6945e178c1ed Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32811)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
}, {
"hash" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
* 1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
}, {
"hash" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32656",
"triggerID" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
* 1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32656)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 73d6764f05ac68f1f665f9783de182d8e3255495 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
}, {
"hash" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
* 1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rkhachatryan commented on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
rkhachatryan commented on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1064008740
Thanks for the review @Myasuka.
I've updated the PR, PTAL.
> What do you mean here?
> I've found experimentally that the distribution isn't that much different (true random being slightly better than Murmur3 hashing).
> Is that to say, new Random().nextInt((int) periodicMaterializeDelay) behaves better than current implementation of this PR?
I generated offset numbers using the two approaches and then analyzed the distributions using K-MEANS.
`new Random().nextInt((int) periodicMaterializeDelay)` creates slightly less clusters.
So the answer to your question is yes, persumably it would work silghtly better. On the downside, it would be more susceptive to bad PRNG and not reproducible to some degree.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pnowojski commented on a change in pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #18989:
URL: https://github.com/apache/flink/pull/18989#discussion_r820659060
##########
File path: flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/PeriodicMaterializationManager.java
##########
@@ -104,7 +105,7 @@ public void start() {
LOG.info("Task {} starts periodic materialization", subtaskName);
- scheduleNextMaterialization();
+ scheduleNextMaterialization(new Random().nextInt((int) periodicMaterializeDelay));
Review comment:
Ok, I see.
Are we scheduling periodic materialisation at fixed "wall clock" rate? Or once every N checkpoints?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
}, {
"hash" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 73d6764f05ac68f1f665f9783de182d8e3255495 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588)
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
* 1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
}, {
"hash" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
* 1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rkhachatryan commented on a change in pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
rkhachatryan commented on a change in pull request #18989:
URL: https://github.com/apache/flink/pull/18989#discussion_r820662408
##########
File path: flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/PeriodicMaterializationManager.java
##########
@@ -104,7 +105,7 @@ public void start() {
LOG.info("Task {} starts periodic materialization", subtaskName);
- scheduleNextMaterialization();
+ scheduleNextMaterialization(new Random().nextInt((int) periodicMaterializeDelay));
Review comment:
Materializations are scheduled with a fixed interval according to "wall clock".
In particular, once materialization finishes, it schedules the next one with a fixed delay. This delay is the same for all subtasks/backends. This PR only adds an initial offset to this delay.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pnowojski commented on a change in pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #18989:
URL: https://github.com/apache/flink/pull/18989#discussion_r820605356
##########
File path: flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/PeriodicMaterializationManager.java
##########
@@ -104,7 +105,7 @@ public void start() {
LOG.info("Task {} starts periodic materialization", subtaskName);
- scheduleNextMaterialization();
+ scheduleNextMaterialization(new Random().nextInt((int) periodicMaterializeDelay));
Review comment:
Isn't the initialisation order of the state backends per operator within a subtask deterministic? So wouldn't be a hash of `env.getTaskInfo().getTaskName() + env.getTaskInfo().getIndexOfThisSubtask() + incrementalId` enough?
##########
File path: flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/PeriodicMaterializationManager.java
##########
@@ -104,7 +105,7 @@ public void start() {
LOG.info("Task {} starts periodic materialization", subtaskName);
- scheduleNextMaterialization();
+ scheduleNextMaterialization(new Random().nextInt((int) periodicMaterializeDelay));
Review comment:
Isn't the initialisation order of the state backends per operator within a subtask deterministic? So wouldn't be a hash of `env.getTaskInfo().getTaskName() + env.getTaskInfo().getIndexOfThisSubtask() + incrementalId++` enough?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rkhachatryan commented on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
rkhachatryan commented on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060912214
The requirements for keyBy partitioning are a bit different: distribute keys across groups as evenly as possible (and there are much more keys than key groups).
For materialization, selected values should be equi-distant from each other as much as possible (and there are much less backends than possible delay values).
(AFAIK, murmur hash is used in keyBy partitioning)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
}, {
"hash" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
* 1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 73d6764f05ac68f1f665f9783de182d8e3255495 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588)
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
}, {
"hash" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
* 1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
}, {
"hash" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32656",
"triggerID" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"triggerType" : "PUSH"
}, {
"hash" : "54b33720f0d51945cf58fabf4c3c6945e178c1ed",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32811",
"triggerID" : "54b33720f0d51945cf58fabf4c3c6945e178c1ed",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32656)
* 54b33720f0d51945cf58fabf4c3c6945e178c1ed Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32811)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rkhachatryan commented on a change in pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
rkhachatryan commented on a change in pull request #18989:
URL: https://github.com/apache/flink/pull/18989#discussion_r820618527
##########
File path: flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/PeriodicMaterializationManager.java
##########
@@ -104,7 +105,7 @@ public void start() {
LOG.info("Task {} starts periodic materialization", subtaskName);
- scheduleNextMaterialization();
+ scheduleNextMaterialization(new Random().nextInt((int) periodicMaterializeDelay));
Review comment:
Yes, the order is deterministic, but I don't see how this can help.
The hash code cab be arbitrarily large, so it needs to be transformed into range `[0..delay)`. For that, the number of operators is needed.
Known is the number of operators in each chain. But it can be the same in different chains, so the delay would be the same.
The number of chains, as well as the total number of operators is unknown on TM AFAIK (at least without involving JM).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 73d6764f05ac68f1f665f9783de182d8e3255495 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588)
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pnowojski commented on a change in pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #18989:
URL: https://github.com/apache/flink/pull/18989#discussion_r820784463
##########
File path: flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/PeriodicMaterializationManager.java
##########
@@ -104,7 +105,7 @@ public void start() {
LOG.info("Task {} starts periodic materialization", subtaskName);
- scheduleNextMaterialization();
+ scheduleNextMaterialization(new Random().nextInt((int) periodicMaterializeDelay));
Review comment:
Anyway, can not we use here:
```
Objects.hash(
env.getTaskInfo().getTaskName(),
env.getTaskInfo().getIndexOfThisSubtask(),
incrementalId++) % periodicMaterializeDelay
```?
##########
File path: flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/PeriodicMaterializationManager.java
##########
@@ -104,7 +105,7 @@ public void start() {
LOG.info("Task {} starts periodic materialization", subtaskName);
- scheduleNextMaterialization();
+ scheduleNextMaterialization(new Random().nextInt((int) periodicMaterializeDelay));
Review comment:
Anyway, can not we use here:
```
Objects.hash(
env.getTaskInfo().getTaskName(),
env.getTaskInfo().getIndexOfThisSubtask(),
incrementalId++) % periodicMaterializeDelay
```
?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
}, {
"hash" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
* 1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pnowojski commented on a change in pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #18989:
URL: https://github.com/apache/flink/pull/18989#discussion_r820521747
##########
File path: flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/PeriodicMaterializationManager.java
##########
@@ -104,7 +105,7 @@ public void start() {
LOG.info("Task {} starts periodic materialization", subtaskName);
- scheduleNextMaterialization();
+ scheduleNextMaterialization(new Random().nextInt((int) periodicMaterializeDelay));
Review comment:
I think it would be better to have this logic deterministic based on for example some combination of uid, task id, subtask id, subtask name. It would make the behaviour less surprisingly to the users. As it is now, for example due to some unrelated failover, job might start miss behaving due to a random chance and debugging/reproducing issues might be impossible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rkhachatryan merged pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
rkhachatryan merged pull request #18989:
URL: https://github.com/apache/flink/pull/18989
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] Myasuka commented on a change in pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
Myasuka commented on a change in pull request #18989:
URL: https://github.com/apache/flink/pull/18989#discussion_r823406204
##########
File path: flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/PeriodicMaterializationManager.java
##########
@@ -105,7 +116,7 @@ public void start() {
LOG.info("Task {} starts periodic materialization", subtaskName);
- scheduleNextMaterialization();
+ scheduleNextMaterialization(initialDelay);
Review comment:
Shall we introduce this another method `scheduleInitialMaterization(initialDelay)` and make all calling of `scheduleNextMaterialization(0)` to `scheduleNextMaterialization()`? I think this looks better.
##########
File path: flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/PeriodicMaterializationManager.java
##########
@@ -96,6 +102,11 @@
Executors.newSingleThreadScheduledExecutor(
new ExecutorThreadFactory(
"periodic-materialization-scheduler-" + subtaskName));
+
+ this.initialDelay =
+ // randomize initial delay to avoid thundering herd problem
+ MathUtils.murmurHash(Objects.hash(operatorId, subtaskIndex))
Review comment:
The `operatorId` which comes from `OperatorSubtaskDescriptionText` already contains the information of `subtaskIndex`, why we still need another `subtaskIndex` here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rkhachatryan commented on a change in pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
rkhachatryan commented on a change in pull request #18989:
URL: https://github.com/apache/flink/pull/18989#discussion_r823426495
##########
File path: flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/PeriodicMaterializationManager.java
##########
@@ -96,6 +102,11 @@
Executors.newSingleThreadScheduledExecutor(
new ExecutorThreadFactory(
"periodic-materialization-scheduler-" + subtaskName));
+
+ this.initialDelay =
+ // randomize initial delay to avoid thundering herd problem
+ MathUtils.murmurHash(Objects.hash(operatorId, subtaskIndex))
Review comment:
Good point, I'll remove `subtaskIndex`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rkhachatryan commented on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
rkhachatryan commented on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1064159246
Thanks for the reviews.
I agree, further improvements on this level wouldn't give much profit.
Instead, we can focus on "prioritization" of async tasks in JM (as discussed in the ticket); or private state ownership, or both.
I'm giong to merge the PR (CI failure unrelated: FLINK-21608; same phase [passed](https://dev.azure.com/khachatryanroman/flink/_build/results?buildId=1463&view=logs&s=ae4f8708-9994-57d3-c2d7-b892156e7812&j=f3dc9b18-b77a-55c1-591e-264c46fe44d1) in private build).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
}, {
"hash" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
* 1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 73d6764f05ac68f1f665f9783de182d8e3255495 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pnowojski commented on a change in pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #18989:
URL: https://github.com/apache/flink/pull/18989#discussion_r820521747
##########
File path: flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/PeriodicMaterializationManager.java
##########
@@ -104,7 +105,7 @@ public void start() {
LOG.info("Task {} starts periodic materialization", subtaskName);
- scheduleNextMaterialization();
+ scheduleNextMaterialization(new Random().nextInt((int) periodicMaterializeDelay));
Review comment:
I think it would be better to have this logic deterministic based on for example some combination of uid, task id, subtask id. It would make the behaviour less surprisingly to the users. As it is now, for example due to some unrelated failover, job might start miss behaving due to a random chance and debugging/reproducing issues might be impossible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pnowojski commented on a change in pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #18989:
URL: https://github.com/apache/flink/pull/18989#discussion_r820659060
##########
File path: flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/PeriodicMaterializationManager.java
##########
@@ -104,7 +105,7 @@ public void start() {
LOG.info("Task {} starts periodic materialization", subtaskName);
- scheduleNextMaterialization();
+ scheduleNextMaterialization(new Random().nextInt((int) periodicMaterializeDelay));
Review comment:
Ok, I see.
Are scheduling periodic materialisation at fixed "wall clock" rate? Or once every N checkpoints?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rkhachatryan commented on a change in pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
rkhachatryan commented on a change in pull request #18989:
URL: https://github.com/apache/flink/pull/18989#discussion_r820559682
##########
File path: flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/PeriodicMaterializationManager.java
##########
@@ -104,7 +105,7 @@ public void start() {
LOG.info("Task {} starts periodic materialization", subtaskName);
- scheduleNextMaterialization();
+ scheduleNextMaterialization(new Random().nextInt((int) periodicMaterializeDelay));
Review comment:
Materializations should be spread evenly across operators, not only subtasks. And while it is possible to use operator ID and e.g. compute hash from it, the total number of operators is unknown. So I don't see how this (hashcode+subtaskIndex) could be transformed into a number with equal probability in range `[0..delay)`. Do you have any ideas?
Besides that:
- operator ID itself is generated randomly (so re-submitted a job will have different order)
- issues with Random number generation will likely affect operator ID too, as well as many other places
- hashing can be thought of as a form of randomization; distribution quality depends on the hash function, instead of RND; which is better in most cases, but not all
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
}, {
"hash" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
* 1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
}, {
"hash" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
* 1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rkhachatryan commented on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
rkhachatryan commented on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1061225590
I've found experimentally that the distribution isn't that much different (true random being slightly better than Murmur3 hashing).
I don't have very strong opinion on the other points, so I'm fine with either approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] rkhachatryan commented on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
rkhachatryan commented on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1061240056
I've updated the PR, please take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
}, {
"hash" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32656",
"triggerID" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"triggerType" : "PUSH"
}, {
"hash" : "54b33720f0d51945cf58fabf4c3c6945e178c1ed",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32811",
"triggerID" : "54b33720f0d51945cf58fabf4c3c6945e178c1ed",
"triggerType" : "PUSH"
}, {
"hash" : "ed26e3dbf81a7f606941c473be3b12348630cc9b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32827",
"triggerID" : "ed26e3dbf81a7f606941c473be3b12348630cc9b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 54b33720f0d51945cf58fabf4c3c6945e178c1ed Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32811)
* ed26e3dbf81a7f606941c473be3b12348630cc9b Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32827)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0f278f1bbd056fbecabbc783479ab1a10ed60628 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
}, {
"hash" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32656",
"triggerID" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"triggerType" : "PUSH"
}, {
"hash" : "54b33720f0d51945cf58fabf4c3c6945e178c1ed",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "54b33720f0d51945cf58fabf4c3c6945e178c1ed",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32656)
* 54b33720f0d51945cf58fabf4c3c6945e178c1ed UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
}, {
"hash" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32656",
"triggerID" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"triggerType" : "PUSH"
}, {
"hash" : "54b33720f0d51945cf58fabf4c3c6945e178c1ed",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32811",
"triggerID" : "54b33720f0d51945cf58fabf4c3c6945e178c1ed",
"triggerType" : "PUSH"
}, {
"hash" : "ed26e3dbf81a7f606941c473be3b12348630cc9b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ed26e3dbf81a7f606941c473be3b12348630cc9b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 54b33720f0d51945cf58fabf4c3c6945e178c1ed Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32811)
* ed26e3dbf81a7f606941c473be3b12348630cc9b UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #18989: [FLINK-26306][state/changelog] Randomly offset materialization
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #18989:
URL: https://github.com/apache/flink/pull/18989#issuecomment-1060121028
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32588",
"triggerID" : "73d6764f05ac68f1f665f9783de182d8e3255495",
"triggerType" : "PUSH"
}, {
"hash" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32652",
"triggerID" : "0f278f1bbd056fbecabbc783479ab1a10ed60628",
"triggerType" : "PUSH"
}, {
"hash" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32656",
"triggerID" : "1da5d40f751ebb3d9bc2382fb7ef5ab52c69bf4f",
"triggerType" : "PUSH"
}, {
"hash" : "54b33720f0d51945cf58fabf4c3c6945e178c1ed",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32811",
"triggerID" : "54b33720f0d51945cf58fabf4c3c6945e178c1ed",
"triggerType" : "PUSH"
}, {
"hash" : "ed26e3dbf81a7f606941c473be3b12348630cc9b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32827",
"triggerID" : "ed26e3dbf81a7f606941c473be3b12348630cc9b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ed26e3dbf81a7f606941c473be3b12348630cc9b Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=32827)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org