You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2021/10/14 07:57:28 UTC
[GitHub] [flink] akalash opened a new pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
akalash opened a new pull request #17473:
URL: https://github.com/apache/flink/pull/17473
<!--
*Thank you very much for contributing to Apache Flink - we are happy that you want to help us improve Flink. To help the community review your contribution in the best possible way, please go through the checklist below, which will get the contribution into a shape in which it can be best reviewed.*
*Please understand that we do not do this to make contributions to Flink a hassle. In order to uphold a high standard of quality for code contributions, while at the same time managing a large number of contributions, we need contributors to prepare the contributions well, and give reviewers enough contextual information for the review. Please also understand that contributions that do not follow this guide will take longer to review and thus typically be picked up with lower priority by the community.*
## Contribution Checklist
- Make sure that the pull request corresponds to a [JIRA issue](https://issues.apache.org/jira/projects/FLINK/issues). Exceptions are made for typos in JavaDoc or documentation files, which need no JIRA issue.
- Name the pull request in the form "[FLINK-XXXX] [component] Title of the pull request", where *FLINK-XXXX* should be replaced by the actual issue number. Skip *component* if you are unsure about which is the best component.
Typo fixes that have no associated JIRA issue should be named following this pattern: `[hotfix] [docs] Fix typo in event time introduction` or `[hotfix] [javadocs] Expand JavaDoc for PuncuatedWatermarkGenerator`.
- Fill out the template below to describe the changes contributed by the pull request. That will give reviewers the context they need to do the review.
- Make sure that the change passes the automated tests, i.e., `mvn clean verify` passes. You can set up Azure Pipelines CI to do that following [this guide](https://cwiki.apache.org/confluence/display/FLINK/Azure+Pipelines#AzurePipelines-Tutorial:SettingupAzurePipelinesforaforkoftheFlinkrepository).
- Each pull request should address only one issue, not mix up code from multiple issues.
- Each commit in the pull request has a meaningful commit message (including the JIRA id)
- Once all items of the checklist are addressed, remove the above text and this checklist, leaving only the filled out template below.
**(The sections below can be removed for hotfixes of typos)**
-->
## What is the purpose of the change
This PR moves the EMA from the throughput calculation to buffer size caclculation
## Brief change log
- *Removed EMA for the throughput*
- *Added EMA for the buffer size calcualtion*
## Verifying this change
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**)
- The serializers: (yes / **no** / don't know)
- The runtime per-record code paths (performance sensitive): (yes / **no** / don't know)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / **no** / don't know)
- The S3 file system connector: (yes / **no** / don't know)
## Documentation
- Does this pull request introduce a new feature? (yes / **no**)
- If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17473:
URL: https://github.com/apache/flink/pull/17473#issuecomment-943109733
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25034",
"triggerID" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"triggerType" : "PUSH"
}, {
"hash" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25059",
"triggerID" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"triggerType" : "PUSH"
}, {
"hash" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25105",
"triggerID" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"triggerType" : "PUSH"
}, {
"hash" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25145",
"triggerID" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"triggerType" : "PUSH"
}, {
"hash" : "18dcbd3ac5aae3d37399c5e038bb0cf0982d4654",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25512",
"triggerID" : "18dcbd3ac5aae3d37399c5e038bb0cf0982d4654",
"triggerType" : "PUSH"
}, {
"hash" : "d29693c8a1eedab92a61067e10bbd225937513a3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25520",
"triggerID" : "d29693c8a1eedab92a61067e10bbd225937513a3",
"triggerType" : "PUSH"
}, {
"hash" : "5ebbdaaeb0298f6ce2a0d2ba20460d4e85509207",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5ebbdaaeb0298f6ce2a0d2ba20460d4e85509207",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d29693c8a1eedab92a61067e10bbd225937513a3 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25520)
* 5ebbdaaeb0298f6ce2a0d2ba20460d4e85509207 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17473:
URL: https://github.com/apache/flink/pull/17473#issuecomment-943109733
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25034",
"triggerID" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"triggerType" : "PUSH"
}, {
"hash" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25059",
"triggerID" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"triggerType" : "PUSH"
}, {
"hash" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25105",
"triggerID" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"triggerType" : "PUSH"
}, {
"hash" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3acc3ea634a7b4fba02a5fbf7315de6028c416a7 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25105)
* 9d44f3d3ef32b0dd154dd209700773ba83ab4486 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17473:
URL: https://github.com/apache/flink/pull/17473#issuecomment-943109733
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25034",
"triggerID" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"triggerType" : "PUSH"
}, {
"hash" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25059",
"triggerID" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"triggerType" : "PUSH"
}, {
"hash" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25105",
"triggerID" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"triggerType" : "PUSH"
}, {
"hash" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25145",
"triggerID" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"triggerType" : "PUSH"
}, {
"hash" : "18dcbd3ac5aae3d37399c5e038bb0cf0982d4654",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25512",
"triggerID" : "18dcbd3ac5aae3d37399c5e038bb0cf0982d4654",
"triggerType" : "PUSH"
}, {
"hash" : "d29693c8a1eedab92a61067e10bbd225937513a3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25520",
"triggerID" : "d29693c8a1eedab92a61067e10bbd225937513a3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 18dcbd3ac5aae3d37399c5e038bb0cf0982d4654 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25512)
* d29693c8a1eedab92a61067e10bbd225937513a3 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25520)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17473:
URL: https://github.com/apache/flink/pull/17473#issuecomment-943109733
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25034",
"triggerID" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"triggerType" : "PUSH"
}, {
"hash" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25059",
"triggerID" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"triggerType" : "PUSH"
}, {
"hash" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25105",
"triggerID" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"triggerType" : "PUSH"
}, {
"hash" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25145",
"triggerID" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"triggerType" : "PUSH"
}, {
"hash" : "18dcbd3ac5aae3d37399c5e038bb0cf0982d4654",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25512",
"triggerID" : "18dcbd3ac5aae3d37399c5e038bb0cf0982d4654",
"triggerType" : "PUSH"
}, {
"hash" : "d29693c8a1eedab92a61067e10bbd225937513a3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25520",
"triggerID" : "d29693c8a1eedab92a61067e10bbd225937513a3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d29693c8a1eedab92a61067e10bbd225937513a3 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25520)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pnowojski commented on a change in pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #17473:
URL: https://github.com/apache/flink/pull/17473#discussion_r735657956
##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/bufferdebloat/BufferDebloater.java
##########
@@ -94,10 +97,10 @@ public void recalculateBufferSize(long currentThroughput) {
return;
}
- lastBufferSize = newSize;
for (IndexedInputGate inputGate : inputGates) {
inputGate.announceBufferSize(newSize);
}
+ lastBufferSize = newSize;
Review comment:
Ok, I see. Can you extract this to a separate commit, with this explanation in the commit message? Maybe (I'm not sure) also adding a short comment in the code?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17473:
URL: https://github.com/apache/flink/pull/17473#issuecomment-943109733
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25034",
"triggerID" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e10ec336a7b9defc556c45c601f55112f8659c89 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25034)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pnowojski commented on a change in pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #17473:
URL: https://github.com/apache/flink/pull/17473#discussion_r735539660
##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/bufferdebloat/BufferDebloater.java
##########
@@ -94,10 +97,10 @@ public void recalculateBufferSize(long currentThroughput) {
return;
}
- lastBufferSize = newSize;
for (IndexedInputGate inputGate : inputGates) {
inputGate.announceBufferSize(newSize);
}
+ lastBufferSize = newSize;
Review comment:
Is this change relevant?
##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/bufferdebloat/BufferSizeEMA.java
##########
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.runtime.tasks.bufferdebloat;
+
+import static org.apache.flink.util.Preconditions.checkArgument;
+
+/** Implementation of 'Exponential moving average' algorithm. */
+public class BufferSizeEMA {
+ private final int maxBufferSize;
+ private final int minBufferSize;
+ /** EMA algorithm specific constant which responsible for speed of reaction. */
+ private final double alpha;
+
+ private int lastBufferSize;
+
+ public BufferSizeEMA(int maxBufferSize, int minBufferSize, long numberOfSamples) {
+ this.maxBufferSize = maxBufferSize;
+ this.minBufferSize = minBufferSize;
+ alpha = 2.0 / (numberOfSamples + 1);
+ this.lastBufferSize = maxBufferSize;
+ }
+
+ /**
+ * Calculating the buffer size over total possible buffers size and number of buffers in use.
+ *
+ * @param totalBufferSizeInBytes Total buffers size.
+ * @param totalBuffers Total number of buffers in use.
+ * @return Throughput calculated according to implemented algorithm.
+ */
+ public int calculateBufferSize(long totalBufferSizeInBytes, int totalBuffers) {
+ checkArgument(totalBufferSizeInBytes >= 0, "Size of buffer should be non negative");
+ checkArgument(totalBuffers > 0, "Number of buffers should be positive");
+
+ // Limit the possible buffer size by twice of current buffer size in order to have the same
+ // changes speed for increasing and decreasing the size.
Review comment:
Can you elaborate on this explanation? Describe what was the problem with sudden throughput spike
?
##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/bufferdebloat/BufferSizeEMA.java
##########
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.runtime.tasks.bufferdebloat;
+
+import static org.apache.flink.util.Preconditions.checkArgument;
+
+/** Implementation of 'Exponential moving average' algorithm. */
+public class BufferSizeEMA {
+ private final int maxBufferSize;
+ private final int minBufferSize;
+ /** EMA algorithm specific constant which responsible for speed of reaction. */
+ private final double alpha;
+
+ private int lastBufferSize;
+
+ public BufferSizeEMA(int maxBufferSize, int minBufferSize, long numberOfSamples) {
+ this.maxBufferSize = maxBufferSize;
+ this.minBufferSize = minBufferSize;
+ alpha = 2.0 / (numberOfSamples + 1);
+ this.lastBufferSize = maxBufferSize;
+ }
+
+ /**
+ * Calculating the buffer size over total possible buffers size and number of buffers in use.
+ *
+ * @param totalBufferSizeInBytes Total buffers size.
+ * @param totalBuffers Total number of buffers in use.
+ * @return Throughput calculated according to implemented algorithm.
+ */
+ public int calculateBufferSize(long totalBufferSizeInBytes, int totalBuffers) {
+ checkArgument(totalBufferSizeInBytes >= 0, "Size of buffer should be non negative");
+ checkArgument(totalBuffers > 0, "Number of buffers should be positive");
+
+ // Limit the possible buffer size by twice of current buffer size in order to have the same
+ // changes speed for increasing and decreasing the size.
+ long desirableBufferSize =
+ Math.min(totalBufferSizeInBytes / totalBuffers, 2L * lastBufferSize);
Review comment:
Shouldn't this be
```
Math.min(totalBufferSizeInBytes / totalBuffers, 2L * maxBufferSize);
```
?
Otherwise won't growing from `minBufferSize` to `maxBufferSize` be actually very slow compared to shrinking from `max` to `min`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pnowojski merged pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
pnowojski merged pull request #17473:
URL: https://github.com/apache/flink/pull/17473
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17473:
URL: https://github.com/apache/flink/pull/17473#issuecomment-943109733
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25034",
"triggerID" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"triggerType" : "PUSH"
}, {
"hash" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25059",
"triggerID" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e10ec336a7b9defc556c45c601f55112f8659c89 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25034)
* e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25059)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17473:
URL: https://github.com/apache/flink/pull/17473#issuecomment-943109733
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25034",
"triggerID" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"triggerType" : "PUSH"
}, {
"hash" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25059",
"triggerID" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"triggerType" : "PUSH"
}, {
"hash" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25105",
"triggerID" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"triggerType" : "PUSH"
}, {
"hash" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25145",
"triggerID" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"triggerType" : "PUSH"
}, {
"hash" : "18dcbd3ac5aae3d37399c5e038bb0cf0982d4654",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25512",
"triggerID" : "18dcbd3ac5aae3d37399c5e038bb0cf0982d4654",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9d44f3d3ef32b0dd154dd209700773ba83ab4486 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25145)
* 18dcbd3ac5aae3d37399c5e038bb0cf0982d4654 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25512)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17473:
URL: https://github.com/apache/flink/pull/17473#issuecomment-943109733
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25034",
"triggerID" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"triggerType" : "PUSH"
}, {
"hash" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25059",
"triggerID" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"triggerType" : "PUSH"
}, {
"hash" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25105",
"triggerID" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"triggerType" : "PUSH"
}, {
"hash" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25145",
"triggerID" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3acc3ea634a7b4fba02a5fbf7315de6028c416a7 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25105)
* 9d44f3d3ef32b0dd154dd209700773ba83ab4486 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25145)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] akalash commented on a change in pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
akalash commented on a change in pull request #17473:
URL: https://github.com/apache/flink/pull/17473#discussion_r735646141
##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/bufferdebloat/BufferDebloater.java
##########
@@ -94,10 +97,10 @@ public void recalculateBufferSize(long currentThroughput) {
return;
}
- lastBufferSize = newSize;
for (IndexedInputGate inputGate : inputGates) {
inputGate.announceBufferSize(newSize);
}
+ lastBufferSize = newSize;
Review comment:
The main idea of this change is to avoid updating the local value if the announcement fails. Right now, after one fix it doesn't make sense because if the announcement fails we restart the task but anyway this order when we announce and only then update the local value looks correct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17473:
URL: https://github.com/apache/flink/pull/17473#issuecomment-943109733
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25034",
"triggerID" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"triggerType" : "PUSH"
}, {
"hash" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25059",
"triggerID" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"triggerType" : "PUSH"
}, {
"hash" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25105",
"triggerID" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"triggerType" : "PUSH"
}, {
"hash" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25145",
"triggerID" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"triggerType" : "PUSH"
}, {
"hash" : "18dcbd3ac5aae3d37399c5e038bb0cf0982d4654",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25512",
"triggerID" : "18dcbd3ac5aae3d37399c5e038bb0cf0982d4654",
"triggerType" : "PUSH"
}, {
"hash" : "d29693c8a1eedab92a61067e10bbd225937513a3",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25520",
"triggerID" : "d29693c8a1eedab92a61067e10bbd225937513a3",
"triggerType" : "PUSH"
}, {
"hash" : "5ebbdaaeb0298f6ce2a0d2ba20460d4e85509207",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25561",
"triggerID" : "5ebbdaaeb0298f6ce2a0d2ba20460d4e85509207",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5ebbdaaeb0298f6ce2a0d2ba20460d4e85509207 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25561)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17473:
URL: https://github.com/apache/flink/pull/17473#issuecomment-943109733
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25034",
"triggerID" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"triggerType" : "PUSH"
}, {
"hash" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25059",
"triggerID" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"triggerType" : "PUSH"
}, {
"hash" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25105",
"triggerID" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"triggerType" : "PUSH"
}, {
"hash" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25145",
"triggerID" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"triggerType" : "PUSH"
}, {
"hash" : "18dcbd3ac5aae3d37399c5e038bb0cf0982d4654",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "18dcbd3ac5aae3d37399c5e038bb0cf0982d4654",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9d44f3d3ef32b0dd154dd209700773ba83ab4486 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25145)
* 18dcbd3ac5aae3d37399c5e038bb0cf0982d4654 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pnowojski commented on a change in pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #17473:
URL: https://github.com/apache/flink/pull/17473#discussion_r736241227
##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/bufferdebloat/BufferSizeEMA.java
##########
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.runtime.tasks.bufferdebloat;
+
+import static org.apache.flink.util.Preconditions.checkArgument;
+
+/** Implementation of 'Exponential moving average' algorithm. */
+public class BufferSizeEMA {
+ private final int maxBufferSize;
+ private final int minBufferSize;
+ /** EMA algorithm specific constant which responsible for speed of reaction. */
+ private final double alpha;
+
+ private int lastBufferSize;
+
+ public BufferSizeEMA(int maxBufferSize, int minBufferSize, long numberOfSamples) {
+ this.maxBufferSize = maxBufferSize;
+ this.minBufferSize = minBufferSize;
+ alpha = 2.0 / (numberOfSamples + 1);
+ this.lastBufferSize = maxBufferSize;
+ }
+
+ /**
+ * Calculating the buffer size over total possible buffers size and number of buffers in use.
+ *
+ * @param totalBufferSizeInBytes Total buffers size.
+ * @param totalBuffers Total number of buffers in use.
+ * @return Throughput calculated according to implemented algorithm.
+ */
+ public int calculateBufferSize(long totalBufferSizeInBytes, int totalBuffers) {
+ checkArgument(totalBufferSizeInBytes >= 0, "Size of buffer should be non negative");
+ checkArgument(totalBuffers > 0, "Number of buffers should be positive");
+
+ // Limit the possible buffer size by twice of current buffer size in order to have the same
+ // changes speed for increasing and decreasing the size.
+ long desirableBufferSize =
+ Math.min(totalBufferSizeInBytes / totalBuffers, 2L * lastBufferSize);
Review comment:
Ok, can you adapt/copy/paste the above explanation into the comment in the code? You can skip comparison to `maxValue`, but stress that this aims to have the same growth and shrink speeds, with this example:
```
Example of value change(the difference for the small value even large):
decrease = 32768, 29647, 26823, 24268, 21956, 19864
increase (2* currentValue) = 19864, 21755, 23826, 26095, 28580, 31301, 32768
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] akalash commented on a change in pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
akalash commented on a change in pull request #17473:
URL: https://github.com/apache/flink/pull/17473#discussion_r735666888
##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/bufferdebloat/BufferSizeEMA.java
##########
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.runtime.tasks.bufferdebloat;
+
+import static org.apache.flink.util.Preconditions.checkArgument;
+
+/** Implementation of 'Exponential moving average' algorithm. */
+public class BufferSizeEMA {
+ private final int maxBufferSize;
+ private final int minBufferSize;
+ /** EMA algorithm specific constant which responsible for speed of reaction. */
+ private final double alpha;
+
+ private int lastBufferSize;
+
+ public BufferSizeEMA(int maxBufferSize, int minBufferSize, long numberOfSamples) {
+ this.maxBufferSize = maxBufferSize;
+ this.minBufferSize = minBufferSize;
+ alpha = 2.0 / (numberOfSamples + 1);
+ this.lastBufferSize = maxBufferSize;
+ }
+
+ /**
+ * Calculating the buffer size over total possible buffers size and number of buffers in use.
+ *
+ * @param totalBufferSizeInBytes Total buffers size.
+ * @param totalBuffers Total number of buffers in use.
+ * @return Throughput calculated according to implemented algorithm.
+ */
+ public int calculateBufferSize(long totalBufferSizeInBytes, int totalBuffers) {
+ checkArgument(totalBufferSizeInBytes >= 0, "Size of buffer should be non negative");
+ checkArgument(totalBuffers > 0, "Number of buffers should be positive");
+
+ // Limit the possible buffer size by twice of current buffer size in order to have the same
+ // changes speed for increasing and decreasing the size.
Review comment:
Should I elaborate here or in the comment? I can expand my comment with "Since the result value is always limited by max buffer size while the instant value is potentially unlimited. It can lead to an instant change from min to max value in case when the instant value is significantly large than the possible max value."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17473:
URL: https://github.com/apache/flink/pull/17473#issuecomment-943109733
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25034",
"triggerID" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"triggerType" : "PUSH"
}, {
"hash" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25059",
"triggerID" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"triggerType" : "PUSH"
}, {
"hash" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25105",
"triggerID" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"triggerType" : "PUSH"
}, {
"hash" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25145",
"triggerID" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"triggerType" : "PUSH"
}, {
"hash" : "18dcbd3ac5aae3d37399c5e038bb0cf0982d4654",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25512",
"triggerID" : "18dcbd3ac5aae3d37399c5e038bb0cf0982d4654",
"triggerType" : "PUSH"
}, {
"hash" : "d29693c8a1eedab92a61067e10bbd225937513a3",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d29693c8a1eedab92a61067e10bbd225937513a3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9d44f3d3ef32b0dd154dd209700773ba83ab4486 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25145)
* 18dcbd3ac5aae3d37399c5e038bb0cf0982d4654 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25512)
* d29693c8a1eedab92a61067e10bbd225937513a3 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17473:
URL: https://github.com/apache/flink/pull/17473#issuecomment-943109733
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25034",
"triggerID" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"triggerType" : "PUSH"
}, {
"hash" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25059",
"triggerID" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"triggerType" : "PUSH"
}, {
"hash" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25105",
"triggerID" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"triggerType" : "PUSH"
}, {
"hash" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25145",
"triggerID" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"triggerType" : "PUSH"
}, {
"hash" : "18dcbd3ac5aae3d37399c5e038bb0cf0982d4654",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25512",
"triggerID" : "18dcbd3ac5aae3d37399c5e038bb0cf0982d4654",
"triggerType" : "PUSH"
}, {
"hash" : "d29693c8a1eedab92a61067e10bbd225937513a3",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25520",
"triggerID" : "d29693c8a1eedab92a61067e10bbd225937513a3",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9d44f3d3ef32b0dd154dd209700773ba83ab4486 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25145)
* 18dcbd3ac5aae3d37399c5e038bb0cf0982d4654 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25512)
* d29693c8a1eedab92a61067e10bbd225937513a3 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25520)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pnowojski commented on a change in pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #17473:
URL: https://github.com/apache/flink/pull/17473#discussion_r735657956
##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/bufferdebloat/BufferDebloater.java
##########
@@ -94,10 +97,10 @@ public void recalculateBufferSize(long currentThroughput) {
return;
}
- lastBufferSize = newSize;
for (IndexedInputGate inputGate : inputGates) {
inputGate.announceBufferSize(newSize);
}
+ lastBufferSize = newSize;
Review comment:
Ok, I see. So it's not related to the EMA calculation and FLINK-24470 after all? If so, can you extract this to a separate commit, with this explanation in the commit message? Maybe (I'm not sure) also adding a short comment in the code?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pnowojski commented on a change in pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
pnowojski commented on a change in pull request #17473:
URL: https://github.com/apache/flink/pull/17473#discussion_r736238309
##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/bufferdebloat/BufferSizeEMA.java
##########
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.runtime.tasks.bufferdebloat;
+
+import static org.apache.flink.util.Preconditions.checkArgument;
+
+/** Implementation of 'Exponential moving average' algorithm. */
+public class BufferSizeEMA {
+ private final int maxBufferSize;
+ private final int minBufferSize;
+ /** EMA algorithm specific constant which responsible for speed of reaction. */
+ private final double alpha;
+
+ private int lastBufferSize;
+
+ public BufferSizeEMA(int maxBufferSize, int minBufferSize, long numberOfSamples) {
+ this.maxBufferSize = maxBufferSize;
+ this.minBufferSize = minBufferSize;
+ alpha = 2.0 / (numberOfSamples + 1);
+ this.lastBufferSize = maxBufferSize;
+ }
+
+ /**
+ * Calculating the buffer size over total possible buffers size and number of buffers in use.
+ *
+ * @param totalBufferSizeInBytes Total buffers size.
+ * @param totalBuffers Total number of buffers in use.
+ * @return Throughput calculated according to implemented algorithm.
+ */
+ public int calculateBufferSize(long totalBufferSizeInBytes, int totalBuffers) {
+ checkArgument(totalBufferSizeInBytes >= 0, "Size of buffer should be non negative");
+ checkArgument(totalBuffers > 0, "Number of buffers should be positive");
+
+ // Limit the possible buffer size by twice of current buffer size in order to have the same
+ // changes speed for increasing and decreasing the size.
Review comment:
I meant to change this in the code :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17473:
URL: https://github.com/apache/flink/pull/17473#issuecomment-943109733
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25034",
"triggerID" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e10ec336a7b9defc556c45c601f55112f8659c89 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25034)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] akalash commented on a change in pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
akalash commented on a change in pull request #17473:
URL: https://github.com/apache/flink/pull/17473#discussion_r735707775
##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/bufferdebloat/BufferSizeEMA.java
##########
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.streaming.runtime.tasks.bufferdebloat;
+
+import static org.apache.flink.util.Preconditions.checkArgument;
+
+/** Implementation of 'Exponential moving average' algorithm. */
+public class BufferSizeEMA {
+ private final int maxBufferSize;
+ private final int minBufferSize;
+ /** EMA algorithm specific constant which responsible for speed of reaction. */
+ private final double alpha;
+
+ private int lastBufferSize;
+
+ public BufferSizeEMA(int maxBufferSize, int minBufferSize, long numberOfSamples) {
+ this.maxBufferSize = maxBufferSize;
+ this.minBufferSize = minBufferSize;
+ alpha = 2.0 / (numberOfSamples + 1);
+ this.lastBufferSize = maxBufferSize;
+ }
+
+ /**
+ * Calculating the buffer size over total possible buffers size and number of buffers in use.
+ *
+ * @param totalBufferSizeInBytes Total buffers size.
+ * @param totalBuffers Total number of buffers in use.
+ * @return Throughput calculated according to implemented algorithm.
+ */
+ public int calculateBufferSize(long totalBufferSizeInBytes, int totalBuffers) {
+ checkArgument(totalBufferSizeInBytes >= 0, "Size of buffer should be non negative");
+ checkArgument(totalBuffers > 0, "Number of buffers should be positive");
+
+ // Limit the possible buffer size by twice of current buffer size in order to have the same
+ // changes speed for increasing and decreasing the size.
+ long desirableBufferSize =
+ Math.min(totalBufferSizeInBytes / totalBuffers, 2L * lastBufferSize);
Review comment:
Not actually. The idea is to have a symmetric for the current value. for example if the instant value is equal to 0 and the current value is 16000 we can decrease it at maximum by 1600(suppose alfa=0.1) . my idea was to allow increase and decrease by the same number. So according to my implementation if the instant value would be large(for example 100000) it will be possible to increase the current value by 1600(the same as decreasing) because the limit will be 2 * currentValue = 32000.
But if we keep the maximum value constant(2 * maxBufferSize) this means that we decrease the value relatively slow but increase value pretty fast especially when it is small(which is not so bad for some number of cases).
I have actually tried both options. They all work the same in most cases but my current solution is a little less sensitive to spikes when buffer size is small which I decided the most useful case.
Example of value change(the difference for the small value even large):
decrease = 32768, 29647, 26823, 24268, 21956, 19864
increase (2* currentValue) = 19864, 21755, 23826, 26095, 28580, 31301, 32768
increase (2* maxValue) = 19864, 24213, 31708, 32768
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17473:
URL: https://github.com/apache/flink/pull/17473#issuecomment-943109733
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25034",
"triggerID" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"triggerType" : "PUSH"
}, {
"hash" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25059",
"triggerID" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"triggerType" : "PUSH"
}, {
"hash" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25105",
"triggerID" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"triggerType" : "PUSH"
}, {
"hash" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25145",
"triggerID" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9d44f3d3ef32b0dd154dd209700773ba83ab4486 Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25145)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17473:
URL: https://github.com/apache/flink/pull/17473#issuecomment-943109733
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25034",
"triggerID" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"triggerType" : "PUSH"
}, {
"hash" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25059",
"triggerID" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"triggerType" : "PUSH"
}, {
"hash" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25105",
"triggerID" : "3acc3ea634a7b4fba02a5fbf7315de6028c416a7",
"triggerType" : "PUSH"
}, {
"hash" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25145",
"triggerID" : "9d44f3d3ef32b0dd154dd209700773ba83ab4486",
"triggerType" : "PUSH"
}, {
"hash" : "18dcbd3ac5aae3d37399c5e038bb0cf0982d4654",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25512",
"triggerID" : "18dcbd3ac5aae3d37399c5e038bb0cf0982d4654",
"triggerType" : "PUSH"
}, {
"hash" : "d29693c8a1eedab92a61067e10bbd225937513a3",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25520",
"triggerID" : "d29693c8a1eedab92a61067e10bbd225937513a3",
"triggerType" : "PUSH"
}, {
"hash" : "5ebbdaaeb0298f6ce2a0d2ba20460d4e85509207",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25561",
"triggerID" : "5ebbdaaeb0298f6ce2a0d2ba20460d4e85509207",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d29693c8a1eedab92a61067e10bbd225937513a3 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25520)
* 5ebbdaaeb0298f6ce2a0d2ba20460d4e85509207 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25561)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17473:
URL: https://github.com/apache/flink/pull/17473#issuecomment-943109733
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25034",
"triggerID" : "e10ec336a7b9defc556c45c601f55112f8659c89",
"triggerType" : "PUSH"
}, {
"hash" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e10ec336a7b9defc556c45c601f55112f8659c89 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=25034)
* e5f5a29ff9c6284a7b3ba16d2657b9b700d3d503 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #17473:
URL: https://github.com/apache/flink/pull/17473#issuecomment-943109733
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #17473: [FLINK-24470][runtime] EMA calculation move to BufferDebloater
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #17473:
URL: https://github.com/apache/flink/pull/17473#issuecomment-943109733
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org