You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/10/30 06:45:12 UTC
[GitHub] [flink] SteNicholas opened a new pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
SteNicholas opened a new pull request #13853:
URL: https://github.com/apache/flink/pull/13853
## What is the purpose of the change
*Currently, both sources and watermark/timestamp operators can emit watermarks that we don't really need. We only need a final watermark in BATCH execution mode. `TimestampsAndWatermarksOperator` should check whether to periodically emit watermarks or only one final watermark at the end of input by the runtime execution mode.*
## Brief change log
- *`TimestampsAndWatermarksOperator` adds `emitProgressiveWatermarks` to check whether to periodically emit watermarks or only one final watermark at the end of input by the runtime execution mode.*
- *`watermarkGenerator` in `TimestampsAndWatermarksOperator` periodically calls the `onPeriodicEmit` method if `emitProgressiveWatermarks` is true.*
## Verifying this change
- *`periodicWatermarksEmitOnPeriodicEmit` and `periodicWatermarksOnlyEmitOnPeriodicEmit` in `TimestampsAndWatermarksOperatorTest` adds the test cases for the batch runtime execution mode to verfiy whether to periodically emit watermarks in batch execution mode.*
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (yes / **no**)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**)
- The serializers: (yes / **no** / don't know)
- The runtime per-record code paths (performance sensitive): (yes / **no** / don't know)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
- The S3 file system connector: (yes / **no** / don't know)
## Documentation
- Does this pull request introduce a new feature? (yes / **no**)
- If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719358730
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8656",
"triggerID" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"triggerType" : "PUSH"
}, {
"hash" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8861",
"triggerID" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"triggerType" : "PUSH"
}, {
"hash" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8875",
"triggerID" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"triggerType" : "PUSH"
}, {
"hash" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8993",
"triggerID" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"triggerType" : "PUSH"
}, {
"hash" : "717de58b51d9369326bedb22240056eaa890acbb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "717de58b51d9369326bedb22240056eaa890acbb",
"triggerType" : "PUSH"
}, {
"hash" : "636a9e0da725b6943b4726687918ee8352724ee4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9003",
"triggerID" : "636a9e0da725b6943b4726687918ee8352724ee4",
"triggerType" : "PUSH"
}, {
"hash" : "75661531fd1abc3f35102b4b6e3af7bac218d981",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9027",
"triggerID" : "75661531fd1abc3f35102b4b6e3af7bac218d981",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 717de58b51d9369326bedb22240056eaa890acbb UNKNOWN
* 75661531fd1abc3f35102b4b6e3af7bac218d981 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9027)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dawidwys closed pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
dawidwys closed pull request #13853:
URL: https://github.com/apache/flink/pull/13853
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719358730
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8656",
"triggerID" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"triggerType" : "PUSH"
}, {
"hash" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8861",
"triggerID" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"triggerType" : "PUSH"
}, {
"hash" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8875",
"triggerID" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"triggerType" : "PUSH"
}, {
"hash" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8993",
"triggerID" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"triggerType" : "PUSH"
}, {
"hash" : "717de58b51d9369326bedb22240056eaa890acbb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "717de58b51d9369326bedb22240056eaa890acbb",
"triggerType" : "PUSH"
}, {
"hash" : "636a9e0da725b6943b4726687918ee8352724ee4",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9003",
"triggerID" : "636a9e0da725b6943b4726687918ee8352724ee4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1190a9b9a01b1665315f710e8afd7230724a61b5 Azure: [CANCELED](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8993)
* 717de58b51d9369326bedb22240056eaa890acbb UNKNOWN
* 636a9e0da725b6943b4726687918ee8352724ee4 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9003)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] SteNicholas commented on a change in pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
SteNicholas commented on a change in pull request #13853:
URL: https://github.com/apache/flink/pull/13853#discussion_r517712768
##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/TimestampsAndWatermarksOperator.java
##########
@@ -83,12 +81,14 @@ public void open() throws Exception {
super.open();
timestampAssigner = watermarkStrategy.createTimestampAssigner(this::getMetricGroup);
- watermarkGenerator = watermarkStrategy.createWatermarkGenerator(this::getMetricGroup);
+ watermarkGenerator = new NoOpWatermarkGenerator<>(
Review comment:
@dawidwys We could reuse `NoWatermarksGenerator` instead of `NoOpWatermarkGenerator `.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719358730
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8656",
"triggerID" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"triggerType" : "PUSH"
}, {
"hash" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8861",
"triggerID" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"triggerType" : "PUSH"
}, {
"hash" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8875",
"triggerID" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"triggerType" : "PUSH"
}, {
"hash" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 462fadcf9b966e3ef627b7173da6036769e2e3cd Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8875)
* 1190a9b9a01b1665315f710e8afd7230724a61b5 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719300959
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit 5fb1b704a6e8daa877405a4e77351d7bd72cd629 (Fri Oct 30 06:47:34 UTC 2020)
**Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up to date!
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] SteNicholas commented on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
SteNicholas commented on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-721775434
> First of all I think your PR does not cover all the cases. You're disabling only the periodic watermarks. We should do the same for the punctuated watermarks (`onEvent`). I think the best way to do it would be to just create a different `watermarkGenerator`. Instead of calling `watermarkStrategy.createWatermarkGenerator` you could create a `NoOpWatermarkGenerator`.
>
> Moreover, could you go over the whole PR and fix indentations of arguments? The argument list should have one more level of indentation than the method body.
@dawidwys , thanks for your detailed suggestions for the punctuated watermarks. It was my mistake that missed this test case for the punctuated watermarks. I have followed your comments to add `NoOpWatermarkGenerator`. If you are available, please help to review again, thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] SteNicholas commented on a change in pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
SteNicholas commented on a change in pull request #13853:
URL: https://github.com/apache/flink/pull/13853#discussion_r516660715
##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/transformations/TimestampsAndWatermarksTransformation.java
##########
@@ -0,0 +1,102 @@
+/*
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+package org.apache.flink.streaming.api.transformations;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.api.common.eventtime.WatermarkStrategy;
+import org.apache.flink.api.common.typeinfo.TypeInformation;
+import org.apache.flink.api.dag.Transformation;
+import org.apache.flink.streaming.api.operators.ChainingStrategy;
+import org.apache.flink.streaming.api.watermark.Watermark;
+
+import org.apache.flink.shaded.guava18.com.google.common.collect.Lists;
+
+import java.util.Collections;
+import java.util.List;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+
+/**
+ * A {@link PhysicalTransformation} for {@link Watermark}.
+ *
+ * @param <IN> The input and output type of the transformation.
+ */
+@Internal
+public class TimestampsAndWatermarksTransformation<IN> extends PhysicalTransformation<IN> {
+
+ private final Transformation<IN> input;
+ private final WatermarkStrategy<IN> watermarkStrategy;
+
+ private ChainingStrategy chainingStrategy = ChainingStrategy.DEFAULT_CHAINING_STRATEGY;
+
+ /**
+ * Creates a new {@code Transformation} with the given name, output type and parallelism.
+ *
+ * @param name The name of the {@code Transformation}, this will be shown in Visualizations
+ * and the Log
+ * @param parallelism The parallelism of this {@code Transformation}
+ * @param input The input transformation of this {@code Transformation}
+ * @param watermarkStrategy The {@link WatermarkStrategy} to use
+ */
+ public TimestampsAndWatermarksTransformation(
+ String name,
Review comment:
@dawidwys sorry for the unmatched idea code style. I would like to modify the indentation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dawidwys commented on a change in pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
dawidwys commented on a change in pull request #13853:
URL: https://github.com/apache/flink/pull/13853#discussion_r517412480
##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/TimestampsAndWatermarksOperator.java
##########
@@ -83,12 +81,14 @@ public void open() throws Exception {
super.open();
timestampAssigner = watermarkStrategy.createTimestampAssigner(this::getMetricGroup);
- watermarkGenerator = watermarkStrategy.createWatermarkGenerator(this::getMetricGroup);
+ watermarkGenerator = new NoOpWatermarkGenerator<>(
+ watermarkStrategy.createWatermarkGenerator(this::getMetricGroup),
+ emitProgressiveWatermarks);
wmOutput = new WatermarkEmitter(output, getContainingTask().getStreamStatusMaintainer());
watermarkInterval = getExecutionConfig().getAutoWatermarkInterval();
- if (watermarkInterval > 0 && emitProgressiveWatermarks) {
+ if (watermarkInterval > 0) {
Review comment:
This check was valid as we do not want to register timers, if we don't need to.
##########
File path: flink-core/src/main/java/org/apache/flink/api/common/eventtime/NoOpWatermarkGenerator.java
##########
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.api.common.eventtime;
+
+import org.apache.flink.annotation.Public;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+
+/**
+ * An implementation of a {@link WatermarkGenerator} that periodically emits watermarks
+ * or generates only one final watermark at the end of input.
+ */
+@Public
+public final class NoOpWatermarkGenerator<E> implements WatermarkGenerator<E> {
Review comment:
I would make it a private class in the `TimestampsAndWatermarksOperator`. Certainly it should not be a `@Public` class.
Moreover I thought we should rather have a class like:
```
/**
* An implementation of a {@link WatermarkGenerator} that does not emit watermarks.
*/
public final class NoOpWatermarkGenerator<E> implements WatermarkGenerator<E> {
@Override
public void onEvent(E event, long eventTimestamp, WatermarkOutput output) {
}
@Override
public void onPeriodicEmit(WatermarkOutput output) {
}
}
```
##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/TimestampsAndWatermarksOperator.java
##########
@@ -83,12 +81,14 @@ public void open() throws Exception {
super.open();
timestampAssigner = watermarkStrategy.createTimestampAssigner(this::getMetricGroup);
- watermarkGenerator = watermarkStrategy.createWatermarkGenerator(this::getMetricGroup);
+ watermarkGenerator = new NoOpWatermarkGenerator<>(
Review comment:
Here I would do:
```
if (emitProgressiveWatermarks) {
watermarkGenerator = watermarkStrategy.createWatermarkGenerator(this::getMetricGroup);
} else {
watermarkGenerator = new NoOpWatermarkGenerator();
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719358730
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8656",
"triggerID" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"triggerType" : "PUSH"
}, {
"hash" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8861",
"triggerID" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"triggerType" : "PUSH"
}, {
"hash" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8875",
"triggerID" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"triggerType" : "PUSH"
}, {
"hash" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8993",
"triggerID" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"triggerType" : "PUSH"
}, {
"hash" : "717de58b51d9369326bedb22240056eaa890acbb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "717de58b51d9369326bedb22240056eaa890acbb",
"triggerType" : "PUSH"
}, {
"hash" : "636a9e0da725b6943b4726687918ee8352724ee4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9003",
"triggerID" : "636a9e0da725b6943b4726687918ee8352724ee4",
"triggerType" : "PUSH"
}, {
"hash" : "75661531fd1abc3f35102b4b6e3af7bac218d981",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9027",
"triggerID" : "75661531fd1abc3f35102b4b6e3af7bac218d981",
"triggerType" : "PUSH"
}, {
"hash" : "a782abc9b502ded66f61dc82f6221ba3d8ad978d",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9094",
"triggerID" : "a782abc9b502ded66f61dc82f6221ba3d8ad978d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 717de58b51d9369326bedb22240056eaa890acbb UNKNOWN
* a782abc9b502ded66f61dc82f6221ba3d8ad978d Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9094)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719358730
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719358730
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8656",
"triggerID" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"triggerType" : "PUSH"
}, {
"hash" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8861",
"triggerID" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"triggerType" : "PUSH"
}, {
"hash" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8875",
"triggerID" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"triggerType" : "PUSH"
}, {
"hash" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8993",
"triggerID" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"triggerType" : "PUSH"
}, {
"hash" : "717de58b51d9369326bedb22240056eaa890acbb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "717de58b51d9369326bedb22240056eaa890acbb",
"triggerType" : "PUSH"
}, {
"hash" : "636a9e0da725b6943b4726687918ee8352724ee4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9003",
"triggerID" : "636a9e0da725b6943b4726687918ee8352724ee4",
"triggerType" : "PUSH"
}, {
"hash" : "75661531fd1abc3f35102b4b6e3af7bac218d981",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9027",
"triggerID" : "75661531fd1abc3f35102b4b6e3af7bac218d981",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 717de58b51d9369326bedb22240056eaa890acbb UNKNOWN
* 636a9e0da725b6943b4726687918ee8352724ee4 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9003)
* 75661531fd1abc3f35102b4b6e3af7bac218d981 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9027)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719358730
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8656",
"triggerID" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"triggerType" : "PUSH"
}, {
"hash" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8861",
"triggerID" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"triggerType" : "PUSH"
}, {
"hash" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8875",
"triggerID" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"triggerType" : "PUSH"
}, {
"hash" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8993",
"triggerID" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"triggerType" : "PUSH"
}, {
"hash" : "717de58b51d9369326bedb22240056eaa890acbb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "717de58b51d9369326bedb22240056eaa890acbb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 462fadcf9b966e3ef627b7173da6036769e2e3cd Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8875)
* 1190a9b9a01b1665315f710e8afd7230724a61b5 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8993)
* 717de58b51d9369326bedb22240056eaa890acbb UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719358730
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8656",
"triggerID" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"triggerType" : "PUSH"
}, {
"hash" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8861",
"triggerID" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"triggerType" : "PUSH"
}, {
"hash" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8875",
"triggerID" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"triggerType" : "PUSH"
}, {
"hash" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8993",
"triggerID" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"triggerType" : "PUSH"
}, {
"hash" : "717de58b51d9369326bedb22240056eaa890acbb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "717de58b51d9369326bedb22240056eaa890acbb",
"triggerType" : "PUSH"
}, {
"hash" : "636a9e0da725b6943b4726687918ee8352724ee4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9003",
"triggerID" : "636a9e0da725b6943b4726687918ee8352724ee4",
"triggerType" : "PUSH"
}, {
"hash" : "75661531fd1abc3f35102b4b6e3af7bac218d981",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "75661531fd1abc3f35102b4b6e3af7bac218d981",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 717de58b51d9369326bedb22240056eaa890acbb UNKNOWN
* 636a9e0da725b6943b4726687918ee8352724ee4 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9003)
* 75661531fd1abc3f35102b4b6e3af7bac218d981 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dawidwys commented on a change in pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
dawidwys commented on a change in pull request #13853:
URL: https://github.com/apache/flink/pull/13853#discussion_r518022227
##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/TimestampsAndWatermarksOperator.java
##########
@@ -64,24 +65,30 @@
/** The interval (in milliseconds) for periodic watermark probes. Initialized during runtime. */
private transient long watermarkInterval;
- public TimestampsAndWatermarksOperator(
- WatermarkStrategy<T> watermarkStrategy) {
+ /** Whether to periodically emit watermarks or only one final watermark at the end of input.*/
Review comment:
It is not only about periodic. I would say `wheter to emit intermediate watermarks or only one ...`
##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/translators/TimestampsAndWatermarksTransformationTranslator.java
##########
@@ -0,0 +1,77 @@
+/*
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+package org.apache.flink.streaming.runtime.translators;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.streaming.api.graph.TransformationTranslator;
+import org.apache.flink.streaming.api.operators.SimpleOperatorFactory;
+import org.apache.flink.streaming.api.transformations.TimestampsAndWatermarksTransformation;
+import org.apache.flink.streaming.runtime.operators.TimestampsAndWatermarksOperator;
+
+import java.util.Collection;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+
+/**
+ * A {@link TransformationTranslator} for the {@link TimestampsAndWatermarksTransformation}.
+ *
+ * @param <IN> The type of the elements in the input {@code Transformation} of the transformation to translate.
+ */
+@Internal
+public class TimestampsAndWatermarksTransformationTranslator<IN>
+ extends AbstractOneInputTransformationTranslator<IN, IN, TimestampsAndWatermarksTransformation<IN>> {
+
+ @Override
+ protected Collection<Integer> translateForBatchInternal(
+ final TimestampsAndWatermarksTransformation<IN> transformation,
+ final Context context) {
+ return translateInternal(transformation, context, false /* emit progressive watermarks */);
Review comment:
The comments are wrong, right? They are inverted. Moreover how do you feel about calling it `intermediate` watermarks? (we would change it both in comments and in the flag name). `Progressive` sound kind of strange to me.
##########
File path: flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/operators/TimestampsAndWatermarksOperatorTest.java
##########
@@ -87,6 +88,18 @@ public void periodicWatermarksEmitOnPeriodicEmit() throws Exception {
assertThat(pollNextStreamRecord(testHarness), streamRecord(4L, 4L));
assertThat(pollNextLegacyWatermark(testHarness), is(legacyWatermark(3L)));
+
+ testHarness = createBatchHarness(WatermarkStrategy.forGenerator((ctx) -> new PeriodicWatermarkGenerator())
Review comment:
Let's extract it to a separate test. It's always better to test a single case at a time. I would rename the existing test to:
`periodicWatermarksEmitOnPeriodicEmitStreamMode` and the batch one `periodicWatermarksEmitOnPeriodicEmitBatchMode`. The same for the other test.
Moreover in the batch case you are not advancing the processing time. Therefore you are not testing the periodic generator. It passes for `STREAM` mode as well.
Lastly could you add tests for the punctuated watermark generator? The `PeriodicWatermarkGenerator` emits only in `onPeriodicEmit`, we should test that we do not emit from `onEvent` as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] aljoscha commented on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
aljoscha commented on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719457511
Thanks for opening the PR! However, this will not work in the current state. We only know if we use `BATCH` or `STREAMING` execution mode when we're translating the `Transformations` to a `StreamGraph`. Doing the check right in `assignTimestampsAndWatermarks()` will not work because we don't yet know if the job contains only bounded sources or not.
This issue is slightly more complicated but not by much. The steps would be:
1. Turn `assignTimestampsAndWatermarks()` into a logical operation. This means adding a `TimestampsAndWatermarksTransformation` that has a `WatermarkStrategy` as a field. Use this instead of creating an operator directly in `assignTimestampsAndWatermarks()`
2. add a `TransformationTranslator` that will be similar to `SourceTransformationTranslator` or `OneInputTransformationTranslator` that translates the `TimestampsAndWatermarksTransformation` to an actual operator. In there, you will now know if we're in `BATCH` or `STREAMING` mode and can create the correct operator with either progressive watermarks or not.
Does that make sense?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719358730
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8656",
"triggerID" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5fb1b704a6e8daa877405a4e77351d7bd72cd629 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8656)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719358730
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8656",
"triggerID" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"triggerType" : "PUSH"
}, {
"hash" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8861",
"triggerID" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"triggerType" : "PUSH"
}, {
"hash" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8875",
"triggerID" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 462fadcf9b966e3ef627b7173da6036769e2e3cd Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8875)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719358730
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8656",
"triggerID" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"triggerType" : "PUSH"
}, {
"hash" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8861",
"triggerID" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"triggerType" : "PUSH"
}, {
"hash" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8875",
"triggerID" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"triggerType" : "PUSH"
}, {
"hash" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8993",
"triggerID" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"triggerType" : "PUSH"
}, {
"hash" : "717de58b51d9369326bedb22240056eaa890acbb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "717de58b51d9369326bedb22240056eaa890acbb",
"triggerType" : "PUSH"
}, {
"hash" : "636a9e0da725b6943b4726687918ee8352724ee4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9003",
"triggerID" : "636a9e0da725b6943b4726687918ee8352724ee4",
"triggerType" : "PUSH"
}, {
"hash" : "75661531fd1abc3f35102b4b6e3af7bac218d981",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9027",
"triggerID" : "75661531fd1abc3f35102b4b6e3af7bac218d981",
"triggerType" : "PUSH"
}, {
"hash" : "a782abc9b502ded66f61dc82f6221ba3d8ad978d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a782abc9b502ded66f61dc82f6221ba3d8ad978d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 717de58b51d9369326bedb22240056eaa890acbb UNKNOWN
* 75661531fd1abc3f35102b4b6e3af7bac218d981 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9027)
* a782abc9b502ded66f61dc82f6221ba3d8ad978d UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] SteNicholas commented on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
SteNicholas commented on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-720985405
> @SteNicholas Will you still be working on this one?
Sorry for laster commit change. I have already followed your steps as you suggested. Please help to review this pull request.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] aljoscha commented on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
aljoscha commented on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-720962616
@SteNicholas Will you still be working on this one?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dawidwys commented on a change in pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
dawidwys commented on a change in pull request #13853:
URL: https://github.com/apache/flink/pull/13853#discussion_r516621376
##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/transformations/TimestampsAndWatermarksTransformation.java
##########
@@ -0,0 +1,102 @@
+/*
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+package org.apache.flink.streaming.api.transformations;
+
+import org.apache.flink.annotation.Internal;
+import org.apache.flink.api.common.eventtime.WatermarkStrategy;
+import org.apache.flink.api.common.typeinfo.TypeInformation;
+import org.apache.flink.api.dag.Transformation;
+import org.apache.flink.streaming.api.operators.ChainingStrategy;
+import org.apache.flink.streaming.api.watermark.Watermark;
+
+import org.apache.flink.shaded.guava18.com.google.common.collect.Lists;
+
+import java.util.Collections;
+import java.util.List;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+
+/**
+ * A {@link PhysicalTransformation} for {@link Watermark}.
+ *
+ * @param <IN> The input and output type of the transformation.
+ */
+@Internal
+public class TimestampsAndWatermarksTransformation<IN> extends PhysicalTransformation<IN> {
+
+ private final Transformation<IN> input;
+ private final WatermarkStrategy<IN> watermarkStrategy;
+
+ private ChainingStrategy chainingStrategy = ChainingStrategy.DEFAULT_CHAINING_STRATEGY;
+
+ /**
+ * Creates a new {@code Transformation} with the given name, output type and parallelism.
+ *
+ * @param name The name of the {@code Transformation}, this will be shown in Visualizations
+ * and the Log
+ * @param parallelism The parallelism of this {@code Transformation}
+ * @param input The input transformation of this {@code Transformation}
+ * @param watermarkStrategy The {@link WatermarkStrategy} to use
+ */
+ public TimestampsAndWatermarksTransformation(
+ String name,
Review comment:
Could you add an indentation here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719358730
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8656",
"triggerID" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"triggerType" : "PUSH"
}, {
"hash" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8861",
"triggerID" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"triggerType" : "PUSH"
}, {
"hash" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8875",
"triggerID" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"triggerType" : "PUSH"
}, {
"hash" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8993",
"triggerID" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"triggerType" : "PUSH"
}, {
"hash" : "717de58b51d9369326bedb22240056eaa890acbb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "717de58b51d9369326bedb22240056eaa890acbb",
"triggerType" : "PUSH"
}, {
"hash" : "636a9e0da725b6943b4726687918ee8352724ee4",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "636a9e0da725b6943b4726687918ee8352724ee4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 462fadcf9b966e3ef627b7173da6036769e2e3cd Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8875)
* 1190a9b9a01b1665315f710e8afd7230724a61b5 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8993)
* 717de58b51d9369326bedb22240056eaa890acbb UNKNOWN
* 636a9e0da725b6943b4726687918ee8352724ee4 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719358730
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8656",
"triggerID" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"triggerType" : "PUSH"
}, {
"hash" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8861",
"triggerID" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"triggerType" : "PUSH"
}, {
"hash" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8875",
"triggerID" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"triggerType" : "PUSH"
}, {
"hash" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8993",
"triggerID" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"triggerType" : "PUSH"
}, {
"hash" : "717de58b51d9369326bedb22240056eaa890acbb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "717de58b51d9369326bedb22240056eaa890acbb",
"triggerType" : "PUSH"
}, {
"hash" : "636a9e0da725b6943b4726687918ee8352724ee4",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9003",
"triggerID" : "636a9e0da725b6943b4726687918ee8352724ee4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 462fadcf9b966e3ef627b7173da6036769e2e3cd Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8875)
* 1190a9b9a01b1665315f710e8afd7230724a61b5 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8993)
* 717de58b51d9369326bedb22240056eaa890acbb UNKNOWN
* 636a9e0da725b6943b4726687918ee8352724ee4 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9003)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719358730
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5fb1b704a6e8daa877405a4e77351d7bd72cd629 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] SteNicholas commented on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
SteNicholas commented on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719462500
@aljoscha , sorry for my bad implementations. I agree with the steps you provided that use `assignTimestampsAndWatermarks()` logical operation instead of creating an operator directly in `assignTimestampsAndWatermarks()` and add a `TransformationTranslator`. I would like to follow your comments of this implementations immediately. Thanks to your detailed suggestion.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dawidwys commented on a change in pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
dawidwys commented on a change in pull request #13853:
URL: https://github.com/apache/flink/pull/13853#discussion_r517872088
##########
File path: flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/TimestampsAndWatermarksOperator.java
##########
@@ -83,12 +81,14 @@ public void open() throws Exception {
super.open();
timestampAssigner = watermarkStrategy.createTimestampAssigner(this::getMetricGroup);
- watermarkGenerator = watermarkStrategy.createWatermarkGenerator(this::getMetricGroup);
+ watermarkGenerator = new NoOpWatermarkGenerator<>(
Review comment:
Yes! I wasn't aware of that class.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] SteNicholas edited a comment on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
SteNicholas edited a comment on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-720985405
> @SteNicholas Will you still be working on this one?
@aljoscha Sorry for laster commit change. I have already followed your steps as you suggested. Please help to review this pull request.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719358730
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8656",
"triggerID" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"triggerType" : "PUSH"
}, {
"hash" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8861",
"triggerID" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"triggerType" : "PUSH"
}, {
"hash" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8875",
"triggerID" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"triggerType" : "PUSH"
}, {
"hash" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8993",
"triggerID" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"triggerType" : "PUSH"
}, {
"hash" : "717de58b51d9369326bedb22240056eaa890acbb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "717de58b51d9369326bedb22240056eaa890acbb",
"triggerType" : "PUSH"
}, {
"hash" : "636a9e0da725b6943b4726687918ee8352724ee4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9003",
"triggerID" : "636a9e0da725b6943b4726687918ee8352724ee4",
"triggerType" : "PUSH"
}, {
"hash" : "75661531fd1abc3f35102b4b6e3af7bac218d981",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9027",
"triggerID" : "75661531fd1abc3f35102b4b6e3af7bac218d981",
"triggerType" : "PUSH"
}, {
"hash" : "a782abc9b502ded66f61dc82f6221ba3d8ad978d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9094",
"triggerID" : "a782abc9b502ded66f61dc82f6221ba3d8ad978d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 717de58b51d9369326bedb22240056eaa890acbb UNKNOWN
* 75661531fd1abc3f35102b4b6e3af7bac218d981 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9027)
* a782abc9b502ded66f61dc82f6221ba3d8ad978d Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9094)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719358730
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8656",
"triggerID" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"triggerType" : "PUSH"
}, {
"hash" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8861",
"triggerID" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"triggerType" : "PUSH"
}, {
"hash" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8875",
"triggerID" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"triggerType" : "PUSH"
}, {
"hash" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8993",
"triggerID" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 462fadcf9b966e3ef627b7173da6036769e2e3cd Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8875)
* 1190a9b9a01b1665315f710e8afd7230724a61b5 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8993)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13853: [FLINK-19837][DataStream] Don't emit intermediate watermarks from watermark operators in BATCH execution mode
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13853:
URL: https://github.com/apache/flink/pull/13853#issuecomment-719358730
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8656",
"triggerID" : "5fb1b704a6e8daa877405a4e77351d7bd72cd629",
"triggerType" : "PUSH"
}, {
"hash" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8861",
"triggerID" : "f113d15b0ab8de3a8c7b99db0b10cca10b454c44",
"triggerType" : "PUSH"
}, {
"hash" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8875",
"triggerID" : "462fadcf9b966e3ef627b7173da6036769e2e3cd",
"triggerType" : "PUSH"
}, {
"hash" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8993",
"triggerID" : "1190a9b9a01b1665315f710e8afd7230724a61b5",
"triggerType" : "PUSH"
}, {
"hash" : "717de58b51d9369326bedb22240056eaa890acbb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "717de58b51d9369326bedb22240056eaa890acbb",
"triggerType" : "PUSH"
}, {
"hash" : "636a9e0da725b6943b4726687918ee8352724ee4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9003",
"triggerID" : "636a9e0da725b6943b4726687918ee8352724ee4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 717de58b51d9369326bedb22240056eaa890acbb UNKNOWN
* 636a9e0da725b6943b4726687918ee8352724ee4 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=9003)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org