You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by ch...@apache.org on 2018/06/28 05:42:06 UTC
[1/5] flink git commit: [hotfix][table] Fix misleading exception
message
Repository: flink
Updated Branches:
refs/heads/master eb525b7f8 -> ce04965b6
[hotfix][table] Fix misleading exception message
This closes #6163.
Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/aa2029c4
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/aa2029c4
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/aa2029c4
Branch: refs/heads/master
Commit: aa2029c4c00d7480adac1019bbe3f86944245632
Parents: fd72b4e
Author: blueszheng <ki...@163.com>
Authored: Thu Jun 14 17:37:14 2018 +0800
Committer: zentol <ch...@apache.org>
Committed: Wed Jun 27 22:01:16 2018 +0200
----------------------------------------------------------------------
.../scala/org/apache/flink/table/plan/logical/groupWindows.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/flink/blob/aa2029c4/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/plan/logical/groupWindows.scala
----------------------------------------------------------------------
diff --git a/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/plan/logical/groupWindows.scala b/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/plan/logical/groupWindows.scala
index 4a8fb52..31d8b4a 100644
--- a/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/plan/logical/groupWindows.scala
+++ b/flink-libraries/flink-table/src/main/scala/org/apache/flink/table/plan/logical/groupWindows.scala
@@ -59,7 +59,7 @@ case class TumblingGroupWindow(
case _: BatchTableEnvironment
if !(isTimePoint(timeField.resultType) || isLong(timeField.resultType)) =>
ValidationFailure(
- "Tumbling window expects a time attribute for grouping in a stream environment.")
+ "Tumbling window expects a time attribute for grouping in a batch environment.")
// check row intervals on event-time
case _: StreamTableEnvironment
[2/5] flink git commit: [hotfix][docs] Remove duplicate line
Posted by ch...@apache.org.
[hotfix][docs] Remove duplicate line
This closes #6179.
Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/fd72b4e7
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/fd72b4e7
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/fd72b4e7
Branch: refs/heads/master
Commit: fd72b4e7835b5d97a2d8b61a3bf12a0369488633
Parents: afcb513
Author: Tobii42 <68...@users.noreply.github.com>
Authored: Mon Jun 18 16:55:59 2018 +0200
Committer: zentol <ch...@apache.org>
Committed: Wed Jun 27 22:01:16 2018 +0200
----------------------------------------------------------------------
docs/dev/stream/operators/windows.md | 1 -
1 file changed, 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/flink/blob/fd72b4e7/docs/dev/stream/operators/windows.md
----------------------------------------------------------------------
diff --git a/docs/dev/stream/operators/windows.md b/docs/dev/stream/operators/windows.md
index ad2b516..f657c80 100644
--- a/docs/dev/stream/operators/windows.md
+++ b/docs/dev/stream/operators/windows.md
@@ -1412,7 +1412,6 @@ val globalResults = resultsPerKey
In this example, the results for time window `[0, 5)` from the first operation will also end up in
time window `[0, 5)` in the subsequent windowed operation. This allows calculating a sum per key
and then calculating the top-k elements within the same window in the second operation.
-and then calculating the top-k elements within the same window in the second operation.
## Useful state size considerations
[4/5] flink git commit: [FLINK-9629][metrics] Include dependencies in
datadog reporter jar
Posted by ch...@apache.org.
[FLINK-9629][metrics] Include dependencies in datadog reporter jar
This closes #6191.
Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/afcb513e
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/afcb513e
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/afcb513e
Branch: refs/heads/master
Commit: afcb513e21afaeab7289c0e51222c261d5d0150a
Parents: b4574c9
Author: Georgii Gobozov <go...@gmail.com>
Authored: Wed Jun 20 17:17:52 2018 -0700
Committer: zentol <ch...@apache.org>
Committed: Wed Jun 27 22:01:16 2018 +0200
----------------------------------------------------------------------
flink-metrics/flink-metrics-datadog/pom.xml | 6 ++++++
1 file changed, 6 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/flink/blob/afcb513e/flink-metrics/flink-metrics-datadog/pom.xml
----------------------------------------------------------------------
diff --git a/flink-metrics/flink-metrics-datadog/pom.xml b/flink-metrics/flink-metrics-datadog/pom.xml
index 8614145..da4c2e5 100644
--- a/flink-metrics/flink-metrics-datadog/pom.xml
+++ b/flink-metrics/flink-metrics-datadog/pom.xml
@@ -66,6 +66,12 @@ under the License.
<goal>shade</goal>
</goals>
<configuration>
+ <artifactSet>
+ <includes>
+ <include>com.squareup.okhttp3:*</include>
+ <include>com.squareup.okio:*</include>
+ </includes>
+ </artifactSet>
<promoteTransitiveDependencies>true</promoteTransitiveDependencies>
<relocations combine.children="append">
<relocation>
[5/5] flink git commit: [FLINK-9677][rest] Cleanup encoder after
request has been processed
Posted by ch...@apache.org.
[FLINK-9677][rest] Cleanup encoder after request has been processed
This closes #6217.
Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/ce04965b
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/ce04965b
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/ce04965b
Branch: refs/heads/master
Commit: ce04965b647c734c9db9f596d1ef6a55c8d28c2b
Parents: aa2029c
Author: zentol <ch...@apache.org>
Authored: Wed Jun 27 09:34:34 2018 +0200
Committer: zentol <ch...@apache.org>
Committed: Wed Jun 27 22:01:17 2018 +0200
----------------------------------------------------------------------
.../org/apache/flink/runtime/rest/FileUploadHandler.java | 7 ++++++-
.../main/java/org/apache/flink/runtime/rest/RestClient.java | 8 ++++----
.../apache/flink/runtime/rest/MultipartUploadResource.java | 5 ++++-
3 files changed, 14 insertions(+), 6 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/flink/blob/ce04965b/flink-runtime/src/main/java/org/apache/flink/runtime/rest/FileUploadHandler.java
----------------------------------------------------------------------
diff --git a/flink-runtime/src/main/java/org/apache/flink/runtime/rest/FileUploadHandler.java b/flink-runtime/src/main/java/org/apache/flink/runtime/rest/FileUploadHandler.java
index 0c910d2..aa21ddf 100644
--- a/flink-runtime/src/main/java/org/apache/flink/runtime/rest/FileUploadHandler.java
+++ b/flink-runtime/src/main/java/org/apache/flink/runtime/rest/FileUploadHandler.java
@@ -174,7 +174,12 @@ public class FileUploadHandler extends SimpleChannelInboundHandler<HttpObject> {
private void reset() {
// destroy() can fail because some data is stored multiple times in the decoder causing an IllegalReferenceCountException
// see https://github.com/netty/netty/issues/7814
- currentHttpPostRequestDecoder.getBodyHttpDatas().clear();
+ try {
+ currentHttpPostRequestDecoder.getBodyHttpDatas().clear();
+ } catch (HttpPostRequestDecoder.NotEnoughDataDecoderException ned) {
+ // this method always fails if not all chunks were offered to the decoder yet
+ LOG.debug("Error while resetting {}.", getClass(), ned);
+ }
currentHttpPostRequestDecoder.destroy();
currentHttpPostRequestDecoder = null;
currentHttpRequest = null;
http://git-wip-us.apache.org/repos/asf/flink/blob/ce04965b/flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestClient.java
----------------------------------------------------------------------
diff --git a/flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestClient.java b/flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestClient.java
index 2e1a784..a119536 100644
--- a/flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestClient.java
+++ b/flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestClient.java
@@ -351,14 +351,14 @@ public class RestClient {
@Override
public void writeTo(Channel channel) {
- channel.writeAndFlush(httpRequest);
+ ChannelFuture future = channel.writeAndFlush(httpRequest);
// this should never be false as we explicitly set the encoder to use multipart messages
if (bodyRequestEncoder.isChunked()) {
- channel.writeAndFlush(bodyRequestEncoder);
+ future = channel.writeAndFlush(bodyRequestEncoder);
}
- // release data and remove temporary files if they were created
- bodyRequestEncoder.cleanFiles();
+ // release data and remove temporary files if they were created, once the writing is complete
+ future.addListener((ignored) -> bodyRequestEncoder.cleanFiles());
}
}
http://git-wip-us.apache.org/repos/asf/flink/blob/ce04965b/flink-runtime/src/test/java/org/apache/flink/runtime/rest/MultipartUploadResource.java
----------------------------------------------------------------------
diff --git a/flink-runtime/src/test/java/org/apache/flink/runtime/rest/MultipartUploadResource.java b/flink-runtime/src/test/java/org/apache/flink/runtime/rest/MultipartUploadResource.java
index c03b85d..1311b80 100644
--- a/flink-runtime/src/test/java/org/apache/flink/runtime/rest/MultipartUploadResource.java
+++ b/flink-runtime/src/test/java/org/apache/flink/runtime/rest/MultipartUploadResource.java
@@ -52,6 +52,7 @@ import javax.annotation.Nonnull;
import java.io.File;
import java.io.IOException;
+import java.io.RandomAccessFile;
import java.net.InetSocketAddress;
import java.nio.file.Files;
import java.nio.file.Path;
@@ -113,7 +114,9 @@ public class MultipartUploadResource extends ExternalResource {
CompletableFuture.completedFuture(mockRestfulGateway);
file1 = temporaryFolder.newFile();
- Files.write(file1.toPath(), "hello".getBytes(ConfigConstants.DEFAULT_CHARSET));
+ try (RandomAccessFile rw = new RandomAccessFile(file1, "rw")) {
+ rw.setLength(1024 * 1024 * 64);
+ }
file2 = temporaryFolder.newFile();
Files.write(file2.toPath(), "world".getBytes(ConfigConstants.DEFAULT_CHARSET));
[3/5] flink git commit: [FLINK-9580][rest] Close streams in
RestClient#readRawResponse
Posted by ch...@apache.org.
[FLINK-9580][rest] Close streams in RestClient#readRawResponse
This closes #6166.
Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/b4574c9b
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/b4574c9b
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/b4574c9b
Branch: refs/heads/master
Commit: b4574c9bb5398713dd5501baf596f284ea19817f
Parents: eb525b7
Author: yanghua <ya...@gmail.com>
Authored: Thu Jun 14 16:16:20 2018 +0800
Committer: zentol <ch...@apache.org>
Committed: Wed Jun 27 22:01:16 2018 +0200
----------------------------------------------------------------------
.../main/java/org/apache/flink/runtime/rest/RestClient.java | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/flink/blob/b4574c9b/flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestClient.java
----------------------------------------------------------------------
diff --git a/flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestClient.java b/flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestClient.java
index c700807..2e1a784 100644
--- a/flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestClient.java
+++ b/flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestClient.java
@@ -415,16 +415,14 @@ public class RestClient {
ByteBuf content = msg.content();
JsonNode rawResponse;
- try {
- InputStream in = new ByteBufInputStream(content);
+ try (InputStream in = new ByteBufInputStream(content)) {
rawResponse = objectMapper.readTree(in);
LOG.debug("Received response {}.", rawResponse);
} catch (JsonParseException je) {
LOG.error("Response was not valid JSON.", je);
// let's see if it was a plain-text message instead
content.readerIndex(0);
- try {
- ByteBufInputStream in = new ByteBufInputStream(content);
+ try (ByteBufInputStream in = new ByteBufInputStream(content)) {
byte[] data = new byte[in.available()];
in.readFully(data);
String message = new String(data);