You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by tillrohrmann <gi...@git.apache.org> on 2018/02/16 10:35:25 UTC
[GitHub] flink pull request #5504: [FLINK-8670] Make MetricRegistryImpl#shutdown non ...
GitHub user tillrohrmann opened a pull request:
https://github.com/apache/flink/pull/5504
[FLINK-8670] Make MetricRegistryImpl#shutdown non blocking
## What is the purpose of the change
This commit makes the MetricRegistryImpl#shutdown method non blocking. Instead
of waiting for the completion of the shutdown procedure, the method returns a
future which is completed once the metric registry has completed the shut down.
This PR is based on #5503.
## Verifying this change
This change is already covered by existing tests.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tillrohrmann/flink metricRegistryShutDown
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5504.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5504
----
commit 51bb532d2490968635214016599475f91c5f97e5
Author: Till Rohrmann <tr...@...>
Date: 2018-02-16T09:08:04Z
[FLINK-8669] Add completeAll and runAfterwards(Async) to FutureUtils
FutureUtils#completeAll(Collection) takes a collection of futures and returns
a future which is completed after all of the given futures are completed. This
also includes exceptional completions. Potentially occurring exceptions are
recorded and combined into a single exception with which the resulting future
is completed.
FutureUtils#runAfterwards takes a future and runs a given action after the
completion of the given future. This also includes an exceptional completion.
In this case, a potentially occurring exception as the result of the provided
action will be combined with the future's exception.
commit 45854d259a5ff0d03fe69a9caabc5c8627be3ba9
Author: Till Rohrmann <tr...@...>
Date: 2018-02-16T07:12:04Z
[FLINK-8670] Make MetricRegistryImpl#shutdown non blocking
This commit makes the MetricRegistryImpl#shutdown method non blocking. Instead
of waiting for the completion of the shutdown procedure, the method returns a
future which is completed once the metric registry has completed the shut down.
----
---
[GitHub] flink pull request #5504: [FLINK-8670] Make MetricRegistryImpl#shutdown non ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/5504
---
[GitHub] flink pull request #5504: [FLINK-8670] Make MetricRegistryImpl#shutdown non ...
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/5504#discussion_r168749048
--- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/metrics/MetricRegistryImpl.java ---
@@ -226,71 +235,72 @@ public int getNumberReporters() {
*/
public boolean isShutdown() {
synchronized (lock) {
- return reporters == null && executor.isShutdown();
+ return isShutdown;
}
}
/**
* Shuts down this registry and the associated {@link MetricReporter}.
+ *
+ * <p>NOTE: This operation is asynchronous and returns a future which is completed
+ * once the shutdown operation has been completed.
+ *
+ * @return Future which is completed once the {@link MetricRegistryImpl}
+ * is shut down.
*/
- public void shutdown() {
+ public CompletableFuture<Void> shutdown() {
--- End diff --
Will do.
---
[GitHub] flink pull request #5504: [FLINK-8670] Make MetricRegistryImpl#shutdown non ...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5504#discussion_r168737362
--- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/metrics/MetricRegistryImpl.java ---
@@ -226,71 +235,72 @@ public int getNumberReporters() {
*/
public boolean isShutdown() {
synchronized (lock) {
- return reporters == null && executor.isShutdown();
+ return isShutdown;
}
}
/**
* Shuts down this registry and the associated {@link MetricReporter}.
+ *
+ * <p>NOTE: This operation is asynchronous and returns a future which is completed
+ * once the shutdown operation has been completed.
+ *
+ * @return Future which is completed once the {@link MetricRegistryImpl}
+ * is shut down.
*/
- public void shutdown() {
+ public CompletableFuture<Void> shutdown() {
--- End diff --
Fair point. What about the renaming though?
---
[GitHub] flink pull request #5504: [FLINK-8670] Make MetricRegistryImpl#shutdown non ...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5504#discussion_r168728890
--- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/metrics/MetricRegistryImpl.java ---
@@ -226,71 +235,72 @@ public int getNumberReporters() {
*/
public boolean isShutdown() {
synchronized (lock) {
- return reporters == null && executor.isShutdown();
+ return isShutdown;
}
}
/**
* Shuts down this registry and the associated {@link MetricReporter}.
+ *
+ * <p>NOTE: This operation is asynchronous and returns a future which is completed
+ * once the shutdown operation has been completed.
+ *
+ * @return Future which is completed once the {@link MetricRegistryImpl}
+ * is shut down.
*/
- public void shutdown() {
+ public CompletableFuture<Void> shutdown() {
--- End diff --
How about renaming this to shutdownAsync? (and possible keep shutdown() that just calls shutdownAsync().get()).
---
[GitHub] flink pull request #5504: [FLINK-8670] Make MetricRegistryImpl#shutdown non ...
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/5504#discussion_r168735349
--- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/metrics/MetricRegistryImpl.java ---
@@ -77,19 +85,20 @@
private ViewUpdater viewUpdater;
- private final ScopeFormats scopeFormats;
- private final char globalDelimiter;
- private final List<Character> delimiters = new ArrayList<>();
+ private boolean isShutdown;
/**
* Creates a new MetricRegistry and starts the configured reporter.
*/
public MetricRegistryImpl(MetricRegistryConfiguration config) {
this.scopeFormats = config.getScopeFormats();
this.globalDelimiter = config.getDelimiter();
+ this.delimiters = new ArrayList<>(10);
--- End diff --
Good point, will change it.
---
[GitHub] flink pull request #5504: [FLINK-8670] Make MetricRegistryImpl#shutdown non ...
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:
https://github.com/apache/flink/pull/5504#discussion_r168735497
--- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/metrics/MetricRegistryImpl.java ---
@@ -226,71 +235,72 @@ public int getNumberReporters() {
*/
public boolean isShutdown() {
synchronized (lock) {
- return reporters == null && executor.isShutdown();
+ return isShutdown;
}
}
/**
* Shuts down this registry and the associated {@link MetricReporter}.
+ *
+ * <p>NOTE: This operation is asynchronous and returns a future which is completed
+ * once the shutdown operation has been completed.
+ *
+ * @return Future which is completed once the {@link MetricRegistryImpl}
+ * is shut down.
*/
- public void shutdown() {
+ public CompletableFuture<Void> shutdown() {
--- End diff --
Hmm, I actually would like to avoid that people can introduce blocking operations into the `ClusterEntrypoint`, for example. By offering a blocking `shutdown` method, I fear that this can easily happen again.
---
[GitHub] flink pull request #5504: [FLINK-8670] Make MetricRegistryImpl#shutdown non ...
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5504#discussion_r168728354
--- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/metrics/MetricRegistryImpl.java ---
@@ -77,19 +85,20 @@
private ViewUpdater viewUpdater;
- private final ScopeFormats scopeFormats;
- private final char globalDelimiter;
- private final List<Character> delimiters = new ArrayList<>();
+ private boolean isShutdown;
/**
* Creates a new MetricRegistry and starts the configured reporter.
*/
public MetricRegistryImpl(MetricRegistryConfiguration config) {
this.scopeFormats = config.getScopeFormats();
this.globalDelimiter = config.getDelimiter();
+ this.delimiters = new ArrayList<>(10);
--- End diff --
the number of delimiters is always equal to the number of reporters, so they should be initialized with the same size.
---