You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by uc...@apache.org on 2017/01/09 13:36:07 UTC
[1/2] flink git commit: [docs] Add TOC to restart strategies page
Repository: flink
Updated Branches:
refs/heads/master 6d963642a -> 384da0e99
[docs] Add TOC to restart strategies page
This closes #3059.
Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/384da0e9
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/384da0e9
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/384da0e9
Branch: refs/heads/master
Commit: 384da0e999dde0667e755c784122addff76ed1f3
Parents: b03ad79
Author: Ufuk Celebi <uc...@apache.org>
Authored: Mon Jan 9 14:35:24 2017 +0100
Committer: Ufuk Celebi <uc...@apache.org>
Committed: Mon Jan 9 14:35:46 2017 +0100
----------------------------------------------------------------------
docs/dev/restart_strategies.md | 18 ++++++++++++++----
docs/setup/config.md | 8 ++++----
2 files changed, 18 insertions(+), 8 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/flink/blob/384da0e9/docs/dev/restart_strategies.md
----------------------------------------------------------------------
diff --git a/docs/dev/restart_strategies.md b/docs/dev/restart_strategies.md
index cc1a68e..a4901ab 100644
--- a/docs/dev/restart_strategies.md
+++ b/docs/dev/restart_strategies.md
@@ -26,6 +26,11 @@ Flink supports different restart strategies which control how the jobs are resta
The cluster can be started with a default restart strategy which is always used when no job specific restart strategy has been defined.
In case that the job is submitted with a restart strategy, this strategy overrides the cluster's default setting.
+* This will be replaced by the TOC
+{:toc}
+
+## Overview
+
The default restart strategy is set via Flink's configuration file `flink-conf.yaml`.
The configuration parameter *restart-strategy* defines which strategy is taken.
If checkpointing is not enabled, the "no restart" strategy is used.
@@ -90,7 +95,11 @@ env.setRestartStrategy(RestartStrategies.fixedDelayRestart(
{% top %}
-## Fixed Delay Restart Strategy
+## Restart Strategies
+
+The following sections describe restart strategy specific configuration options.
+
+### Fixed Delay Restart Strategy
The fixed delay restart strategy attempts a given number of times to restart the job.
If the maximum number of attempts is exceeded, the job eventually fails.
@@ -125,6 +134,7 @@ restart-strategy: fixed-delay
</table>
For example:
+
~~~
restart-strategy.fixed-delay.attempts: 3
restart-strategy.fixed-delay.delay: 10 s
@@ -155,7 +165,7 @@ env.setRestartStrategy(RestartStrategies.fixedDelayRestart(
{% top %}
-## Failure Rate Restart Strategy
+### Failure Rate Restart Strategy
The failure rate restart strategy restarts job after failure, but when `failure rate` (failures per time interval) is exceeded, the job eventually fails.
In-between two consecutive restart attempts, the restart strategy waits a fixed amount of time.
@@ -226,7 +236,7 @@ env.setRestartStrategy(RestartStrategies.failureRateRestart(
{% top %}
-## No Restart Strategy
+### No Restart Strategy
The job fails directly and no restart is attempted.
@@ -251,7 +261,7 @@ env.setRestartStrategy(RestartStrategies.noRestart())
</div>
</div>
-## Fallback Restart Strategy
+### Fallback Restart Strategy
The cluster defined restart strategy is used.
This helpful for streaming programs which enable checkpointing.
http://git-wip-us.apache.org/repos/asf/flink/blob/384da0e9/docs/setup/config.md
----------------------------------------------------------------------
diff --git a/docs/setup/config.md b/docs/setup/config.md
index 8af3e6b..724fe33 100644
--- a/docs/setup/config.md
+++ b/docs/setup/config.md
@@ -171,11 +171,11 @@ will be used under the directory specified by jobmanager.web.tmpdir.
- `restart-strategy`: Default [restart strategy]({{site.baseurl}}/dev/restart_strategies.html) to use in case no
restart strategy has been specified for the job.
The options are:
- - fixed delay strategy: "fixed-delay".
- - failure rate strategy: "failure-rate".
- - no restarts: "none"
+ - fixed delay strategy: `fixed-delay`.
+ - failure rate strategy: `failure-rate`.
+ - no restarts: `none`
- Default value is "none" unless checkpointing is enabled for the job in which case the default is "fixed-delay".
+ Default value is `none` unless checkpointing is enabled for the job in which case the default is `fixed-delay` with `Integer.MAX_VALUE` restart attempts and `10s` delay.
- `restart-strategy.fixed-delay.attempts`: Number of restart attempts, used if the default restart strategy is set to "fixed-delay".
Default value is 1, unless "fixed-delay" was activated by enabling checkpoints, in which case the default is `Integer.MAX_VALUE`.
[2/2] flink git commit: [docs] Clarify restart strategy defaults set
by checkpointing
Posted by uc...@apache.org.
[docs] Clarify restart strategy defaults set by checkpointing
- Added info about checkpointing changing the default restart
strategy in places where it was missing: the config page and the
section about the fixed-delay strategy
- Replaced no-restart with "no restart" so people don't think we're
referring to a config value
- Replaced invalid <it> html tag with <code>
- Fixed bad link to restart strategies page from state.md
Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/b03ad793
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/b03ad793
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/b03ad793
Branch: refs/heads/master
Commit: b03ad793db9bd7857628cefb0da9f7f30e1e4545
Parents: 6d96364
Author: Shannon Carey <re...@gmail.com>
Authored: Wed Jan 4 13:20:12 2017 -0600
Committer: Ufuk Celebi <uc...@apache.org>
Committed: Mon Jan 9 14:35:46 2017 +0100
----------------------------------------------------------------------
docs/dev/restart_strategies.md | 14 ++++++++------
docs/dev/state.md | 3 ++-
docs/setup/config.md | 23 +++++++++++++----------
3 files changed, 23 insertions(+), 17 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/flink/blob/b03ad793/docs/dev/restart_strategies.md
----------------------------------------------------------------------
diff --git a/docs/dev/restart_strategies.md b/docs/dev/restart_strategies.md
index 5965157..cc1a68e 100644
--- a/docs/dev/restart_strategies.md
+++ b/docs/dev/restart_strategies.md
@@ -28,8 +28,9 @@ In case that the job is submitted with a restart strategy, this strategy overrid
The default restart strategy is set via Flink's configuration file `flink-conf.yaml`.
The configuration parameter *restart-strategy* defines which strategy is taken.
-Per default, the no-restart strategy is used.
-When checkpointing is activated and no restart strategy has been configured, the job will be restarted infinitely often.
+If checkpointing is not enabled, the "no restart" strategy is used.
+If checkpointing is activated and the restart strategy has not been configured, the fixed-delay strategy is used with
+`Integer.MAX_VALUE` restart attempts.
See the following list of available restart strategies to learn what values are supported.
Each restart strategy comes with its own set of parameters which control its behaviour.
@@ -111,18 +112,19 @@ restart-strategy: fixed-delay
</thead>
<tbody>
<tr>
- <td><it>restart-strategy.fixed-delay.attempts</it></td>
+ <td><code>restart-strategy.fixed-delay.attempts</code></td>
<td>The number of times that Flink retries the execution before the job is declared as failed.</td>
- <td>1</td>
+ <td>1, or <code>Integer.MAX_VALUE</code> if activated by checkpointing</td>
</tr>
<tr>
- <td><it>restart-strategy.fixed-delay.delay</it></td>
+ <td><code>restart-strategy.fixed-delay.delay</code></td>
<td>Delaying the retry means that after a failed execution, the re-execution does not start immediately, but only after a certain delay. Delaying the retries can be helpful when the program interacts with external systems where for example connections or pending transactions should reach a timeout before re-execution is attempted.</td>
- <td><it>akka.ask.timeout</it></td>
+ <td><code>akka.ask.timeout</code>, or 10s if activated by checkpointing</td>
</tr>
</tbody>
</table>
+For example:
~~~
restart-strategy.fixed-delay.attempts: 3
restart-strategy.fixed-delay.delay: 10 s
http://git-wip-us.apache.org/repos/asf/flink/blob/b03ad793/docs/dev/state.md
----------------------------------------------------------------------
diff --git a/docs/dev/state.md b/docs/dev/state.md
index 99f9055..73a4ceb 100644
--- a/docs/dev/state.md
+++ b/docs/dev/state.md
@@ -358,4 +358,5 @@ Please note that records in flight in the loop edges (and the state changes asso
## Restart Strategies
-Flink supports different restart strategies which control how the jobs are restarted in case of a failure. For more information, see [Restart Strategies]({{ site.baseurl }}/dev/restart_strategies).
+Flink supports different restart strategies which control how the jobs are restarted in case of a failure. For more
+information, see [Restart Strategies]({{ site.baseurl }}/dev/restart_strategies.html).
http://git-wip-us.apache.org/repos/asf/flink/blob/b03ad793/docs/setup/config.md
----------------------------------------------------------------------
diff --git a/docs/setup/config.md b/docs/setup/config.md
index ecb8c25..8af3e6b 100644
--- a/docs/setup/config.md
+++ b/docs/setup/config.md
@@ -162,24 +162,27 @@ will be used under the directory specified by jobmanager.web.tmpdir.
- `high-availability.zookeeper.storageDir`: Required for HA. Directory for storing JobManager metadata; this is persisted in the state backend and only a pointer to this state is stored in ZooKeeper. Exactly like the checkpoint directory it must be accessible from the JobManager and a local filesystem should only be used for local deployments. Previously this key was named `recovery.zookeeper.storageDir`.
-- `blob.storage.directory`: Directory for storing blobs (such as user jar's) on the TaskManagers.
+- `blob.storage.directory`: Directory for storing blobs (such as user JARs) on the TaskManagers.
-- `blob.server.port`: Port definition for the blob server (serving user jar's) on the Taskmanagers. By default the port is set to 0, which means that the operating system is picking an ephemeral port. Flink also accepts a list of ports ("50100,50101"), ranges ("50100-50200") or a combination of both. It is recommended to set a range of ports to avoid collisions when multiple JobManagers are running on the same machine.
+- `blob.server.port`: Port definition for the blob server (serving user JARs) on the TaskManagers. By default the port is set to 0, which means that the operating system is picking an ephemeral port. Flink also accepts a list of ports ("50100,50101"), ranges ("50100-50200") or a combination of both. It is recommended to set a range of ports to avoid collisions when multiple JobManagers are running on the same machine.
- `blob.service.ssl.enabled`: Flag to enable ssl for the blob client/server communication. This is applicable only when the global ssl flag security.ssl.enabled is set to true (DEFAULT: true).
-- `restart-strategy`: Default restart strategy to use in case that no restart strategy has been specified for the submitted job.
-Currently, it can be chosen from fixed delay restart strategy, failure rate restart strategy or no restart strategy.
-To use the fixed delay strategy you have to specify "fixed-delay".
-To use the failure rate strategy you have to specify "failure-rate".
-To turn the restart behaviour off you have to specify "none".
-Default value "none".
+- `restart-strategy`: Default [restart strategy]({{site.baseurl}}/dev/restart_strategies.html) to use in case no
+restart strategy has been specified for the job.
+The options are:
+ - fixed delay strategy: "fixed-delay".
+ - failure rate strategy: "failure-rate".
+ - no restarts: "none"
+
+ Default value is "none" unless checkpointing is enabled for the job in which case the default is "fixed-delay".
- `restart-strategy.fixed-delay.attempts`: Number of restart attempts, used if the default restart strategy is set to "fixed-delay".
-Default value is 1.
+Default value is 1, unless "fixed-delay" was activated by enabling checkpoints, in which case the default is `Integer.MAX_VALUE`.
- `restart-strategy.fixed-delay.delay`: Delay between restart attempts, used if the default restart strategy is set to "fixed-delay".
-Default value is the `akka.ask.timeout`.
+Default value is the `akka.ask.timeout`, unless "fixed-delay" was activated by enabling checkpoints, in which case
+the default is 10s.
- `restart-strategy.failure-rate.max-failures-per-interval`: Maximum number of restarts in given time interval before failing a job in "failure-rate" strategy.
Default value is 1.