You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by mx...@apache.org on 2016/12/07 13:18:44 UTC
[1/2] flink git commit: [docs] Note that numberOfExecutionRetries and
executionRetryDelay are deprecated.
Repository: flink
Updated Branches:
refs/heads/master af3bf837a -> c024b0b6c
[docs] Note that numberOfExecutionRetries and executionRetryDelay are deprecated.
And some other minor fixes and deduplication.
Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/c024b0b6
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/c024b0b6
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/c024b0b6
Branch: refs/heads/master
Commit: c024b0b6cae16a0b668d864c77e923820c262087
Parents: 91414d9
Author: Rohit Agarwal <mi...@gmail.com>
Authored: Tue Dec 6 12:21:38 2016 -0800
Committer: Maximilian Michels <mx...@apache.org>
Committed: Wed Dec 7 14:19:04 2016 +0100
----------------------------------------------------------------------
docs/dev/api_concepts.md | 4 ++--
docs/dev/datastream_api.md | 6 +++---
docs/setup/fault_tolerance.md | 21 ++-------------------
3 files changed, 7 insertions(+), 24 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/flink/blob/c024b0b6/docs/dev/api_concepts.md
----------------------------------------------------------------------
diff --git a/docs/dev/api_concepts.md b/docs/dev/api_concepts.md
index 07a81e7..35e4d3a 100644
--- a/docs/dev/api_concepts.md
+++ b/docs/dev/api_concepts.md
@@ -962,9 +962,9 @@ With the closure cleaner disabled, it might happen that an anonymous user functi
- `getMaxParallelism()` / `setMaxParallelism(int parallelism)` Set the default maximum parallelism for the job. This setting determines the maximum degree of parallelism and specifies the upper limit for dynamic scaling.
-- `getNumberOfExecutionRetries()` / `setNumberOfExecutionRetries(int numberOfExecutionRetries)` Sets the number of times that failed tasks are re-executed. A value of zero effectively disables fault tolerance. A value of `-1` indicates that the system default value (as defined in the configuration) should be used.
+- `getNumberOfExecutionRetries()` / `setNumberOfExecutionRetries(int numberOfExecutionRetries)` Sets the number of times that failed tasks are re-executed. A value of zero effectively disables fault tolerance. A value of `-1` indicates that the system default value (as defined in the configuration) should be used. This is deprecated, use [restart strategies]({{ site.baseurl }}/setup/fault_tolerance.html#restart-strategies) instead.
-- `getExecutionRetryDelay()` / `setExecutionRetryDelay(long executionRetryDelay)` Sets the delay in milliseconds that the system waits after a job has failed, before re-executing it. The delay starts after all tasks have been successfully been stopped on the TaskManagers, and once the delay is past, the tasks are re-started. This parameter is useful to delay re-execution in order to let certain time-out related failures surface fully (like broken connections that have not fully timed out), before attempting a re-execution and immediately failing again due to the same problem. This parameter only has an effect if the number of execution re-tries is one or more.
+- `getExecutionRetryDelay()` / `setExecutionRetryDelay(long executionRetryDelay)` Sets the delay in milliseconds that the system waits after a job has failed, before re-executing it. The delay starts after all tasks have been successfully been stopped on the TaskManagers, and once the delay is past, the tasks are re-started. This parameter is useful to delay re-execution in order to let certain time-out related failures surface fully (like broken connections that have not fully timed out), before attempting a re-execution and immediately failing again due to the same problem. This parameter only has an effect if the number of execution re-tries is one or more. This is deprecated, use [restart strategies]({{ site.baseurl }}/setup/fault_tolerance.html#restart-strategies) instead.
- `getExecutionMode()` / `setExecutionMode()`. The default execution mode is PIPELINED. Sets the execution mode to execute the program. The execution mode defines whether data exchanges are performed in a batch or on a pipelined manner.
http://git-wip-us.apache.org/repos/asf/flink/blob/c024b0b6/docs/dev/datastream_api.md
----------------------------------------------------------------------
diff --git a/docs/dev/datastream_api.md b/docs/dev/datastream_api.md
index ef4a394..1b167ac 100644
--- a/docs/dev/datastream_api.md
+++ b/docs/dev/datastream_api.md
@@ -342,7 +342,7 @@ allWindowedStream.apply (new AllWindowFunction<Tuple2<String,Integer>, Integer,
<td>
<p>Applies a functional reduce function to the window and returns the reduced value.</p>
{% highlight java %}
-windowedStream.reduce (new ReduceFunction<Tuple2<String,Integer>() {
+windowedStream.reduce (new ReduceFunction<Tuple2<String,Integer>>() {
public Tuple2<String, Integer> reduce(Tuple2<String, Integer> value1, Tuple2<String, Integer> value2) throws Exception {
return new Tuple2<String,Integer>(value1.f0, value1.f1 + value2.f1);
}
@@ -388,7 +388,7 @@ windowedStream.maxBy("key");
<tr>
<td><strong>Union</strong><br>DataStream* → DataStream</td>
<td>
- <p>Union of two or more data streams creating a new stream containing all the elements from all the streams. Node: If you union a data stream
+ <p>Union of two or more data streams creating a new stream containing all the elements from all the streams. Note: If you union a data stream
with itself you will get each element twice in the resulting stream.</p>
{% highlight java %}
dataStream.union(otherStream1, otherStream2, ...);
@@ -737,7 +737,7 @@ windowedStream.maxBy("key")
<tr>
<td><strong>Union</strong><br>DataStream* → DataStream</td>
<td>
- <p>Union of two or more data streams creating a new stream containing all the elements from all the streams. Node: If you union a data stream
+ <p>Union of two or more data streams creating a new stream containing all the elements from all the streams. Note: If you union a data stream
with itself you will get each element twice in the resulting stream.</p>
{% highlight scala %}
dataStream.union(otherStream1, otherStream2, ...)
http://git-wip-us.apache.org/repos/asf/flink/blob/c024b0b6/docs/setup/fault_tolerance.md
----------------------------------------------------------------------
diff --git a/docs/setup/fault_tolerance.md b/docs/setup/fault_tolerance.md
index e123e60..7fb3df4 100644
--- a/docs/setup/fault_tolerance.md
+++ b/docs/setup/fault_tolerance.md
@@ -45,9 +45,6 @@ To enable checkpointing, call `enableCheckpointing(n)` on the `StreamExecutionEn
Other parameters for checkpointing include:
-- *Number of retries*: The `setNumberOfExecutionRerties()` method defines how many times the job is restarted after a failure.
- When checkpointing is activated, but this value is not explicitly set, the job is restarted infinitely often.
-
- *exactly-once vs. at-least-once*: You can optionally pass a mode to the `enableCheckpointing(n)` method to choose between the two guarantee levels.
Exactly-once is preferrable for most applications. At-least-once may be relevant for certain super-low-latency (consistently few milliseconds) applications.
@@ -307,12 +304,12 @@ restart-strategy: fixed-delay
<tbody>
<tr>
<td><it>restart-strategy.fixed-delay.attempts</it></td>
- <td>Number of restart attempts</td>
+ <td>The number of times that Flink retries the execution before the job is declared as failed</td>
<td>1</td>
</tr>
<tr>
<td><it>restart-strategy.fixed-delay.delay</it></td>
- <td>Delay between two consecutive restart attempts</td>
+ <td>Delay between two consecutive restart attempts. Delaying the retry means that after a failed execution, the re-execution does not start immediately, but only after a certain delay. Delaying the retries can be helpful when the program interacts with external systems where for example connections or pending transactions should reach a timeout before re-execution is attempted.</td>
<td><it>akka.ask.timeout</it></td>
</tr>
</tbody>
@@ -346,20 +343,6 @@ env.setRestartStrategy(RestartStrategies.fixedDelayRestart(
</div>
</div>
-#### Restart Attempts
-
-The number of times that Flink retries the execution before the job is declared as failed is configurable via the *restart-strategy.fixed-delay.attempts* parameter.
-
-The default value is **1**.
-
-#### Retry Delays
-
-Execution retries can be configured to be delayed. Delaying the retry means that after a failed execution, the re-execution does not start immediately, but only after a certain delay.
-
-Delaying the retries can be helpful when the program interacts with external systems where for example connections or pending transactions should reach a timeout before re-execution is attempted.
-
-The default value is the value of *akka.ask.timeout*.
-
{% top %}
### Failure Rate Restart Strategy
[2/2] flink git commit: [docs] clarify default restart behavior when
checkpointing is enabled
Posted by mx...@apache.org.
[docs] clarify default restart behavior when checkpointing is enabled
Project: http://git-wip-us.apache.org/repos/asf/flink/repo
Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/91414d96
Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/91414d96
Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/91414d96
Branch: refs/heads/master
Commit: 91414d9611a5a14d4d73ae5ea090c16406dcf025
Parents: af3bf83
Author: Maximilian Michels <mx...@apache.org>
Authored: Tue Dec 6 15:12:22 2016 +0100
Committer: Maximilian Michels <mx...@apache.org>
Committed: Wed Dec 7 14:19:04 2016 +0100
----------------------------------------------------------------------
docs/setup/fault_tolerance.md | 1 +
1 file changed, 1 insertion(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/flink/blob/91414d96/docs/setup/fault_tolerance.md
----------------------------------------------------------------------
diff --git a/docs/setup/fault_tolerance.md b/docs/setup/fault_tolerance.md
index fa1c821..e123e60 100644
--- a/docs/setup/fault_tolerance.md
+++ b/docs/setup/fault_tolerance.md
@@ -224,6 +224,7 @@ In case that the job is submitted with a restart strategy, this strategy overrid
The default restart strategy is set via Flink's configuration file `flink-conf.yaml`.
The configuration parameter *restart-strategy* defines which strategy is taken.
Per default, the no-restart strategy is used.
+When checkpointing is activated and no restart strategy has been configured, the job will be restarted infinitely often.
See the following list of available restart strategies to learn what values are supported.
Each restart strategy comes with its own set of parameters which control its behaviour.