You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2018/11/17 19:50:54 UTC
[GitHub] tillrohrmann closed pull request #7129: [FLINK-10880] Exclude
JobManagerOptions#EXECUTION_FAILOVER_STRATEGY from documentation
tillrohrmann closed pull request #7129: [FLINK-10880] Exclude JobManagerOptions#EXECUTION_FAILOVER_STRATEGY from documentation
URL: https://github.com/apache/flink/pull/7129
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/docs/_includes/generated/job_manager_configuration.html b/docs/_includes/generated/job_manager_configuration.html
index 0458af24c06..99eec1dcec7 100644
--- a/docs/_includes/generated/job_manager_configuration.html
+++ b/docs/_includes/generated/job_manager_configuration.html
@@ -17,11 +17,6 @@
<td style="word-wrap: break-word;">16</td>
<td>The maximum number of prior execution attempts kept in history.</td>
</tr>
- <tr>
- <td><h5>jobmanager.execution.failover-strategy</h5></td>
- <td style="word-wrap: break-word;">"full"</td>
- <td>This option specifies how the job computation recovers from task failures. Accepted values are:<ul><li>'full': Restarts all tasks.</li><li>'individual': Restarts only the failed task. Should only be used if all tasks are independent components.</li><li>'region': Restarts all tasks that could be affected by the task failure.</li></ul></td>
- </tr>
<tr>
<td><h5>jobmanager.heap.size</h5></td>
<td style="word-wrap: break-word;">"1024m"</td>
diff --git a/docs/release-notes/flink-1.5.md b/docs/release-notes/flink-1.5.md
index 4cee5774cdf..ed5f2c2ad26 100644
--- a/docs/release-notes/flink-1.5.md
+++ b/docs/release-notes/flink-1.5.md
@@ -80,5 +80,13 @@ The Kinesis dependencies of Flink’s Kinesis connector have been updated to the
<aws.kinesis-kpl.version>0.12.9</aws.kinesis-kcl.version>
```
+<!-- Remove once FLINK-10712 has been fixed -->
+### Limitations of failover strategies
+Flink's non-default failover strategies are still a very experimental feature which come with a set of limitations.
+You should only use this feature if you are executing a stateless streaming job.
+In any other cases, it is highly recommended to remove the config option `jobmanager.execution.failover-strategy` from your `flink-conf.yaml` or set it to `"full"`.
+
+In order to avoid future problems, this feature has been removed from the documentation until it will be fixed.
+See [FLINK-10880](https://issues.apache.org/jira/browse/FLINK-10880) for more details.
{% top %}
diff --git a/docs/release-notes/flink-1.6.md b/docs/release-notes/flink-1.6.md
index 34cd6135511..7c22b3f4de6 100644
--- a/docs/release-notes/flink-1.6.md
+++ b/docs/release-notes/flink-1.6.md
@@ -29,6 +29,15 @@ The default value of the slot idle timeout `slot.idle.timeout` is set to the def
### Changed ElasticSearch 5.x Sink API
Previous APIs in the Flink ElasticSearch 5.x Sink's `RequestIndexer` interface have been deprecated in favor of new signatures.
-When adding requests to the `RequestIndexer`, the requests now must be of type `IndexRequest`, `DeleteRequest`, or `UpdateRequest`, instead of the base `ActionRequest`.
+When adding requests to the `RequestIndexer`, the requests now must be of type `IndexRequest`, `DeleteRequest`, or `UpdateRequest`, instead of the base `ActionRequest`.
+
+<!-- Remove once FLINK-10712 has been fixed -->
+### Limitations of failover strategies
+Flink's non-default failover strategies are still a very experimental feature which come with a set of limitations.
+You should only use this feature if you are executing a stateless streaming job.
+In any other cases, it is highly recommended to remove the config option `jobmanager.execution.failover-strategy` from your `flink-conf.yaml` or set it to `"full"`.
+
+In order to avoid future problems, this feature has been removed from the documentation until it will be fixed.
+See [FLINK-10880](https://issues.apache.org/jira/browse/FLINK-10880) for more details.
{% top %}
diff --git a/docs/release-notes/flink-1.7.md b/docs/release-notes/flink-1.7.md
index f9e7425601a..8cdfe9d5400 100644
--- a/docs/release-notes/flink-1.7.md
+++ b/docs/release-notes/flink-1.7.md
@@ -30,4 +30,13 @@ Therefore, the module `flink-scala-shell` is not being released for Scala 2.12.
See [FLINK-10911](https://issues.apache.org/jira/browse/FLINK-10911) for more details.
+<!-- Remove once FLINK-10712 has been fixed -->
+### Limitations of failover strategies
+Flink's non-default failover strategies are still a very experimental feature which come with a set of limitations.
+You should only use this feature if you are executing a stateless streaming job.
+In any other cases, it is highly recommended to remove the config option `jobmanager.execution.failover-strategy` from your `flink-conf.yaml` or set it to `"full"`.
+
+In order to avoid future problems, this feature has been removed from the documentation until it will be fixed.
+See [FLINK-10880](https://issues.apache.org/jira/browse/FLINK-10880) for more details.
+
{% top %}
diff --git a/flink-annotations/src/main/java/org/apache/flink/annotation/docs/Documentation.java b/flink-annotations/src/main/java/org/apache/flink/annotation/docs/Documentation.java
index c193c4e0554..aed0d2975f4 100644
--- a/flink-annotations/src/main/java/org/apache/flink/annotation/docs/Documentation.java
+++ b/flink-annotations/src/main/java/org/apache/flink/annotation/docs/Documentation.java
@@ -59,6 +59,19 @@
int position() default Integer.MAX_VALUE;
}
+ /**
+ * Annotation used on config option fields to exclude the config option from documentation.
+ */
+ @Target(ElementType.FIELD)
+ @Retention(RetentionPolicy.RUNTIME)
+ @Internal
+ public @interface ExcludeFromDocumentation {
+ /**
+ * The optional reason why the config option is excluded from documentation.
+ */
+ String value() default "";
+ }
+
private Documentation(){
}
}
diff --git a/flink-core/src/main/java/org/apache/flink/configuration/JobManagerOptions.java b/flink-core/src/main/java/org/apache/flink/configuration/JobManagerOptions.java
index 1666f213d18..cb2dbe61631 100644
--- a/flink-core/src/main/java/org/apache/flink/configuration/JobManagerOptions.java
+++ b/flink-core/src/main/java/org/apache/flink/configuration/JobManagerOptions.java
@@ -106,6 +106,7 @@
/**
* This option specifies the failover strategy, i.e. how the job computation recovers from task failures.
*/
+ @Documentation.ExcludeFromDocumentation("The failover strategy feature is highly experimental.")
public static final ConfigOption<String> EXECUTION_FAILOVER_STRATEGY =
key("jobmanager.execution.failover-strategy")
.defaultValue("full")
diff --git a/flink-docs/src/main/java/org/apache/flink/docs/configuration/ConfigOptionsDocGenerator.java b/flink-docs/src/main/java/org/apache/flink/docs/configuration/ConfigOptionsDocGenerator.java
index d0f665a5051..36a2203b7e4 100644
--- a/flink-docs/src/main/java/org/apache/flink/docs/configuration/ConfigOptionsDocGenerator.java
+++ b/flink-docs/src/main/java/org/apache/flink/docs/configuration/ConfigOptionsDocGenerator.java
@@ -202,7 +202,7 @@ static void processConfigOptions(String rootDir, String module, String packageNa
List<OptionWithMetaInfo> configOptions = new ArrayList<>(8);
Field[] fields = clazz.getFields();
for (Field field : fields) {
- if (field.getType().equals(ConfigOption.class) && field.getAnnotation(Deprecated.class) == null) {
+ if (isConfigOption(field) && shouldBeDocumented(field)) {
configOptions.add(new OptionWithMetaInfo((ConfigOption<?>) field.get(null), field));
}
}
@@ -213,6 +213,15 @@ static void processConfigOptions(String rootDir, String module, String packageNa
}
}
+ private static boolean isConfigOption(Field field) {
+ return field.getType().equals(ConfigOption.class);
+ }
+
+ private static boolean shouldBeDocumented(Field field) {
+ return field.getAnnotation(Deprecated.class) == null &&
+ field.getAnnotation(Documentation.ExcludeFromDocumentation.class) == null;
+ }
+
/**
* Transforms this configuration group into HTML formatted table.
* Options are sorted alphabetically by key.
diff --git a/flink-docs/src/test/java/org/apache/flink/docs/configuration/ConfigOptionsDocGeneratorTest.java b/flink-docs/src/test/java/org/apache/flink/docs/configuration/ConfigOptionsDocGeneratorTest.java
index a0eade69d45..c0fb7c99eda 100644
--- a/flink-docs/src/test/java/org/apache/flink/docs/configuration/ConfigOptionsDocGeneratorTest.java
+++ b/flink-docs/src/test/java/org/apache/flink/docs/configuration/ConfigOptionsDocGeneratorTest.java
@@ -266,4 +266,45 @@ public void testCommonOptions() throws IOException, ClassNotFoundException {
assertEquals(expected, output);
}
+
+ static class TestConfigGroupWithExclusion {
+ public static ConfigOption<Integer> firstOption = ConfigOptions
+ .key("first.option.a")
+ .defaultValue(2)
+ .withDescription("This is example description for the first option.");
+
+ @Documentation.ExcludeFromDocumentation
+ public static ConfigOption<String> excludedOption = ConfigOptions
+ .key("excluded.option.a")
+ .noDefaultValue()
+ .withDescription("This should not be documented.");
+ }
+
+ /**
+ * Tests that {@link ConfigOption} annotated with {@link Documentation.ExcludeFromDocumentation}
+ * are not documented.
+ */
+ @Test
+ public void testConfigOptionExclusion() {
+ final String expectedTable =
+ "<table class=\"table table-bordered\">\n" +
+ " <thead>\n" +
+ " <tr>\n" +
+ " <th class=\"text-left\" style=\"width: 20%\">Key</th>\n" +
+ " <th class=\"text-left\" style=\"width: 15%\">Default</th>\n" +
+ " <th class=\"text-left\" style=\"width: 65%\">Description</th>\n" +
+ " </tr>\n" +
+ " </thead>\n" +
+ " <tbody>\n" +
+ " <tr>\n" +
+ " <td><h5>first.option.a</h5></td>\n" +
+ " <td style=\"word-wrap: break-word;\">2</td>\n" +
+ " <td>This is example description for the first option.</td>\n" +
+ " </tr>\n" +
+ " </tbody>\n" +
+ "</table>\n";
+ final String htmlTable = ConfigOptionsDocGenerator.generateTablesForClass(TestConfigGroupWithExclusion.class).get(0).f1;
+
+ assertEquals(expectedTable, htmlTable);
+ }
}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services