You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "pgaref (via GitHub)" <gi...@apache.org> on 2023/04/24 02:08:21 UTC
[GitHub] [flink] pgaref opened a new pull request, #22468: FLINK-31889: Add documentation for implementing/loading enrichers
pgaref opened a new pull request, #22468:
URL: https://github.com/apache/flink/pull/22468
[https://issues.apache.org/jira/browse/FLINK-31889](https://issues.apache.org/jira/browse/FLINK-31889)
FailureEnrichers documentation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] zentol commented on a diff in pull request #22468: [FLINK-31889] Add documentation for implementing/loading enrichers
Posted by "zentol (via GitHub)" <gi...@apache.org>.
zentol commented on code in PR #22468:
URL: https://github.com/apache/flink/pull/22468#discussion_r1318898982
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,112 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all the external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise may be ignored.
+{{< /hint >}}
+
+FailureEnricherFactory example:
+
+``` java
+public class TestFailureEnricherFactory implements FailureEnricherFactory {
+
+ @Override
+ public FailureEnricher createFailureEnricher(Configuration conf) {
+ return new CustomEnricher();
+ }
+}
+```
+
+FailureEnricher example:
+
+``` java
+public class CustomEnricher implements FailureEnricher {
+ private final Set<String> outputKeys;
+
+ public CustomEnricher() {
+ this.outputKeys = Collections.singleton("labelKey");
+ }
+
+ @Override
+ public Set<String> getOutputKeys() {
+ return outputKeys;
+ }
+
+ @Override
+ public CompletableFuture<Map<String, String>> processFailure(
+ Throwable cause, Context context) {
+ return CompletableFuture.completedFuture(Collections.singletonMap("labelKey", "labelValue"));
+ }
+}
+```
+
+### Configuration
+
+JobManager loads FailureEnricher plugins at startup. To make sure your FailureEnrichers are loaded:
+* All class names should be defined as part of {{< javadoc file="org/apache/flink/configuration/JobManagerOptions.html#FAILURE_ENRICHERS_LIST" name="jobmanager.failure-enrichers">}} configuration.
+ If this configuration is empty, NO enrichers will be started. Example:
+```
+ jobmanager.failure-enrichers = org.apache.flink.test.plugin.jar.failure.CustomEnricher
Review Comment:
Did you miss this one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dmvk commented on a diff in pull request #22468: FLINK-31889: Add documentation for implementing/loading enrichers
Posted by "dmvk (via GitHub)" <gi...@apache.org>.
dmvk commented on code in PR #22468:
URL: https://github.com/apache/flink/pull/22468#discussion_r1183548843
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,53 @@
+---
+title: "Pluggable Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+Flink enables users to extend the default failure handling behavior using the plugin framework.
+
+## Custom failure enrichers
+Flink provides a pluggable failure enricher interface for users to register their custom logic.
+The goal is to give flexibility to developers who can now implement their own plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+Failure enrichers are triggered every time an exception is reported at runtime by the job manager.
+Every failure enricher may optionally return labels (kv string pairs) associated with the failure that are then exposed via the job manager's Rest interface (e.g., a 'System' tag implying the failure considered as a system error).
+For instance, when a Flink runtime failure occurs caused by network error, we can increment the appropriate counter.
+With accurate metrics, we now have better insight of platform level metrics e.g., network failures, platform reliability, etc.
+The default CountingFailureEnricher just records the failure count and then emits the metric "numJobFailure" for the job.
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom plugin for your use-case:
+
+- Add your own FailureEnricher by implementing the `org.apache.flink.core.failure.FailureEnricher` interface.
+
+- Add your own FailureEnricherFactory by implementing the `org.apache.flink.core.failure.FailureEnricherFactory` interface.
Review Comment:
It would be neat to add links into javadocs.
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,53 @@
+---
+title: "Pluggable Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+Flink enables users to extend the default failure handling behavior using the plugin framework.
+
+## Custom failure enrichers
+Flink provides a pluggable failure enricher interface for users to register their custom logic.
+The goal is to give flexibility to developers who can now implement their own plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+Failure enrichers are triggered every time an exception is reported at runtime by the job manager.
+Every failure enricher may optionally return labels (kv string pairs) associated with the failure that are then exposed via the job manager's Rest interface (e.g., a 'System' tag implying the failure considered as a system error).
+For instance, when a Flink runtime failure occurs caused by network error, we can increment the appropriate counter.
+With accurate metrics, we now have better insight of platform level metrics e.g., network failures, platform reliability, etc.
+The default CountingFailureEnricher just records the failure count and then emits the metric "numJobFailure" for the job.
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom plugin for your use-case:
+
+- Add your own FailureEnricher by implementing the `org.apache.flink.core.failure.FailureEnricher` interface.
+
+- Add your own FailureEnricherFactory by implementing the `org.apache.flink.core.failure.FailureEnricherFactory` interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory`
+ which contains the class name of your exception classifier factory class (see the [Java Service Loader](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all the external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "exception-classification", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+As a plugin will be loaded in different classloader, log4j is not able to initialized correctly in your plugin. In this case,
+you should add the config below in `flink-conf.yaml`:
+- `plugin.classloader.parent-first-patterns.additional: org.slf4j`
Review Comment:
Also `org.apache.flink.configuration.CoreOptions#PARENT_FIRST_LOGGING_PATTERNS` already contains it
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,53 @@
+---
+title: "Pluggable Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+Flink enables users to extend the default failure handling behavior using the plugin framework.
+
+## Custom failure enrichers
+Flink provides a pluggable failure enricher interface for users to register their custom logic.
+The goal is to give flexibility to developers who can now implement their own plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+Failure enrichers are triggered every time an exception is reported at runtime by the job manager.
+Every failure enricher may optionally return labels (kv string pairs) associated with the failure that are then exposed via the job manager's Rest interface (e.g., a 'System' tag implying the failure considered as a system error).
+For instance, when a Flink runtime failure occurs caused by network error, we can increment the appropriate counter.
+With accurate metrics, we now have better insight of platform level metrics e.g., network failures, platform reliability, etc.
+The default CountingFailureEnricher just records the failure count and then emits the metric "numJobFailure" for the job.
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom plugin for your use-case:
+
+- Add your own FailureEnricher by implementing the `org.apache.flink.core.failure.FailureEnricher` interface.
+
+- Add your own FailureEnricherFactory by implementing the `org.apache.flink.core.failure.FailureEnricherFactory` interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory`
+ which contains the class name of your exception classifier factory class (see the [Java Service Loader](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all the external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "exception-classification", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+As a plugin will be loaded in different classloader, log4j is not able to initialized correctly in your plugin. In this case,
+you should add the config below in `flink-conf.yaml`:
+- `plugin.classloader.parent-first-patterns.additional: org.slf4j`
Review Comment:
This feels weird, you shouldn't bundle slf4j in the plugin
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] wangzzu commented on a diff in pull request #22468: FLINK-31889: Add documentation for implementing/loading enrichers
Posted by "wangzzu (via GitHub)" <gi...@apache.org>.
wangzzu commented on code in PR #22468:
URL: https://github.com/apache/flink/pull/22468#discussion_r1314608313
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,93 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html) docs for more details).
Review Comment:
Here I think the user needs to be reminded that the file name must be named `org.apache.flink.core.failure.FailureEnricherFactory`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] zentol commented on a diff in pull request #22468: FLINK-31889: Add documentation for implementing/loading enrichers
Posted by "zentol (via GitHub)" <gi...@apache.org>.
zentol commented on code in PR #22468:
URL: https://github.com/apache/flink/pull/22468#discussion_r1318386858
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,112 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all the external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise may be ignored.
Review Comment:
> otherwise may be ignored
Otherwise _what_ may be ignored?
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,112 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
Review Comment:
```suggestion
Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string key-value pairs).
```
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,112 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all the external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise may be ignored.
+{{< /hint >}}
+
+FailureEnricherFactory example:
+
+``` java
+public class TestFailureEnricherFactory implements FailureEnricherFactory {
+
+ @Override
+ public FailureEnricher createFailureEnricher(Configuration conf) {
+ return new CustomEnricher();
+ }
+}
+```
+
+FailureEnricher example:
+
+``` java
+public class CustomEnricher implements FailureEnricher {
+ private final Set<String> outputKeys;
+
+ public CustomEnricher() {
+ this.outputKeys = Collections.singleton("labelKey");
+ }
+
+ @Override
+ public Set<String> getOutputKeys() {
+ return outputKeys;
+ }
+
+ @Override
+ public CompletableFuture<Map<String, String>> processFailure(
+ Throwable cause, Context context) {
+ return CompletableFuture.completedFuture(Collections.singletonMap("labelKey", "labelValue"));
+ }
+}
+```
+
+### Configuration
+
+JobManager loads FailureEnricher plugins at startup. To make sure your FailureEnrichers are loaded:
+* All class names should be defined as part of {{< javadoc file="org/apache/flink/configuration/JobManagerOptions.html#FAILURE_ENRICHERS_LIST" name="jobmanager.failure-enrichers">}} configuration.
Review Comment:
Please link to the config docs instead of the javadocs.
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,112 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all the external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise may be ignored.
+{{< /hint >}}
+
+FailureEnricherFactory example:
+
+``` java
+public class TestFailureEnricherFactory implements FailureEnricherFactory {
+
+ @Override
+ public FailureEnricher createFailureEnricher(Configuration conf) {
+ return new CustomEnricher();
+ }
+}
+```
+
+FailureEnricher example:
+
+``` java
+public class CustomEnricher implements FailureEnricher {
+ private final Set<String> outputKeys;
+
+ public CustomEnricher() {
+ this.outputKeys = Collections.singleton("labelKey");
+ }
+
+ @Override
+ public Set<String> getOutputKeys() {
+ return outputKeys;
+ }
+
+ @Override
+ public CompletableFuture<Map<String, String>> processFailure(
+ Throwable cause, Context context) {
+ return CompletableFuture.completedFuture(Collections.singletonMap("labelKey", "labelValue"));
+ }
+}
+```
+
+### Configuration
+
+JobManager loads FailureEnricher plugins at startup. To make sure your FailureEnrichers are loaded:
Review Comment:
```suggestion
The JobManager loads FailureEnricher plugins at startup. To make sure your FailureEnrichers are loaded:
```
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,112 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all the external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise may be ignored.
+{{< /hint >}}
+
+FailureEnricherFactory example:
+
+``` java
+public class TestFailureEnricherFactory implements FailureEnricherFactory {
+
+ @Override
+ public FailureEnricher createFailureEnricher(Configuration conf) {
+ return new CustomEnricher();
+ }
+}
+```
+
+FailureEnricher example:
+
+``` java
+public class CustomEnricher implements FailureEnricher {
+ private final Set<String> outputKeys;
+
+ public CustomEnricher() {
+ this.outputKeys = Collections.singleton("labelKey");
+ }
+
+ @Override
+ public Set<String> getOutputKeys() {
+ return outputKeys;
+ }
+
+ @Override
+ public CompletableFuture<Map<String, String>> processFailure(
+ Throwable cause, Context context) {
+ return CompletableFuture.completedFuture(Collections.singletonMap("labelKey", "labelValue"));
+ }
+}
+```
+
+### Configuration
+
+JobManager loads FailureEnricher plugins at startup. To make sure your FailureEnrichers are loaded:
+* All class names should be defined as part of {{< javadoc file="org/apache/flink/configuration/JobManagerOptions.html#FAILURE_ENRICHERS_LIST" name="jobmanager.failure-enrichers">}} configuration.
+ If this configuration is empty, NO enrichers will be started. Example:
+```
+ jobmanager.failure-enrichers = org.apache.flink.test.plugin.jar.failure.CustomEnricher
Review Comment:
This would be more complete if the CustomEnricher example included a `package`
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,112 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
Review Comment:
```suggestion
This enables users to implement their own failure enrichment plugins to categorize job failures, expose custom metrics, or make calls to external notification systems.
```
Consistent target audience, clarify that you _can_ do that stuff and that it's not just a "goal" (which _may_ not have been reached).
I'd also remove "and more" because it doesn't add any value. You don't want users guessing what this can be used for.
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,112 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
Review Comment:
```suggestion
Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's REST API (e.g., a 'type:System' label implying the failure is categorized as a system error).
```
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,112 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all the external dependencies.
Review Comment:
```suggestion
Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all external dependencies.
```
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,112 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all the external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise may be ignored.
+{{< /hint >}}
+
+FailureEnricherFactory example:
+
+``` java
+public class TestFailureEnricherFactory implements FailureEnricherFactory {
+
+ @Override
+ public FailureEnricher createFailureEnricher(Configuration conf) {
+ return new CustomEnricher();
+ }
+}
+```
+
+FailureEnricher example:
+
+``` java
+public class CustomEnricher implements FailureEnricher {
+ private final Set<String> outputKeys;
+
+ public CustomEnricher() {
+ this.outputKeys = Collections.singleton("labelKey");
+ }
+
+ @Override
+ public Set<String> getOutputKeys() {
+ return outputKeys;
+ }
+
+ @Override
+ public CompletableFuture<Map<String, String>> processFailure(
+ Throwable cause, Context context) {
+ return CompletableFuture.completedFuture(Collections.singletonMap("labelKey", "labelValue"));
+ }
+}
+```
+
+### Configuration
+
+JobManager loads FailureEnricher plugins at startup. To make sure your FailureEnrichers are loaded:
+* All class names should be defined as part of {{< javadoc file="org/apache/flink/configuration/JobManagerOptions.html#FAILURE_ENRICHERS_LIST" name="jobmanager.failure-enrichers">}} configuration.
+ If this configuration is empty, NO enrichers will be started. Example:
+```
+ jobmanager.failure-enrichers = org.apache.flink.test.plugin.jar.failure.CustomEnricher
+```
+
+### Validation
+
+To validate that your FailureEnricher is loaded, you can check the JobManager logs for the following line:
+```
+ Found failure enricher org.apache.flink.test.plugin.jar.failure.CustomEnricher at jar:file:/path/to/flink/plugins/failure-enrichment/flink-test-plugin.jar!/org/apache/flink/test/plugin/jar/failure/CustomEnricher.class
+```
+
+Moreover, you can query the JobManager's Rest API using the following endpoint:
+```
+ http://localhost:8081/jobs/<jobid>/exceptions
Review Comment:
You could link to the REST API docs instead.
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,112 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all the external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise may be ignored.
+{{< /hint >}}
+
+FailureEnricherFactory example:
+
+``` java
+public class TestFailureEnricherFactory implements FailureEnricherFactory {
+
+ @Override
+ public FailureEnricher createFailureEnricher(Configuration conf) {
+ return new CustomEnricher();
+ }
+}
+```
+
+FailureEnricher example:
+
+``` java
+public class CustomEnricher implements FailureEnricher {
+ private final Set<String> outputKeys;
+
+ public CustomEnricher() {
+ this.outputKeys = Collections.singleton("labelKey");
+ }
+
+ @Override
+ public Set<String> getOutputKeys() {
+ return outputKeys;
+ }
+
+ @Override
+ public CompletableFuture<Map<String, String>> processFailure(
+ Throwable cause, Context context) {
+ return CompletableFuture.completedFuture(Collections.singletonMap("labelKey", "labelValue"));
+ }
+}
+```
+
+### Configuration
+
+JobManager loads FailureEnricher plugins at startup. To make sure your FailureEnrichers are loaded:
+* All class names should be defined as part of {{< javadoc file="org/apache/flink/configuration/JobManagerOptions.html#FAILURE_ENRICHERS_LIST" name="jobmanager.failure-enrichers">}} configuration.
Review Comment:
Feels weird to have a list containing a single item.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pgaref commented on pull request #22468: [FLINK-31889] Add documentation for implementing/loading enrichers
Posted by "pgaref (via GitHub)" <gi...@apache.org>.
pgaref commented on PR #22468:
URL: https://github.com/apache/flink/pull/22468#issuecomment-1712584944
@zentol just addressed latest comments -- please let me know what you think
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] zentol commented on a diff in pull request #22468: [FLINK-31889] Add documentation for implementing/loading enrichers
Posted by "zentol (via GitHub)" <gi...@apache.org>.
zentol commented on code in PR #22468:
URL: https://github.com/apache/flink/pull/22468#discussion_r1319493242
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,112 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all the external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise may be ignored.
Review Comment:
> When enrichers output keys overlap, only the first one is loaded and the rest are dropped with an Error message
That doesn't seem to match `org.apache.flink.runtime.failure.FailureEnricherUtils#filterInvalidEnrichers`. `invalidEnrichers` will contain all enrichers that clashed, and those are all thrown out.
```
final Set<Class<?>> invalidEnrichers =
enrichersByKey.entrySet().stream()
.filter(entry -> entry.getValue().size() > 1)
.flatMap(
entry -> {
LOG.warn(
"Following enrichers have have registered duplicate output key [%s] and will be ignored: {}.",
entry.getValue().stream()
.map(Class::getName)
.collect(Collectors.joining(", ")));
return entry.getValue().stream();
})
.collect(Collectors.toSet());
```
`org.apache.flink.runtime.failure.FailureEnricherUtilsTest#testValidatedEnrichersWithInvalidEntries` confirms that, as it uses 3 enrichers, with 2 having clashing keys, and asserting that only 1 is loaded.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pgaref commented on a diff in pull request #22468: [FLINK-31889] Add documentation for implementing/loading enrichers
Posted by "pgaref (via GitHub)" <gi...@apache.org>.
pgaref commented on code in PR #22468:
URL: https://github.com/apache/flink/pull/22468#discussion_r1318964767
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,112 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all the external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise may be ignored.
+{{< /hint >}}
+
+FailureEnricherFactory example:
+
+``` java
+public class TestFailureEnricherFactory implements FailureEnricherFactory {
+
+ @Override
+ public FailureEnricher createFailureEnricher(Configuration conf) {
+ return new CustomEnricher();
+ }
+}
+```
+
+FailureEnricher example:
+
+``` java
+public class CustomEnricher implements FailureEnricher {
+ private final Set<String> outputKeys;
+
+ public CustomEnricher() {
+ this.outputKeys = Collections.singleton("labelKey");
+ }
+
+ @Override
+ public Set<String> getOutputKeys() {
+ return outputKeys;
+ }
+
+ @Override
+ public CompletableFuture<Map<String, String>> processFailure(
+ Throwable cause, Context context) {
+ return CompletableFuture.completedFuture(Collections.singletonMap("labelKey", "labelValue"));
+ }
+}
+```
+
+### Configuration
+
+JobManager loads FailureEnricher plugins at startup. To make sure your FailureEnrichers are loaded:
+* All class names should be defined as part of {{< javadoc file="org/apache/flink/configuration/JobManagerOptions.html#FAILURE_ENRICHERS_LIST" name="jobmanager.failure-enrichers">}} configuration.
+ If this configuration is empty, NO enrichers will be started. Example:
+```
+ jobmanager.failure-enrichers = org.apache.flink.test.plugin.jar.failure.CustomEnricher
Review Comment:
`org.apache.flink.test.plugin.jar.failure` is the package? Not sure what you are referring to here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pgaref commented on a diff in pull request #22468: [FLINK-31889] Add documentation for implementing/loading enrichers
Posted by "pgaref (via GitHub)" <gi...@apache.org>.
pgaref commented on code in PR #22468:
URL: https://github.com/apache/flink/pull/22468#discussion_r1318963910
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,112 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all the external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise may be ignored.
Review Comment:
When enrichers output keys overlap, only the first one is loaded and the rest are dropped with an Error message -- that was the meaning of `This set of keys has to be unique across enrichers otherwise may be ignored.`
Rephrasing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] zentol commented on a diff in pull request #22468: [FLINK-31889] Add documentation for implementing/loading enrichers
Posted by "zentol (via GitHub)" <gi...@apache.org>.
zentol commented on code in PR #22468:
URL: https://github.com/apache/flink/pull/22468#discussion_r1318907314
##########
docs/content.zh/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,106 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string key-value pairs).
+This enables users to implement their own failure enrichment plugins to categorize job failures, expose custom metrics, or make calls to external notification systems.
Review Comment:
side-note: bit surprised that something that is advertised as being able to talk to external systems has no lifecycle hooks for managing resources (like a http client).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #22468: FLINK-31889: Add documentation for implementing/loading enrichers
Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #22468:
URL: https://github.com/apache/flink/pull/22468#issuecomment-1519286076
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "a0c85e227d20bcaaa791801f97ee809b44572446",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a0c85e227d20bcaaa791801f97ee809b44572446",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a0c85e227d20bcaaa791801f97ee809b44572446 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] zentol commented on a diff in pull request #22468: [FLINK-31889] Add documentation for implementing/loading enrichers
Posted by "zentol (via GitHub)" <gi...@apache.org>.
zentol commented on code in PR #22468:
URL: https://github.com/apache/flink/pull/22468#discussion_r1319494494
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,112 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all the external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise may be ignored.
+{{< /hint >}}
+
+FailureEnricherFactory example:
+
+``` java
+public class TestFailureEnricherFactory implements FailureEnricherFactory {
+
+ @Override
+ public FailureEnricher createFailureEnricher(Configuration conf) {
+ return new CustomEnricher();
+ }
+}
+```
+
+FailureEnricher example:
+
+``` java
+public class CustomEnricher implements FailureEnricher {
+ private final Set<String> outputKeys;
+
+ public CustomEnricher() {
+ this.outputKeys = Collections.singleton("labelKey");
+ }
+
+ @Override
+ public Set<String> getOutputKeys() {
+ return outputKeys;
+ }
+
+ @Override
+ public CompletableFuture<Map<String, String>> processFailure(
+ Throwable cause, Context context) {
+ return CompletableFuture.completedFuture(Collections.singletonMap("labelKey", "labelValue"));
+ }
+}
+```
+
+### Configuration
+
+JobManager loads FailureEnricher plugins at startup. To make sure your FailureEnrichers are loaded:
+* All class names should be defined as part of {{< javadoc file="org/apache/flink/configuration/JobManagerOptions.html#FAILURE_ENRICHERS_LIST" name="jobmanager.failure-enrichers">}} configuration.
+ If this configuration is empty, NO enrichers will be started. Example:
+```
+ jobmanager.failure-enrichers = org.apache.flink.test.plugin.jar.failure.CustomEnricher
Review Comment:
Where did you add the package to the example implementation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] zentol merged pull request #22468: [FLINK-31889] Add documentation for implementing/loading enrichers
Posted by "zentol (via GitHub)" <gi...@apache.org>.
zentol merged PR #22468:
URL: https://github.com/apache/flink/pull/22468
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] zentol commented on a diff in pull request #22468: [FLINK-31889] Add documentation for implementing/loading enrichers
Posted by "zentol (via GitHub)" <gi...@apache.org>.
zentol commented on code in PR #22468:
URL: https://github.com/apache/flink/pull/22468#discussion_r1318909983
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,106 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string key-value pairs).
+This enables users to implement their own failure enrichment plugins to categorize job failures, expose custom metrics, or make calls to external notification systems.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's REST API (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise values associated with them may be ignored.
+{{< /hint >}}
+
+FailureEnricherFactory example:
+
+``` java
+public class TestFailureEnricherFactory implements FailureEnricherFactory {
+
+ @Override
+ public FailureEnricher createFailureEnricher(Configuration conf) {
+ return new CustomEnricher();
+ }
+}
+```
+
+FailureEnricher example:
+
+``` java
+public class CustomEnricher implements FailureEnricher {
+ private final Set<String> outputKeys;
+
+ public CustomEnricher() {
+ this.outputKeys = Collections.singleton("labelKey");
+ }
+
+ @Override
+ public Set<String> getOutputKeys() {
+ return outputKeys;
+ }
+
+ @Override
+ public CompletableFuture<Map<String, String>> processFailure(
+ Throwable cause, Context context) {
+ return CompletableFuture.completedFuture(Collections.singletonMap("labelKey", "labelValue"));
+ }
+}
+```
+
+### Configuration
+
+The JobManager loads FailureEnricher plugins at startup. To make sure your FailureEnrichers are loaded all class names should be defined as part of [jobmanager.failure-enrichers configuration]({{< ref "docs/deployment/config#jobmanager-failure-enrichers" >}}).
+ If this configuration is empty, NO enrichers will be started. Example:
+```
+ jobmanager.failure-enrichers = org.apache.flink.test.plugin.jar.failure.CustomEnricher
Review Comment:
Oh my it does work like that :(
Better hope the enrichers don't setup any resources in their constructor :/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] zentol commented on a diff in pull request #22468: [FLINK-31889] Add documentation for implementing/loading enrichers
Posted by "zentol (via GitHub)" <gi...@apache.org>.
zentol commented on code in PR #22468:
URL: https://github.com/apache/flink/pull/22468#discussion_r1318898608
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,112 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all the external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise may be ignored.
Review Comment:
I don't understand what the implications "ignoring values" is. In practice all that happens if that one enricher would override the label of another one, right? Can we just write that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] zentol commented on a diff in pull request #22468: [FLINK-31889] Add documentation for implementing/loading enrichers
Posted by "zentol (via GitHub)" <gi...@apache.org>.
zentol commented on code in PR #22468:
URL: https://github.com/apache/flink/pull/22468#discussion_r1318921748
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,112 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all the external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise may be ignored.
Review Comment:
oh wow all enrichers that clash in any way are just completely dropped. That doesn't seem to match the documented behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] wangzzu commented on a diff in pull request #22468: FLINK-31889: Add documentation for implementing/loading enrichers
Posted by "wangzzu (via GitHub)" <gi...@apache.org>.
wangzzu commented on code in PR #22468:
URL: https://github.com/apache/flink/pull/22468#discussion_r1314383990
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,93 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html) docs for more details).
Review Comment:
maybe use `https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html` is better, now flink has spported jdk17
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,93 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all the external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise may be ignored.
+{{< /hint >}}
+
+FailureEnricherFactory example:
+
+``` java
+public class TestFailureEnricherFactory implements FailureEnricherFactory {
+
+ @Override
+ public FailureEnricher createFailureEnricher(Configuration conf) {
+ return new CustomEnricher();
+ }
+}
+```
+
+FailureEnricher example:
+
+``` java
+public class CustomEnricher implements FailureEnricher {
+ private final Set<String> outputKeys;
+
+ public CustomEnricher() {
+ this.outputKeys = Collections.singleton("labelKey");
+ }
+
+ @Override
+ public Set<String> getOutputKeys() {
+ return outputKeys;
+ }
+
+ @Override
+ public CompletableFuture<Map<String, String>> processFailure(
+ Throwable cause, Context context) {
+ return CompletableFuture.completedFuture(Collections.singletonMap("labelKey", "labelValue"));
+ }
+}
+```
+
+### Configuration
+
+JobManager loads FailureEnricher plugins at startup. To make sure your FailureEnrichers are loaded:
+* All class names should be defined as part of {{< javadoc file="org/apache/flink/configuration/JobManagerOptions.html#FAILURE_ENRICHERS_LIST" name="jobmanager.failure-enrichers">}} configuration.
+ If this configuration is empty, NO enrichers will be started. Example:
+```
+ jobmanager.failure-enrichers = org.apache.flink.test.plugin.jar.failure.CustomEnricher
+```
Review Comment:
Should we add a chapter here to explain how to confirm that this mechanism is in effect?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pgaref commented on pull request #22468: FLINK-31889: Add documentation for implementing/loading enrichers
Posted by "pgaref (via GitHub)" <gi...@apache.org>.
pgaref commented on PR #22468:
URL: https://github.com/apache/flink/pull/22468#issuecomment-1708649704
> Overall LGTM, some little comments
Thanks for taking a look @wangzzu!
Let me know how you feel about the latest changes!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] zentol commented on a diff in pull request #22468: [FLINK-31889] Add documentation for implementing/loading enrichers
Posted by "zentol (via GitHub)" <gi...@apache.org>.
zentol commented on code in PR #22468:
URL: https://github.com/apache/flink/pull/22468#discussion_r1318921748
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,112 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all the external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise may be ignored.
Review Comment:
oh wow all enrichers that clash in any way are just completely dropped if there is a clash. That doesn't seem to match the documented behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pgaref commented on a diff in pull request #22468: [FLINK-31889] Add documentation for implementing/loading enrichers
Posted by "pgaref (via GitHub)" <gi...@apache.org>.
pgaref commented on code in PR #22468:
URL: https://github.com/apache/flink/pull/22468#discussion_r1318968366
##########
docs/content.zh/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,106 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string key-value pairs).
+This enables users to implement their own failure enrichment plugins to categorize job failures, expose custom metrics, or make calls to external notification systems.
Review Comment:
like a setup/close methods? Main functionality is enriching/labeling exceptions but that would be a useful addition indeed. Lets see what users do with it first :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] zentol commented on a diff in pull request #22468: [FLINK-31889] Add documentation for implementing/loading enrichers
Posted by "zentol (via GitHub)" <gi...@apache.org>.
zentol commented on code in PR #22468:
URL: https://github.com/apache/flink/pull/22468#discussion_r1318902851
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,106 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string key-value pairs).
+This enables users to implement their own failure enrichment plugins to categorize job failures, expose custom metrics, or make calls to external notification systems.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's REST API (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise values associated with them may be ignored.
+{{< /hint >}}
+
+FailureEnricherFactory example:
+
+``` java
+public class TestFailureEnricherFactory implements FailureEnricherFactory {
+
+ @Override
+ public FailureEnricher createFailureEnricher(Configuration conf) {
+ return new CustomEnricher();
+ }
+}
+```
+
+FailureEnricher example:
+
+``` java
+public class CustomEnricher implements FailureEnricher {
+ private final Set<String> outputKeys;
+
+ public CustomEnricher() {
+ this.outputKeys = Collections.singleton("labelKey");
+ }
+
+ @Override
+ public Set<String> getOutputKeys() {
+ return outputKeys;
+ }
+
+ @Override
+ public CompletableFuture<Map<String, String>> processFailure(
+ Throwable cause, Context context) {
+ return CompletableFuture.completedFuture(Collections.singletonMap("labelKey", "labelValue"));
+ }
+}
+```
+
+### Configuration
+
+The JobManager loads FailureEnricher plugins at startup. To make sure your FailureEnrichers are loaded all class names should be defined as part of [jobmanager.failure-enrichers configuration]({{< ref "docs/deployment/config#jobmanager-failure-enrichers" >}}).
+ If this configuration is empty, NO enrichers will be started. Example:
+```
+ jobmanager.failure-enrichers = org.apache.flink.test.plugin.jar.failure.CustomEnricher
Review Comment:
side-note: If multiple enrichers are used this option can become a bit cumbersome for users, and a structure like we use for metrics reporters could've solved that (and paved a way to make them configurable!)
```
jobmanager.failure-enrichers.enricher1.class = org.apache.flink.test.plugin.jar.failure.CustomEnricher
```
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,106 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string key-value pairs).
+This enables users to implement their own failure enrichment plugins to categorize job failures, expose custom metrics, or make calls to external notification systems.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's REST API (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise values associated with them may be ignored.
+{{< /hint >}}
+
+FailureEnricherFactory example:
+
+``` java
+public class TestFailureEnricherFactory implements FailureEnricherFactory {
+
+ @Override
+ public FailureEnricher createFailureEnricher(Configuration conf) {
+ return new CustomEnricher();
+ }
+}
+```
+
+FailureEnricher example:
+
+``` java
+public class CustomEnricher implements FailureEnricher {
+ private final Set<String> outputKeys;
+
+ public CustomEnricher() {
+ this.outputKeys = Collections.singleton("labelKey");
+ }
+
+ @Override
+ public Set<String> getOutputKeys() {
+ return outputKeys;
+ }
+
+ @Override
+ public CompletableFuture<Map<String, String>> processFailure(
+ Throwable cause, Context context) {
+ return CompletableFuture.completedFuture(Collections.singletonMap("labelKey", "labelValue"));
+ }
+}
+```
+
+### Configuration
+
+The JobManager loads FailureEnricher plugins at startup. To make sure your FailureEnrichers are loaded all class names should be defined as part of [jobmanager.failure-enrichers configuration]({{< ref "docs/deployment/config#jobmanager-failure-enrichers" >}}).
+ If this configuration is empty, NO enrichers will be started. Example:
+```
+ jobmanager.failure-enrichers = org.apache.flink.test.plugin.jar.failure.CustomEnricher
Review Comment:
Is it correct that you configure the enricher itself, and not the factory? How does the factory lookup work? Do we just load all factories, have them all instantiate the enricher and do classname checks to find the right one...?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pgaref commented on a diff in pull request #22468: [FLINK-31889] Add documentation for implementing/loading enrichers
Posted by "pgaref (via GitHub)" <gi...@apache.org>.
pgaref commented on code in PR #22468:
URL: https://github.com/apache/flink/pull/22468#discussion_r1318975346
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,106 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string key-value pairs).
+This enables users to implement their own failure enrichment plugins to categorize job failures, expose custom metrics, or make calls to external notification systems.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's REST API (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise values associated with them may be ignored.
+{{< /hint >}}
+
+FailureEnricherFactory example:
+
+``` java
+public class TestFailureEnricherFactory implements FailureEnricherFactory {
+
+ @Override
+ public FailureEnricher createFailureEnricher(Configuration conf) {
+ return new CustomEnricher();
+ }
+}
+```
+
+FailureEnricher example:
+
+``` java
+public class CustomEnricher implements FailureEnricher {
+ private final Set<String> outputKeys;
+
+ public CustomEnricher() {
+ this.outputKeys = Collections.singleton("labelKey");
+ }
+
+ @Override
+ public Set<String> getOutputKeys() {
+ return outputKeys;
+ }
+
+ @Override
+ public CompletableFuture<Map<String, String>> processFailure(
+ Throwable cause, Context context) {
+ return CompletableFuture.completedFuture(Collections.singletonMap("labelKey", "labelValue"));
+ }
+}
+```
+
+### Configuration
+
+The JobManager loads FailureEnricher plugins at startup. To make sure your FailureEnrichers are loaded all class names should be defined as part of [jobmanager.failure-enrichers configuration]({{< ref "docs/deployment/config#jobmanager-failure-enrichers" >}}).
+ If this configuration is empty, NO enrichers will be started. Example:
+```
+ jobmanager.failure-enrichers = org.apache.flink.test.plugin.jar.failure.CustomEnricher
Review Comment:
So what you are implying here is by filtering on the Factory level we could avoid instantiating unwanted enrichers that could be potentialy setting up resources?
Happy to add that as a followup ticket and follow an approach similar to MetricsReporter setup
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pgaref commented on pull request #22468: FLINK-31889: Add documentation for implementing/loading enrichers
Posted by "pgaref (via GitHub)" <gi...@apache.org>.
pgaref commented on PR #22468:
URL: https://github.com/apache/flink/pull/22468#issuecomment-1710421376
> This needs a copy in the chinese version of the docs. (that may be in english; translation is a follow-up)
Thanks for the review @zentol !
Addressed your comments, also created a copy as the chinese version of the docs that will be addressed as part of https://issues.apache.org/jira/browse/FLINK-33061 by @wangzzu
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pgaref commented on a diff in pull request #22468: [FLINK-31889] Add documentation for implementing/loading enrichers
Posted by "pgaref (via GitHub)" <gi...@apache.org>.
pgaref commented on code in PR #22468:
URL: https://github.com/apache/flink/pull/22468#discussion_r1318966093
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,106 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string key-value pairs).
+This enables users to implement their own failure enrichment plugins to categorize job failures, expose custom metrics, or make calls to external notification systems.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's REST API (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise values associated with them may be ignored.
+{{< /hint >}}
+
+FailureEnricherFactory example:
+
+``` java
+public class TestFailureEnricherFactory implements FailureEnricherFactory {
+
+ @Override
+ public FailureEnricher createFailureEnricher(Configuration conf) {
+ return new CustomEnricher();
+ }
+}
+```
+
+FailureEnricher example:
+
+``` java
+public class CustomEnricher implements FailureEnricher {
+ private final Set<String> outputKeys;
+
+ public CustomEnricher() {
+ this.outputKeys = Collections.singleton("labelKey");
+ }
+
+ @Override
+ public Set<String> getOutputKeys() {
+ return outputKeys;
+ }
+
+ @Override
+ public CompletableFuture<Map<String, String>> processFailure(
+ Throwable cause, Context context) {
+ return CompletableFuture.completedFuture(Collections.singletonMap("labelKey", "labelValue"));
+ }
+}
+```
+
+### Configuration
+
+The JobManager loads FailureEnricher plugins at startup. To make sure your FailureEnrichers are loaded all class names should be defined as part of [jobmanager.failure-enrichers configuration]({{< ref "docs/deployment/config#jobmanager-failure-enrichers" >}}).
+ If this configuration is empty, NO enrichers will be started. Example:
+```
+ jobmanager.failure-enrichers = org.apache.flink.test.plugin.jar.failure.CustomEnricher
Review Comment:
Yes, this is the first version of the feature -- if many enrichers are loaded we could easily change that to what metrics reporters do but looked like an overkill for now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pgaref commented on a diff in pull request #22468: [FLINK-31889] Add documentation for implementing/loading enrichers
Posted by "pgaref (via GitHub)" <gi...@apache.org>.
pgaref commented on code in PR #22468:
URL: https://github.com/apache/flink/pull/22468#discussion_r1320633358
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,112 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all the external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise may be ignored.
+{{< /hint >}}
+
+FailureEnricherFactory example:
+
+``` java
+public class TestFailureEnricherFactory implements FailureEnricherFactory {
+
+ @Override
+ public FailureEnricher createFailureEnricher(Configuration conf) {
+ return new CustomEnricher();
+ }
+}
+```
+
+FailureEnricher example:
+
+``` java
+public class CustomEnricher implements FailureEnricher {
+ private final Set<String> outputKeys;
+
+ public CustomEnricher() {
+ this.outputKeys = Collections.singleton("labelKey");
+ }
+
+ @Override
+ public Set<String> getOutputKeys() {
+ return outputKeys;
+ }
+
+ @Override
+ public CompletableFuture<Map<String, String>> processFailure(
+ Throwable cause, Context context) {
+ return CompletableFuture.completedFuture(Collections.singletonMap("labelKey", "labelValue"));
+ }
+}
+```
+
+### Configuration
+
+JobManager loads FailureEnricher plugins at startup. To make sure your FailureEnrichers are loaded:
+* All class names should be defined as part of {{< javadoc file="org/apache/flink/configuration/JobManagerOptions.html#FAILURE_ENRICHERS_LIST" name="jobmanager.failure-enrichers">}} configuration.
+ If this configuration is empty, NO enrichers will be started. Example:
+```
+ jobmanager.failure-enrichers = org.apache.flink.test.plugin.jar.failure.CustomEnricher
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] pgaref commented on a diff in pull request #22468: [FLINK-31889] Add documentation for implementing/loading enrichers
Posted by "pgaref (via GitHub)" <gi...@apache.org>.
pgaref commented on code in PR #22468:
URL: https://github.com/apache/flink/pull/22468#discussion_r1320633268
##########
docs/content/docs/deployment/advanced/failure_enrichers.md:
##########
@@ -0,0 +1,112 @@
+---
+title: "Failure Enrichers"
+nav-title: failure-enrichers
+nav-parent_id: advanced
+nav-pos: 3
+---
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Custom failure enrichers
+Flink provides a pluggable interface for users to register their custom logic and enrich failures with extra metadata labels (string kv pairs).
+The goal is to enable developers implement their own failure enrichment plugins to categorize job failures, expose custom metrics, make calls to external notification systems, and more.
+
+FailureEnrichers are triggered every time an exception is reported at runtime by the JobManager.
+Every FailureEnricher may asynchronously return labels associated with the failure that are then exposed via the JobManager's Rest interface (e.g., a 'type:System' label implying the failure is categorized as a system error).
+
+
+### Implement a plugin for your custom enricher
+
+To implement a custom FailureEnricher plugin, you need to:
+
+- Add your own FailureEnricher by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="FailureEnricher" >}} interface.
+
+- Add your own FailureEnricherFactory by implementing the {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricherFactory.java" name="FailureEnricherFactory" >}} interface.
+
+- Add a service entry. Create a file `META-INF/services/org.apache.flink.core.failure.FailureEnricherFactory` which contains the class name of your failure enricher factory class (see [Java Service Loader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html) docs for more details).
+
+
+Then, create a jar which includes your `FailureEnricher`, `FailureEnricherFactory`, `META-INF/services/` and all the external dependencies.
+Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "failure-enrichment", and put the jar into this directory.
+See [Flink Plugin]({% link deployment/filesystems/plugins.md %}) for more details.
+
+{{< hint warning >}}
+Note that every FailureEnricher should have defined a set of {{< gh_link file="/flink-core/src/main/java/org/apache/flink/core/failure/FailureEnricher.java" name="output keys" >}} that may be associated with values. This set of keys has to be unique across enrichers otherwise may be ignored.
Review Comment:
You are right, that was the initial logic that we then reworked before merging.
Updating README accordingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org