You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by ke...@apache.org on 2025/06/03 15:54:10 UTC

(beam) branch master updated: add extra_transforms block documentation to chain transform documentation (#35101)

This is an automated email from the ASF dual-hosted git repository.

kenn pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
     new 64ea938877c add extra_transforms block documentation to chain transform documentation (#35101)
64ea938877c is described below

commit 64ea938877c6c0ed6ab02abff25dc00340ac1a05
Author: Derrick Williams <de...@google.com>
AuthorDate: Tue Jun 3 11:54:03 2025 -0400

    add extra_transforms block documentation to chain transform documentation (#35101)
---
 .../www/site/content/en/documentation/sdks/yaml.md | 46 ++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/website/www/site/content/en/documentation/sdks/yaml.md b/website/www/site/content/en/documentation/sdks/yaml.md
index 3d266ade616..73d1eebaae9 100644
--- a/website/www/site/content/en/documentation/sdks/yaml.md
+++ b/website/www/site/content/en/documentation/sdks/yaml.md
@@ -273,6 +273,52 @@ pipeline:
         path: /path/to/output.json
 ```
 
+If a `chain` pipeline has required error consumption or needs additional
+transforms not supported in a typical `chain` context, use an
+`extra_transforms` block.
+
+```
+pipeline:
+  type: chain
+  transforms:
+    - type: ReadFromCsv
+      config:
+        path: /path/to/input*.csv
+
+    - type: MapToFields
+      name: SomeStep
+      config:
+        language: python
+        fields:
+          col1: col1
+          # This could raise a divide-by-zero error.
+          ratio: col2 / col3
+        error_handling:
+          output: errors
+
+    - type: MapToFields
+      name: AnotherStep
+      config:
+        language: python
+        fields:
+          col1: col1
+          # This could raise a divide-by-zero error.
+          inverse_ratio: 1 / ratio
+        error_handling:
+          output: errors
+
+    - type: WriteToJson
+      config:
+        path: /path/to/output.json
+
+  extra_transforms:
+    - type: WriteToJson
+      name: WriteErrors
+      input: [SomeStep.errors, AnotherStep.errors]
+      config:
+        path: /path/to/errors.json
+```
+
 ### Source and sink transforms
 
 As syntactic sugar, you can name the first and last transforms in your pipeline