You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by tv...@apache.org on 2023/09/07 20:58:35 UTC
[beam] branch master updated: Edit for clarity (#28360)
This is an automated email from the ASF dual-hosted git repository.
tvalentyn pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 5afb54e8f29 Edit for clarity (#28360)
5afb54e8f29 is described below
commit 5afb54e8f29ff2f71614ae15c9d0200b0d2b5541
Author: tvalentyn <tv...@users.noreply.github.com>
AuthorDate: Thu Sep 7 13:58:28 2023 -0700
Edit for clarity (#28360)
---
.../site/content/en/documentation/sdks/python-pipeline-dependencies.md | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/website/www/site/content/en/documentation/sdks/python-pipeline-dependencies.md b/website/www/site/content/en/documentation/sdks/python-pipeline-dependencies.md
index 025540cfb52..378032ab6b5 100644
--- a/website/www/site/content/en/documentation/sdks/python-pipeline-dependencies.md
+++ b/website/www/site/content/en/documentation/sdks/python-pipeline-dependencies.md
@@ -224,8 +224,9 @@ Use a constraint file to ensure that Beam dependencies in the launch environment
The launch environment translates the pipeline graph into a [runner-independent representation](https://github.com/apache/beam/blob/master/model/pipeline/src/main/proto/org/apache/beam/model/pipeline/v1/beam_runner_api.proto). This process involves serializing (or pickling) the code of the transforms. The serialized content is deserialized on the workers. If the runtime worker environment significantly differs from the launch environment, runtime errors might occur for the following reasons:
+* The Apache Beam version must match in the submission and runtime environments. Python major.minor versions must match as well. Otherwise, the pipeline might fail with errors like `Pipeline construction environment and pipeline runtime environment are not compatible`. On older SDK versions, the error might be reported as `SystemError: unknown opcode`.
+
* Versions of `protobuf` in the submission and runtime environment need to match or be compatible.
-The Apache Beam version and the Python major.minor versions must match in the submission and runtime environments. Otherwise, the pipeline might fail with errors like `Pipeline construction environment and pipeline runtime environment are not compatible`. On older SDK versions, the error might be reported as `SystemError: unknown opcode`.
* Libraries used in the pipeline code might need to match. If serialized pipeline code has references to functions or modules that aren’t available on the workers, the pipeline might fail with `ModuleNotFound` or `AttributeError` exceptions on the remote runner. If you encounter such errors, make sure that the affected libraries are available on the remote worker, and check whether you need to [save the main session]( https://beam.apache.org/documentation/sdks/python-pipeline-dependenci [...]