You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2022/05/02 22:16:56 UTC
[beam] branch asf-site updated: Publishing website 2022/05/02 22:16:51 at commit 40673dc
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 8ac17123195 Publishing website 2022/05/02 22:16:51 at commit 40673dc
8ac17123195 is described below
commit 8ac171231953a709cdd93c7674a46bb81e01b81d
Author: jenkins <bu...@apache.org>
AuthorDate: Mon May 2 22:16:51 2022 +0000
Publishing website 2022/05/02 22:16:51 at commit 40673dc
---
.../sdks/python-pipeline-dependencies/index.html | 11 ++---------
website/generated-content/sitemap.xml | 2 +-
2 files changed, 3 insertions(+), 10 deletions(-)
diff --git a/website/generated-content/documentation/sdks/python-pipeline-dependencies/index.html b/website/generated-content/documentation/sdks/python-pipeline-dependencies/index.html
index df7ce1f92e8..3201e8495d4 100644
--- a/website/generated-content/documentation/sdks/python-pipeline-dependencies/index.html
+++ b/website/generated-content/documentation/sdks/python-pipeline-dependencies/index.html
@@ -45,15 +45,8 @@ package then you can build the tarball with the following command:</p><pre><code
other_files_dir/
</code></pre><p>See the <a href=https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/complete/juliaset>Juliaset</a> project for an example that follows this required project structure.</p></li><li><p>Run your pipeline with the following command-line option:</p><pre><code> --setup_file /path/to/setup.py
</code></pre></li></ol><p><strong>Note:</strong> Because custom commands execute after the dependencies for your workflow are installed (by <code>pip</code>), you should omit the PyPI package dependency from the pipeline’s <code>requirements.txt</code> file and from the <code>install_requires</code> parameter in the <code>setuptools.setup()</code> call of your <code>setup.py</code> file.</p><h2 id=pre-building-sdk-container-image>Pre-building SDK container image</h2><p>In pipeline [...]
-However, it may be possible to pre-build the SDK containers and perform the dependency installation once before the workers start. To pre-build the container image before pipeline submission, provide the pipeline options mentioned below.</p><ol><li><p>Provide the container engine. Beam supports <code>local_docker</code>(requires local installation of Docker) and <code>cloud_build</code>(requires a GCP project with Cloud Build API enabled).</p><pre><code>--prebuild_sdk_container_engine=&l [...]
-</code></pre></li><li><p>If using <code>local_docker</code> engine, provide a URL for the remote registry to which the image will be pushed by passing</p><pre><code>--docker_registry_push_url=<remote_registry_url>
-# Example: --docker_registry_push_url=<registry_name>/beam
-# pre-built image will be pushed to the <registry_name>/beam/beam_python_prebuilt_sdk:<unique_image_tag>
-# <unique_image_tag> tag is generated by Beam SDK.
-</code></pre><p><strong>NOTE:</strong> <code>docker_registry_push_url</code> must be a remote registry.</p></li></ol><blockquote><p>The pre-building feature requires the Apache Beam SDK for Python, version 2.25.0 or later.
-The container images created during prebuilding will persist beyond the pipeline runtime.
-Once your job is finished or stopped, you can remove the pre-built image from the container registry.</p></blockquote><blockquote><p>If your pipeline is using a custom container image, most likely you will not benefit from pre-building step as extra dependencies can be preinstalled in the custom image at build time. If you still would like to use pre-building with custom images, use Apache Beam SDK 2.38.0 or newer and
-supply your custom image via <code>--sdk_container_image</code> pipeline option.</p></blockquote><p><strong>NOTE</strong>: This feature is available only for the <code>Dataflow Runner v2</code>.</p></div></div><footer class=footer><div class=footer__contained><div class=footer__cols><div class="footer__cols__col footer__cols__col__logos"><div class=footer__cols__col__logo><img src=/images/beam_logo_circle.svg class=footer__logo alt="Beam logo"></div><div class=footer__cols__col__logo><im [...]
+However, it may be possible to pre-build the SDK containers and perform the dependency installation once before the workers start with <code>--prebuild_sdk_container_engine</code>. For instructions of how to use pre-building with Google Cloud
+Dataflow, see <a href=https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild>Pre-building the python SDK custom container image with extra dependencies</a>.</p><p><strong>NOTE</strong>: This feature is available only for the <code>Dataflow Runner v2</code>.</p></div></div><footer class=footer><div class=footer__contained><div class=footer__cols><div class="footer__cols__col footer__cols__col__logos"><div class=footer__cols__col__logo><img src=/images/beam_logo_cir [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></div></div></div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml
index 64a1dd4389a..6eb3a9963ae 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/categories/blog/</loc><lastmod>2022-04-29T15:54:00-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2022-04-29T15:54:00-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2022-04-29T15:54:00-07:00</lastmod></url><url><loc>/blog/beam-sql-with-notebooks/</loc><lastmod>2022-04-29T15:54:00-07:00</lastmod></url><url> [...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/categories/blog/</loc><lastmod>2022-04-29T15:54:00-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2022-04-29T15:54:00-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2022-04-29T15:54:00-07:00</lastmod></url><url><loc>/blog/beam-sql-with-notebooks/</loc><lastmod>2022-04-29T15:54:00-07:00</lastmod></url><url> [...]
\ No newline at end of file