You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2022/10/26 22:15:50 UTC
[beam] branch asf-site updated: Publishing website 2022/10/26 22:15:43 at commit 9c44241
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 91ea99deeb9 Publishing website 2022/10/26 22:15:43 at commit 9c44241
91ea99deeb9 is described below
commit 91ea99deeb9dda9079d8f80c8de156f82ca006d3
Author: jenkins <bu...@apache.org>
AuthorDate: Wed Oct 26 22:15:43 2022 +0000
Publishing website 2022/10/26 22:15:43 at commit 9c44241
---
website/generated-content/contribute/index.xml | 24 +++++++-------
.../contribute/release-guide/index.html | 24 +++++++-------
.../sdks/python-machine-learning/index.html | 37 ++++------------------
.../get-started/downloads/index.html | 6 ++--
website/generated-content/get-started/index.xml | 6 ++++
website/generated-content/sitemap.xml | 2 +-
6 files changed, 44 insertions(+), 55 deletions(-)
diff --git a/website/generated-content/contribute/index.xml b/website/generated-content/contribute/index.xml
index fcbaefbe6ed..edbfaea2df4 100644
--- a/website/generated-content/contribute/index.xml
+++ b/website/generated-content/contribute/index.xml
@@ -1250,7 +1250,7 @@ is perfectly safe since the script does not depend on the current working tree.&
<ol>
<li>Visit <a href="https://hub.docker.com/search?q=apache%2Fbeam&amp;type=image">https://hub.docker.com/u/apache</a></li>
<li>Visit each repository and navigate to <em>tags</em> tab.</li>
-<li>Verify images are pushed with tags: ${RELEASE}_rc{RC_NUM}</li>
+<li>Verify images are pushed with tags: ${RELEASE_VERSION}_rc{RC_NUM}</li>
</ol>
</li>
<li>Verify that third party licenses are included in Docker containers by logging in to the images.
@@ -1259,12 +1259,12 @@ is perfectly safe since the script does not depend on the current working tree.&
Please note that dependencies for the SDKs with different Python versions vary.
Need to verify all Python images by replacing <code>${ver}</code> with each supported Python version <code>X.Y</code>.</li>
</ul>
-<pre><code>docker run --rm -it --entrypoint=/bin/bash apache/beam_python${ver}_sdk:${RELEASE}_rc{RC_NUM}
+<pre><code>docker run --rm -it --entrypoint=/bin/bash apache/beam_python${ver}_sdk:${RELEASE_VERSION}_rc{RC_NUM}
ls -al /opt/apache/beam/third_party_licenses/ | wc -l
</code></pre><ul>
<li>For Java SDK images, there should be around 200 dependencies.</li>
</ul>
-<pre><code>docker run --rm -it --entrypoint=/bin/bash apache/beam_java${ver}_sdk:${RELEASE}_rc{RC_NUM}
+<pre><code>docker run --rm -it --entrypoint=/bin/bash apache/beam_java${ver}_sdk:${RELEASE_VERSION}_rc{RC_NUM}
ls -al /opt/apache/beam/third_party_licenses/ | wc -l
</code></pre></li>
</ol>
@@ -1413,7 +1413,7 @@ ${CONTRIBUTORS}
<li>Maven artifacts deployed to the staging repository of <a href="https://repository.apache.org/content/repositories/">repository.apache.org</a></li>
<li>Source distribution deployed to the dev repository of <a href="https://dist.apache.org/repos/dist/dev/beam/">dist.apache.org</a></li>
<li>Website pull request proposed to list the <a href="https://beam.apache.org/get-started/downloads/">release</a>, publish the <a href="https://beam.apache.org/releases/javadoc/">Java API reference manual</a>, and publish the <a href="https://beam.apache.org/releases/pydoc/">Python API reference manual</a>.</li>
-<li>Docker images are published to <a href="https://hub.docker.com/search?q=apache%2Fbeam&amp;type=image">DockerHub</a> with tags: {RELEASE}_rc{RC_NUM}.</li>
+<li>Docker images are published to <a href="https://hub.docker.com/search?q=apache%2Fbeam&amp;type=image">DockerHub</a> with tags: {RELEASE_VERSION}_rc{RC_NUM}.</li>
</ol>
<p>You can (optionally) also do additional verification by:</p>
<ol>
@@ -1449,8 +1449,9 @@ The complete staging area is available for your review, which includes:
* website pull request listing the release [6], the blog post [6], and publishing the API reference manual [7].
* Java artifacts were built with Gradle GRADLE_VERSION and OpenJDK/Oracle JDK JDK_VERSION.
* Python artifacts are deployed along with the source release to the dist.apache.org [2] and PyPI[8].
-* Validation sheet with a tab for 1.2.3 release to help with validation [9].
-* Docker images published to Docker Hub [10].
+* Go artifacts and documentation are available at pkg.go.dev [9]
+* Validation sheet with a tab for 1.2.3 release to help with validation [10].
+* Docker images published to Docker Hub [11].
The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes.
For guidelines on how to try the release in your projects, check out our blog post at https://beam.apache.org/blog/validate-beam-release/.
Thanks,
@@ -1463,8 +1464,9 @@ Release Manager
[6] https://github.com/apache/beam/pull/...
[7] https://github.com/apache/beam-site/pull/...
[8] https://pypi.org/project/apache-beam/1.2.3rc3/
-[9] https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=...
-[10] https://hub.docker.com/search?q=apache%2Fbeam&amp;type=image
+[9] https://pkg.go.dev/github.com/apache/beam/sdks/v2@v1.2.3-RC3/go/pkg/beam
+[10] https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=...
+[11] https://hub.docker.com/search?q=apache%2Fbeam&amp;type=image
</code></pre>
<p>If there are any issues found in the release candidate, reply on the vote thread to cancel the vote.
There’s no need to wait 72 hours.
@@ -1792,8 +1794,8 @@ All wheels should be published, in addition to the zip of the release source.
</code></pre><ul>
<li><strong>Verify that:</strong>
<ul>
-<li>Images are published at <a href="https://hub.docker.com/search?q=apache%2Fbeam&amp;type=image">DockerHub</a> with tags {RELEASE} and <em>latest</em>.</li>
-<li>Images with <em>latest</em> tag are pointing to current release by confirming the digest of the image with <em>latest</em> tag is the same as the one with {RELEASE} tag.</li>
+<li>Images are published at <a href="https://hub.docker.com/search?q=apache%2Fbeam&amp;type=image">DockerHub</a> with tags {RELEASE_VERSION} and <em>latest</em>.</li>
+<li>Images with <em>latest</em> tag are pointing to current release by confirming the digest of the image with <em>latest</em> tag is the same as the one with {RELEASE_VERSION} tag.</li>
</ul>
</li>
</ul>
@@ -1809,7 +1811,7 @@ All wheels should be published, in addition to the zip of the release source.
<p>Create and push a new signed tag for the released version by copying the tag for the final release candidate, as follows:</p>
<pre><code># Optional: unlock the signing key by signing an arbitrary file.
gpg --output ~/doc.sig --sign ~/.bashrc
-VERSION_TAG=&quot;v${RELEASE}&quot;
+VERSION_TAG=&quot;v${RELEASE_VERSION}&quot;
# Tag for Go SDK
git tag -s &quot;sdks/$VERSION_TAG&quot; &quot;$RC_TAG&quot;
git push https://github.com/apache/beam &quot;sdks/$VERSION_TAG&quot;
diff --git a/website/generated-content/contribute/release-guide/index.html b/website/generated-content/contribute/release-guide/index.html
index 4fd6338478a..06acf40985a 100644
--- a/website/generated-content/contribute/release-guide/index.html
+++ b/website/generated-content/contribute/release-guide/index.html
@@ -143,11 +143,11 @@ The final state of the repository should match this diagram:</p><p><img src=/ima
adjust the version, and add the tag locally. If it looks good, run it again with <code>--push-tag</code>.
If you already have a clone that includes the <code>${COMMIT_REF}</code> then you can omit <code>--clone</code>. This
is perfectly safe since the script does not depend on the current working tree.</p><p>See the source of the script for more details, or to run commands manually in case of a problem.</p><h3 id=run-build_release_candidatesh-to-create-a-release-candidate>Run build_release_candidate.sh to create a release candidate</h3><ul><li><p><strong>Script:</strong> <a href=https://github.com/apache/beam/blob/master/release/src/main/scripts/build_release_candidate.sh>build_release_candidate.sh</a></p>< [...]
-</code></pre></li><li><p><strong>The script will:</strong></p><ol><li>Clone the repo at the selected RC tag.</li><li>Run gradle publish to push java artifacts into Maven staging repo.</li><li>Stage source release into dist.apache.org dev <a href=https://dist.apache.org/repos/dist/dev/beam/>repo</a>.</li><li>Stage, sign and hash python source distribution and wheels into dist.apache.org dev repo python dir</li><li>Stage SDK docker images to <a href="https://hub.docker.com/search?q=apache% [...]
+</code></pre></li><li><p><strong>The script will:</strong></p><ol><li>Clone the repo at the selected RC tag.</li><li>Run gradle publish to push java artifacts into Maven staging repo.</li><li>Stage source release into dist.apache.org dev <a href=https://dist.apache.org/repos/dist/dev/beam/>repo</a>.</li><li>Stage, sign and hash python source distribution and wheels into dist.apache.org dev repo python dir</li><li>Stage SDK docker images to <a href="https://hub.docker.com/search?q=apache% [...]
Please note that dependencies for the SDKs with different Python versions vary.
-Need to verify all Python images by replacing <code>${ver}</code> with each supported Python version <code>X.Y</code>.</li></ul><pre><code>docker run --rm -it --entrypoint=/bin/bash apache/beam_python${ver}_sdk:${RELEASE}_rc{RC_NUM}
+Need to verify all Python images by replacing <code>${ver}</code> with each supported Python version <code>X.Y</code>.</li></ul><pre><code>docker run --rm -it --entrypoint=/bin/bash apache/beam_python${ver}_sdk:${RELEASE_VERSION}_rc{RC_NUM}
ls -al /opt/apache/beam/third_party_licenses/ | wc -l
-</code></pre><ul><li>For Java SDK images, there should be around 200 dependencies.</li></ul><pre><code>docker run --rm -it --entrypoint=/bin/bash apache/beam_java${ver}_sdk:${RELEASE}_rc{RC_NUM}
+</code></pre><ul><li>For Java SDK images, there should be around 200 dependencies.</li></ul><pre><code>docker run --rm -it --entrypoint=/bin/bash apache/beam_java${ver}_sdk:${RELEASE_VERSION}_rc{RC_NUM}
ls -al /opt/apache/beam/third_party_licenses/ | wc -l
</code></pre></li></ol></li><li>Publish staging artifacts<ol><li>Log in to the <a href=https://repository.apache.org/#stagingRepositories>Apache Nexus</a> website.</li><li>Navigate to Build Promotion -> Staging Repositories (in the left sidebar).</li><li>Select repository <code>orgapachebeam-NNNN</code>.</li><li>Click the Close button.</li><li>When prompted for a description, enter “Apache Beam, version X, release candidate Y”.</li><li>Review all staged artifacts on <code>https://reposit [...]
They should contain all relevant parts for each module, including <code>pom.xml</code>, jar, test jar, javadoc, etc.
@@ -235,7 +235,7 @@ For more information on changes in {$RELEASE_VERSION}, check out the [detailed r
According to git shortlog, the following people contributed to the {$RELEASE_VERSION} release. Thank you to all contributors!
${CONTRIBUTORS}
-</code></pre><h4 id=checklist-to-proceed-to-the-next-step-1>Checklist to proceed to the next step</h4><ol><li>Maven artifacts deployed to the staging repository of <a href=https://repository.apache.org/content/repositories/>repository.apache.org</a></li><li>Source distribution deployed to the dev repository of <a href=https://dist.apache.org/repos/dist/dev/beam/>dist.apache.org</a></li><li>Website pull request proposed to list the <a href=https://beam.apache.org/get-started/downloads/>re [...]
+</code></pre><h4 id=checklist-to-proceed-to-the-next-step-1>Checklist to proceed to the next step</h4><ol><li>Maven artifacts deployed to the staging repository of <a href=https://repository.apache.org/content/repositories/>repository.apache.org</a></li><li>Source distribution deployed to the dev repository of <a href=https://dist.apache.org/repos/dist/dev/beam/>dist.apache.org</a></li><li>Website pull request proposed to list the <a href=https://beam.apache.org/get-started/downloads/>re [...]
docker pull apache/beam_python3.7_sdk:2.39.0_rc1
</code></pre><hr><h2 id=10-vote-and-validate-release-candidate>10. Vote and validate release candidate</h2><p>Once you have built and individually reviewed the release candidate, please share it for the community-wide review.
Please review foundation-wide <a href=https://www.apache.org/foundation/voting.html>voting guidelines</a> for more information.</p><p>Start the review-and-vote thread on the dev@ mailing list.
@@ -260,8 +260,9 @@ The complete staging area is available for your review, which includes:
* website pull request listing the release [6], the blog post [6], and publishing the API reference manual [7].
* Java artifacts were built with Gradle GRADLE_VERSION and OpenJDK/Oracle JDK JDK_VERSION.
* Python artifacts are deployed along with the source release to the dist.apache.org [2] and PyPI[8].
-* Validation sheet with a tab for 1.2.3 release to help with validation [9].
-* Docker images published to Docker Hub [10].
+* Go artifacts and documentation are available at pkg.go.dev [9]
+* Validation sheet with a tab for 1.2.3 release to help with validation [10].
+* Docker images published to Docker Hub [11].
The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes.
@@ -278,8 +279,9 @@ Release Manager
[6] https://github.com/apache/beam/pull/...
[7] https://github.com/apache/beam-site/pull/...
[8] https://pypi.org/project/apache-beam/1.2.3rc3/
-[9] https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=...
-[10] https://hub.docker.com/search?q=apache%2Fbeam&type=image
+[9] https://pkg.go.dev/github.com/apache/beam/sdks/v2@v1.2.3-RC3/go/pkg/beam
+[10] https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=...
+[11] https://hub.docker.com/search?q=apache%2Fbeam&type=image
</code></pre><p>If there are any issues found in the release candidate, reply on the vote thread to cancel the vote.
There’s no need to wait 72 hours.
Proceed to the <code>Fix Issues</code> step below and address the problem.
@@ -409,10 +411,10 @@ please follow <a href=https://help.github.com/articles/creating-a-personal-acces
</code></pre><ul><li>Verify that the files at <a href=https://pypi.org/project/apache-beam/#files>https://pypi.org/project/apache-beam/#files</a> are correct.
All wheels should be published, in addition to the zip of the release source.
(Signatures and hashes do <em>not</em> need to be uploaded.)</li></ul><h3 id=deploy-docker-images-to-dockerhub>Deploy docker images to DockerHub</h3><ul><li><strong>Script:</strong> <a href=https://github.com/apache/beam/blob/master/release/src/main/scripts/publish_docker_images.sh>publish_docker_images.sh</a></li><li><strong>Usage</strong></li></ul><pre><code>./beam/release/src/main/scripts/publish_docker_images.sh
-</code></pre><ul><li><strong>Verify that:</strong><ul><li>Images are published at <a href="https://hub.docker.com/search?q=apache%2Fbeam&type=image">DockerHub</a> with tags {RELEASE} and <em>latest</em>.</li><li>Images with <em>latest</em> tag are pointing to current release by confirming the digest of the image with <em>latest</em> tag is the same as the one with {RELEASE} tag.</li></ul></li></ul><p>(Optional) Clean up any unneeded local images afterward to save disk space.</p><h3 id=me [...]
+</code></pre><ul><li><strong>Verify that:</strong><ul><li>Images are published at <a href="https://hub.docker.com/search?q=apache%2Fbeam&type=image">DockerHub</a> with tags {RELEASE_VERSION} and <em>latest</em>.</li><li>Images with <em>latest</em> tag are pointing to current release by confirming the digest of the image with <em>latest</em> tag is the same as the one with {RELEASE_VERSION} tag.</li></ul></li></ul><p>(Optional) Clean up any unneeded local images afterward to save disk spa [...]
gpg --output ~/doc.sig --sign ~/.bashrc
-VERSION_TAG="v${RELEASE}"
+VERSION_TAG="v${RELEASE_VERSION}"
# Tag for Go SDK
git tag -s "sdks/$VERSION_TAG" "$RC_TAG"
@@ -429,7 +431,7 @@ This should happen automatically: <a href=https://lists.apache.org/thread.html/3
Ask other contributors to do the same.</p><p>Also, update <a href=https://en.wikipedia.org/wiki/Apache_Beam>the Wikipedia article on Apache Beam</a>.</p><h3 id=checklist-to-declare-the-process-completed>Checklist to declare the process completed</h3><ol><li>Release announced on the user@ mailing list.</li><li>Blog post published, if applicable.</li><li>Release recorded in reporter.apache.org.</li><li>Release announced on social media.</li><li>Completion declared on the dev@ mailing list. [...]
Once you’ve finished the release, please take a step back and look what areas of this process and be improved. Perhaps some part of the process can be simplified.
Perhaps parts of this guide can be clarified.</p><p>If we have specific ideas, please start a discussion on the dev@ mailing list and/or propose a pull request to update this guide.
-Thanks!</p><div class=feedback><p class=update>Last updated on 2022/10/24</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all useful and clear? Is there anything that you would like to change? Let us know!</p><button class=load-button><a href="mailto:dev@beam.apache.org?subject=Beam Website Feedback">SEND FEEDBACK</a></button></div></div></div><footer class=footer><div class=footer__contained><div class=footer__cols><div class="footer__cols__col foo [...]
+Thanks!</p><div class=feedback><p class=update>Last updated on 2022/10/26</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all useful and clear? Is there anything that you would like to change? Let us know!</p><button class=load-button><a href="mailto:dev@beam.apache.org?subject=Beam Website Feedback">SEND FEEDBACK</a></button></div></div></div><footer class=footer><div class=footer__contained><div class=footer__cols><div class="footer__cols__col foo [...]
<a href=https://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></div><div class="footer__cols__col footer__cols__col__logos"><div class=footer__cols__col--group><div class=footer__cols__col__logo><a href=https://github.com/apache/beam><im [...]
\ No newline at end of file
diff --git a/website/generated-content/documentation/sdks/python-machine-learning/index.html b/website/generated-content/documentation/sdks/python-machine-learning/index.html
index 8000f10c54d..a7c1aa141ce 100644
--- a/website/generated-content/documentation/sdks/python-machine-learning/index.html
+++ b/website/generated-content/documentation/sdks/python-machine-learning/index.html
@@ -19,7 +19,7 @@
function addPlaceholder(){$('input:text').attr('placeholder',"What are you looking for?");}
function endSearch(){var search=document.querySelector(".searchBar");search.classList.add("disappear");var icons=document.querySelector("#iconsBar");icons.classList.remove("disappear");}
function blockScroll(){$("body").toggleClass("fixedPosition");}
-function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfix container-main-content"><div class="section-nav closed" data-offset-top=90 data-offset-bottom=500><span class="section-nav-back glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list data-section-nav><li><span class=section-nav-list-main-title>Languages</span></li><li><span class=section-nav-list-title>Java</span><ul class=section-nav-list><li><a href=/documentation/sdks/java/>Java SDK overvi [...]
+function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfix container-main-content"><div class="section-nav closed" data-offset-top=90 data-offset-bottom=500><span class="section-nav-back glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list data-section-nav><li><span class=section-nav-list-main-title>Languages</span></li><li><span class=section-nav-list-title>Java</span><ul class=section-nav-list><li><a href=/documentation/sdks/java/>Java SDK overvi [...]
Pydoc</a></td></table><p><br><br><br></p><p>You can use Apache Beam with the RunInference API to use machine learning (ML) models to do local and remote inference with batch and streaming pipelines. Starting with Apache Beam 2.40.0, PyTorch and Scikit-learn frameworks are supported. Tensorflow models are supported through tfx-bsl.</p><p>You can create multiple types of transforms using the RunInference API: the API takes multiple types of setup parameters from model handlers, and the par [...]
<a href=https://github.com/apache/beam/blob/master/sdks/python/apache_beam/utils/shared.py#L20><code>Shared</code> class documentation</a>.</p><h3 id=multi-model-pipelines>Multi-model pipelines</h3><p>The RunInference API can be composed into multi-model pipelines. Multi-model pipelines can be useful for A/B testing or for building out cascade models made up of models that perform tokenization, sentence segmentation, part-of-speech tagging, named entity extraction, language detection, co [...]
with pipeline as p:
@@ -30,35 +30,12 @@ from apache_beam.ml.inference.sklearn_inference import SklearnModelHandlerPandas
from apache_beam.ml.inference.pytorch_inference import PytorchModelHandlerTensor
from apache_beam.ml.inference.pytorch_inference import PytorchModelHandlerKeyedTensor
from tfx_bsl.public.beam.run_inference import CreateModelHandler
-</code></pre><h3 id=use-pre-trained-models>Use pre-trained models</h3><p>The section provides requirements for using pre-trained models with PyTorch and Scikit-learn</p><h4 id=pytorch>PyTorch</h4><p>You need to provide a path to a file that contains the model’s saved weights. This path must be accessible by the pipeline. To use pre-trained models with the RunInference API and the PyTorch framework, complete the following steps:</p><ol><li>Download the pre-trained weights and host t [...]
+</code></pre><h3 id=use-pre-trained-models>Use pre-trained models</h3><p>The section provides requirements for using pre-trained models with PyTorch and Scikit-learn</p><h4 id=pytorch>PyTorch</h4><p>You need to provide a path to a file that contains the model’s saved weights. This path must be accessible by the pipeline. To use pre-trained models with the RunInference API and the PyTorch framework, complete the following steps:</p><ol><li>Download the pre-trained weights and host t [...]
+that illustrates running PyTorch models with Apache Beam.</p><h4 id=scikit-learn>Scikit-learn</h4><p>You need to provide a path to a file that contains the pickled Scikit-learn model. This path must be accessible by the pipeline. To use pre-trained models with the RunInference API and the Scikit-learn framework, complete the following steps:</p><ol><li>Download the pickled model class and host it in a location that the pipeline can access.</li><li>Pass the path of the model to the Sklear [...]
<code>model_uri=<path_to_pickled_file></code> and <code>model_file_type: <ModelFileType></code>, where you can specify
-<code>ModelFileType.PICKLE</code> or <code>ModelFileType.JOBLIB</code>, depending on how the model was serialized.</li></ol><h4 id=tensorflow>TensorFlow</h4><p>To use TensorFlow with the RunInference API, you need to do the following:</p><ul><li>Use <code>tfx_bsl</code> version 1.10.0 or later.</li><li>Create a model handler using <code>tfx_bsl.public.beam.run_inference.CreateModelHandler()</code>.</li><li>Use the model handler with the <a href=/releases/pydoc/current/apache_beam.ml.infe [...]
-from apache_beam.ml.inference.base import RunInference
-from tensorflow_serving.apis import prediction_log_pb2
-from tfx_bsl.public.proto import model_spec_pb2
-from tfx_bsl.public.tfxio import TFExampleRecord
-from tfx_bsl.public.beam.run_inference import CreateModelHandler
-
-pipeline = beam.Pipeline()
-tfexample_beam_record = TFExampleRecord(file_pattern='/path/to/examples')
-saved_model_spec = model_spec_pb2.SavedModelSpec(model_path='/path/to/model')
-inference_spec_type = model_spec_pb2.InferenceSpecType(saved_model_spec=saved_model_spec)
-model_handler = CreateModelHandler(inference_spec_type)
-with pipeline as p:
- _ = (p | tfexample_beam_record.RawRecordBeamSource()
- | RunInference(model_handler)
- | beam.Map(print)
- )
-</code></pre><p>Note: A model handler that is created with <code>CreateModelHander()</code> is always unkeyed.</p><h3 id=keyed-model-handlers>Keyed Model Handlers</h3><p>To make a keyed model handler, wrap any unkeyed model handler in the keyed model handler. For example:</p><pre><code>from apache_beam.ml.inference.base import RunInference
-from apache_beam.ml.inference.base import KeyedModelHandler
-model_handler = <Instantiate your model handler>
-keyed_model_handler = KeyedModelHandler(model_handler)
-
-with pipeline as p:
- p | ( <Your Pipeline>
- RunInference(keyed_model_handler)
- )
-</code></pre><p>If you are unsure if your data is keyed, you can also use <code>MaybeKeyedModelHandler</code>.</p><p>For more information, see <a href=https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.KeyedModelHandler><code>KeyedModelHander</code></a>.</p><h3 id=use-custom-models>Use custom models</h3><p>If you would like to use a model that isn’t specified by one of the supported frameworks, the RunInference API is des [...]
+<code>ModelFileType.PICKLE</code> or <code>ModelFileType.JOBLIB</code>, depending on how the model was serialized.</li></ol><p>See <a href=https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_sklearn.ipynb>this notebook</a>
+that illustrates running Scikit-learn models with Apache Beam.</p><h4 id=tensorflow>TensorFlow</h4><p>To use TensorFlow with the RunInference API, you need to do the following:</p><ul><li>Use <code>tfx_bsl</code> version 1.10.0 or later.</li><li>Create a model handler using <code>tfx_bsl.public.beam.run_inference.CreateModelHandler()</code>.</li><li>Use the model handler with the <a href=/releases/pydoc/current/apache_beam.ml.inference.base.html><code>apache_beam.ml.inference.base.RunInf [...]
+that illustrates running TensorFlow models with Apache Beam and tfx-bsl.</p><h3 id=use-custom-models>Use custom models</h3><p>If you would like to use a model that isn’t specified by one of the supported frameworks, the RunInference API is designed flexibly to allow you to use any custom machine learning models.
You only need to create your own <code>ModelHandler</code> or <code>KeyedModelHandler</code> with logic to load your model and use it to run the inference.</p><p>A simple example can be found in <a href=https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_custom_inference.ipynb>this notebook</a>.
The <code>load_model</code> method shows how to load the model using a popular <code>spaCy</code> package while <code>run_inference</code> shows how to run the inference on a batch of examples.</p><h3 id=use-multiple-models>Use multiple models</h3><p>You can also use the RunInference transform to add multiple inference models to your pipeline.</p><h4 id=ab-pattern>A/B Pattern</h4><pre><code>with pipeline as p:
data = p | 'Read' >> beam.ReadFromSource('a_source')
@@ -88,7 +65,7 @@ with pipeline as p:
('img3', torch.tensor([[1,2,3],[4,5,6],...])),
])
predictions = data | RunInference(keyed_model_handler)
-</code></pre><h3 id=use-the-predictionresults-object>Use the PredictionResults object</h3><p>When doing a prediction in Apache Beam, the output <code>PCollection</code> includes both the keys of the input examples and the inferences. Including both these items in the output allows you to find the input that determined the predictions.</p><p>The <code>PredictionResult</code> is a <code>NamedTuple</code> object that contains both the input and the inferences, named <code>example</code> and [...]
+</code></pre><p>If you are unsure if your data is keyed, you can also use <code>MaybeKeyedModelHandler</code>.</p><p>For more information, see <a href=https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.KeyedModelHandler><code>KeyedModelHander</code></a>.</p><h3 id=use-the-predictionresults-object>Use the PredictionResults object</h3><p>When doing a prediction in Apache Beam, the output <code>PCollection</code> includes both the [...]
def process(self, element: Tuple[str, PredictionResult]):
key, prediction_result = element
inputs = prediction_result.example
diff --git a/website/generated-content/get-started/downloads/index.html b/website/generated-content/get-started/downloads/index.html
index 80950b44743..8ee09dd6c6a 100644
--- a/website/generated-content/get-started/downloads/index.html
+++ b/website/generated-content/get-started/downloads/index.html
@@ -38,7 +38,9 @@ with the <code>DirectRunner</code>, add the following dependencies to your <code
with <code>DirectRunner</code>, add the following requirement to your <code>setup.py</code> file:</p><pre><code>apache-beam==2.42.0
</code></pre><p>Additionally, you may want to depend on additional SDK modules, such as IO
connectors or other extensions, and additional runners to execute your pipeline
-at scale.</p><h2 id=downloading-source-code>Downloading source code</h2><p>You can download the source code package for a release from the links in the
+at scale.</p><p>The Go SDK is accessible via Go Modules and calling <code>go get</code> from a module subdirectory:</p><pre><code> go get github.com/apache/beam/sdks/v2/go/pkg/beam
+</code></pre><p>Specific versions can be depended on similarly:</p><pre><code> go get github.com/apache/beam/sdks/v2@v2.42.0/go/pkg/beam
+</code></pre><h2 id=downloading-source-code>Downloading source code</h2><p>You can download the source code package for a release from the links in the
<a href=#releases>Releases</a> section.</p><h3 id=release-integrity>Release integrity</h3><p>You <em>must</em> <a href=https://www.apache.org/info/verification.html>verify</a> the integrity
of downloaded files. We provide OpenPGP signatures for every release file. This
signature should be matched against the
@@ -180,7 +182,7 @@ versions denoted <code>0.x.y</code>.</p><h2 id=releases>Releases</h2><h3 id=2420
<a href=https://archive.apache.org/dist/beam/2.1.0/apache-beam-2.1.0-source-release.zip.asc>signature</a>.</p><p><a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12340528">Release notes</a>.</p><h3 id=200-2017-05-17>2.0.0 (2017-05-17)</h3><p>Official <a href=https://archive.apache.org/dist/beam/2.0.0/apache-beam-2.0.0-source-release.zip>source code download</a>.
<a href=https://archive.apache.org/dist/beam/2.0.0/apache-beam-2.0.0-source-release.zip.sha1>SHA-1</a>.
<a href=https://archive.apache.org/dist/beam/2.0.0/apache-beam-2.0.0-source-release.zip.md5>MD5</a>.
-<a href=https://archive.apache.org/dist/beam/2.0.0/apache-beam-2.0.0-source-release.zip.asc>signature</a>.</p><p><a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12339746">Release notes</a>.</p><div class=feedback><p class=update>Last updated on 2022/10/17</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all useful and clear? Is there anything that you would like to change? Let us know!</p><button class=load- [...]
+<a href=https://archive.apache.org/dist/beam/2.0.0/apache-beam-2.0.0-source-release.zip.asc>signature</a>.</p><p><a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12339746">Release notes</a>.</p><div class=feedback><p class=update>Last updated on 2022/10/26</p><h3>Have you found everything you were looking for?</h3><p class=description>Was it all useful and clear? Is there anything that you would like to change? Let us know!</p><button class=load- [...]
<a href=https://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></div><div class="footer__cols__col footer__cols__col__logos"><div class=footer__cols__col--group><div class=footer__cols__col__logo><a href=https://github.com/apache/beam><im [...]
\ No newline at end of file
diff --git a/website/generated-content/get-started/index.xml b/website/generated-content/get-started/index.xml
index d4f7b99e23f..9c6585f4ee4 100644
--- a/website/generated-content/get-started/index.xml
+++ b/website/generated-content/get-started/index.xml
@@ -1576,6 +1576,12 @@ with <code>DirectRunner</code>, add the following requirement to your <
<p>Additionally, you may want to depend on additional SDK modules, such as IO
connectors or other extensions, and additional runners to execute your pipeline
at scale.</p>
+<p>The Go SDK is accessible via Go Modules and calling <code>go get</code> from a module subdirectory:</p>
+<pre><code> go get github.com/apache/beam/sdks/v2/go/pkg/beam
+</code></pre>
+<p>Specific versions can be depended on similarly:</p>
+<pre><code> go get github.com/apache/beam/sdks/v2@v2.42.0/go/pkg/beam
+</code></pre>
<h2 id="downloading-source-code">Downloading source code</h2>
<p>You can download the source code package for a release from the links in the
<a href="#releases">Releases</a> section.</p>
diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml
index a73b53aa8f2..1ff12253687 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.42.0/</loc><lastmod>2022-10-17T09:50:38-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2022-10-17T09:50:38-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2022-10-17T09:50:38-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2022-10-17T09:50:38-07:00</lastmod></url><url><loc>/catego [...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.42.0/</loc><lastmod>2022-10-17T09:50:38-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2022-10-17T09:50:38-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2022-10-17T09:50:38-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2022-10-17T09:50:38-07:00</lastmod></url><url><loc>/catego [...]
\ No newline at end of file