You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2024/03/11 23:37:34 UTC

(beam) branch asf-site updated: Publishing website 2024/03/11 23:37:27 at commit d22a7e7

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/asf-site by this push:
     new dde043d4d78 Publishing website 2024/03/11 23:37:27 at commit d22a7e7
dde043d4d78 is described below

commit dde043d4d78f4affaf5329450e3a96f92f60c58e
Author: runner <ru...@main-runner-zt478-tf6m6>
AuthorDate: Mon Mar 11 23:37:28 2024 +0000

    Publishing website 2024/03/11 23:37:27 at commit d22a7e7
---
 website/generated-content/documentation/index.xml           | 13 ++++++-------
 .../documentation/programming-guide/index.html              |  7 +++++--
 website/generated-content/sitemap.xml                       |  2 +-
 3 files changed, 12 insertions(+), 10 deletions(-)

diff --git a/website/generated-content/documentation/index.xml b/website/generated-content/documentation/index.xml
index 432429ce373..d48748176b7 100644
--- a/website/generated-content/documentation/index.xml
+++ b/website/generated-content/documentation/index.xml
@@ -15484,10 +15484,11 @@ $ jar -jar /path/to/expansion_service.jar &amp;lt;PORT_NUMBER&amp;gt;&lt;/code>&
 &lt;/span>&lt;/span>&lt;span class="line">&lt;span class="cl"> &lt;span class="n">PythonExternalTransform&lt;/span>&lt;span class="o">.&amp;lt;&lt;/span>&lt;span class="n">PBegin&lt;/span>&lt;span class="o">,&lt;/span> &lt;span class="n">PCollection&lt;/span>&lt;span class="o">&amp;lt;&lt;/span>&lt;span class="n">String&lt;/span>&lt;span class="o">&amp;gt;&amp;gt;&lt;/span>&lt;span class="n">from&lt;/span>&lt;span class="o">(&lt;/span>&lt;span class="s">&amp;#34;apache_beam.io.ReadFromTe [...]
 &lt;/span>&lt;/span>&lt;span class="line">&lt;span class="cl"> &lt;span class="o">.&lt;/span>&lt;span class="na">withKwarg&lt;/span>&lt;span class="o">(&lt;/span>&lt;span class="s">&amp;#34;file_pattern&amp;#34;&lt;/span>&lt;span class="o">,&lt;/span> &lt;span class="n">options&lt;/span>&lt;span class="o">.&lt;/span>&lt;span class="na">getInputFile&lt;/span>&lt;span class="o">())&lt;/span>
 &lt;/span>&lt;/span>&lt;span class="line">&lt;span class="cl"> &lt;span class="o">.&lt;/span>&lt;span class="na">withKwarg&lt;/span>&lt;span class="o">(&lt;/span>&lt;span class="s">&amp;#34;validate&amp;#34;&lt;/span>&lt;span class="o">,&lt;/span> &lt;span class="kc">false&lt;/span>&lt;span class="o">))&lt;/span>
-&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;blockquote>
-&lt;p>&lt;strong>Note:&lt;/strong> &lt;code>PythonExternalTransform&lt;/code> has other useful methods such as &lt;code>withExtraPackages&lt;/code> for staging PyPI package dependencies and &lt;code>withOutputCoder&lt;/code> for setting an output coder.&lt;/p>
-&lt;/blockquote>
-&lt;p>Alternatively, you may want to create a Python module that registers an existing Python transform as a cross-language transform for use with the Python expansion service and calls into that existing transform to perform its intended operation. A registered URN can be used later in an expansion request for indicating an expansion target.&lt;/p>
+&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>&lt;code>PythonExternalTransform&lt;/code> has other useful methods such as &lt;code>withExtraPackages&lt;/code> for staging PyPI package dependencies and &lt;code>withOutputCoder&lt;/code> for setting an output coder. If your transform exists in an external package, make sure to specify that package using &lt;code>withExtraPackages&lt;/code>, for example:&lt;/p>
+&lt;div class="highlight">&lt;pre tabindex="0" class="chroma">&lt;code class="language-java" data-lang="java">&lt;span class="line">&lt;span class="cl">&lt;span class="n">p&lt;/span>&lt;span class="o">.&lt;/span>&lt;span class="na">apply&lt;/span>&lt;span class="o">(&lt;/span>&lt;span class="s">&amp;#34;Read&amp;#34;&lt;/span>&lt;span class="o">,&lt;/span>
+&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="cl"> &lt;span class="n">PythonExternalTransform&lt;/span>&lt;span class="o">.&amp;lt;&lt;/span>&lt;span class="n">PBegin&lt;/span>&lt;span class="o">,&lt;/span> &lt;span class="n">PCollection&lt;/span>&lt;span class="o">&amp;lt;&lt;/span>&lt;span class="n">String&lt;/span>&lt;span class="o">&amp;gt;&amp;gt;&lt;/span>&lt;span class="n">from&lt;/span>&lt;span class="o">(&lt;/span>&lt;span class="s">&amp;#34;my_python_package.BeamRea [...]
+&lt;/span>&lt;/span>&lt;span class="line">&lt;span class="cl"> &lt;span class="o">.&lt;/span>&lt;span class="na">withExtraPackages&lt;/span>&lt;span class="o">(&lt;/span>&lt;span class="n">ImmutableList&lt;/span>&lt;span class="o">.&lt;/span>&lt;span class="na">of&lt;/span>&lt;span class="o">(&lt;/span>&lt;span class="s">&amp;#34;my_python_package&amp;#34;&lt;/span>&lt;span class="o">)))&lt;/span>
+&lt;/span>&lt;/span>&lt;/code>&lt;/pre>&lt;/div>&lt;p>Alternatively, you may want to create a Python module that registers an existing Python transform as a cross-language transform for use with the Python expansion service and calls into that existing transform to perform its intended operation. A registered URN can be used later in an expansion request for indicating an expansion target.&lt;/p>
 &lt;p>&lt;strong>Defining the Python module&lt;/strong>&lt;/p>
 &lt;ol>
 &lt;li>
@@ -15551,11 +15552,9 @@ $ jar -jar /path/to/expansion_service.jar &amp;lt;PORT_NUMBER&amp;gt;&lt;/code>&
 &lt;/div>
 &lt;/li>
 &lt;li>
-&lt;p>This expansion service is now ready to serve up transforms on the address &lt;code>localhost:$PORT_FOR_EXPANSION_SERVICE&lt;/code>.&lt;/p>
+&lt;p>This expansion service is now ready to serve up transforms on the address `localhost:$PORT_FOR_EXPANSION_SERVICE&lt;/p>
 &lt;/li>
 &lt;/ol>
-&lt;p>&lt;strong>Including dependencies&lt;/strong>&lt;/p>
-&lt;p>Currently Python external transforms are limited to dependencies available in the core Beam SDK harness.&lt;/p>
 &lt;h4 id="1313-creating-cross-language-go-transforms">13.1.3. Creating cross-language Go transforms&lt;/h4>
 &lt;p>Go currently does not support creating cross-language transforms, only using cross-language
 transforms from other languages; see more at &lt;a href="https://github.com/apache/beam/issues/21767">Issue 21767&lt;/a>.&lt;/p>
diff --git a/website/generated-content/documentation/programming-guide/index.html b/website/generated-content/documentation/programming-guide/index.html
index 45ab1f54854..3f1c5feab3a 100644
--- a/website/generated-content/documentation/programming-guide/index.html
+++ b/website/generated-content/documentation/programming-guide/index.html
@@ -5595,7 +5595,10 @@ $ jar -jar /path/to/expansion_service.jar &lt;PORT_NUMBER&gt;</code></pre></div>
 </span></span><span class=line><span class=cl>    <span class=n>PythonExternalTransform</span><span class=o>.&lt;</span><span class=n>PBegin</span><span class=o>,</span> <span class=n>PCollection</span><span class=o>&lt;</span><span class=n>String</span><span class=o>&gt;&gt;</span><span class=n>from</span><span class=o>(</span><span class=s>&#34;apache_beam.io.ReadFromText&#34;</span><span class=o>)</span>
 </span></span><span class=line><span class=cl>    <span class=o>.</span><span class=na>withKwarg</span><span class=o>(</span><span class=s>&#34;file_pattern&#34;</span><span class=o>,</span> <span class=n>options</span><span class=o>.</span><span class=na>getInputFile</span><span class=o>())</span>
 </span></span><span class=line><span class=cl>    <span class=o>.</span><span class=na>withKwarg</span><span class=o>(</span><span class=s>&#34;validate&#34;</span><span class=o>,</span> <span class=kc>false</span><span class=o>))</span>
-</span></span></code></pre></div><blockquote><p><strong>Note:</strong> <code>PythonExternalTransform</code> has other useful methods such as <code>withExtraPackages</code> for staging PyPI package dependencies and <code>withOutputCoder</code> for setting an output coder.</p></blockquote><p>Alternatively, you may want to create a Python module that registers an existing Python transform as a cross-language transform for use with the Python expansion service and calls into that existing tr [...]
+</span></span></code></pre></div><p><code>PythonExternalTransform</code> has other useful methods such as <code>withExtraPackages</code> for staging PyPI package dependencies and <code>withOutputCoder</code> for setting an output coder. If your transform exists in an external package, make sure to specify that package using <code>withExtraPackages</code>, for example:</p><div class=highlight><pre tabindex=0 class=chroma><code class=language-java data-lang=java><span class=line><span clas [...]
+</span></span><span class=line><span class=cl>    <span class=n>PythonExternalTransform</span><span class=o>.&lt;</span><span class=n>PBegin</span><span class=o>,</span> <span class=n>PCollection</span><span class=o>&lt;</span><span class=n>String</span><span class=o>&gt;&gt;</span><span class=n>from</span><span class=o>(</span><span class=s>&#34;my_python_package.BeamReadPTransform&#34;</span><span class=o>)</span>
+</span></span><span class=line><span class=cl>    <span class=o>.</span><span class=na>withExtraPackages</span><span class=o>(</span><span class=n>ImmutableList</span><span class=o>.</span><span class=na>of</span><span class=o>(</span><span class=s>&#34;my_python_package&#34;</span><span class=o>)))</span>
+</span></span></code></pre></div><p>Alternatively, you may want to create a Python module that registers an existing Python transform as a cross-language transform for use with the Python expansion service and calls into that existing transform to perform its intended operation. A registered URN can be used later in an expansion request for indicating an expansion target.</p><p><strong>Defining the Python module</strong></p><ol><li><p>Define a Uniform Resource Name (URN) for your transfo [...]
 </span></span></code></pre></div></li><li><p>For an existing Python transform, create a new class to register the URN with the Python expansion service.</p><div class=highlight><pre tabindex=0 class=chroma><code class=language-py data-lang=py><span class=line><span class=cl><span class=nd>@ptransform.PTransform.register_urn</span><span class=p>(</span><span class=n>TEST_COMPK_URN</span><span class=p>,</span> <span class=kc>None</span><span class=p>)</span>
 </span></span><span class=line><span class=cl><span class=k>class</span> <span class=nc>CombinePerKeyTransform</span><span class=p>(</span><span class=n>ptransform</span><span class=o>.</span><span class=n>PTransform</span><span class=p>):</span>
 </span></span></code></pre></div></li><li><p>From within the class, define an expand method that takes an input PCollection, runs the Python transform, and then returns the output PCollection.</p><div class=highlight><pre tabindex=0 class=chroma><code class=language-py data-lang=py><span class=line><span class=cl><span class=k>def</span> <span class=nf>expand</span><span class=p>(</span><span class=bp>self</span><span class=p>,</span> <span class=n>pcoll</span><span class=p>):</span>
@@ -5610,7 +5613,7 @@ $ jar -jar /path/to/expansion_service.jar &lt;PORT_NUMBER&gt;</code></pre></div>
 </span></span><span class=line><span class=cl>    <span class=k>return</span> <span class=n>CombinePerKeyTransform</span><span class=p>()</span>
 </span></span></code></pre></div></li></ol><p><strong>Starting the expansion service</strong></p><p>An expansion service can be used with multiple transforms in the same pipeline. The Beam Python SDK provides a default expansion service for you to use with your Python transforms. You are free to write your own expansion service, but that is generally not needed, so it is not covered in this section.</p><p>Perform the following steps to start up the default Python expansion service direct [...]
     </code></pre></div></div></li><li><p>Import any modules that contain transforms to be made available using the expansion service.</p><div class=snippet><div class="notebook-skip code-snippet without_switcher"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><pre tabindex=0><code>$ python -m apache_beam.runners.portability.expansion_service_test -p $PORT_FOR_EXPANSION_SERVICE --pickle_library=cloudpickle
-    </code></pre></div></div></li><li><p>This expansion service is now ready to serve up transforms on the address <code>localhost:$PORT_FOR_EXPANSION_SERVICE</code>.</p></li></ol><p><strong>Including dependencies</strong></p><p>Currently Python external transforms are limited to dependencies available in the core Beam SDK harness.</p><h4 id=1313-creating-cross-language-go-transforms>13.1.3. Creating cross-language Go transforms</h4><p>Go currently does not support creating cross-languag [...]
+    </code></pre></div></div></li><li><p>This expansion service is now ready to serve up transforms on the address `localhost:$PORT_FOR_EXPANSION_SERVICE</p></li></ol><h4 id=1313-creating-cross-language-go-transforms>13.1.3. Creating cross-language Go transforms</h4><p>Go currently does not support creating cross-language transforms, only using cross-language
 transforms from other languages; see more at <a href=https://github.com/apache/beam/issues/21767>Issue 21767</a>.</p><h4 id=1314-defining-a-urn>13.1.4. Defining a URN</h4><p>Developing a cross-language transform involves defining a URN for registering the transform with an expansion service. In this section
 we provide a convention for defining such URNs. Following this convention is optional but it will ensure that your transform
 will not run into conflicts when registering in an expansion service along with transforms developed by other developers.</p><h5 id=13141-schema>13.1.4.1. Schema</h5><p>A URN should consist of the following components:</p><ul><li><strong>ns-id</strong>: A namespace identifier. Default recommendation is <code>beam:transform</code>.</li><li><strong>org-identifier</strong>: Identifies the organization where the transform was defined. Transforms defined in Apache Beam use <code>org.apache.be [...]
diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml
index bf227246272..86bcc10879c 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.54.0/</loc><lastmod>2024-03-11T13:21:18-04:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2024-03-11T13:21:18-04:00</lastmod></url><url><loc>/blog/</loc><lastmod>2024-03-11T13:21:18-04:00</lastmod></url><url><loc>/categories/</loc><lastmod>2024-03-11T13:21:18-04:00</lastmod></url><url><loc>/catego [...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.54.0/</loc><lastmod>2024-03-11T17:26:54-04:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2024-03-11T17:26:54-04:00</lastmod></url><url><loc>/blog/</loc><lastmod>2024-03-11T17:26:54-04:00</lastmod></url><url><loc>/categories/</loc><lastmod>2024-03-11T17:26:54-04:00</lastmod></url><url><loc>/catego [...]
\ No newline at end of file