You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2021/04/27 18:03:59 UTC
[beam] branch asf-site updated: Publishing website 2021/04/27
18:03:10 at commit a2230e0
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 54a6958 Publishing website 2021/04/27 18:03:10 at commit a2230e0
54a6958 is described below
commit 54a6958f843f95081e5166d057c393e6b9514cd9
Author: jenkins <bu...@apache.org>
AuthorDate: Tue Apr 27 18:03:11 2021 +0000
Publishing website 2021/04/27 18:03:10 at commit a2230e0
---
website/generated-content/documentation/index.xml | 23 ++++++++++++++++++----
.../documentation/programming-guide/index.html | 14 ++++++++-----
website/generated-content/sitemap.xml | 2 +-
3 files changed, 29 insertions(+), 10 deletions(-)
diff --git a/website/generated-content/documentation/index.xml b/website/generated-content/documentation/index.xml
index 0b5c467..6b29bf0 100644
--- a/website/generated-content/documentation/index.xml
+++ b/website/generated-content/documentation/index.xml
@@ -7029,10 +7029,25 @@ values for a pipeline.</p>
</div>
</div>
<h4 id="annotating-custom-type-default-coder">7.2.3. Annotating a custom data type with a default coder</h4>
-<p class="language-java">If your pipeline program defines a custom data type, you can use the
-<code>@DefaultCoder</code> annotation to specify the coder to use with that type. For
-example, let&rsquo;s say you have a custom data type for which you want to use
-<code>SerializableCoder</code>. You can use the <code>@DefaultCoder</code> annotation as follows:</p>
+<p class="language-java"><p>If your pipeline program defines a custom data type, you can use the
+<code>@DefaultCoder</code> annotation to specify the coder to use with that type.
+By default, Beam will use <code>SerializableCoder</code> which uses Java serialization,
+but it has drawbacks:</p>
+<ol>
+<li>
+<p>It is inefficient in encoding size and speed.
+See this <a href="https://blog.softwaremill.com/the-best-serialization-strategy-for-event-sourcing-9321c299632b">comparison of Java serialization methods.</a></p>
+</li>
+<li>
+<p>It is non-deterministic: it may produce different binary encodings for two
+equivalent objects.</p>
+<p>For key/value pairs, the correctness of key-based operations
+(GroupByKey, Combine) and per-key State depends on having a deterministic
+coder for the key.</p>
+</li>
+</ol>
+<p>You can use the <code>@DefaultCoder</code> annotation to set a new default as follows:</p>
+</p>
<div class='language-java snippet'>
<div class="notebook-skip code-snippet">
<a class="copy" type="button" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Copy to clipboard">
diff --git a/website/generated-content/documentation/programming-guide/index.html b/website/generated-content/documentation/programming-guide/index.html
index 4b083a3..b578a5d 100644
--- a/website/generated-content/documentation/programming-guide/index.html
+++ b/website/generated-content/documentation/programming-guide/index.html
@@ -1793,10 +1793,14 @@ values for a pipeline.</p><div class="language-java snippet"><div class="noteboo
<span class=n>Pipeline</span> <span class=n>p</span> <span class=o>=</span> <span class=n>Pipeline</span><span class=o>.</span><span class=na>create</span><span class=o>(</span><span class=n>options</span><span class=o>);</span>
<span class=n>CoderRegistry</span> <span class=n>cr</span> <span class=o>=</span> <span class=n>p</span><span class=o>.</span><span class=na>getCoderRegistry</span><span class=o>();</span>
-<span class=n>cr</span><span class=o>.</span><span class=na>registerCoder</span><span class=o>(</span><span class=n>Integer</span><span class=o>.</span><span class=na>class</span><span class=o>,</span> <span class=n>BigEndianIntegerCoder</span><span class=o>.</span><span class=na>class</span><span class=o>);</span></code></pre></div></div></div><div class="language-py snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=botto [...]
-<code>@DefaultCoder</code> annotation to specify the coder to use with that type. For
-example, let’s say you have a custom data type for which you want to use
-<code>SerializableCoder</code>. You can use the <code>@DefaultCoder</code> annotation as follows:</p><div class="language-java snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=nd>@DefaultCoder</span><span class=o>(</span><span class=n>AvroCoder</span><span class=o> [...]
+<span class=n>cr</span><span class=o>.</span><span class=na>registerCoder</span><span class=o>(</span><span class=n>Integer</span><span class=o>.</span><span class=na>class</span><span class=o>,</span> <span class=n>BigEndianIntegerCoder</span><span class=o>.</span><span class=na>class</span><span class=o>);</span></code></pre></div></div></div><div class="language-py snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=botto [...]
+<code>@DefaultCoder</code> annotation to specify the coder to use with that type.
+By default, Beam will use <code>SerializableCoder</code> which uses Java serialization,
+but it has drawbacks:</p><ol><li><p>It is inefficient in encoding size and speed.
+See this <a href=https://blog.softwaremill.com/the-best-serialization-strategy-for-event-sourcing-9321c299632b>comparison of Java serialization methods.</a></p></li><li><p>It is non-deterministic: it may produce different binary encodings for two
+equivalent objects.</p><p>For key/value pairs, the correctness of key-based operations
+(GroupByKey, Combine) and per-key State depends on having a deterministic
+coder for the key.</p></li></ol><p>You can use the <code>@DefaultCoder</code> annotation to set a new default as follows:</p></p><div class="language-java snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=nd>@DefaultCoder</span><span class=o>(</span><span class=n>Av [...]
<span class=kd>public</span> <span class=kd>class</span> <span class=nc>MyCustomDataType</span> <span class=o>{</span>
<span class=o>...</span>
<span class=o>}</span></code></pre></div></div></div><p class=language-java>If you’ve created a custom coder to match your data type, and you want to use
@@ -3408,7 +3412,7 @@ kafka_records = (
ImplicitSchemaPayloadBuilder({'data': u'0'}),
<Address of expansion service>))
assert_that(res, equal_to(['0a', '0b']))
- </code></pre></div></div></li><li><p>After the job has been submitted to the Beam runner, shutdown the expansion service by terminating the expansion service process.</p></li></ol><h3 id=x-lang-transform-runner-support>13.3. Runner Support</h3><p>Currently, portable runners such as Flink, Spark, and the Direct runner can be used with multi-language pipelines.</p><p>Google Cloud Dataflow supports multi-language pipelines through the Dataflow Runner v2 backend architecture.</p><div cla [...]
+ </code></pre></div></div></li><li><p>After the job has been submitted to the Beam runner, shutdown the expansion service by terminating the expansion service process.</p></li></ol><h3 id=x-lang-transform-runner-support>13.3. Runner Support</h3><p>Currently, portable runners such as Flink, Spark, and the Direct runner can be used with multi-language pipelines.</p><p>Google Cloud Dataflow supports multi-language pipelines through the Dataflow Runner v2 backend architecture.</p><div cla [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></div></div></div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml
index 5c4d226..2085cb1 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.28.0/</loc><lastmod>2021-02-22T11:40:20-08:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2021-03-31T21:00:41-04:00</lastmod></url><url><loc>/blog/</loc><lastmod>2021-03-31T21:00:41-04:00</lastmod></url><url><loc>/categories/</loc><lastmod>2021-03-31T21:00:41-04:00</lastmod></url><url><loc>/blog/k [...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.28.0/</loc><lastmod>2021-02-22T11:40:20-08:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2021-03-31T21:00:41-04:00</lastmod></url><url><loc>/blog/</loc><lastmod>2021-03-31T21:00:41-04:00</lastmod></url><url><loc>/categories/</loc><lastmod>2021-03-31T21:00:41-04:00</lastmod></url><url><loc>/blog/k [...]
\ No newline at end of file