You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "dependabot[bot] (via GitHub)" <gi...@apache.org> on 2024/03/26 12:21:46 UTC
[PR] Bump apache-beam from 2.54.0 to 2.55.0 [beam-starter-python]
dependabot[bot] opened a new pull request, #30:
URL: https://github.com/apache/beam-starter-python/pull/30
Bumps [apache-beam](https://github.com/apache/beam) from 2.54.0 to 2.55.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/apache/beam/releases">apache-beam's releases</a>.</em></p>
<blockquote>
<h2>Beam 2.55.0 release</h2>
<p>We are happy to present the new 2.55.0 release of Beam.
This release includes both improvements and new functionality.
See the <a href="https://beam.apache.org/get-started/downloads/">download page</a> for this release.</p>
<!-- raw HTML omitted -->
<p>For more information on changes in 2.55.0, check out the <a href="https://github.com/apache/beam/milestone/19">detailed release notes</a>.</p>
<h2>Highlights</h2>
<ul>
<li>The Python SDK will now include automatically generated wrappers for external Java transforms! (<a href="https://redirect.github.com/apache/beam/pull/29834">#29834</a>)</li>
</ul>
<h2>I/Os</h2>
<ul>
<li>Added support for handling bad records to BigQueryIO (<a href="https://redirect.github.com/apache/beam/pull/30081">#30081</a>).
<ul>
<li>Full Support for Storage Read and Write APIs</li>
<li>Partial Support for File Loads (Failures writing to files supported, failures loading files to BQ unsupported)</li>
<li>No Support for Extract or Streaming Inserts</li>
</ul>
</li>
<li>Added support for handling bad records to PubSubIO (<a href="https://redirect.github.com/apache/beam/pull/30372">#30372</a>).
<ul>
<li>Support is not available for handling schema mismatches, and enabling error handling for writing to Pub/Sub topics with schemas is not recommended</li>
</ul>
</li>
<li><code>--enableBundling</code> pipeline option for BigQueryIO DIRECT_READ is replaced by <code>--enableStorageReadApiV2</code>. Both were considered experimental and subject to change (Java) (<a href="https://redirect.github.com/apache/beam/issues/26354">#26354</a>).</li>
</ul>
<h2>New Features / Improvements</h2>
<ul>
<li>Allow writing clustered and not time-partitioned BigQuery tables (Java) (<a href="https://redirect.github.com/apache/beam/pull/30094">#30094</a>).</li>
<li>Redis cache support added to RequestResponseIO and Enrichment transform (Python) (<a href="https://redirect.github.com/apache/beam/pull/30307">#30307</a>)</li>
<li>Merged <code>sdks/java/fn-execution</code> and <code>runners/core-construction-java</code> into the main SDK. These artifacts were never meant for users, but noting
that they no longer exist. These are steps to bring portability into the core SDK alongside all other core functionality.</li>
<li>Added Vertex AI Feature Store handler for Enrichment transform (Python) (<a href="https://redirect.github.com/apache/beam/pull/30388">#30388</a>)</li>
</ul>
<h2>Breaking Changes</h2>
<ul>
<li>Arrow version was bumped to 15.0.0 from 5.0.0 (<a href="https://redirect.github.com/apache/beam/pull/30181">#30181</a>).</li>
<li>Go SDK users who build custom worker containers may run into issues with the move to distroless containers as a base (see Security Fixes).
<ul>
<li>The issue stems from distroless containers lacking additional tools, which current custom container processes may rely on.</li>
<li>See <a href="https://beam.apache.org/documentation/runtime/environments/#from-scratch-go">https://beam.apache.org/documentation/runtime/environments/#from-scratch-go</a> for instructions on building and using a custom container.</li>
</ul>
</li>
<li>Python SDK has changed the default value for the <code>--max_cache_memory_usage_mb</code> pipeline option from 100 to 0. This option was first introduced in the 2.52.0 SDK version. This change restores the behavior of the 2.51.0 SDK, which does not use the state cache. If your pipeline uses iterable side inputs views, consider increasing the cache size by setting the option manually. (<a href="https://redirect.github.com/apache/beam/issues/30360">#30360</a>).</li>
</ul>
<h2>Deprecations</h2>
<ul>
<li>N/A</li>
</ul>
<h2>Bug fixes</h2>
<ul>
<li>Fixed <code>SpannerIO.readChangeStream</code> to support propagating credentials from pipeline options
to the <code>getDialect</code> calls for authenticating with Spanner (Java) (<a href="https://redirect.github.com/apache/beam/pull/30361">#30361</a>).</li>
<li>Reduced the number of HTTP requests in GCSIO function calls (Python) (<a href="https://redirect.github.com/apache/beam/pull/30205">#30205</a>)</li>
</ul>
<h2>Security Fixes</h2>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/apache/beam/blob/master/CHANGES.md">apache-beam's changelog</a>.</em></p>
<blockquote>
<h1>[2.55.0] - 2024-03-25</h1>
<h2>Highlights</h2>
<ul>
<li>The Python SDK will now include automatically generated wrappers for external Java transforms! (<a href="https://redirect.github.com/apache/beam/pull/29834">#29834</a>)</li>
</ul>
<h2>I/Os</h2>
<ul>
<li>Added support for handling bad records to BigQueryIO (<a href="https://redirect.github.com/apache/beam/pull/30081">#30081</a>).
<ul>
<li>Full Support for Storage Read and Write APIs</li>
<li>Partial Support for File Loads (Failures writing to files supported, failures loading files to BQ unsupported)</li>
<li>No Support for Extract or Streaming Inserts</li>
</ul>
</li>
<li>Added support for handling bad records to PubSubIO (<a href="https://redirect.github.com/apache/beam/pull/30372">#30372</a>).
<ul>
<li>Support is not available for handling schema mismatches, and enabling error handling for writing to pubsub topics with schemas is not recommended</li>
</ul>
</li>
<li><code>--enableBundling</code> pipeline option for BigQueryIO DIRECT_READ is replaced by <code>--enableStorageReadApiV2</code>. Both were considered experimental and may subject to change (Java) (<a href="https://redirect.github.com/apache/beam/issues/26354">#26354</a>).</li>
</ul>
<h2>New Features / Improvements</h2>
<ul>
<li>Allow writing clustered and not time partitioned BigQuery tables (Java) (<a href="https://redirect.github.com/apache/beam/pull/30094">#30094</a>).</li>
<li>Redis cache support added to RequestResponseIO and Enrichment transform (Python) (<a href="https://redirect.github.com/apache/beam/pull/30307">#30307</a>)</li>
<li>Merged sdks/java/fn-execution and runners/core-construction-java into the main SDK. These artifacts were never meant for users, but noting
that they no longer exist. These are steps to bring portability into the core SDK alongside all other core functionality.</li>
<li>Added Vertex AI Feature Store handler for Enrichment transform (Python) (<a href="https://redirect.github.com/apache/beam/pull/30388">#30388</a>)</li>
</ul>
<h2>Breaking Changes</h2>
<ul>
<li>Arrow version was bumped to 15.0.0 from 5.0.0 (<a href="https://redirect.github.com/apache/beam/pull/30181">#30181</a>).</li>
<li>Go SDK users who build custom worker containers may run into issues with the move to distroless containers as a base (see Security Fixes).
<ul>
<li>The issue stems from distroless containers lacking additional tools, which current custom container processes may rely on.</li>
<li>See <a href="https://beam.apache.org/documentation/runtime/environments/#from-scratch-go">https://beam.apache.org/documentation/runtime/environments/#from-scratch-go</a> for instructions on building and using a custom container.</li>
</ul>
</li>
<li>Python SDK has changed the default value for the <code>--max_cache_memory_usage_mb</code> pipeline option from 100 to 0. This option was first introduced in 2.52.0 SDK. This change restores the behavior of 2.51.0 SDK, which does not use the state cache. If your pipeline uses iterable side inputs views, consider increasing the cache size by setting the option manually. (<a href="https://redirect.github.com/apache/beam/issues/30360">#30360</a>).</li>
</ul>
<h2>Bugfixes</h2>
<ul>
<li>Fixed SpannerIO.readChangeStream to support propagating credentials from pipeline options
to the getDialect calls for authenticating with Spanner (Java) (<a href="https://redirect.github.com/apache/beam/pull/30361">#30361</a>).</li>
<li>Reduced the number of HTTP requests in GCSIO function calls (Python) (<a href="https://redirect.github.com/apache/beam/pull/30205">#30205</a>)</li>
</ul>
<h2>Security Fixes</h2>
<ul>
<li>Go SDK base container image moved to distroless/base-nossl-debian12, reducing vulnerable container surface to kernel and glibc (<a href="https://redirect.github.com/apache/beam/pull/30011">#30011</a>).</li>
</ul>
<h2>Known Issues</h2>
<ul>
<li>In Python pipelines, when shutting down inactive bundle processors, shutdown logic can overaggressively hold the lock, blocking acceptance of new work. Symptoms of this issue include slowness or stuckness in long-running jobs. Fixed in 2.56.0 (<a href="https://redirect.github.com/apache/beam/pull/30679">#30679</a>).</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/apache/beam/commit/dda0549e5e848cbc2a47a86c9ee9494040130c1f"><code>dda0549</code></a> Set version for 2.55.0 RC3</li>
<li><a href="https://github.com/apache/beam/commit/0dc330e6d53cce51b13bcb652e80859c1b3a5975"><code>0dc330e</code></a> Revert "Implementing lull reporting at bundle level processing (<a href="https://redirect.github.com/apache/beam/issues/29882">#29882</a>)" (<a href="https://redirect.github.com/apache/beam/issues/30">#30</a>...</li>
<li><a href="https://github.com/apache/beam/commit/d9897b0fc334f5804af54aeedad6c78935a4237b"><code>d9897b0</code></a> Include byte size of stateKey in estimated weight of WindmillBag, WindmillVal...</li>
<li><a href="https://github.com/apache/beam/commit/18d5bd119114b9a1cdf2fab09f0dd439b7a48e69"><code>18d5bd1</code></a> Add hamcrest as an implementation dependency (<a href="https://redirect.github.com/apache/beam/issues/30635">#30635</a>) (<a href="https://redirect.github.com/apache/beam/issues/30637">#30637</a>)</li>
<li><a href="https://github.com/apache/beam/commit/8dcaf128a1f10314c49df4c2c935b6cb1883b3fc"><code>8dcaf12</code></a> Redo spark_runner exclude test (<a href="https://redirect.github.com/apache/beam/issues/30589">#30589</a>) (<a href="https://redirect.github.com/apache/beam/issues/30598">#30598</a>)</li>
<li><a href="https://github.com/apache/beam/commit/5e1920f217f2cb9599a79e7dab16687c521bf68c"><code>5e1920f</code></a> Disable remote gradle cache until it is cleaned (<a href="https://redirect.github.com/apache/beam/issues/30584">#30584</a>) (<a href="https://redirect.github.com/apache/beam/issues/30596">#30596</a>)</li>
<li><a href="https://github.com/apache/beam/commit/93140cb113b673fd9ee1a1282a6ec64fe817c5c2"><code>93140cb</code></a> Resolve Any environment types before optimization. (<a href="https://redirect.github.com/apache/beam/issues/30568">#30568</a>) (<a href="https://redirect.github.com/apache/beam/issues/30576">#30576</a>)</li>
<li><a href="https://github.com/apache/beam/commit/b67fc15f5d5e7d21056054036197d68b7b070a54"><code>b67fc15</code></a> Bump InfluxDB PostCommit timeout (<a href="https://redirect.github.com/apache/beam/issues/30575">#30575</a>)</li>
<li><a href="https://github.com/apache/beam/commit/3c8e0cd1bbf578d22be1fd6f52e075abe2e324da"><code>3c8e0cd</code></a> Try fix Dataflow PreCommit on release branch (<a href="https://redirect.github.com/apache/beam/issues/30574">#30574</a>) (<a href="https://redirect.github.com/apache/beam/issues/30578">#30578</a>)</li>
<li><a href="https://github.com/apache/beam/commit/5d89a331647939934a4aeba501b8badb270600b2"><code>5d89a33</code></a> Fixes an upgrade imcompatiblity of BQ read/write transforms (<a href="https://redirect.github.com/apache/beam/issues/30562">#30562</a>)</li>
<li>Additional commits viewable in <a href="https://github.com/apache/beam/compare/v2.54.0...v2.55.0">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Bump apache-beam from 2.54.0 to 2.55.0 [beam-starter-python]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm merged PR #30:
URL: https://github.com/apache/beam-starter-python/pull/30
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org