You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2023/12/06 03:54:41 UTC
(beam) branch nightly-refs/heads/master updated (c9c89fe7d5f -> e16d401987d)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch nightly-refs/heads/master
in repository https://gitbox.apache.org/repos/asf/beam.git
from c9c89fe7d5f Use UUIDs instead of object hashes to avoid collisions (#29542)
add 6973255b3d0 Bump cloud.google.com/go/spanner from 1.53.0 to 1.53.1 in /sdks (#29610)
add c53f0af09f9 Bump golang.org/x/net from 0.18.0 to 0.19.0 in /sdks (#29574)
add 17f3f0532c2 [#29568] Use gcr clone of images by default on Dataflow. (#29608)
add 59930cd2584 Bump github.com/aws/aws-sdk-go-v2 from 1.23.4 to 1.23.5 in /sdks (#29611)
add 4b4cc5d8088 Esacpe strings in integrationTestPipelineOptions in Java Tests
add 5ebd5d823d8 Merge pull request #29620: Escape strings in integrationTestPipelineOptions in Java Tests
add 169dce40670 Initial RC Testing Guide Documentation (#29595)
add eedbfe1fce4 Dask runner: Use bag sqrt scaling (#29366)
add 68bd67fb084 Bump go.mongodb.org/mongo-driver from 1.12.1 to 1.13.0 in /sdks (#29343)
add 70e41f4ebef Add docs and warning about using SpannerIO.Read in streaming pipelines (#29601)
add 7d997348b55 [YAML] - PubSubLite Normalization (#29525)
add 22ae2e35b98 Feature/dead letter queue core (#29164)
add 40685cdd0ee [RRIO] [Call] Implement the Repeater (#29490)
add 50904522a5c [yaml] Unmark as experimental
add 665ede8b0f6 Merge pull request #29625 [yaml] Unmark as experimental
add e16d401987d [YAML] Clean up some confusing error messages. (#29481)
No new revisions were added by this update.
Summary of changes:
.github/workflows/java_tests.yml | 4 +-
CHANGES.md | 1 +
contributor-docs/rc-testing-guide.md | 60 ++++
contributor-docs/release-guide.md | 8 +-
.../core/construction/PTransformTranslation.java | 7 +
sdks/go.mod | 12 +-
sdks/go.sum | 24 +-
sdks/go/pkg/beam/runners/dataflow/dataflow.go | 13 +-
sdks/go/pkg/beam/runners/dataflow/dataflow_test.go | 10 +
.../main/java/org/apache/beam/sdk/Pipeline.java | 24 ++
.../org/apache/beam/sdk/transforms/PTransform.java | 14 +
.../sdk/transforms/errorhandling/BadRecord.java | 218 ++++++++++++
.../transforms/errorhandling/BadRecordRouter.java | 85 +++++
.../sdk/transforms/errorhandling/ErrorHandler.java | 227 +++++++++++++
.../transforms/errorhandling}/package-info.java | 5 +-
.../errorhandling/BRHEnabledPTransform.java | 90 +++++
.../errorhandling/BadRecordRouterTest.java | 180 ++++++++++
.../transforms/errorhandling/ErrorHandlerTest.java | 136 ++++++++
...ueryStorageWriteApiSchemaTransformProvider.java | 11 +-
.../PubsubLiteReadSchemaTransformProvider.java | 311 +++++++++++++++--
.../PubsubLiteWriteSchemaTransformProvider.java | 245 ++++++++++++--
.../apache/beam/sdk/io/gcp/spanner/SpannerIO.java | 87 ++++-
.../gcp/pubsublite/internal/PubsubLiteDlqTest.java | 373 +++++++++++++++++++--
.../internal/PubsubLiteWriteDlqTest.java | 176 +++++++++-
.../apache/beam/io/requestresponse/Repeater.java | 140 ++++++++
...ion.java => UserCodeRemoteSystemException.java} | 15 +-
.../beam/io/requestresponse/RepeaterTest.java | 345 +++++++++++++++++++
.../runners/dask/transform_evaluator.py | 6 +-
.../apache_beam/runners/direct/direct_runner.py | 11 +
sdks/python/apache_beam/yaml/README.md | 33 +-
sdks/python/apache_beam/yaml/readme_test.py | 4 +-
sdks/python/apache_beam/yaml/standard_io.yaml | 30 ++
32 files changed, 2762 insertions(+), 143 deletions(-)
create mode 100644 contributor-docs/rc-testing-guide.md
create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/errorhandling/BadRecord.java
create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/errorhandling/BadRecordRouter.java
create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/errorhandling/ErrorHandler.java
copy {runners/core-java/src/main/java/org/apache/beam/runners/core => sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/errorhandling}/package-info.java (89%)
create mode 100644 sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/errorhandling/BRHEnabledPTransform.java
create mode 100644 sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/errorhandling/BadRecordRouterTest.java
create mode 100644 sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/errorhandling/ErrorHandlerTest.java
create mode 100644 sdks/java/io/rrio/src/main/java/org/apache/beam/io/requestresponse/Repeater.java
copy sdks/java/io/rrio/src/main/java/org/apache/beam/io/requestresponse/{UserCodeQuotaException.java => UserCodeRemoteSystemException.java} (70%)
create mode 100644 sdks/java/io/rrio/src/test/java/org/apache/beam/io/requestresponse/RepeaterTest.java