You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2023/12/06 03:54:41 UTC

(beam) branch nightly-refs/heads/master updated (c9c89fe7d5f -> e16d401987d)

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch nightly-refs/heads/master
in repository https://gitbox.apache.org/repos/asf/beam.git


    from c9c89fe7d5f Use UUIDs instead of object hashes to avoid collisions (#29542)
     add 6973255b3d0 Bump cloud.google.com/go/spanner from 1.53.0 to 1.53.1 in /sdks (#29610)
     add c53f0af09f9 Bump golang.org/x/net from 0.18.0 to 0.19.0 in /sdks (#29574)
     add 17f3f0532c2 [#29568] Use gcr clone of images by default on Dataflow. (#29608)
     add 59930cd2584 Bump github.com/aws/aws-sdk-go-v2 from 1.23.4 to 1.23.5 in /sdks (#29611)
     add 4b4cc5d8088 Esacpe strings in integrationTestPipelineOptions in Java Tests
     add 5ebd5d823d8 Merge pull request #29620: Escape strings in integrationTestPipelineOptions in Java Tests
     add 169dce40670 Initial RC Testing Guide Documentation  (#29595)
     add eedbfe1fce4 Dask runner: Use bag sqrt scaling (#29366)
     add 68bd67fb084 Bump go.mongodb.org/mongo-driver from 1.12.1 to 1.13.0 in /sdks (#29343)
     add 70e41f4ebef Add docs and warning about using SpannerIO.Read in streaming pipelines (#29601)
     add 7d997348b55 [YAML] - PubSubLite Normalization (#29525)
     add 22ae2e35b98 Feature/dead letter queue core (#29164)
     add 40685cdd0ee [RRIO] [Call] Implement the Repeater (#29490)
     add 50904522a5c [yaml] Unmark as experimental
     add 665ede8b0f6 Merge pull request #29625 [yaml] Unmark as experimental
     add e16d401987d [YAML] Clean up some confusing error messages. (#29481)

No new revisions were added by this update.

Summary of changes:
 .github/workflows/java_tests.yml                   |   4 +-
 CHANGES.md                                         |   1 +
 contributor-docs/rc-testing-guide.md               |  60 ++++
 contributor-docs/release-guide.md                  |   8 +-
 .../core/construction/PTransformTranslation.java   |   7 +
 sdks/go.mod                                        |  12 +-
 sdks/go.sum                                        |  24 +-
 sdks/go/pkg/beam/runners/dataflow/dataflow.go      |  13 +-
 sdks/go/pkg/beam/runners/dataflow/dataflow_test.go |  10 +
 .../main/java/org/apache/beam/sdk/Pipeline.java    |  24 ++
 .../org/apache/beam/sdk/transforms/PTransform.java |  14 +
 .../sdk/transforms/errorhandling/BadRecord.java    | 218 ++++++++++++
 .../transforms/errorhandling/BadRecordRouter.java  |  85 +++++
 .../sdk/transforms/errorhandling/ErrorHandler.java | 227 +++++++++++++
 .../transforms/errorhandling}/package-info.java    |   5 +-
 .../errorhandling/BRHEnabledPTransform.java        |  90 +++++
 .../errorhandling/BadRecordRouterTest.java         | 180 ++++++++++
 .../transforms/errorhandling/ErrorHandlerTest.java | 136 ++++++++
 ...ueryStorageWriteApiSchemaTransformProvider.java |  11 +-
 .../PubsubLiteReadSchemaTransformProvider.java     | 311 +++++++++++++++--
 .../PubsubLiteWriteSchemaTransformProvider.java    | 245 ++++++++++++--
 .../apache/beam/sdk/io/gcp/spanner/SpannerIO.java  |  87 ++++-
 .../gcp/pubsublite/internal/PubsubLiteDlqTest.java | 373 +++++++++++++++++++--
 .../internal/PubsubLiteWriteDlqTest.java           | 176 +++++++++-
 .../apache/beam/io/requestresponse/Repeater.java   | 140 ++++++++
 ...ion.java => UserCodeRemoteSystemException.java} |  15 +-
 .../beam/io/requestresponse/RepeaterTest.java      | 345 +++++++++++++++++++
 .../runners/dask/transform_evaluator.py            |   6 +-
 .../apache_beam/runners/direct/direct_runner.py    |  11 +
 sdks/python/apache_beam/yaml/README.md             |  33 +-
 sdks/python/apache_beam/yaml/readme_test.py        |   4 +-
 sdks/python/apache_beam/yaml/standard_io.yaml      |  30 ++
 32 files changed, 2762 insertions(+), 143 deletions(-)
 create mode 100644 contributor-docs/rc-testing-guide.md
 create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/errorhandling/BadRecord.java
 create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/errorhandling/BadRecordRouter.java
 create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/errorhandling/ErrorHandler.java
 copy {runners/core-java/src/main/java/org/apache/beam/runners/core => sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/errorhandling}/package-info.java (89%)
 create mode 100644 sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/errorhandling/BRHEnabledPTransform.java
 create mode 100644 sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/errorhandling/BadRecordRouterTest.java
 create mode 100644 sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/errorhandling/ErrorHandlerTest.java
 create mode 100644 sdks/java/io/rrio/src/main/java/org/apache/beam/io/requestresponse/Repeater.java
 copy sdks/java/io/rrio/src/main/java/org/apache/beam/io/requestresponse/{UserCodeQuotaException.java => UserCodeRemoteSystemException.java} (70%)
 create mode 100644 sdks/java/io/rrio/src/test/java/org/apache/beam/io/requestresponse/RepeaterTest.java