You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2023/12/15 03:54:26 UTC
(beam) branch nightly-refs/heads/master updated (def026bfe11 -> 52f4fc0c378)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch nightly-refs/heads/master
in repository https://gitbox.apache.org/repos/asf/beam.git
from def026bfe11 Bump google.golang.org/api from 0.153.0 to 0.154.0 in /sdks (#29734)
add 921e40a12f4 Dynamic SchemaTransform wrapper provider (#29561)
add 4264c2c3e25 Fixes breakages of the upgrade feature (#29731)
add 75746c6fe4a Bump cloud.google.com/go/storage from 1.35.1 to 1.36.0 in /sdks (#29767)
add 00462b13cc1 Bump google.golang.org/grpc from 1.59.0 to 1.60.0 in /sdks (#29714)
add 1ce707c3865 Add user-agent string for Beam in the GCS request header. (#29765)
add bc42a637d56 [#28543][prism] Implement State API (#29712)
add 9e8a310f0c0 Bump Precommit Java Examples Timeouts (#29771)
add c4c9bb82723 Add 2.54.0 release section to CHANGES.md (#29757)
add 543b013a548 Bump com.gradle.enterprise from 3.15.1 to 3.16 (#29740)
add a81c9e8e38b Bump actions/setup-python from 4 to 5 (#29654)
add 970a0bd9a92 Bump actions/stale from 8 to 9 (#29682)
add d297cd531ee Bump org.javacc.javacc from 3.0.0 to 3.0.2 (#29739)
add 1c7d178dd2b Bump github.com/google/uuid from 1.4.0 to 1.5.0 in /sdks (#29733)
add 52f4fc0c378 [Go SDK] Decrease sampling frequency for streaming jobs to avoid oversampling (#29774)
No new revisions were added by this update.
Summary of changes:
.../beam_PreCommit_Java_Examples_Dataflow.yml | 2 +-
...eam_PreCommit_Java_Examples_Dataflow_Java21.yml | 2 +-
.github/workflows/build_release_candidate.yml | 4 +-
.github/workflows/build_wheels.yml | 4 +-
.github/workflows/dask_runner_tests.yml | 4 +-
.github/workflows/local_env_tests.yml | 4 +-
.github/workflows/python_dependency_tests.yml | 2 +-
.github/workflows/python_tests.yml | 8 +-
.github/workflows/run_perf_alert_tool.yml | 2 +-
.github/workflows/run_rc_validation.yml | 14 +-
.github/workflows/stale.yml | 2 +-
.github/workflows/typescript_tests.yml | 4 +-
CHANGES.md | 34 +
.../core/construction/TransformUpgrader.java | 13 +-
sdks/go.mod | 8 +-
sdks/go.sum | 17 +-
sdks/go/pkg/beam/core/runtime/exec/datasink.go | 4 +-
sdks/go/pkg/beam/core/runtime/exec/pcollection.go | 14 +-
.../pkg/beam/runners/prism/internal/engine/data.go | 176 +++++
.../prism/internal/engine/elementmanager.go | 226 ++++++-
.../runners/prism/internal/engine/engine_test.go | 159 +++++
sdks/go/pkg/beam/runners/prism/internal/execute.go | 22 +
.../beam/runners/prism/internal/execute_test.go | 8 +-
.../pkg/beam/runners/prism/internal/handlepardo.go | 9 +-
.../beam/runners/prism/internal/jobservices/job.go | 3 +-
.../prism/internal/jobservices/management.go | 20 +-
.../pkg/beam/runners/prism/internal/preprocess.go | 14 +-
sdks/go/pkg/beam/runners/prism/internal/stage.go | 22 +-
.../runners/prism/internal/unimplemented_test.go | 48 +-
.../pkg/beam/runners/prism/internal/urns/urns.go | 5 +
.../beam/runners/prism/internal/worker/bundle.go | 6 +-
.../beam/runners/prism/internal/worker/worker.go | 89 ++-
sdks/go/test/integration/primitives/state.go | 53 +-
.../GenerateSequenceSchemaTransformProvider.java | 201 ++++++
.../apache/beam/sdk/providers}/package-info.java | 5 +-
...enerateSequenceSchemaTransformProviderTest.java | 61 ++
.../sdk/io/gcp/bigquery/BigQueryIOTranslation.java | 705 +++++++++++----------
sdks/java/io/kafka/upgrade/build.gradle | 2 -
.../sdk/io/kafka/upgrade/KafkaIOTranslation.java | 414 ++++++------
.../io/kafka/upgrade/KafkaIOTranslationTest.java | 11 +
.../io/external/xlang_kafkaio_it_test.py | 4 +-
sdks/python/apache_beam/io/gcp/gcsio.py | 7 +-
.../runners/dataflow/internal/apiclient.py | 6 +-
.../apache_beam/runners/interactive/utils.py | 8 +-
.../runners/portability/sdk_container_builder.py | 6 +-
sdks/python/apache_beam/transforms/external.py | 1 +
.../external_schematransform_provider.py | 277 ++++++++
.../external_schematransform_provider_test.py | 140 ++++
sdks/python/apache_beam/typehints/schemas.py | 3 +
sdks/python/pytest.ini | 2 +-
sdks/python/setup.py | 2 +-
sdks/python/test-suites/dataflow/build.gradle | 4 +-
sdks/python/test-suites/direct/build.gradle | 8 +-
sdks/python/test-suites/gradle.properties | 4 +-
sdks/python/test-suites/xlang/build.gradle | 9 +-
settings.gradle.kts | 4 +-
56 files changed, 2205 insertions(+), 681 deletions(-)
create mode 100644 sdks/go/pkg/beam/runners/prism/internal/engine/engine_test.go
create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/providers/GenerateSequenceSchemaTransformProvider.java
copy {it/google-cloud-platform/src/main/java/org/apache/beam/it/gcp/dlp => sdks/java/core/src/main/java/org/apache/beam/sdk/providers}/package-info.java (83%)
create mode 100644 sdks/java/core/src/test/java/org/apache/beam/sdk/providers/GenerateSequenceSchemaTransformProviderTest.java
create mode 100644 sdks/python/apache_beam/transforms/external_schematransform_provider.py
create mode 100644 sdks/python/apache_beam/transforms/external_schematransform_provider_test.py