You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2022/03/17 04:36:14 UTC
[beam] branch nightly-refs/heads/master updated (fc442c1 -> d08cef6)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch nightly-refs/heads/master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from fc442c1 Merge pull request #17010: [BEAM-14047] Updates x-lang release validation to use staged jars
add 800cf63 feat: add more custom metrics
add 7a05f76 Merge pull request #17064 from [BEAM-12164]: add metrics for partition reads, queries, and latencies for Spanner Change Streams Connector
add 950dc9f [BEAM-14103][Playgrounf][Bugfix] Fix google analytics id (#17092)
add 284bc7c Minor: Make ScopedReadStateSupplier final (#16992)
add ed8bb85 [BEAM-14113] Improve SamzaJobServerDriver extensibility (#17099)
add 8ae7628 [BEAM-14116] Chunk commit requests dynamically (#17004)
add f8a7a18 Merge pull request #17079 from [BEAM-13660] Add types and queries in Spanner's Postgresql dialect
add 15e636f [BEAM-13888] Add unit testing to ioutilx (#17058)
add ad21d83 Merge pull request #16822 from [BEAM-13841][Playground] Add Application and Infrastructure main modules to Terraform builder
add f7d9e6a Minor: Make serializableCoder warning gramatically correct english (#17020)
add 9c17960 [BEAM-14091] Fixing Interactive Beam show/collect for remote runners (#17072)
add c1a49c7 fixed typo in typehints
add 358916b Merge branch 'apache:master' into typehints_typo
add 6665638 Typehints typo #17059
add 02d9657 Remove unused prebuild_sdk_container_base_iamge option from validate container test
add d46bd07 Remove unused prebuild_sdk_container_base_image option from validate … #17086
add d1e98e5 Mapped JOB_STATE_RESOURCE_CLEANING_UP to State.RUNNING.
add d08cef6 Merge pull request #17055: [BEAM-14087] Mapped JOB_STATE_RESOURCE_CLEANING_UP to State.RUNNING
No new revisions were added by this update.
Summary of changes:
.github/workflows/build_playground_backend.yml | 52 +--
.github/workflows/build_playground_frontend.yml | 17 +-
.github/workflows/playground_deploy_examples.yml | 13 +-
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 2 +-
playground/CI.md | 117 ++++++
playground/backend/containers/router/Dockerfile | 9 +
playground/frontend/build.gradle | 1 +
playground/frontend/gradle.properties | 2 +-
playground/frontend/lib/config.g.dart | 2 +-
playground/terraform/README.md | 58 ++-
playground/terraform/applications/README.md | 59 ---
.../applications/{ => backend}/backend-go/main.tf | 21 +-
.../backend/backend-go/output.tf} | 6 +-
.../backend-go}/variables.tf | 49 ++-
.../{ => backend}/backend-java/main.tf | 17 +-
.../backend/backend-java/output.tf} | 6 +-
.../{ => backend}/backend-java/variables.tf | 43 +-
.../{ => backend}/backend-python/main.tf | 21 +-
.../backend/backend-python/output.tf} | 6 +-
.../{ => backend}/backend-python/variables.tf | 45 +++
.../{ => backend}/backend-router/main.tf | 29 +-
.../backend/backend-router/output.tf} | 6 +-
.../backend-router}/variables.tf | 49 ++-
.../{ => backend}/backend-scio/main.tf | 15 +-
.../backend/backend-scio/output.tf} | 6 +-
.../{ => backend}/backend-scio/variables.tf | 39 ++
playground/terraform/applications/backend/main.tf | 118 ++++++
.../backend/output.tf} | 20 +-
.../terraform/applications/backend/variables.tf | 227 +++++++++++
.../applications/{frontend => default}/main.tf | 44 ++-
.../setup => applications/default}/variables.tf | 13 +-
.../{ => applications/frontend/dispatch}/README.md | 22 +-
.../frontend/dispatch/dispatch.yaml} | 23 +-
playground/terraform/applications/frontend/main.tf | 7 +-
.../frontend/output.tf} | 7 +-
.../terraform/applications/frontend/variables.tf | 17 +-
playground/terraform/applications/main.tf | 87 +++++
.../buckets => applications}/output.tf | 32 +-
playground/terraform/applications/variables.tf | 244 ++++++++++++
playground/terraform/build.gradle.kts | 397 +++++++++++++++++++
.../beta/state.tfbackend} | 5 +-
.../infrastructure/artifact_registry/main.tf | 4 +-
.../infrastructure/artifact_registry/output.tf | 2 +-
.../infrastructure/artifact_registry/variables.tf | 4 +-
.../terraform/infrastructure/buckets/main.tf | 12 +-
.../terraform/infrastructure/buckets/output.tf | 19 -
.../terraform/infrastructure/buckets/variables.tf | 22 +-
.../terraform/infrastructure/gke/variables.tf | 2 +-
playground/terraform/infrastructure/main.tf | 79 ++--
.../terraform/infrastructure/memorystore/main.tf | 31 +-
.../{setup/provider.tf => memorystore/outputs.tf} | 7 +-
.../infrastructure/memorystore/variables.tf | 40 +-
.../terraform/infrastructure/network/output.tf | 12 +-
.../terraform/infrastructure/network/variables.tf | 4 +-
playground/terraform/infrastructure/output.tf | 55 +++
playground/terraform/infrastructure/provider.tf | 33 --
.../terraform/infrastructure/setup/services.tf | 1 +
.../terraform/infrastructure/setup/variables.tf | 7 +-
playground/terraform/infrastructure/variables.tf | 62 +--
playground/terraform/main.tf | 91 +++++
playground/terraform/output.tf | 79 ++++
playground/terraform/provider.tf | 30 +-
playground/terraform/variables.tf | 329 ++++++++++++++++
.../beam/runners/dataflow/util/MonitoringUtil.java | 1 +
.../worker/StreamingModeExecutionContext.java | 36 +-
.../worker/windmill/GrpcWindmillServer.java | 105 +++--
.../beam/runners/samza/SamzaJobServerDriver.java | 10 +-
sdks/go/pkg/beam/core/util/ioutilx/read_test.go | 94 +++++
sdks/go/pkg/beam/core/util/ioutilx/write_test.go | 43 ++
.../apache/beam/sdk/coders/SerializableCoder.java | 9 +-
.../sdk/io/gcp/spanner/MutationKeyEncoder.java | 7 +
.../sdk/io/gcp/spanner/MutationSizeEstimator.java | 2 +
.../beam/sdk/io/gcp/spanner/ReadSpannerSchema.java | 74 +++-
.../apache/beam/sdk/io/gcp/spanner/SpannerIO.java | 27 +-
.../beam/sdk/io/gcp/spanner/SpannerSchema.java | 126 ++++--
.../spanner/changestreams/ChangeStreamMetrics.java | 67 +++-
.../changestreams/action/ActionFactory.java | 7 +-
.../action/QueryChangeStreamAction.java | 9 +-
.../dofn/PostProcessingMetricsDoFn.java | 2 +
.../dofn/ReadChangeStreamPartitionDoFn.java | 4 +-
.../sdk/io/gcp/spanner/MutationKeyEncoderTest.java | 435 +++++++++++++++++++++
.../io/gcp/spanner/MutationSizeEstimatorTest.java | 47 +++
.../sdk/io/gcp/spanner/ReadSpannerSchemaTest.java | 54 ++-
.../sdk/io/gcp/spanner/SpannerIOWriteTest.java | 74 ++++
.../beam/sdk/io/gcp/spanner/SpannerReadIT.java | 106 ++++-
.../beam/sdk/io/gcp/spanner/SpannerSchemaTest.java | 36 ++
.../beam/sdk/io/gcp/spanner/SpannerWriteIT.java | 132 ++++++-
.../action/QueryChangeStreamActionTest.java | 6 +-
.../dofn/ReadChangeStreamPartitionDoFnTest.java | 3 +-
.../runners/interactive/interactive_environment.py | 15 +-
.../interactive/interactive_environment_test.py | 20 -
.../runners/interactive/recording_manager.py | 8 +-
.../runners/interactive/recording_manager_test.py | 33 ++
.../apache_beam/runners/interactive/utils.py | 13 +
.../apache_beam/runners/interactive/utils_test.py | 19 +
sdks/python/apache_beam/typehints/typehints.py | 2 +-
sdks/python/container/run_validatescontainer.sh | 1 -
settings.gradle.kts | 4 +
98 files changed, 3878 insertions(+), 631 deletions(-)
create mode 100644 playground/CI.md
delete mode 100644 playground/terraform/applications/README.md
rename playground/terraform/applications/{ => backend}/backend-go/main.tf (79%)
copy playground/terraform/{infrastructure/setup/provider.tf => applications/backend/backend-go/output.tf} (89%)
rename playground/terraform/applications/{backend-router => backend/backend-go}/variables.tf (55%)
rename playground/terraform/applications/{ => backend}/backend-java/main.tf (82%)
copy playground/terraform/{infrastructure/setup/provider.tf => applications/backend/backend-java/output.tf} (89%)
rename playground/terraform/applications/{ => backend}/backend-java/variables.tf (64%)
rename playground/terraform/applications/{ => backend}/backend-python/main.tf (79%)
copy playground/terraform/{infrastructure/setup/provider.tf => applications/backend/backend-python/output.tf} (88%)
rename playground/terraform/applications/{ => backend}/backend-python/variables.tf (59%)
rename playground/terraform/applications/{ => backend}/backend-router/main.tf (70%)
copy playground/terraform/{infrastructure/setup/provider.tf => applications/backend/backend-router/output.tf} (88%)
rename playground/terraform/applications/{backend-go => backend/backend-router}/variables.tf (55%)
rename playground/terraform/applications/{ => backend}/backend-scio/main.tf (85%)
copy playground/terraform/{infrastructure/setup/provider.tf => applications/backend/backend-scio/output.tf} (89%)
rename playground/terraform/applications/{ => backend}/backend-scio/variables.tf (66%)
create mode 100644 playground/terraform/applications/backend/main.tf
copy playground/terraform/{infrastructure/artifact_registry/variables.tf => applications/backend/output.tf} (69%)
create mode 100644 playground/terraform/applications/backend/variables.tf
copy playground/terraform/applications/{frontend => default}/main.tf (57%)
copy playground/terraform/{infrastructure/setup => applications/default}/variables.tf (79%)
copy playground/terraform/{ => applications/frontend/dispatch}/README.md (59%)
copy playground/terraform/{infrastructure/setup/variables.tf => applications/frontend/dispatch/dispatch.yaml} (70%)
copy playground/terraform/{infrastructure/setup/provider.tf => applications/frontend/output.tf} (89%)
create mode 100644 playground/terraform/applications/main.tf
copy playground/terraform/{infrastructure/buckets => applications}/output.tf (52%)
create mode 100644 playground/terraform/applications/variables.tf
create mode 100644 playground/terraform/build.gradle.kts
copy playground/terraform/{infrastructure/setup/provider.tf => environment/beta/state.tfbackend} (91%)
rename playground/terraform/infrastructure/{setup/provider.tf => memorystore/outputs.tf} (91%)
create mode 100644 playground/terraform/infrastructure/output.tf
delete mode 100644 playground/terraform/infrastructure/provider.tf
create mode 100644 playground/terraform/main.tf
create mode 100644 playground/terraform/output.tf
create mode 100644 playground/terraform/variables.tf
create mode 100644 sdks/go/pkg/beam/core/util/ioutilx/read_test.go
create mode 100644 sdks/go/pkg/beam/core/util/ioutilx/write_test.go