You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2022/09/29 04:33:13 UTC
[beam] branch nightly-refs/heads/master updated (5cc47cdf56d -> b59df6c7614)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch nightly-refs/heads/master
in repository https://gitbox.apache.org/repos/asf/beam.git
from 5cc47cdf56d Add ISSUE#23071 to CHANGES.md (#23297)
add 91d79d973a3 Pin objsize version to avoid regression in 0.6.0 (#23396)
add a393efa61d4 [Tour Of Beam] API adjustments (#23349)
add 406da4ae224 Adds support in Samza Runner to run DoFn.processElement in parallel inside Samza tasks (#23313)
add e53466f8c4f Regenerate Go Protos (#23408)
add 1427b7dd93f Support google-cloud-spanner v3 and fixes broken unit tests (#23365)
add b7fe0f965c2 Add relevant docs to Cloud Profiler exceptions. (#23404)
add b59df6c7614 Update state cache to not fail when measuring object sizes. (#23391)
No new revisions were added by this update.
Summary of changes:
.github/workflows/tour_of_beam_backend.yml | 1 +
.../workflows/tour_of_beam_backend_integration.yml | 5 +-
learning/tour-of-beam/backend/README.md | 4 +-
learning/tour-of-beam/backend/docker-compose.yml | 2 +-
learning/tour-of-beam/backend/function.go | 71 +-
.../tour-of-beam/backend/integration_tests/api.go | 21 +-
.../backend/integration_tests/client.go | 28 +-
.../backend/integration_tests/function_test.go | 19 +-
.../backend/integration_tests/local.sh | 4 +-
learning/tour-of-beam/backend/internal/entity.go | 21 +-
.../backend/internal/fs_content/builders.go | 2 +-
.../backend/internal/fs_content/load.go | 4 +-
.../backend/internal/fs_content/load_test.go | 18 +-
learning/tour-of-beam/backend/internal/sdk.go | 43 +-
learning/tour-of-beam/backend/internal/sdk_test.go | 50 +-
.../backend/internal/storage/adapter.go | 36 +-
.../backend/internal/storage/datastore.go | 6 +-
.../backend/internal/storage/index.yaml | 2 +-
.../backend/internal/storage/schema.go | 14 +-
learning/tour-of-beam/backend/middleware.go | 79 +
.../backend/samples/api/get_content_tree.json | 20 +-
.../backend/samples/api/get_sdk_list.json | 8 +
.../backend/samples/api/get_unit_content.json | 4 +-
.../backend/samples/api/get_unit_content_full.json | 4 +-
.../beam/runners/samza/SamzaPipelineOptions.java | 31 +
.../samza/SamzaPipelineOptionsValidator.java | 7 +-
.../runners/samza/runtime/AsyncDoFnRunner.java | 118 ++
.../apache/beam/runners/samza/runtime/DoFnOp.java | 51 +-
.../runners/samza/runtime/FutureCollector.java | 7 +
.../beam/runners/samza/runtime/OpAdapter.java | 83 +-
.../beam/runners/samza/runtime/OpEmitter.java | 6 +
.../runners/samza/runtime/SamzaDoFnRunners.java | 13 +-
.../runners/samza/translation/ConfigBuilder.java | 19 +
.../beam/runners/samza/util/FutureUtils.java | 17 +
.../runners/samza/runtime/AsyncDoFnRunnerTest.java | 171 ++
.../beam/model/fnexecution_v1/beam_fn_api.pb.go | 107 +-
.../model/fnexecution_v1/beam_fn_api_grpc.pb.go | 2 +-
.../model/fnexecution_v1/beam_provision_api.pb.go | 2 +-
.../fnexecution_v1/beam_provision_api_grpc.pb.go | 2 +-
.../model/jobmanagement_v1/beam_artifact_api.pb.go | 2 +-
.../jobmanagement_v1/beam_artifact_api_grpc.pb.go | 2 +-
.../jobmanagement_v1/beam_expansion_api.pb.go | 129 +-
.../jobmanagement_v1/beam_expansion_api_grpc.pb.go | 2 +-
.../beam/model/jobmanagement_v1/beam_job_api.pb.go | 2 +-
.../model/jobmanagement_v1/beam_job_api_grpc.pb.go | 2 +-
.../beam/model/pipeline_v1/beam_runner_api.pb.go | 2242 ++++++++++----------
.../model/pipeline_v1/beam_runner_api_grpc.pb.go | 2 +-
sdks/go/pkg/beam/model/pipeline_v1/endpoints.pb.go | 2 +-
.../model/pipeline_v1/external_transforms.pb.go | 2 +-
sdks/go/pkg/beam/model/pipeline_v1/metrics.pb.go | 2 +-
sdks/go/pkg/beam/model/pipeline_v1/schema.pb.go | 732 ++++---
.../model/pipeline_v1/standard_window_fns.pb.go | 2 +-
.../apache_beam/io/gcp/experimental/spannerio.py | 7 +-
.../io/gcp/experimental/spannerio_test.py | 422 ++--
sdks/python/apache_beam/metrics/execution.py | 5 +
.../apache_beam/runners/worker/sdk_worker_main.py | 6 +-
.../apache_beam/runners/worker/sdk_worker_test.py | 2 +-
.../apache_beam/runners/worker/statecache.py | 109 +-
.../apache_beam/runners/worker/statecache_test.py | 69 +
sdks/python/setup.py | 6 +-
60 files changed, 2915 insertions(+), 1936 deletions(-)
create mode 100644 learning/tour-of-beam/backend/middleware.go
create mode 100644 learning/tour-of-beam/backend/samples/api/get_sdk_list.json
create mode 100644 runners/samza/src/main/java/org/apache/beam/runners/samza/runtime/AsyncDoFnRunner.java
create mode 100644 runners/samza/src/test/java/org/apache/beam/runners/samza/runtime/AsyncDoFnRunnerTest.java