You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by ro...@apache.org on 2022/11/10 17:10:50 UTC
[beam] branch master updated (38a85b288ad -> 74f87b0282a)
This is an automated email from the ASF dual-hosted git repository.
robertwb pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
from 38a85b288ad Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks (#24077)
add ed7b560d7ce Compute element counts for all PCollections.
add 9283512544b Add the ability to schedule splits on the ULR via a pipeline option.
add ed520243f47 Add the a Reshuffle operation and use it in Create.
add 3bd697561aa Add dynamic splitting support to the worker.
add 60fe3cdddf9 Update style sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py
add 5f587b9339b Clarifying comments.
add 7f7713de09b Merge branch 'master' into javascript-liquid-sharding
add a84b7dc179f Make mypy happy.
add 383ea77bc98 Reduce flakiness of time-based split manager test.
new 74f87b0282a Merge pull request #23985 Support dynamic sharding in the worker.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../python/apache_beam/options/pipeline_options.py | 11 +++
.../runners/portability/fn_api_runner/execution.py | 13 +++-
.../runners/portability/fn_api_runner/fn_runner.py | 68 +++++++++++++-----
.../portability/fn_api_runner/fn_runner_test.py | 28 ++++++++
.../src/apache_beam/transforms/create.ts | 13 +++-
.../apache_beam/transforms/{create.ts => utils.ts} | 19 +++--
sdks/typescript/src/apache_beam/worker/metrics.ts | 54 +++++++++++---
.../typescript/src/apache_beam/worker/operators.ts | 81 ++++++++++++++++++++-
.../src/apache_beam/worker/pardo_context.ts | 2 +-
sdks/typescript/src/apache_beam/worker/worker.ts | 83 +++++++++++++++++++---
sdks/typescript/test/primitives_test.ts | 70 +++++++++++++++++-
11 files changed, 393 insertions(+), 49 deletions(-)
copy sdks/typescript/src/apache_beam/transforms/{create.ts => utils.ts} (74%)
[beam] 01/01: Merge pull request #23985 Support dynamic sharding in the worker.
Posted by ro...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
robertwb pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
commit 74f87b0282a23aa01ecb63288013311d207960aa
Merge: 38a85b288ad 383ea77bc98
Author: Robert Bradshaw <ro...@gmail.com>
AuthorDate: Thu Nov 10 09:10:37 2022 -0800
Merge pull request #23985 Support dynamic sharding in the worker.
.../python/apache_beam/options/pipeline_options.py | 11 +++
.../runners/portability/fn_api_runner/execution.py | 13 +++-
.../runners/portability/fn_api_runner/fn_runner.py | 68 +++++++++++++-----
.../portability/fn_api_runner/fn_runner_test.py | 28 ++++++++
.../src/apache_beam/transforms/create.ts | 13 +++-
.../apache_beam/transforms/{create.ts => utils.ts} | 19 +++--
sdks/typescript/src/apache_beam/worker/metrics.ts | 54 +++++++++++---
.../typescript/src/apache_beam/worker/operators.ts | 81 ++++++++++++++++++++-
.../src/apache_beam/worker/pardo_context.ts | 2 +-
sdks/typescript/src/apache_beam/worker/worker.ts | 83 +++++++++++++++++++---
sdks/typescript/test/primitives_test.ts | 70 +++++++++++++++++-
11 files changed, 393 insertions(+), 49 deletions(-)