You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2022/11/11 05:34:11 UTC
[beam] branch nightly-refs/heads/master updated (4522f4ce916 -> 4b96193250a)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch nightly-refs/heads/master
in repository https://gitbox.apache.org/repos/asf/beam.git
from 4522f4ce916 Update datastore_wordcount.py (#23724)
add c5110bae06f Fix FhirIO javadoc format broken (#24072)
add 38a85b288ad Bump github.com/aws/aws-sdk-go-v2/service/s3 in /sdks (#24077)
add ed7b560d7ce Compute element counts for all PCollections.
add 9283512544b Add the ability to schedule splits on the ULR via a pipeline option.
add ed520243f47 Add the a Reshuffle operation and use it in Create.
add 3bd697561aa Add dynamic splitting support to the worker.
add 60fe3cdddf9 Update style sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py
add 5f587b9339b Clarifying comments.
add 7f7713de09b Merge branch 'master' into javascript-liquid-sharding
add a84b7dc179f Make mypy happy.
add 383ea77bc98 Reduce flakiness of time-based split manager test.
add 74f87b0282a Merge pull request #23985 Support dynamic sharding in the worker.
add 08b6a524fec [BEAM-12792] Install pipline dependencies to temporary venv (#16658)
add 8d585242e15 [Python]Set pickle library at the Pipeline creation stage (#24069)
add e439f4120ef Improving stale container cleanup script (#24040)
add df553d1a4c6 Add random string at the end of BigQuery query job name to make it resilient to retries (#24041)
add 36b0c8f1a76 [Playground] update snippet by persistence_key (#24056)
add 2341f61d48d [Tour Of Beam] handle CORS pre-flight requests (#24083)
add 92cef32915c Num failed inferences (#23830)
add 38742d40c89 Bump github.com/aws/aws-sdk-go-v2/config from 1.5.0 to 1.17.10 in /sdks (#24080)
add 156a6099d71 Add blog post on new ML resources (#24071)
add a4a94da1936 fixing linter error (#24104)
add b3186ba91f9 Support using BigQueryIO Storage Read API with SchemaTransforms (#23827)
add 8f8f089a5f5 Wire SamzaPipelineOptions to Exeption listener interface (#24109)
add 4b96193250a Remove TheNeuralBit from the pool of Python reviewers (#24108)
No new revisions were added by this update.
Summary of changes:
.github/REVIEWERS.yml | 1 -
.../tools/stale_dataflow_prebuilt_image_cleaner.sh | 37 ++-
CHANGES.md | 1 +
learning/tour-of-beam/backend/README.md | 30 +-
.../backend/integration_tests/client.go | 57 +++-
learning/tour-of-beam/backend/middleware.go | 32 +-
playground/api/v1/api.proto | 1 +
playground/backend/cmd/server/controller.go | 16 +-
playground/backend/cmd/server/controller_test.go | 68 ++++
playground/backend/containers/java/Dockerfile | 3 +-
playground/backend/go.mod | 33 +-
playground/backend/go.sum | 215 ++-----------
playground/backend/internal/api/v1/api.pb.go | 351 ++++++++++----------
playground/backend/internal/api/v1/api_grpc.pb.go | 4 +
.../internal/constants/datastore_constants.go | 1 +
.../backend/internal/db/datastore/datastore_db.go | 169 +++++-----
.../internal/db/datastore/datastore_db_test.go | 103 +++---
playground/backend/internal/db/entity/snippet.go | 31 +-
.../backend/internal/db/mapper/datastore_mapper.go | 22 +-
playground/index.yaml | 5 +
.../beam/runners/samza/runtime/OpAdapter.java | 15 +-
.../samza/util/SamzaPipelineExceptionListener.java | 3 +-
sdks/go.mod | 30 +-
sdks/go.sum | 48 ++-
.../sdk/io/gcp/bigquery/BigQueryQueryHelper.java | 11 +-
.../BigQueryDirectReadSchemaTransformProvider.java | 230 ++++++++++++++
.../io/gcp/bigquery/providers}/package-info.java | 4 +-
.../apache/beam/sdk/io/gcp/healthcare/FhirIO.java | 106 ++++---
...QueryDirectReadSchemaTransformProviderTest.java | 352 +++++++++++++++++++++
.../transforms/aggregation/groupby_test.py | 134 +++++---
sdks/python/apache_beam/ml/inference/base.py | 10 +-
sdks/python/apache_beam/ml/inference/base_test.py | 31 ++
.../python/apache_beam/options/pipeline_options.py | 11 +
sdks/python/apache_beam/pipeline.py | 3 +
.../runners/portability/fn_api_runner/execution.py | 13 +-
.../runners/portability/fn_api_runner/fn_runner.py | 68 +++-
.../portability/fn_api_runner/fn_runner_test.py | 28 ++
.../apache_beam/runners/worker/worker_pool_main.py | 36 ++-
sdks/python/container/Dockerfile | 2 +-
sdks/python/container/boot.go | 230 +++++++++-----
sdks/python/container/piputil.go | 34 +-
.../src/apache_beam/transforms/create.ts | 13 +-
.../apache_beam/transforms/{create.ts => utils.ts} | 19 +-
sdks/typescript/src/apache_beam/worker/metrics.ts | 54 +++-
.../typescript/src/apache_beam/worker/operators.ts | 81 ++++-
.../src/apache_beam/worker/pardo_context.ts | 2 +-
sdks/typescript/src/apache_beam/worker/worker.ts | 83 ++++-
sdks/typescript/test/primitives_test.ts | 70 +++-
website/www/site/content/en/blog/ml-resources.md | 61 ++++
.../static/images/blog/ensemble-model-notebook.png | Bin 0 -> 1858102 bytes
website/www/site/static/images/blog/ml-landing.png | Bin 0 -> 322079 bytes
51 files changed, 2124 insertions(+), 838 deletions(-)
create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/providers/BigQueryDirectReadSchemaTransformProvider.java
copy {examples/java/src/main/java/org/apache/beam/examples/complete/kafkatopubsub/options => sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/providers}/package-info.java (85%)
create mode 100644 sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/providers/BigQueryDirectReadSchemaTransformProviderTest.java
copy sdks/typescript/src/apache_beam/transforms/{create.ts => utils.ts} (74%)
create mode 100644 website/www/site/content/en/blog/ml-resources.md
create mode 100644 website/www/site/static/images/blog/ensemble-model-notebook.png
create mode 100644 website/www/site/static/images/blog/ml-landing.png