You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2021/12/14 03:34:28 UTC
[beam] branch nightly-refs/heads/master updated (6e2d39f -> d41ae47)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch nightly-refs/heads/master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from 6e2d39f [BEAM-13015] Plumb the cache through contexts and transform executors. (#16188)
add 2547dbe [BEAM-13441] Use quiet delete for S3 batch deletes. In quiet mode only errors will be returned in the service response.
add 3985d8d Merge pull request #16197: [BEAM-13441] Use S3 quiet mode for deletes
add cc11936 Merge pull request #16140 from [BEAM-13377][Playground] Update CI/CD python scripts to work with specific SDK
add 0257792 Merge pull request #16120 from [BEAM-13333][Playground] Save Python logs to the cache
add e183c14 Merge pull request #16185 from [BEAM-13425][Playground][Bugfix] Support logs while storing to bucket
add b225f60 change context type
add 79d6418 Merge pull request #16211 from [BEAM-13207][Playground] Change context type
add 97a2acc Merge pull request #16121 from [BEAM-13334][Playground] Save Go logs to the cache
add 290fc87 [BEAM-13423]: fix frontend failure if no examples
add ad28b00 Merge pull request #16193 from [BEAM-13423] [Bugiix] [Playground]: fix frontend failure if no examples
add 9a68b80 Docs for validators tests
add bf77781 Merge pull request #16206 from [BEAM-13449][Playground] Docs for validators tests
add 22e761c change return type of 2 methods
add ba35c86 Merge pull request #16196 from [BEAM-13408][Playground] Change return type of 2 methods
add d7e1a50 Merge pull request #16179 from [BEAM-13344][Playground] support python unit tests
add 3e5630f Merge pull request #16204 from [BEAM-13444][Playground] Add processing of an error
add 626e7a9 Merge pull request #16208 from [BEAM-13442][Playground] Filepath to log error
add f969a7b [BEAM-13445] Correctly set data limit when flushing S3 upload buffer and avoid temporary copy to array when writing source buffer to upload buffer
add e2ac013 Merge pull request #16205: [BEAM-13445] Correctly set data limit when flushing S3 upload buffer
add cf35f72 [BEAM-13276] bump jackson-core to 2.13.0 for .test-infra (#16062)
add 512ce86 Change Pub/Sub Lite PollResult to set explicit watermark (#16216)
add 3cfec5b Updating Grafana from v8.1.2 to v8.1.6
add 37d1d99 Merge pull request #16201 from apache/aaltay-patch-1
add e056d73 Include name of missing tag in error message.
add 25e6008 Merge pull request #16175 from ibzib/tag-name
add 98c5266 Better type hints for Count combiners.
add 0111905 Merge pull request #16145 Better type hints for Count combiners.
add da11b84 [BEAM-13454] Fix and test dataframe read_fwf. (#16064)
add d41ae47 [BEAM-12976] Pipeline visitor to discover pushdown opportunities. (#16176)
No new revisions were added by this update.
Summary of changes:
.test-infra/metrics/grafana/Dockerfile | 2 +-
.test-infra/validate-runner/build.gradle | 4 +-
playground/backend/cmd/server/controller.go | 7 +-
playground/backend/configs/SDK_PYTHON.json | 4 +-
.../internal/cloud_bucket/precompiled_objects.go | 14 +-
.../internal/code_processing/code_processing.go | 113 ++++++---
.../internal/preparators/python_preparators.go | 77 +++++-
.../preparators/python_preparators_test.go | 101 ++++++++
.../internal/setup_tools/builder/setup_builder.go | 2 +-
.../setup_tools/builder/setup_builder_test.go | 3 +
.../internal/streaming/run_output_writer.go | 3 +
playground/backend/internal/utils/cache_utils.go | 2 +-
.../backend/internal/utils/validators_utils.go | 2 +-
.../internal/validators/python_validators.go | 34 ++-
.../backend/internal/validators/validator_test.go | 3 +
.../components/playground_page_providers.dart | 3 +
playground/frontend/pubspec.lock | 10 +-
playground/infrastructure/api/v1/api_pb2_grpc.py | 2 +-
playground/infrastructure/cd_helper.py | 15 +-
playground/infrastructure/ci_cd.py | 67 +++--
playground/infrastructure/ci_helper.py | 18 +-
playground/infrastructure/config.py | 12 +-
playground/infrastructure/grpc_client.py | 15 ++
playground/infrastructure/helper.py | 68 +++--
playground/infrastructure/requirements.txt | 1 +
playground/infrastructure/test_cd_helper.py | 35 +--
playground/infrastructure/test_ci_cd.py | 42 +---
playground/infrastructure/test_helper.py | 129 +++++-----
.../graph/ProjectionProducerVisitor.java | 83 +++++++
.../graph/ProjectionProducerVisitorTest.java | 274 +++++++++++++++++++++
.../apache/beam/sdk/values/PCollectionTuple.java | 3 +-
.../apache/beam/sdk/io/aws/s3/S3FileSystem.java | 3 +-
.../beam/sdk/io/aws/s3/S3WritableByteChannel.java | 25 +-
.../sdk/io/aws/s3/S3WritableByteChannelTest.java | 39 +--
.../apache/beam/sdk/io/aws2/s3/S3FileSystem.java | 2 +-
.../beam/sdk/io/aws2/s3/S3WritableByteChannel.java | 25 +-
.../apache/beam/sdk/io/aws2/s3/S3FileSystemIT.java | 18 +-
.../sdk/io/aws2/s3/S3WritableByteChannelTest.java | 17 +-
.../internal/SubscriptionPartitionLoader.java | 3 +-
sdks/python/apache_beam/dataframe/io.py | 47 +++-
sdks/python/apache_beam/dataframe/io_test.py | 29 ++-
sdks/python/apache_beam/transforms/combiners.py | 6 +
.../apache_beam/transforms/ptransform_test.py | 10 +-
43 files changed, 1046 insertions(+), 326 deletions(-)
create mode 100644 playground/backend/internal/preparators/python_preparators_test.go
create mode 100644 runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/graph/ProjectionProducerVisitor.java
create mode 100644 runners/core-construction-java/src/test/java/org/apache/beam/runners/core/construction/graph/ProjectionProducerVisitorTest.java