You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2020/11/05 02:57:55 UTC
[beam] tag nightly-master updated (c737128 -> 1fe8791)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to tag nightly-master
in repository https://gitbox.apache.org/repos/asf/beam.git.
*** WARNING: tag nightly-master was modified! ***
from c737128 (commit)
to 1fe8791 (commit)
from c737128 [BEAM-10409] Remap all PCollections in KeyWithNone elimination (#13204)
add fd4d92a [BEAM-10124] Compute number of records before each offset using a Multimap sideinput keyed with a filename. (#13198)
add bdeb4af [BEAM-10869] Remove unused PubSubSink with_attributes property (#13254)
add eeb1497 [BEAM-11091] Allow to specify coder for HadoopFormatIO.Read (#13166)
add d90f926 [BEAM-11162] Fetch missing projectId from options (#13234)
add a2f22df Fix GroupIntoBathces.test_buffering_timer_in_fixed_window_streaming
add d535e6d Merge pull request #13253 from nehsyc/gib_timeout_py
add eb58694 Adding display data to BQ File Loads transform
add 7dcbdf1 Merge pull request #13249 from Adding display data to BQ File Loads transform
add dfbeeca [BEAM-3736] Add CombineFn.setup and CombineFn.teardown to Python SDK (#13048)
add c3407f3 [BEAM-11190] Fix grouping on categorical columns (#13256)
add 70ee750 [BEAM-2717] Implement ProtoCoder.to_type_hint
add 1fe8791 Merge pull request #13226 [BEAM-2717] Implement ProtoCoder.to_type_hint
No new revisions were added by this update.
Summary of changes:
CHANGES.md | 1 +
.../pipeline/src/main/proto/beam_runner_api.proto | 4 -
.../sdk/io/contextualtextio/ContextualTextIO.java | 51 +++++---
.../io/contextualtextio/RecordWithMetadata.java | 2 +-
.../gcp/bigquery/DynamicDestinationsHelpers.java | 3 +
.../sdk/io/gcp/testing/FakeDatasetService.java | 3 +
.../sdk/io/gcp/bigquery/BigQueryIOWriteTest.java | 15 ++-
.../beam/sdk/io/hadoop/format/HadoopFormatIO.java | 34 +++++-
.../io/hadoop/format/HadoopFormatIOReadTest.java | 104 ++++++++++++++++
sdks/python/apache_beam/coders/coders.py | 3 +
sdks/python/apache_beam/coders/coders_test.py | 1 +
sdks/python/apache_beam/dataframe/frames.py | 27 ++++-
sdks/python/apache_beam/dataframe/frames_test.py | 11 ++
.../apache_beam/io/gcp/bigquery_file_loads.py | 7 +-
sdks/python/apache_beam/io/gcp/pubsub.py | 65 +++++-----
sdks/python/apache_beam/io/gcp/pubsub_test.py | 5 -
sdks/python/apache_beam/io/iobase.py | 4 +-
.../runners/dataflow/dataflow_runner.py | 40 ++++++-
.../runners/dataflow/dataflow_runner_test.py | 19 +++
.../apache_beam/runners/direct/direct_userstate.py | 16 ++-
.../runners/direct/helper_transforms.py | 12 ++
.../runners/direct/transform_evaluator.py | 1 +
.../apache_beam/runners/worker/bundle_processor.py | 13 +-
.../apache_beam/runners/worker/operations.py | 30 +++++
.../transforms/combinefn_lifecycle_pipeline.py | 131 +++++++++++++++++++++
.../transforms/combinefn_lifecycle_test.py | 98 +++++++++++++++
sdks/python/apache_beam/transforms/combiners.py | 52 ++++++--
sdks/python/apache_beam/transforms/core.py | 52 ++++++--
sdks/python/apache_beam/transforms/trigger.py | 2 +
sdks/python/apache_beam/transforms/userstate.py | 4 +
sdks/python/apache_beam/transforms/util_test.py | 19 +--
sdks/python/apache_beam/typehints/typecheck.py | 6 +
32 files changed, 725 insertions(+), 110 deletions(-)
create mode 100644 sdks/python/apache_beam/transforms/combinefn_lifecycle_pipeline.py
create mode 100644 sdks/python/apache_beam/transforms/combinefn_lifecycle_test.py