You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2021/03/06 02:55:06 UTC
[beam] tag nightly-master updated (d780ebd -> 33d59b3)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to tag nightly-master
in repository https://gitbox.apache.org/repos/asf/beam.git.
*** WARNING: tag nightly-master was modified! ***
from d780ebd (commit)
to 33d59b3 (commit)
from d780ebd Remove pip version check to avoid an extra request and unactionable warning. (#14129)
add d79cd82 [BEAM-7929] Support column projection for Parquet Tables
add 2447679 Merge pull request #14117: [BEAM-7929] Support column projection for Parquet Tables
add e039ca2 Merge pull request #14136: [BEAM-11648] Add conversion utilities for BigQuery Storage API sink
add 38fbc8c Add a deadline of 30s to SplitReadStream
add adf85a1 Merge pull request #13896 from [BEAM-11326] Add a deadline of 30s to SplitReadStream
add 14e2ad4 Add topic and subscription overridden field into pubsub proto.
add d9fd515 Merge pull request #14143 from Add topic and subscription overridden field into pubsub proto.
add c47ce74 Remove experiement --enable_streaming_auto_sharding
add 4390b70 Merge pull request #14133 from [BEAM-11772, BEAM-11408] Remove Dataflow experiment --enable_streaming_auto_sharding
add ea21fad [BEAM-11748] Fix PViewToIdMapper in Samza runner to not include hashcode as part of the id (#14105)
add 73650e3 [BEAM-11924] Add Java TextIO.read() to TestExpansionService
add b9bb3a6 Merge pull request #14149 from ihji/BEAM-11924
add e75e75e Bump pandas version to 1.2.3 in python container (#14157)
add 33d59b3 Increase the timeout for ValidatesContainer suite (#14151)
No new revisions were added by this update.
Summary of changes:
...ommit_Python_ValidatesContainer_Dataflow.groovy | 2 +-
.../pipeline/src/main/proto/beam_runner_api.proto | 12 +
.../beam/runners/dataflow/DataflowRunner.java | 7 +-
.../dataflow/DataflowPipelineTranslatorTest.java | 46 +-
.../beam/runners/dataflow/DataflowRunnerTest.java | 1 -
.../runners/samza/translation/PViewToIdMapper.java | 17 +-
sdks/java/extensions/sql/build.gradle | 1 +
.../sql/meta/provider/parquet/ParquetTable.java | 132 +++++
.../provider/parquet/ParquetTableProvider.java | 22 +-
.../provider/parquet/ParquetTableProviderTest.java | 35 +-
.../io/gcp/bigquery/BeamRowToStorageApiProto.java | 336 +++++++++++
.../sdk/io/gcp/bigquery/BigQueryServicesImpl.java | 26 +-
.../beam/sdk/io/gcp/bigquery/BigQueryUtils.java | 221 +------
.../beam/sdk/io/gcp/bigquery/CivilTimeEncoder.java | 648 +++++++++++++++++++++
.../io/gcp/bigquery/TableRowToStorageApiProto.java | 304 ++++++++++
.../gcp/bigquery/BeamRowToStorageApiProtoTest.java | 391 +++++++++++++
.../sdk/io/gcp/bigquery/BigQueryUtilsTest.java | 260 ---------
.../bigquery/TableRowToStorageApiProtoTest.java | 312 ++++++++++
.../sdk/io/parquet/ParquetSchemaIOProvider.java | 127 ----
.../testing/expansion/TestExpansionService.java | 17 +
.../runners/dataflow/dataflow_runner_test.py | 43 +-
.../runners/dataflow/ptransform_overrides.py | 15 +-
sdks/python/container/base_image_requirements.txt | 2 +-
23 files changed, 2277 insertions(+), 700 deletions(-)
create mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/parquet/ParquetTable.java
create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BeamRowToStorageApiProto.java
create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/CivilTimeEncoder.java
create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableRowToStorageApiProto.java
create mode 100644 sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/BeamRowToStorageApiProtoTest.java
create mode 100644 sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/TableRowToStorageApiProtoTest.java
delete mode 100644 sdks/java/io/parquet/src/main/java/org/apache/beam/sdk/io/parquet/ParquetSchemaIOProvider.java