You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by bh...@apache.org on 2020/07/22 19:22:56 UTC
[beam] branch master updated (8aca8cc -> 33f14ec)
This is an automated email from the ASF dual-hosted git repository.
bhulette pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.
from 8aca8cc Merge pull request #12264: Enable spotless for all groovy files in .test-infra/jenkins directory
add 2045d93 Added SchemaIOTableProviderWrapper, used for PubsubJsonTableProvider
add 6c2aed7 Implemented SchemaIO and SchemaIOProvider for Avro
add bd915c7 Implemented SchemaIO and SchemaIOProvider for Parquet
new 33f14ec [BEAM-10407,10408] SchemaIOTableProviderWrapper and implementations for Avro and Parquet (#12202)
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../apache/beam/sdk/io/AvroSchemaIOProvider.java | 126 +++++++++++++++++
.../beam/sdk/io}/GenericRecordWriteConverter.java | 2 +-
.../beam/sdk/schemas/io/SchemaIOProvider.java | 3 +
.../provider/SchemaIOTableProviderWrapper.java | 129 +++++++++++++++++
.../sql/meta/provider/avro/AvroTable.java | 82 -----------
.../sql/meta/provider/avro/AvroTableProvider.java | 23 +--
.../sql/meta/provider/parquet/ParquetTable.java | 68 ---------
.../provider/parquet/ParquetTableProvider.java | 23 +--
.../meta/provider/pubsub/PubsubIOJsonTable.java | 154 ---------------------
.../provider/pubsub/PubsubJsonTableProvider.java | 56 ++------
...IOProvider.java => PubsubSchemaIOProvider.java} | 7 +-
.../sdk/io/gcp/pubsub/PubsubMessageToRowTest.java | 2 +-
.../io}/parquet/GenericRecordReadConverter.java | 4 +-
.../sdk/io/parquet/ParquetSchemaIOProvider.java | 112 +++++++++++++++
.../sdk/io}/parquet/GenericRecordToRowTest.java | 2 +-
15 files changed, 422 insertions(+), 371 deletions(-)
create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/io/AvroSchemaIOProvider.java
rename sdks/java/{extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/avro => core/src/main/java/org/apache/beam/sdk/io}/GenericRecordWriteConverter.java (97%)
create mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/SchemaIOTableProviderWrapper.java
delete mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/avro/AvroTable.java
delete mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/parquet/ParquetTable.java
delete mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider/pubsub/PubsubIOJsonTable.java
rename sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/{PubsubSchemaCapableIOProvider.java => PubsubSchemaIOProvider.java} (98%)
rename sdks/java/{extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/meta/provider => io/parquet/src/main/java/org/apache/beam/sdk/io}/parquet/GenericRecordReadConverter.java (95%)
create mode 100644 sdks/java/io/parquet/src/main/java/org/apache/beam/sdk/io/parquet/ParquetSchemaIOProvider.java
rename sdks/java/{extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/meta/provider => io/parquet/src/test/java/org/apache/beam/sdk/io}/parquet/GenericRecordToRowTest.java (98%)
[beam] 01/01: [BEAM-10407,
10408] SchemaIOTableProviderWrapper and implementations for Avro
and Parquet (#12202)
Posted by bh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
bhulette pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
commit 33f14ec1b315996b4569b4694a0ff5bf95626f8a
Merge: 8aca8cc bd915c7
Author: Brian Hulette <bh...@google.com>
AuthorDate: Wed Jul 22 12:22:29 2020 -0700
[BEAM-10407,10408] SchemaIOTableProviderWrapper and implementations for Avro and Parquet (#12202)
[BEAM-10407,10408] SchemaIOTableProviderWrapper and implementations for Avro and Parquet
.../apache/beam/sdk/io/AvroSchemaIOProvider.java | 126 +++++++++++++++++
.../beam/sdk/io}/GenericRecordWriteConverter.java | 2 +-
.../beam/sdk/schemas/io/SchemaIOProvider.java | 3 +
.../provider/SchemaIOTableProviderWrapper.java | 129 +++++++++++++++++
.../sql/meta/provider/avro/AvroTable.java | 82 -----------
.../sql/meta/provider/avro/AvroTableProvider.java | 23 +--
.../sql/meta/provider/parquet/ParquetTable.java | 68 ---------
.../provider/parquet/ParquetTableProvider.java | 23 +--
.../meta/provider/pubsub/PubsubIOJsonTable.java | 154 ---------------------
.../provider/pubsub/PubsubJsonTableProvider.java | 56 ++------
...IOProvider.java => PubsubSchemaIOProvider.java} | 7 +-
.../sdk/io/gcp/pubsub/PubsubMessageToRowTest.java | 2 +-
.../io}/parquet/GenericRecordReadConverter.java | 4 +-
.../sdk/io/parquet/ParquetSchemaIOProvider.java | 112 +++++++++++++++
.../sdk/io}/parquet/GenericRecordToRowTest.java | 2 +-
15 files changed, 422 insertions(+), 371 deletions(-)