You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by ch...@apache.org on 2022/09/20 03:50:21 UTC
[beam] branch master updated (98589f259d4 -> d578e3df7c9)
This is an automated email from the ASF dual-hosted git repository.
chamikara pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
from 98589f259d4 [BEAM-13416] Introduce Schema provider for AWS models and deprecate low level coders (#16947)
add d578e3df7c9 [BEAM-14378] [CdapIO] SparkReceiverIO Read via SDF (#17828)
No new revisions were added by this update.
Summary of changes:
sdks/java/io/sparkreceiver/build.gradle | 8 +
.../ReadFromSparkReceiverWithOffsetDoFn.java | 223 +++++++++++++++++++++
.../beam/sdk/io/sparkreceiver/SparkConsumer.java} | 23 ++-
.../beam/sdk/io/sparkreceiver/SparkReceiverIO.java | 166 +++++++++++++++
.../sdk/io/sparkreceiver/WrappedSupervisor.java | 15 +-
.../io/sparkreceiver/CustomReceiverWithOffset.java | 91 +++++++++
.../sdk/io/sparkreceiver/SparkReceiverIOTest.java | 156 ++++++++++++++
7 files changed, 673 insertions(+), 9 deletions(-)
create mode 100644 sdks/java/io/sparkreceiver/src/main/java/org/apache/beam/sdk/io/sparkreceiver/ReadFromSparkReceiverWithOffsetDoFn.java
copy sdks/java/{extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/auth/CredentialFactory.java => io/sparkreceiver/src/main/java/org/apache/beam/sdk/io/sparkreceiver/SparkConsumer.java} (69%)
create mode 100644 sdks/java/io/sparkreceiver/src/main/java/org/apache/beam/sdk/io/sparkreceiver/SparkReceiverIO.java
create mode 100644 sdks/java/io/sparkreceiver/src/test/java/org/apache/beam/sdk/io/sparkreceiver/CustomReceiverWithOffset.java
create mode 100644 sdks/java/io/sparkreceiver/src/test/java/org/apache/beam/sdk/io/sparkreceiver/SparkReceiverIOTest.java