You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Esun Kim (Jira)" <ji...@apache.org> on 2019/12/04 23:52:00 UTC

[jira] [Created] (BEAM-8889) Make GcsUtil use GoogleCloudStorage

Esun Kim created BEAM-8889:
------------------------------

             Summary: Make GcsUtil use GoogleCloudStorage
                 Key: BEAM-8889
                 URL: https://issues.apache.org/jira/browse/BEAM-8889
             Project: Beam
          Issue Type: Improvement
          Components: io-java-gcp
    Affects Versions: 2.16.0
            Reporter: Esun Kim


[GcsUtil|https://github.com/apache/beam/blob/master/sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/util/GcsUtil.java] is a primary class to access Google Cloud Storage on Apache Beam. Current implementation directly creates GoogleCloudStorageReadChannel and GoogleCloudStorageWriteChannel by itself to read and write GCS data rather than using [GoogleCloudStorage|https://github.com/GoogleCloudPlatform/bigdata-interop/blob/master/gcsio/src/main/java/com/google/cloud/hadoop/gcsio/GoogleCloudStorage.java] which is an abstract class providing basic IO capability which eventually creates channel objects. This request is about updating GcsUtil to use GoogleCloudStorage to create read and write channel, which is expected flexible because it can easily pick up the new change; e.g. new channel implementation using new protocol without code change.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)