You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@beam.apache.org by Fabian Strunz <fa...@techmatrix.de> on 2018/02/22 14:12:42 UTC

Get file from S3 bucket

Hello!



Is there already a how-to on how to read a file from a S3 bucket?



Can you only read public buckets or private buckets as well?







Regards

Re: Get file from S3 bucket

Posted by Ismaël Mejía <ie...@gmail.com>.

Hello,

Beam 2.3.0 introduced a native reader for S3, see the module
https://mvnrepository.com/artifact/org.apache.beam/beam-sdks-java-io-amazon-web-services/2.3.0

You should add this dependency to your project and then you can read
using the Read transform.
It supports authentication so you can read both public and private buckets.

You can find an example here
https://github.com/jbonofre/beam-samples/blob/master/amazon-web-services/src/main/java/org/apache/beam/samples/ingest/amazon/IngestToS3.java

Regards,
Ismaël


On Thu, Feb 22, 2018 at 3:12 PM, Fabian Strunz
<fa...@techmatrix.de> wrote:
> Hello!
>
>
>
> Is there already a how-to on how to read a file from a S3 bucket?
>
>
>
> Can you only read public buckets or private buckets as well?
>
>
>
>
>
>
>
> Regards
>
>