[jira] [Closed] (BEAM-3958) beam-sdks-java-io-amazon-web-services may be global pollution.

["Jacob Marble (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jacob Marble closed BEAM-3958.
------------------------------
    Resolution: Information Provided

> beam-sdks-java-io-amazon-web-services may be global pollution.
> --------------------------------------------------------------
>
>                 Key: BEAM-3958
>                 URL: https://issues.apache.org/jira/browse/BEAM-3958
>             Project: Beam
>          Issue Type: Bug
>          Components: io-java-aws, io-java-gcp, runner-dataflow
>    Affects Versions: 2.3.0
>         Environment: Hardware Overview:
>   Model Name:	MacBook Pro
>   Model Identifier:	MacBookPro14,3
>   Processor Name:	Intel Core i7
>   Processor Speed:	2.8 GHz
> maven-compiler-plugin: 3.6.1
>   - source: 1.8
>   - target: 1.8
> maven-shade-plugin: 3.1.0
> exec-maven-plugin: 1.5.0
> slf4j-api: 1.7.14
> slf4j-jdk14: 1.7.14
> google-cloud-dataflow-java-sdk-all: 2.3.0
> google-cloud-bigquery: 0.26.0-beta
> grpc-google-common-protos: 1.0.0
> beam-sdks-java-io-amazon-web-services: 2.3.0 and 2.4.0
>            Reporter: Hideyuki Okada
>            Assignee: Ismaël Mejía
>            Priority: Major
>             Fix For: 2.3.0
>
>
> Note:
>  I am sorry if it is difficult to read this report because I am not good at English.
> Thank you for implementing S3FileSystem.
> I tried implementing a program which performs FileIO with AWS S3 on Dataflow, and, It works.
>  But other Dataflow Pipeline which moved correctly until adding the SDK to dependencies has not working.
> Specifically, the next log will flow after program that has not working execution starts.
> _Info: The AWS S3 Beam extension was included in this build, but the awsRegion flag was not specified. If you do not plan to use S3, then ignore this message. [Date]_
> In practice, jobs that do not end on Dataflow are created. It keeps running without spilling out errors or logs.
>  And, If you pass 'awsRegion' as an argument, this will works successfully. But it is a strange workaround.
>  This means that aws sdk is requesting the connection information to a program not accessing S3. Is not it contaminated?
> As far as I've investigated, this Log seems to be spitting out in this part
>  [https://github.com/apache/beam/blob/7fa6292a21564744011fe94a7e50f7e074564b71/sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/s3/S3FileSystem.java#L108-L112]
> It must pass the region as an argument?
> I want you to tell me if I'm wrong. And If it is contaminated, I hope this problem will be fixed.
> The version of sdks that I tried.
> google-cloud-dataflow-java-sdk-all: 2.3.0
>  beam-sdks-java-io-amazon-web-services: 2.3.0 and 2.4.0
> Thank you for reading.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)