[jira] [Updated] (BEAM-12739) Flink JobServer should bundle AWS IO libraries to support staging on S3

["Luke Cwik (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-12739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Luke Cwik updated BEAM-12739:
-----------------------------
    Fix Version/s: 2.33.0
         Assignee: Jeremy Lewi
       Resolution: Fixed
           Status: Resolved  (was: Triage Needed)

> Flink JobServer should bundle AWS IO libraries to support staging on S3
> -----------------------------------------------------------------------
>
>                 Key: BEAM-12739
>                 URL: https://issues.apache.org/jira/browse/BEAM-12739
>             Project: Beam
>          Issue Type: Improvement
>          Components: jobserver
>    Affects Versions: 2.31.0
>            Reporter: Jeremy Lewi
>            Assignee: Jeremy Lewi
>            Priority: P2
>             Fix For: 2.33.0
>
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> It doesn't look like the AWS Filesystem libraries are included in the Flink JobServer.
> There is a TODO in the code to update the dependencies to support the AWS filesystem.
> [https://github.com/apache/beam/blob/master/runners/flink/job-server/flink_job_server.gradle#L89]
> As reported in this [mailing list thread|https://lists.apache.org/x/thread.html/rec91c120cb5ead637d434ca99ae98e5119721de0789f0b8d814f1306@%3Cuser.beam.apache.org%3E] trying to use S3 to stage artifacts results in the exception
> {code:java}
> Aug 06, 2021 1:52:51 AM
> org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$2
> finishStaging
> SEVERE: Error staging artifacts
> java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: No filesystem found for scheme s3
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>  {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)