You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Aitozi (Jira)" <ji...@apache.org> on 2022/06/12 09:41:00 UTC
[jira] [Commented] (FLINK-28012) Add built-in support for fetching jar from GCS (Google Cloud Storage)
[ https://issues.apache.org/jira/browse/FLINK-28012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17553249#comment-17553249 ]
Aitozi commented on FLINK-28012:
--------------------------------
I think the gcs should work with the filesystem based fetcher, have you tried by adding the flink-gs-fs-hadoop dependency ?
> Add built-in support for fetching jar from GCS (Google Cloud Storage)
> ---------------------------------------------------------------------
>
> Key: FLINK-28012
> URL: https://issues.apache.org/jira/browse/FLINK-28012
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
> Reporter: Xin Hao
> Priority: Minor
>
> I think it's easy and meaningful to add built-in support in session job mode for downloading jar from GCS now.
> The logic should look like the code below
> {code:java}
> if ("http".equals(uri.getScheme()) || "https".equals(uri.getScheme())) {
> return HttpArtifactFetcher.INSTANCE.fetch(jarURI, flinkConfiguration, targetDir);
> } else if ("gs".equals(uri.getScheme())) {
> return GcsArtifactFetcher.INSTANCE.fetch(jarURI, flinkConfiguration, targetDir);
> } else {
> return FileSystemBasedArtifactFetcher.INSTANCE.fetch(
> jarURI, flinkConfiguration, targetDir);
> } {code}
> We only need to extend the ArtifactManager and implement the new GcsArtifactFetcher.
> Also, the users can set up the GCS authorization credentials by ENV and ConfigMap so that no need for any change inner the operator.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)