You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ming li (Jira)" <ji...@apache.org> on 2022/05/27 12:52:00 UTC
[jira] [Commented] (FLINK-20681) Support specifying the hdfs path when ship archives or files
[ https://issues.apache.org/jira/browse/FLINK-20681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17542901#comment-17542901 ]
ming li commented on FLINK-20681:
---------------------------------
hi, [~RocMarshal], what is the current progress of this issue? We also want to use this feature to add some third-party packages to the job.:)
> Support specifying the hdfs path when ship archives or files
> -------------------------------------------------------------
>
> Key: FLINK-20681
> URL: https://issues.apache.org/jira/browse/FLINK-20681
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / YARN
> Affects Versions: 1.12.0
> Reporter: Ruguo Yu
> Assignee: RocMarshal
> Priority: Minor
> Labels: auto-deprioritized-major, auto-unassigned, pull-request-available, pull-requests-available, stale-assigned
> Attachments: image-2020-12-23-20-58-41-234.png, image-2020-12-24-01-01-10-021.png
>
>
> Currently, our team try to submit flink job that depends extra resource with yarn-application target, and use two options: "yarn.ship-archives" and "yarn.ship-files".
> But above options only support specifying local resource and shiping them to hdfs, besides if it can support remote resource on distributed filesystem (such as hdfs), then get the following benefits：
> * client will exclude the local resource uploading to accelerate the job submission process
> * yarn will cache them on the nodes so that they doesn't need to be downloaded for application
--
This message was sent by Atlassian Jira
(v8.20.7#820007)