You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Qi Zhu (Jira)" <ji...@apache.org> on 2021/04/28 05:55:00 UTC
[jira] [Comment Edited] (YARN-7713) Add parallel copying of
directories into FSDownload
[ https://issues.apache.org/jira/browse/YARN-7713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17334461#comment-17334461 ]
Qi Zhu edited comment on YARN-7713 at 4/28/21, 5:54 AM:
--------------------------------------------------------
[~ChrisKarampeazis]
I agree with [~ebadger] that we'd better split the list based on the file size. But how we can handle the dir size is 0?
was (Author: zhuqi):
[~ChrisKarampeazis]
I agree with [~ebadger] that we'd better split the list based on the file size, just like the DistCp.
> Add parallel copying of directories into FSDownload
> ---------------------------------------------------
>
> Key: YARN-7713
> URL: https://issues.apache.org/jira/browse/YARN-7713
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Miklos Szegedi
> Assignee: Christos Karampeazis-Papadakis
> Priority: Major
> Labels: newbie, pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> YARN currently copies directories sequentially when localizing. This could be improved to do in parallel, since the source blocks are normally on different nodes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org