You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Eric Yang (JIRA)" <ji...@apache.org> on 2018/12/14 18:28:00 UTC
[jira] [Commented] (YARN-9126) Container reinit always fails in
branch-3.2 and trunk
[ https://issues.apache.org/jira/browse/YARN-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721666#comment-16721666 ]
Eric Yang commented on YARN-9126:
---------------------------------
We have a couple options here:
Option 1 - Force file copy to overwrite existing files in container-executor.
Option 2 - Delete the entire working directory, and allow the reinit process to reconstruct the whole thing.
Option 3 - Delete the files that will be localized prior to launching container-executor. The only problem is the files are owned by application runner. This approach will require calling container-executor to delete the targeted list of files, then reinit again during relaunch. It's a more expensive process than option 1. The only rationale to go with this solution is to delete files as application runner to protect against overwriting other people's files.
> Container reinit always fails in branch-3.2 and trunk
> -----------------------------------------------------
>
> Key: YARN-9126
> URL: https://issues.apache.org/jira/browse/YARN-9126
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Eric Yang
> Priority: Major
> Labels: docker
>
> When upgrading container, container reinitialization always failed with code 33. This error code means the localizing file already exist while copying resource files. The container will retry with another container ID, hence the problem is masked.
> Hadoop 3.1.x relaunch logic seem to have some way to prevent this bug from happening. The same logic might be useful in branch 3.2 and trunk.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org