[jira] [Updated] (YARN-6315) Improve LocalResourcesTrackerImpl#isResourcePresent to return false for corrupted files

["Kuhu Shukla (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YARN-6315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kuhu Shukla updated YARN-6315:
------------------------------
    Attachment: YARN-6315.005.patch

Updated patch with a revised approach to keep track of actual size of the file via downloadSize. Changes were also made to YARNRunner and LocalResourceProto for this added field. If download size is not updated and is -1 (it could be changed to a constant to indicate that the value was not set at any point), we ignore the file attribute mismatch. Would appreciate any initial comments/modifications on the approach. Thanks a lot!

> Improve LocalResourcesTrackerImpl#isResourcePresent to return false for corrupted files
> ---------------------------------------------------------------------------------------
>
>                 Key: YARN-6315
>                 URL: https://issues.apache.org/jira/browse/YARN-6315
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.7.3, 2.8.1
>            Reporter: Kuhu Shukla
>            Assignee: Kuhu Shukla
>         Attachments: YARN-6315.001.patch, YARN-6315.002.patch, YARN-6315.003.patch, YARN-6315.004.patch, YARN-6315.005.patch
>
>
> We currently check if a resource is present by making sure that the file exists locally. There can be a case where the LocalizationTracker thinks that it has the resource if the file exists but with size 0 or less than the "expected" size of the LocalResource. This JIRA tracks the change to harden the isResourcePresent call to address that case.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org