You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Hongyuan Li (JIRA)" <ji...@apache.org> on 2017/07/07 14:45:00 UTC
[jira] [Updated] (HADOOP-14631) Distcp should add a default
AtomicWorkPath properties when using atomic
[ https://issues.apache.org/jira/browse/HADOOP-14631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hongyuan Li updated HADOOP-14631:
---------------------------------
Description:
Distcp should add a default AtomicWorkPath properties when using atomic
{{Distcp}}#{{configureOutputFormat}} using code below to generate atomic work path
{code}
if (context.shouldAtomicCommit()) {
Path workDir = context.getAtomicWorkPath();
if (workDir == null) {
workDir = targetPath.getParent();
}
workDir = new Path(workDir, WIP_PREFIX + targetPath.getName()
+ rand.nextInt());
[code}
When atomic is set and {{AtomicWorkPath}} == null, distcp will get the parent of current {{WorkDir}}. In this case, if {{workdir}} is {{"/"}}, the parent will be {{null}}, wich means
{{workDir = new Path(workDir, WIP_PREFIX + targetPath.getName() + rand.nextInt());}} will throw a nullpoint exception.
was:
Distcp should add a default AtomicWorkPath properties when using atomic
{{Distcp}}#{{configureOutputFormat}} using code below to generate atomic work path
{code}
if (context.shouldAtomicCommit()) {
Path workDir = context.getAtomicWorkPath();
if (workDir == null) {
workDir = targetPath.getParent();
}
workDir = new Path(workDir, WIP_PREFIX + targetPath.getName()
+ rand.nextInt());
[code}
When atomic is set and tAtomicWorkPath == null, distcp will get the parent of current WorkDir. In this case, if {{workdir}} is {{"/"}}, the parent will be {{null}}, wich means
{{ workDir = new Path(workDir, WIP_PREFIX + targetPath.getName() + rand.nextInt());}} will throw a nullpoint exception.
> Distcp should add a default AtomicWorkPath properties when using atomic
> ------------------------------------------------------------------------
>
> Key: HADOOP-14631
> URL: https://issues.apache.org/jira/browse/HADOOP-14631
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Hongyuan Li
>
> Distcp should add a default AtomicWorkPath properties when using atomic
> {{Distcp}}#{{configureOutputFormat}} using code below to generate atomic work path
> {code}
> if (context.shouldAtomicCommit()) {
> Path workDir = context.getAtomicWorkPath();
> if (workDir == null) {
> workDir = targetPath.getParent();
> }
> workDir = new Path(workDir, WIP_PREFIX + targetPath.getName()
> + rand.nextInt());
> [code}
> When atomic is set and {{AtomicWorkPath}} == null, distcp will get the parent of current {{WorkDir}}. In this case, if {{workdir}} is {{"/"}}, the parent will be {{null}}, wich means
> {{workDir = new Path(workDir, WIP_PREFIX + targetPath.getName() + rand.nextInt());}} will throw a nullpoint exception.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org