You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Íñigo Goiri (JIRA)" <ji...@apache.org> on 2018/07/03 01:50:00 UTC
[jira] [Commented] (HADOOP-15578) GridmixTestUtils uses the wrong
staging directory in windows
[ https://issues.apache.org/jira/browse/HADOOP-15578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16530672#comment-16530672 ]
Íñigo Goiri commented on HADOOP-15578:
--------------------------------------
The error would be:
{code}
Exception in thread "StressJobFactory" java.lang.IllegalArgumentException: Pathname /D:/hadoop-2.9/hadoop-tools/hadoop-gridmix/target/tmp/mapred/staging from D:/hadoop-2.9/hadoop-tools/hadoop-gridmix/target/tmp/mapred/staging is not a valid DFS filename.
at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:221)
at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1248)
at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1245)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:1262)
at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:1237)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:2216)
at org.apache.hadoop.mapred.gridmix.GridmixTestUtils.createHomeAndStagingDirectory(GridmixTestUtils.java:115)
at org.apache.hadoop.mapred.gridmix.DebugJobProducer$MockJob.getUser(DebugJobProducer.java:217)
at org.apache.hadoop.mapred.gridmix.JobFactory$FilterJobStory.getUser(JobFactory.java:147)
at org.apache.hadoop.mapred.gridmix.StressJobFactory$StressReaderThread.run(StressJobFactory.java:210)
{code}
[~surmountian], do you see any other way to fix this issue?/
> GridmixTestUtils uses the wrong staging directory in windows
> ------------------------------------------------------------
>
> Key: HADOOP-15578
> URL: https://issues.apache.org/jira/browse/HADOOP-15578
> Project: Hadoop Common
> Issue Type: Test
> Reporter: Íñigo Goiri
> Priority: Major
> Attachments: HADOOP-15578.000.patch
>
>
> {{GridmixTestUtils#createHomeAndStagingDirectory}} gets the staging area from the configuration key {{mapreduce.jobtracker.staging.root.dir}}. This variable depends on {{hadoop.tmp.dir}} which in Windows is set to a local Windows folder. When the test tries to create the path in HDFS it gets an error because the path is not compliant.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org