You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Konstantin Boudnik (JIRA)" <ji...@apache.org> on 2011/05/31 03:06:47 UTC
[jira] [Created] (MAPREDUCE-2547) TestDFSIO failes on a physical
cluster
TestDFSIO failes on a physical cluster
--------------------------------------
Key: MAPREDUCE-2547
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2547
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Affects Versions: 0.22.0
Environment: Physical cluster based on 0.22-SNAPSHOT
Reporter: Konstantin Boudnik
An attempt to run TestDSFIO on cluster fails because TestDFSIO tries to run MR job with local runner. If JT is explicitly specified via {{-jt}} cmd. arg. then everything is working as expected.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-2547) TestDFSIO fails on a physical
cluster
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik resolved MAPREDUCE-2547.
-------------------------------------------
Resolution: Not A Problem
> TestDFSIO fails on a physical cluster
> -------------------------------------
>
> Key: MAPREDUCE-2547
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2547
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.22.0
> Environment: Physical cluster based on 0.22-SNAPSHOT
> Reporter: Konstantin Boudnik
> Assignee: Konstantin Boudnik
> Priority: Blocker
>
> An attempt to run TestDSFIO on cluster fails because TestDFSIO tries to run MR job with local runner. If JT is explicitly specified via {{-jt}} cmd. arg. then everything is working as expected.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2547) TestDFSIO failes on a physical
cluster
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13041359#comment-13041359 ]
Konstantin Boudnik commented on MAPREDUCE-2547:
-----------------------------------------------
You'll see the following error message
{noformat}
11/05/31 01:13:57 INFO util.NativeCodeLoader: Loaded the native-hadoop library
11/05/31 01:13:57 INFO mapreduce.JobSubmitter: Cleaning up the staging area file:/user/cos1445850318/.staging/job_local_0001
ENOENT: No such file or directory
{noformat}
If the same test is executed with {{-jt jobtracker:port}} command line argument everything works fine.
It would seem like a configuration problem unless this is the only test/example with doesn't work properly. Sleep jobs, Pi, randomwriter jobs are working just fine.
> TestDFSIO failes on a physical cluster
> --------------------------------------
>
> Key: MAPREDUCE-2547
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2547
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.22.0
> Environment: Physical cluster based on 0.22-SNAPSHOT
> Reporter: Konstantin Boudnik
> Priority: Blocker
>
> An attempt to run TestDSFIO on cluster fails because TestDFSIO tries to run MR job with local runner. If JT is explicitly specified via {{-jt}} cmd. arg. then everything is working as expected.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2547) TestDFSIO failes on a physical
cluster
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated MAPREDUCE-2547:
------------------------------------------
Priority: Blocker (was: Major)
> TestDFSIO failes on a physical cluster
> --------------------------------------
>
> Key: MAPREDUCE-2547
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2547
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.22.0
> Environment: Physical cluster based on 0.22-SNAPSHOT
> Reporter: Konstantin Boudnik
> Priority: Blocker
>
> An attempt to run TestDSFIO on cluster fails because TestDFSIO tries to run MR job with local runner. If JT is explicitly specified via {{-jt}} cmd. arg. then everything is working as expected.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2547) TestDFSIO fails on a physical
cluster
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050788#comment-13050788 ]
Konstantin Boudnik commented on MAPREDUCE-2547:
-----------------------------------------------
I don't remember exact build numbers but they all were happening after 5/23 for sure.
> TestDFSIO fails on a physical cluster
> -------------------------------------
>
> Key: MAPREDUCE-2547
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2547
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.22.0
> Environment: Physical cluster based on 0.22-SNAPSHOT
> Reporter: Konstantin Boudnik
> Assignee: Konstantin Boudnik
> Priority: Blocker
>
> An attempt to run TestDSFIO on cluster fails because TestDFSIO tries to run MR job with local runner. If JT is explicitly specified via {{-jt}} cmd. arg. then everything is working as expected.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2547) TestDFSIO fails on a physical
cluster
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated MAPREDUCE-2547:
------------------------------------------
Summary: TestDFSIO fails on a physical cluster (was: TestDFSIO failes on a physical cluster)
> TestDFSIO fails on a physical cluster
> -------------------------------------
>
> Key: MAPREDUCE-2547
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2547
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.22.0
> Environment: Physical cluster based on 0.22-SNAPSHOT
> Reporter: Konstantin Boudnik
> Priority: Blocker
>
> An attempt to run TestDSFIO on cluster fails because TestDFSIO tries to run MR job with local runner. If JT is explicitly specified via {{-jt}} cmd. arg. then everything is working as expected.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2547) TestDFSIO fails on a physical
cluster
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik reassigned MAPREDUCE-2547:
---------------------------------------------
Assignee: Konstantin Boudnik
> TestDFSIO fails on a physical cluster
> -------------------------------------
>
> Key: MAPREDUCE-2547
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2547
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.22.0
> Environment: Physical cluster based on 0.22-SNAPSHOT
> Reporter: Konstantin Boudnik
> Assignee: Konstantin Boudnik
> Priority: Blocker
>
> An attempt to run TestDSFIO on cluster fails because TestDFSIO tries to run MR job with local runner. If JT is explicitly specified via {{-jt}} cmd. arg. then everything is working as expected.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2547) TestDFSIO fails on a physical
cluster
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048473#comment-13048473 ]
Todd Lipcon commented on MAPREDUCE-2547:
----------------------------------------
The deprecated config ought to work. When was this particular build from? Did it include the rather-recent fix for HADOOP-7287 (committed May 23)? If so and this still didn't work, it seems like a bug to me.
> TestDFSIO fails on a physical cluster
> -------------------------------------
>
> Key: MAPREDUCE-2547
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2547
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.22.0
> Environment: Physical cluster based on 0.22-SNAPSHOT
> Reporter: Konstantin Boudnik
> Assignee: Konstantin Boudnik
> Priority: Blocker
>
> An attempt to run TestDSFIO on cluster fails because TestDFSIO tries to run MR job with local runner. If JT is explicitly specified via {{-jt}} cmd. arg. then everything is working as expected.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2547) TestDFSIO fails on a physical
cluster
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048242#comment-13048242 ]
Konstantin Boudnik commented on MAPREDUCE-2547:
-----------------------------------------------
Turns out not a problem of the test. Rather the configuration was using an old way of configuring jobtracker address (mapred.job.tracker). Is this an intended behavior to simply ignore the old configuration parameter rather than issue a deprecation warning?
> TestDFSIO fails on a physical cluster
> -------------------------------------
>
> Key: MAPREDUCE-2547
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2547
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.22.0
> Environment: Physical cluster based on 0.22-SNAPSHOT
> Reporter: Konstantin Boudnik
> Assignee: Konstantin Boudnik
> Priority: Blocker
>
> An attempt to run TestDSFIO on cluster fails because TestDFSIO tries to run MR job with local runner. If JT is explicitly specified via {{-jt}} cmd. arg. then everything is working as expected.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2547) TestDFSIO failes on a physical
cluster
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13041358#comment-13041358 ]
Konstantin Boudnik commented on MAPREDUCE-2547:
-----------------------------------------------
Steps to reproduce:
- setup cluster from [these bits|https://builds.apache.org/job/Hadoop-22-Build/]
- apply [simple configs|https://github.com/rvs/hdeploy/tree/master/conf]
- start cluster
- run TestDFSIO:
{noformat}
$HADOOP_HOME/bin/hadoop jar $HADOOP_HOME/hadoop-mapred-test-0.22.0-SNAPSHOT.jar TestDFSIO -write -nrFiles 100 -fileSize 10MB
{noformat}
> TestDFSIO failes on a physical cluster
> --------------------------------------
>
> Key: MAPREDUCE-2547
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2547
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 0.22.0
> Environment: Physical cluster based on 0.22-SNAPSHOT
> Reporter: Konstantin Boudnik
> Priority: Blocker
>
> An attempt to run TestDSFIO on cluster fails because TestDFSIO tries to run MR job with local runner. If JT is explicitly specified via {{-jt}} cmd. arg. then everything is working as expected.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira