You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by "Konstantin Boudnik (JIRA)" <ji...@apache.org> on 2014/05/28 02:24:03 UTC
[jira] [Commented] (BIGTOP-764) Bigtop mapreduce job fails in
ubuntu precise - Invalid shuffle port number
[ https://issues.apache.org/jira/browse/BIGTOP-764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14010568#comment-14010568 ]
Konstantin Boudnik commented on BIGTOP-764:
-------------------------------------------
[~mackrorysd], is this still an issue in 0.8.0 ?
> Bigtop mapreduce job fails in ubuntu precise - Invalid shuffle port number
> --------------------------------------------------------------------------
>
> Key: BIGTOP-764
> URL: https://issues.apache.org/jira/browse/BIGTOP-764
> Project: Bigtop
> Issue Type: Bug
> Components: Deployment, VM
> Affects Versions: 0.4.0, 0.6.0
> Environment: Ubuntu Precise 12.04 LTS 64 bit running on VMWare Fusion
> Reporter: P Mohan
> Assignee: Sean Mackrory
> Priority: Minor
> Labels: newbie, patch
>
> i installed bigtop 0.4.0 following the instructions in the wiki. Installation was fine. i ran into "Invalid shuffle port number" when i tried to run the pi calculation example. I did a initial check on the internet and found - https://issues.apache.org/jira/browse/MAPREDUCE-2983 . but this did not help.
> did anyone face this error before ?
> hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples*.jar pi 10 1000
> error message:
> 12/10/21 19:26:03 INFO mapreduce.Job: map 0% reduce 0%
> 12/10/21 19:26:04 INFO mapreduce.Job: Task Id : attempt_1350872346733_0001_m_000004_0, Status : FAILED
> Container launch failed for container_1350872346733_0001_01_000006 : java.lang.IllegalStateException: Invalid shuffle port number -1 returned for attempt_1350872346733_0001_m_000004_0
> at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$Container.launch(ContainerLauncherImpl.java:162)
> at org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl$EventProcessor.run(ContainerLauncherImpl.java:373)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> 12/10/21 19:26:04 WARN mapreduce.Job: Error reading task output Connection refused
> I am attaching the installation instructions i followed below. Any help would be highly appreciated.
> installation instructions
> #Environment : Ubuntu 12.04 LTS 64 bit running on VMWare fusion
> #Install JAVA
> sudo add-apt-repository ppa:webupd8team/java
> sudo apt-get update
> sudo apt-get install oracle-java7-installer
> #Set Java Home
> JAVA_HOME=/usr/lib/jvm/java-7-oracle/
> export JAVA_HOME
> # add big top repos
> wget -O- http://www.apache.org/dist/incubator/bigtop/bigtop-0.4.0-incubating/repos/GPG-KEY-bigtop | sudo apt-key add -
> sudo wget -O /etc/apt/sources.list.d/bigtop.list http://bigtop01.cloudera.org:8080/view/Releases/job/Bigtop-0.4.0/label=precise/lastSuccessfulBuild/artifact/output/bigtop.list
> sudo apt-get update
> apt-cache search hadoop
> #install hadoop
> sudo apt-get install hadoop\* flume-* mahout\* oozie\* whirr-*
> #initialize namenode - format
> sudo /etc/init.d/hadoop-hdfs-namenode init
> for i in hadoop-hdfs-namenode hadoop-hdfs-datanode ; do sudo service $i start ; done
> #setup folders necessary for hadoop operarions
> sudo -u hdfs hadoop fs -mkdir /user/$USER
> sudo -u hdfs hadoop fs -chown $USER:$USER /user/$USER
> sudo -u hdfs hadoop fs -mkdir /tmp
> sudo -u hdfs hadoop fs -chmod -R 1777 /tmp
> sudo -u hdfs hadoop fs -mkdir /var/log/hadoop-yarn
> sudo -u hdfs hadoop fs -chown yarn:mapred /var/log/hadoop-yarn
> sudo -u hdfs hadoop fs -mkdir /user/history
> sudo -u hdfs hadoop fs -chown mapred:mapred /user/history
> sudo -u hdfs hadoop fs -chmod 770 /user/history
> sudo -u hdfs hadoop fs -mkdir /tmp/hadoop-yarn/staging
> sudo -u hdfs hadoop fs -chmod -R 1777 /tmp/hadoop-yarn/staging
> sudo -u hdfs hadoop fs -mkdir /tmp/hadoop-yarn/staging/history/done_intermediate
> sudo -u hdfs hadoop fs -chmod -R 1777 /tmp/hadoop-yarn/staging/history/done_intermediate
> sudo -u hdfs hadoop fs -chown -R mapred:mapred /tmp/hadoop-yarn/staging
> sudo service hadoop-yarn-resourcemanager start
> sudo service hadoop-yarn-nodemanager start
> #test
> hadoop fs -lsr /
--
This message was sent by Atlassian JIRA
(v6.2#6252)