You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Sean Bigdatafun <se...@gmail.com> on 2012/07/13 04:42:33 UTC

Reducer hangs in a pseudo-distributed environment

I ran the example to test if my cluster was installed correctly and I found
the the reducer hanged there. It looks like the network connection timed
out. After taking a look at the log, I thought maybe I did not configure my
DNS well, but I found the DNS resolving is correct. Any idea and suggestion
to further check?

#host  cdh-pseudo.myhadoop.com
10.20.20.141




2012-07-12 19:23:48,984 INFO org.apache.hadoop.mapred.ReduceTask:
attempt_201207121742_0003_r_000000_0 Scheduled 0 outputs (0 slow hosts and1
dup hosts)
2012-07-12 19:23:53,014 WARN org.apache.hadoop.mapred.ReduceTask:
attempt_201207121742_0003_r_000000_0 copy failed:
attempt_201207121742_0003_m_000001_0 from cdh-pseudo.myhadoop.com
2012-07-12 19:23:53,015 WARN org.apache.hadoop.mapred.ReduceTask:
java.net.SocketTimeoutException: connect timed out
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
        at
java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
        at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
        at java.net.Socket.connect(Socket.java:529)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:158)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)
        at sun.net.www.http.HttpClient.<init>(HttpClient.java:233)
        at sun.net.www.http.HttpClient.New(HttpClient.java:306)
        at sun.net.www.http.HttpClient.New(HttpClient.java:323)
        at
sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:970)
        at
sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:911)
        at
sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:836)
        at
org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.getInputStream(ReduceTask.java:1545)
        at
org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.setupSecureConnection(ReduceTask.java:1502)
        at
org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.getMapOutput(ReduceTask.java:1410)
        at
org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.copyOutput(ReduceTask.java:1321)
        at
org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.run(ReduceTask.java:1253)