Re: Unexpected NULL output from the MapReduce Framework in a clustered environement

[Eric Sammer <es...@cloudera.com>]

Sam:

>From your counters, it looks like you're not outputting any records
from the mapper. My guess is that you're printing output records to
stderr or stdout and not using the output collector. Check out
http://www.cloudera.com/videos/programming_with_hadoop to learn about
the basics.

Hope that helps.

On Mon, Jun 21, 2010 at 7:21 PM, samanthula <bh...@gmail.com> wrote:
>
> Hello friends,
>                      I'new to the hadoop environment.  My application uses
> a small text file (i know that hadoop is not designed for smaller files, but
> i am just using it for testing purposes) and generates an intermediate file
> which is passed as an input to the mapper function.
>
> I ran my application on single machine and it worked pretty good. In the
> case of hadoop cluster, i exported the intermediate file(within the my java
> application) to HDFS and passed it as an argument to mapper function.
> Whenever i ran my application, it just creates an empty file in the output
> directory.
>
> The counters from my output are as follows:
>
> 10/06/20 16:15:50 INFO mapred.JobClient: Counters: 18
> 10/06/20 16:15:50 INFO mapred.JobClient:   Job Counters
> 10/06/20 16:15:50 INFO mapred.JobClient:     Launched reduce tasks=1
> 10/06/20 16:15:50 INFO mapred.JobClient:     Rack-local map tasks=9
> 10/06/20 16:15:50 INFO mapred.JobClient:     Launched map tasks=92
> 10/06/20 16:15:50 INFO mapred.JobClient:     Data-local map tasks=83
> 10/06/20 16:15:50 INFO mapred.JobClient:   FileSystemCounters
> 10/06/20 16:15:50 INFO mapred.JobClient:     FILE_BYTES_READ=6
> 10/06/20 16:15:50 INFO mapred.JobClient:     HDFS_BYTES_READ=1308951
> 10/06/20 16:15:50 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=3426
> 10/06/20 16:15:50 INFO mapred.JobClient:   Map-Reduce Framework
> 10/06/20 16:15:50 INFO mapred.JobClient:     Reduce input groups=0
> 10/06/20 16:15:50 INFO mapred.JobClient:     Combine output records=0
> 10/06/20 16:15:50 INFO mapred.JobClient:     Map input records=34
> 10/06/20 16:15:50 INFO mapred.JobClient:     Reduce shuffle bytes=534
> 10/06/20 16:15:50 INFO mapred.JobClient:     Reduce output records=0
> 10/06/20 16:15:50 INFO mapred.JobClient:     Spilled Records=0
> 10/06/20 16:15:50 INFO mapred.JobClient:     Map output bytes=0
> 10/06/20 16:15:50 INFO mapred.JobClient:     Map input bytes=664666
> 10/06/20 16:15:50 INFO mapred.JobClient:     Combine input records=0
> 10/06/20 16:15:50 INFO mapred.JobClient:     Map output records=0
> 10/06/20 16:15:50 INFO mapred.JobClient:     Reduce input records=0
>
> I checked my configuration files and the output path in the application.
> They are specified correctly. And even i ran some tutorial examples which
> are working fine in my cluster environment.
>
> Master node : namenode, JobTracker, Secondary Namenode
> DataNodes: TaskTrackers.
>
> I have included 9 data nodes in my cluster setup and one master node where
> the client's application is launched.
>
> After i ram my application, the output directory has some log files and an
> empty part-00000 file. I wonder why the output is not stored in part-00000
> file.
>
> Could any one of you point me the error here. Any suggestions are valuable
> to me.
>
> Thanks,
> Sam
> --
> View this message in context: http://old.nabble.com/Unexpected-NULL-output-from-the-MapReduce-Framework-in-a-clustered-environement-tp28953218p28953218.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>
>



-- 
Eric Sammer
twitter: esammer
data: www.cloudera.com