Re: Program is running well in pseudo distributed mode on Hadoop-0.18.3, but it is not running in distributed mode on 4 nodes(each running Redhat linux 9)

[Ravi <ra...@gmail.com>]

Thanks a lot steve. I will go through the link.
Ravi.

On Mon, Jan 4, 2010 at 8:14 PM, Steve Loughran <st...@apache.org> wrote:

> Ravi wrote:
>
>> Hi,
>>  I have designed a mapreduce algorithm for all pairs shortest paths
>> problem. As a part of the implementation of this algorithm, I have written
>> the following mapreduce job. It is running well and producing desired output
>> in pseudo distributed mode. I have used a machine with ubuntu 8.04 and
>> hadoop-0.18.3 to run the job in pseudo distributed mode. When I tried to run
>> the same program on a cluster of 4 machines(each running Redhat linux 9)
>> with the same version of hadoop(hadooop-0.18.3), the program is not giving
>> any errors but its not giving any output as well(The output file is blank).
>> This is the first time I am facing this kind of problem.
>> I am attaching the jar file of the program and sample inputs: out1 and
>> out2 as well.(The program need to read input from these two files)
>> I have searched the archive but didn't find any mail mentioning this
>> problem. I have googled, but it was of no use.
>> I am not able to find out what am I missing in the code.
>>
>
> First, the bad news: nobody is going to debug your program for you. It's
> your program, you get to learn about distributed debugging. We all have our
> own programs and their bugs to deal with, and if its a problem with your
> physical cluster, then nobody but you are in a psition to fix.
>
> Now, the good news: the skills you learn on this simple app scale well to
> bigger clusters and more complex programs. Accordingly, it is absolutely
> essential that you do learn this process now, while your problem is still
> small.
>
> http://wiki.apache.org/hadoop/HowToDebugMapReducePrograms
>
> * Tell Hadoop to save all failed outputs with keep.failed.task.files
> * See what gets retained on the worker machines, and their logs
> * log at log4j's debug level in your code, run the nodes with your classes
> set to log at debug level (leave the rest at info, for now). Leave the
> logging settings in, they may come in useful later, just check with
> log.isDebugEnabled() before constructing strings or other work to create the
> log entries.
>
>
> > Should I be using hadoop-0.20?
> >
>
> It's not likely to magically make your problem go away, if that is what you
> were wondering.
>