minimum time to kick off MapReduce job

[Scott Lindner <sc...@gmail.com>]

Hi,

I am experiencing slow startup times when submitting a MapReduce job
(single node cluster for testing purposes) and was hoping someone could
confirm that this was expected behavior.  Even a job that does absolutely
no processing takes 20-30 seconds to complete.  Is this expected?  If not,
what is the rough minimum time to kick off a newly submitted job?

Thanks,
-Scott

Re: minimum time to kick off MapReduce job

[Scott Lindner <sc...@gmail.com>]

I am really just using the default settings w/ HADOOP_HEAPSIZE=2000.  I'm
running on a 4-core node w/ 4GB ram for testing purposes.  CPU isn't being
used by anything else and there is plenty of RAM available.

I've seen similar behavior across all OS (windows, linux, os x) and just
wanted to confirm the rough times.  From everything that I've read & from
responses in this thread it appears what I'm seeing is expected.

Thanks,
-Scott

On Tue, Jan 10, 2012 at 3:46 PM, David Ginzburg <gi...@hotmail.com> wrote:

>  It is possible
> What kind of node are you using, what other  processes are running? what
> are the Xmx for NN and JT ?
>
> ------------------------------
> Date: Tue, 10 Jan 2012 10:35:43 -0500
> Subject: minimum time to kick off MapReduce job
> From: scott.a.lindner@gmail.com
> To: mapreduce-user@hadoop.apache.org
>
>
> Hi,
>
> I am experiencing slow startup times when submitting a MapReduce job
> (single node cluster for testing purposes) and was hoping someone could
> confirm that this was expected behavior.  Even a job that does absolutely
> no processing takes 20-30 seconds to complete.  Is this expected?  If not,
> what is the rough minimum time to kick off a newly submitted job?
>
> Thanks,
> -Scott
>

RE: minimum time to kick off MapReduce job

[David Ginzburg <gi...@hotmail.com>]

It is possible
What kind of node are you using, what other  processes are running? what are the Xmx for NN and JT ?

Date: Tue, 10 Jan 2012 10:35:43 -0500
Subject: minimum time to kick off MapReduce job
From: scott.a.lindner@gmail.com
To: mapreduce-user@hadoop.apache.org

Hi,
I am experiencing slow startup times when submitting a MapReduce job (single node cluster for testing purposes) and was hoping someone could confirm that this was expected behavior.  Even a job that does absolutely no processing takes 20-30 seconds to complete.  Is this expected?  If not, what is the rough minimum time to kick off a newly submitted job?

Thanks,-Scott