TableOutputFormat

[Billy Pearson <sa...@pearsonwholesale.com>]

I am running jobs and I am noticing that the % done on the reduce jobs is 
hitting 100% way before the job is done writing to the table

I run a map and reduce and use setOutputFormat(TableOutputFormat.class) to 
write the reducer's BatchUpdates to the table I need
but on say 10M rows to update the reduce show 100% done at say a few mins 
but do not complete for 30+ mins.

Any idea on why this is happening now it was not acting like this before in 
0.2?
Is there a new buffer holding the reduce output or something like that?

Billy

Re: TableOutputFormat

[Billy Pearson <sa...@pearsonwholesale.com>]

Appears to be a problem on the reduce task reporting in higher then 100% 
done
I see task reporting as high as 377% done when viewing running task.

Billy

"Billy Pearson" <sa...@pearsonwholesale.com> 
wrote in message news:gcj7nq$ubd$1@ger.gmane.org...
>I am running jobs and I am noticing that the % done on the reduce jobs is 
>hitting 100% way before the job is done writing to the table
>
> I run a map and reduce and use setOutputFormat(TableOutputFormat.class) to 
> write the reducer's BatchUpdates to the table I need
> but on say 10M rows to update the reduce show 100% done at say a few mins 
> but do not complete for 30+ mins.
>
> Any idea on why this is happening now it was not acting like this before 
> in 0.2?
> Is there a new buffer holding the reduce output or something like that?
>
> Billy
>
>
> 

Re: TableOutputFormat

[Ryan Smith <ry...@gmail.com>]

+1 ive seen this too.

On Wed, Oct 8, 2008 at 5:12 PM, Billy Pearson <sa...@pearsonwholesale.com>wrote:

> I am running jobs and I am noticing that the % done on the reduce jobs is
> hitting 100% way before the job is done writing to the table
>
> I run a map and reduce and use setOutputFormat(TableOutputFormat.class) to
> write the reducer's BatchUpdates to the table I need
> but on say 10M rows to update the reduce show 100% done at say a few mins
> but do not complete for 30+ mins.
>
> Any idea on why this is happening now it was not acting like this before in
> 0.2?
> Is there a new buffer holding the reduce output or something like that?
>
> Billy
>
>
>