map error (MultipleOutputs and firstbadlink related )

[Yair Even-Zohar <ya...@audiencescience.com>]

I'm running just the map part of a MR and I'm writing the output of that
map using MultipleOutputs.  That is, each job writes to X number of
files via MultipleOutputs.

 

The default is 2 map jobs and it works well for about X=30.

Now, when I increase the number of output files per job (from 30 to 60 )
I get IOExeption with "firstbadlink" and hbase/hadoop claim it has
problems reading data.
Not surprisingly, the same behavior happens if I force a split of the
data to 4 etc.
 
I have upped the "dfs.datanode.max.xcievers" to 1024 but that made no
change.
Any ideas?

 

Thanks

-Yair

 

Re: map error (MultipleOutputs and firstbadlink related )

[jason hadoop <ja...@gmail.com>]

The usual cause in this case is a shortage of file descriptors.
Either through application leakage or just to few available to the server
processes such as the Datanode and the Tasktracker.
Try upping the limit to something huge like 500,000
Set  the hard and soft limit in /etc/security/limits.conf on a linux
machine.
search this forum for /etc/security/limits.conf to get more detailed
instructions.



On Fri, Mar 6, 2009 at 11:42 AM, Yair Even-Zohar
<ya...@audiencescience.com>wrote:

> I'm running just the map part of a MR and I'm writing the output of that
> map using MultipleOutputs.  That is, each job writes to X number of
> files via MultipleOutputs.
>
>
>
> The default is 2 map jobs and it works well for about X=30.
>
> Now, when I increase the number of output files per job (from 30 to 60 )
> I get IOExeption with "firstbadlink" and hbase/hadoop claim it has
> problems reading data.
> Not surprisingly, the same behavior happens if I force a split of the
> data to 4 etc.
>
> I have upped the "dfs.datanode.max.xcievers" to 1024 but that made no
> change.
> Any ideas?
>
>
>
> Thanks
>
> -Yair
>
>
>
>


-- 
Alpha Chapters of my book on Hadoop are available
http://www.apress.com/book/view/9781430219422