You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-user@hadoop.apache.org by "S.L" <si...@gmail.com> on 2014/04/18 05:13:43 UTC

Hadoop Spill Exception

Hi All,

I am running Nutch on Hadoop2.3.0 and I get Hadoop Spill Exception even
though the disk space that is being utilized is only 30% of the available.

I am looking at the syslog file on the userlogs directory however , I do
not see much information there  except the following line. How do I know
whats causing this spill exception as disk space does not seem to be the
problem.

I see the following entry in the syslog file.

2014-04-17 12:17:28,517 INFO [FetcherThread]
org.apache.hadoop.mapred.MapTask: kvstart = 6159120(24636480); kvend =
6152164(24608656); length = 6957/6553600
2014-04-17 12:17:28,528 INFO [FetcherThread]
org.apache.hadoop.mapred.MapTask: (EQUATOR) 3661729 kvi 915428(3661712)
2014-04-17 12:17:28,529 ERROR [FetcherThread]
org.apache.nutch.fetcher.Fetcher: fetcher caught:java.io.IOException: Spill
failed

Thanks for your help.