You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by chethan <ch...@gmail.com> on 2012/08/29 08:19:58 UTC

Crawl fails when run as a background process

Hi,

When I run the crawl as a background process on Ubuntu in local mode, the
Fetcher aborts with hung threads. The message is something like:

*WARN  fetcher.Fetcher - Aborting with "X" hung threads.*
*
*
I start off the script using nohup and & as I want to log off from the
session and have the crawler still run on the server. Else, when the crawl
finishes at a certain depth and when the crawldb is being updated, the SSH
session times out. I've tried configuring "keep alive" messages without
much help. The command is something like:

*nohup ./bin/nutch crawl* ....... *&*
*
*
Has anybody experienced this before? It seems to happen only when I use
nohup or &.

Thanks
Chethan
*
*
*
*