You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@nutch.apache.org by v0id null <v0...@gmail.com> on 2016/09/08 14:03:08 UTC

Segment/CrawlDB in Nutch 1.x, how is it stored?

I haven't realy been able to find this information on the wiki. How are
crawled segments stored by Nutch 1.x? Is it using HDFS?

thanks,
--alex

RE: Segment/CrawlDB in Nutch 1.x, how is it stored?

Posted by Markus Jelsma <ma...@openindex.io>.

Yes, plain Hadoop map or sequence files on local storage or HDFS.
M.

 
 
-----Original message-----
> From:v0id null <v0...@gmail.com>
> Sent: Thursday 8th September 2016 16:03
> To: user@nutch.apache.org
> Subject: Segment/CrawlDB in Nutch 1.x, how is it stored?
> 
> I haven't realy been able to find this information on the wiki. How are
> crawled segments stored by Nutch 1.x? Is it using HDFS?
> 
> thanks,
> --alex
>