You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Christopher Gross <co...@gmail.com> on 2011/12/16 20:56:15 UTC

updates to runbot.sh

http://wiki.apache.org/nutch/Crawl
This script no longer works.  See:echo "----- Index (Step 5 of $steps)
-----"$NUTCH_HOME/bin/nutch index crawl/NEWindexes crawl/crawldb
crawl/linkdb \   crawl/segments/*
The "index" call doesn't exist....so what does this line get
replacedwith?  Is there an updated runbot.sh script?  Has anyone
created a newone that will work?  I've done some changes on it, but I
just don'tknow what to do for this part.
Thanks!

-- Chris

Re: updates to runbot.sh

Posted by Christopher Gross <co...@gmail.com>.
I'm having the opposite problem, links that should get indexed aren't
showing up, and I'm having trouble figuring out why.  I'm trying to
use the command line options
(http://wiki.apache.org/nutch/CommandLineOptions) to dump out the data
in the segments & link db, but I'm not really seeing why items aren't
appearing in the index.

Anybody have other tools or examples or anything to help figure out
these types of problems?

-- Chris



On Fri, Dec 16, 2011 at 3:40 PM, remi tassing <ta...@gmail.com> wrote:
> Same question here, while (re)running the script, I see URLs that are
> supposed to be filtered out. Not sure how to make it right...
>
> On Friday, December 16, 2011, Christopher Gross <co...@gmail.com> wrote:
>> http://wiki.apache.org/nutch/Crawl
>> This script no longer works.  See:echo "----- Index (Step 5 of $steps)
>> -----"$NUTCH_HOME/bin/nutch index crawl/NEWindexes crawl/crawldb
>> crawl/linkdb \   crawl/segments/*
>> The "index" call doesn't exist....so what does this line get
>> replacedwith?  Is there an updated runbot.sh script?  Has anyone
>> created a newone that will work?  I've done some changes on it, but I
>> just don'tknow what to do for this part.
>> Thanks!
>>
>> -- Chris
>>

Re: updates to runbot.sh

Posted by remi tassing <ta...@gmail.com>.
Same question here, while (re)running the script, I see URLs that are
supposed to be filtered out. Not sure how to make it right...

On Friday, December 16, 2011, Christopher Gross <co...@gmail.com> wrote:
> http://wiki.apache.org/nutch/Crawl
> This script no longer works.  See:echo "----- Index (Step 5 of $steps)
> -----"$NUTCH_HOME/bin/nutch index crawl/NEWindexes crawl/crawldb
> crawl/linkdb \   crawl/segments/*
> The "index" call doesn't exist....so what does this line get
> replacedwith?  Is there an updated runbot.sh script?  Has anyone
> created a newone that will work?  I've done some changes on it, but I
> just don'tknow what to do for this part.
> Thanks!
>
> -- Chris
>