You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by "Yves S. Garret" <yo...@gmail.com> on 2013/04/24 04:01:29 UTC

Unable to crawl a series of pages in tutorial

Hello,

I'm following this tutorial.

http://wiki.apache.org/nutch/NutchTutorial#A3.2_Using_Individual_Commands_for_Whole-Web_Crawling

When I get to the following command, I'm getting this error:
$ bin/nutch inject crawl/crawldb dmoz
Unrecognized arg dmoz

I realize that this tutorial was written for Nutch 1.x, but why would I get
an error message in this case?

Re: Unable to crawl a series of pages in tutorial

Posted by "Yves S. Garret" <yo...@gmail.com>.
2.1


On Wed, Apr 24, 2013 at 3:14 PM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:

> Which version of Nutch are you using?
>
>
> On Wed, Apr 24, 2013 at 11:45 AM, Yves S. Garret <
> yoursurrogategod@gmail.com
> > wrote:
>
> > Oh, before I forget, thanks for the new script in the previous mail.
> >
> > I got to this point in the tutorial.
> >
> > bin/nutch org.apache.nutch.tools.DmozParser content.rdf.u8 -subset
> > 5000 > dmoz/urls
> >
> >
> > When I run the next command [ bin/nutch inject crawl/crawldb dmoz ], I
> get
> > this error.
> > Unrecognized arg dmoz
> > I'm running this in the ${NUTCH_HOME}/runtime/local directory.
> >
> > The dmoz directory is there, so this should not be an issue.
> >
> >
> > On Tue, Apr 23, 2013 at 10:09 PM, Lewis John Mcgibbney <
> > lewis.mcgibbney@gmail.com> wrote:
> >
> > > The DmozParser should have created a flat file similar to a bootstrap
> > file
> > > which you can inject.
> > > The flat file should be inside a the dmoz directory (if you've followed
> > the
> > > tutorial). Please make sure the file is present, and that the CLI
> syntax
> > is
> > > correct.
> > > If you are using Nutch 2.x then drop the arguments for crawl/crawldb as
> > > Nutchn does not maintain a local crawldb in 2.x. We delegate Gora to
> deal
> > > with the abstraction here.
> > > hth
> > > Lewis
> > >
> >
>
>
>
> --
> *Lewis*
>

Re: Unable to crawl a series of pages in tutorial

Posted by Lewis John Mcgibbney <le...@gmail.com>.
Which version of Nutch are you using?


On Wed, Apr 24, 2013 at 11:45 AM, Yves S. Garret <yoursurrogategod@gmail.com
> wrote:

> Oh, before I forget, thanks for the new script in the previous mail.
>
> I got to this point in the tutorial.
>
> bin/nutch org.apache.nutch.tools.DmozParser content.rdf.u8 -subset
> 5000 > dmoz/urls
>
>
> When I run the next command [ bin/nutch inject crawl/crawldb dmoz ], I get
> this error.
> Unrecognized arg dmoz
> I'm running this in the ${NUTCH_HOME}/runtime/local directory.
>
> The dmoz directory is there, so this should not be an issue.
>
>
> On Tue, Apr 23, 2013 at 10:09 PM, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com> wrote:
>
> > The DmozParser should have created a flat file similar to a bootstrap
> file
> > which you can inject.
> > The flat file should be inside a the dmoz directory (if you've followed
> the
> > tutorial). Please make sure the file is present, and that the CLI syntax
> is
> > correct.
> > If you are using Nutch 2.x then drop the arguments for crawl/crawldb as
> > Nutchn does not maintain a local crawldb in 2.x. We delegate Gora to deal
> > with the abstraction here.
> > hth
> > Lewis
> >
>



-- 
*Lewis*

Re: Unable to crawl a series of pages in tutorial

Posted by "Yves S. Garret" <yo...@gmail.com>.
Oh, before I forget, thanks for the new script in the previous mail.

I got to this point in the tutorial.

bin/nutch org.apache.nutch.tools.DmozParser content.rdf.u8 -subset
5000 > dmoz/urls


When I run the next command [ bin/nutch inject crawl/crawldb dmoz ], I get
this error.
Unrecognized arg dmoz
I'm running this in the ${NUTCH_HOME}/runtime/local directory.

The dmoz directory is there, so this should not be an issue.


On Tue, Apr 23, 2013 at 10:09 PM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:

> The DmozParser should have created a flat file similar to a bootstrap file
> which you can inject.
> The flat file should be inside a the dmoz directory (if you've followed the
> tutorial). Please make sure the file is present, and that the CLI syntax is
> correct.
> If you are using Nutch 2.x then drop the arguments for crawl/crawldb as
> Nutchn does not maintain a local crawldb in 2.x. We delegate Gora to deal
> with the abstraction here.
> hth
> Lewis
>

Re: Unable to crawl a series of pages in tutorial

Posted by Lewis John Mcgibbney <le...@gmail.com>.
Yes


On Wed, Apr 24, 2013 at 11:41 AM, Yves S. Garret <yoursurrogategod@gmail.com
> wrote:

> The dmoz directory, it should be located here, yes?
>
> ${APACHE_NUTCH_HOME}/runtime/local
>
>
> On Tue, Apr 23, 2013 at 10:09 PM, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com> wrote:
>
> > The DmozParser should have created a flat file similar to a bootstrap
> file
> > which you can inject.
> > The flat file should be inside a the dmoz directory (if you've followed
> the
> > tutorial). Please make sure the file is present, and that the CLI syntax
> is
> > correct.
> > If you are using Nutch 2.x then drop the arguments for crawl/crawldb as
> > Nutchn does not maintain a local crawldb in 2.x. We delegate Gora to deal
> > with the abstraction here.
> > hth
> > Lewis
> >
>



-- 
*Lewis*

Re: Unable to crawl a series of pages in tutorial

Posted by "Yves S. Garret" <yo...@gmail.com>.
The dmoz directory, it should be located here, yes?

${APACHE_NUTCH_HOME}/runtime/local


On Tue, Apr 23, 2013 at 10:09 PM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:

> The DmozParser should have created a flat file similar to a bootstrap file
> which you can inject.
> The flat file should be inside a the dmoz directory (if you've followed the
> tutorial). Please make sure the file is present, and that the CLI syntax is
> correct.
> If you are using Nutch 2.x then drop the arguments for crawl/crawldb as
> Nutchn does not maintain a local crawldb in 2.x. We delegate Gora to deal
> with the abstraction here.
> hth
> Lewis
>

Re: Unable to crawl a series of pages in tutorial

Posted by Lewis John Mcgibbney <le...@gmail.com>.
The DmozParser should have created a flat file similar to a bootstrap file
which you can inject.
The flat file should be inside a the dmoz directory (if you've followed the
tutorial). Please make sure the file is present, and that the CLI syntax is
correct.
If you are using Nutch 2.x then drop the arguments for crawl/crawldb as
Nutchn does not maintain a local crawldb in 2.x. We delegate Gora to deal
with the abstraction here.
hth
Lewis