You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by "Yves S. Garret" <yo...@gmail.com> on 2013/04/24 04:01:29 UTC
Unable to crawl a series of pages in tutorial
Hello,
I'm following this tutorial.
http://wiki.apache.org/nutch/NutchTutorial#A3.2_Using_Individual_Commands_for_Whole-Web_Crawling
When I get to the following command, I'm getting this error:
$ bin/nutch inject crawl/crawldb dmoz
Unrecognized arg dmoz
I realize that this tutorial was written for Nutch 1.x, but why would I get
an error message in this case?
Re: Unable to crawl a series of pages in tutorial
Posted by "Yves S. Garret" <yo...@gmail.com>.
2.1
On Wed, Apr 24, 2013 at 3:14 PM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:
> Which version of Nutch are you using?
>
>
> On Wed, Apr 24, 2013 at 11:45 AM, Yves S. Garret <
> yoursurrogategod@gmail.com
> > wrote:
>
> > Oh, before I forget, thanks for the new script in the previous mail.
> >
> > I got to this point in the tutorial.
> >
> > bin/nutch org.apache.nutch.tools.DmozParser content.rdf.u8 -subset
> > 5000 > dmoz/urls
> >
> >
> > When I run the next command [ bin/nutch inject crawl/crawldb dmoz ], I
> get
> > this error.
> > Unrecognized arg dmoz
> > I'm running this in the ${NUTCH_HOME}/runtime/local directory.
> >
> > The dmoz directory is there, so this should not be an issue.
> >
> >
> > On Tue, Apr 23, 2013 at 10:09 PM, Lewis John Mcgibbney <
> > lewis.mcgibbney@gmail.com> wrote:
> >
> > > The DmozParser should have created a flat file similar to a bootstrap
> > file
> > > which you can inject.
> > > The flat file should be inside a the dmoz directory (if you've followed
> > the
> > > tutorial). Please make sure the file is present, and that the CLI
> syntax
> > is
> > > correct.
> > > If you are using Nutch 2.x then drop the arguments for crawl/crawldb as
> > > Nutchn does not maintain a local crawldb in 2.x. We delegate Gora to
> deal
> > > with the abstraction here.
> > > hth
> > > Lewis
> > >
> >
>
>
>
> --
> *Lewis*
>
Re: Unable to crawl a series of pages in tutorial
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Which version of Nutch are you using?
On Wed, Apr 24, 2013 at 11:45 AM, Yves S. Garret <yoursurrogategod@gmail.com
> wrote:
> Oh, before I forget, thanks for the new script in the previous mail.
>
> I got to this point in the tutorial.
>
> bin/nutch org.apache.nutch.tools.DmozParser content.rdf.u8 -subset
> 5000 > dmoz/urls
>
>
> When I run the next command [ bin/nutch inject crawl/crawldb dmoz ], I get
> this error.
> Unrecognized arg dmoz
> I'm running this in the ${NUTCH_HOME}/runtime/local directory.
>
> The dmoz directory is there, so this should not be an issue.
>
>
> On Tue, Apr 23, 2013 at 10:09 PM, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com> wrote:
>
> > The DmozParser should have created a flat file similar to a bootstrap
> file
> > which you can inject.
> > The flat file should be inside a the dmoz directory (if you've followed
> the
> > tutorial). Please make sure the file is present, and that the CLI syntax
> is
> > correct.
> > If you are using Nutch 2.x then drop the arguments for crawl/crawldb as
> > Nutchn does not maintain a local crawldb in 2.x. We delegate Gora to deal
> > with the abstraction here.
> > hth
> > Lewis
> >
>
--
*Lewis*
Re: Unable to crawl a series of pages in tutorial
Posted by "Yves S. Garret" <yo...@gmail.com>.
Oh, before I forget, thanks for the new script in the previous mail.
I got to this point in the tutorial.
bin/nutch org.apache.nutch.tools.DmozParser content.rdf.u8 -subset
5000 > dmoz/urls
When I run the next command [ bin/nutch inject crawl/crawldb dmoz ], I get
this error.
Unrecognized arg dmoz
I'm running this in the ${NUTCH_HOME}/runtime/local directory.
The dmoz directory is there, so this should not be an issue.
On Tue, Apr 23, 2013 at 10:09 PM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:
> The DmozParser should have created a flat file similar to a bootstrap file
> which you can inject.
> The flat file should be inside a the dmoz directory (if you've followed the
> tutorial). Please make sure the file is present, and that the CLI syntax is
> correct.
> If you are using Nutch 2.x then drop the arguments for crawl/crawldb as
> Nutchn does not maintain a local crawldb in 2.x. We delegate Gora to deal
> with the abstraction here.
> hth
> Lewis
>
Re: Unable to crawl a series of pages in tutorial
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Yes
On Wed, Apr 24, 2013 at 11:41 AM, Yves S. Garret <yoursurrogategod@gmail.com
> wrote:
> The dmoz directory, it should be located here, yes?
>
> ${APACHE_NUTCH_HOME}/runtime/local
>
>
> On Tue, Apr 23, 2013 at 10:09 PM, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com> wrote:
>
> > The DmozParser should have created a flat file similar to a bootstrap
> file
> > which you can inject.
> > The flat file should be inside a the dmoz directory (if you've followed
> the
> > tutorial). Please make sure the file is present, and that the CLI syntax
> is
> > correct.
> > If you are using Nutch 2.x then drop the arguments for crawl/crawldb as
> > Nutchn does not maintain a local crawldb in 2.x. We delegate Gora to deal
> > with the abstraction here.
> > hth
> > Lewis
> >
>
--
*Lewis*
Re: Unable to crawl a series of pages in tutorial
Posted by "Yves S. Garret" <yo...@gmail.com>.
The dmoz directory, it should be located here, yes?
${APACHE_NUTCH_HOME}/runtime/local
On Tue, Apr 23, 2013 at 10:09 PM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:
> The DmozParser should have created a flat file similar to a bootstrap file
> which you can inject.
> The flat file should be inside a the dmoz directory (if you've followed the
> tutorial). Please make sure the file is present, and that the CLI syntax is
> correct.
> If you are using Nutch 2.x then drop the arguments for crawl/crawldb as
> Nutchn does not maintain a local crawldb in 2.x. We delegate Gora to deal
> with the abstraction here.
> hth
> Lewis
>
Re: Unable to crawl a series of pages in tutorial
Posted by Lewis John Mcgibbney <le...@gmail.com>.
The DmozParser should have created a flat file similar to a bootstrap file
which you can inject.
The flat file should be inside a the dmoz directory (if you've followed the
tutorial). Please make sure the file is present, and that the CLI syntax is
correct.
If you are using Nutch 2.x then drop the arguments for crawl/crawldb as
Nutchn does not maintain a local crawldb in 2.x. We delegate Gora to deal
with the abstraction here.
hth
Lewis