You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by Tejas Patil <te...@gmail.com> on 2013/06/11 01:30:43 UTC
Fwd: Nutch Compilation Error with Eclipse
Hi @nutch-dev,
I want to put out this [0] tutorial over Nutch wiki.
1. Do you see anything wrong in it or any improvements ?
2. Where do I upload the images ? Wiki will allow me to just specify the
url.
[0] :
https://docs.google.com/document/d/1qvJwrZ9Sc0NAF9p3ie4uV7JsfCHxnrh9QF19HINw48c/edit?usp=sharing
---------- Forwarded message ----------
From: Tejas Patil <te...@gmail.com>
Date: Mon, Jun 10, 2013 at 2:58 PM
Subject: Re: Nutch Compilation Error with Eclipse
To: "user@nutch.apache.org" <us...@nutch.apache.org>
I have created a google doc [0] with several snapshots describing how to
setup nutch 2.x + eclipse. This is different from the one over the wiki
page and tailored for Nutch 2.x. Please try it out, let us know if you
still have issues with that. Based on your comments, I would add the same
over nutch wiki.
[0] :
https://docs.google.com/document/d/1qvJwrZ9Sc0NAF9p3ie4uV7JsfCHxnrh9QF19HINw48c/edit?usp=sharing
On Mon, Jun 10, 2013 at 6:23 AM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:
> Hi,
> It is (IMHO) kind of fruitless running the crawl class (which is deprecated
> now and we highly suggest you use and amend the /src/bin/crawl script for
> your usecase) within Eclipse. You will learn far more setting breakpoints
> within individual classes and watching them execute on that basis. I notice
> you've not provided an URL directory to the crawl argument anyway so you
> will need to sort this one out.
> Best
> Lewis
>
> On Monday, June 10, 2013, Jamshaid Ashraf <ja...@gmail.com> wrote:
> > I'm performing following tasks:
> >
> > Commands in Arguments tab:
> >
> > Program Arguments=urls -dir crawl -depth 3 -topN 50
> >
> > VM Arguments:-Dhadoop.log.dir=logs -Dhadoop.log.file=hadoop.log
> >
> > And then just running the code.
> >
> > Regards,
> > Jamshaid
> >
> >
> > On Mon, Jun 10, 2013 at 4:54 PM, Sznajder ForMailingList <
> > bs4mailinglist@gmail.com> wrote:
> >
> >> Hi
> >>
> >> Which task do you try to launch?
> >>
> >> Benjamin
> >>
> >>
> >> On Mon, Jun 10, 2013 at 1:57 PM, Jamshaid Ashraf <jamshaid.qe@gmail.com
> >> >wrote:
> >>
> >> > Hi,
> >> >
> >> > I am new to Nutch. I am trying to use Nutch with Cassandra and have
> >> > successfully build the Nutch 2.x but shows following error when I run
> it
> >> > from latest eclipse.
> >> >
> >> >
> >> > java.lang.NullPointerException
> >> > at org.apache.avro.util.Utf8.<init>(Utf8.java:37)
> >> > at
> >> org.apache.nutch.crawl.GeneratorReducer.setup(GeneratorReducer.java:100)
> >> > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> >> > at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:650)
> >> > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418)
> >> > at
> >> >
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:260).
> >> >
> >> > I will be grateful for any help if someone can provide.
> >> >
> >> >
> >> > Thanks.
> >> >
> >>
> >
>
> --
> *Lewis*
>
Re: Fwd: Nutch Compilation Error with Eclipse
Posted by Tejas Patil <te...@gmail.com>.
Hi Seb,
You rock !!! It worked like a charm :)
Thanks,
Tejas
On Tue, Jun 11, 2013 at 12:36 PM, Sebastian Nagel <
wastl.nagel@googlemail.com> wrote:
> Hi Tejas,
>
> you should be able to add images as "Attachments":
> there is a tab/link left of "More Actions:".
>
> Cheers,
> Sebastian
>
> On 06/11/2013 01:30 AM, Tejas Patil wrote:
> > Hi @nutch-dev,
> >
> > I want to put out this [0] tutorial over Nutch wiki.
> >
> > 1. Do you see anything wrong in it or any improvements ?
> > 2. Where do I upload the images ? Wiki will allow me to just specify the
> > url.
> >
> > [0] :
> >
> https://docs.google.com/document/d/1qvJwrZ9Sc0NAF9p3ie4uV7JsfCHxnrh9QF19HINw48c/edit?usp=sharing
> >
> > ---------- Forwarded message ----------
> > From: Tejas Patil <te...@gmail.com>
> > Date: Mon, Jun 10, 2013 at 2:58 PM
> > Subject: Re: Nutch Compilation Error with Eclipse
> > To: "user@nutch.apache.org" <us...@nutch.apache.org>
> >
> >
> > I have created a google doc [0] with several snapshots describing how to
> > setup nutch 2.x + eclipse. This is different from the one over the wiki
> > page and tailored for Nutch 2.x. Please try it out, let us know if you
> > still have issues with that. Based on your comments, I would add the same
> > over nutch wiki.
> >
> > [0] :
> >
> https://docs.google.com/document/d/1qvJwrZ9Sc0NAF9p3ie4uV7JsfCHxnrh9QF19HINw48c/edit?usp=sharing
> >
> >
> > On Mon, Jun 10, 2013 at 6:23 AM, Lewis John Mcgibbney <
> > lewis.mcgibbney@gmail.com> wrote:
> >
> >> Hi,
> >> It is (IMHO) kind of fruitless running the crawl class (which is
> deprecated
> >> now and we highly suggest you use and amend the /src/bin/crawl script
> for
> >> your usecase) within Eclipse. You will learn far more setting
> breakpoints
> >> within individual classes and watching them execute on that basis. I
> notice
> >> you've not provided an URL directory to the crawl argument anyway so you
> >> will need to sort this one out.
> >> Best
> >> Lewis
> >>
> >> On Monday, June 10, 2013, Jamshaid Ashraf <ja...@gmail.com>
> wrote:
> >>> I'm performing following tasks:
> >>>
> >>> Commands in Arguments tab:
> >>>
> >>> Program Arguments=urls -dir crawl -depth 3 -topN 50
> >>>
> >>> VM Arguments:-Dhadoop.log.dir=logs -Dhadoop.log.file=hadoop.log
> >>>
> >>> And then just running the code.
> >>>
> >>> Regards,
> >>> Jamshaid
> >>>
> >>>
> >>> On Mon, Jun 10, 2013 at 4:54 PM, Sznajder ForMailingList <
> >>> bs4mailinglist@gmail.com> wrote:
> >>>
> >>>> Hi
> >>>>
> >>>> Which task do you try to launch?
> >>>>
> >>>> Benjamin
> >>>>
> >>>>
> >>>> On Mon, Jun 10, 2013 at 1:57 PM, Jamshaid Ashraf <
> jamshaid.qe@gmail.com
> >>>>> wrote:
> >>>>
> >>>>> Hi,
> >>>>>
> >>>>> I am new to Nutch. I am trying to use Nutch with Cassandra and have
> >>>>> successfully build the Nutch 2.x but shows following error when I run
> >> it
> >>>>> from latest eclipse.
> >>>>>
> >>>>>
> >>>>> java.lang.NullPointerException
> >>>>> at org.apache.avro.util.Utf8.<init>(Utf8.java:37)
> >>>>> at
> >>>>
> org.apache.nutch.crawl.GeneratorReducer.setup(GeneratorReducer.java:100)
> >>>>> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> >>>>> at
> >> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:650)
> >>>>> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418)
> >>>>> at
> >>>>>
> >>
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:260).
> >>>>>
> >>>>> I will be grateful for any help if someone can provide.
> >>>>>
> >>>>>
> >>>>> Thanks.
> >>>>>
> >>>>
> >>>
> >>
> >> --
> >> *Lewis*
> >>
> >
>
>
Re: Fwd: Nutch Compilation Error with Eclipse
Posted by Sebastian Nagel <wa...@googlemail.com>.
Hi Tejas,
you should be able to add images as "Attachments":
there is a tab/link left of "More Actions:".
Cheers,
Sebastian
On 06/11/2013 01:30 AM, Tejas Patil wrote:
> Hi @nutch-dev,
>
> I want to put out this [0] tutorial over Nutch wiki.
>
> 1. Do you see anything wrong in it or any improvements ?
> 2. Where do I upload the images ? Wiki will allow me to just specify the
> url.
>
> [0] :
> https://docs.google.com/document/d/1qvJwrZ9Sc0NAF9p3ie4uV7JsfCHxnrh9QF19HINw48c/edit?usp=sharing
>
> ---------- Forwarded message ----------
> From: Tejas Patil <te...@gmail.com>
> Date: Mon, Jun 10, 2013 at 2:58 PM
> Subject: Re: Nutch Compilation Error with Eclipse
> To: "user@nutch.apache.org" <us...@nutch.apache.org>
>
>
> I have created a google doc [0] with several snapshots describing how to
> setup nutch 2.x + eclipse. This is different from the one over the wiki
> page and tailored for Nutch 2.x. Please try it out, let us know if you
> still have issues with that. Based on your comments, I would add the same
> over nutch wiki.
>
> [0] :
> https://docs.google.com/document/d/1qvJwrZ9Sc0NAF9p3ie4uV7JsfCHxnrh9QF19HINw48c/edit?usp=sharing
>
>
> On Mon, Jun 10, 2013 at 6:23 AM, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com> wrote:
>
>> Hi,
>> It is (IMHO) kind of fruitless running the crawl class (which is deprecated
>> now and we highly suggest you use and amend the /src/bin/crawl script for
>> your usecase) within Eclipse. You will learn far more setting breakpoints
>> within individual classes and watching them execute on that basis. I notice
>> you've not provided an URL directory to the crawl argument anyway so you
>> will need to sort this one out.
>> Best
>> Lewis
>>
>> On Monday, June 10, 2013, Jamshaid Ashraf <ja...@gmail.com> wrote:
>>> I'm performing following tasks:
>>>
>>> Commands in Arguments tab:
>>>
>>> Program Arguments=urls -dir crawl -depth 3 -topN 50
>>>
>>> VM Arguments:-Dhadoop.log.dir=logs -Dhadoop.log.file=hadoop.log
>>>
>>> And then just running the code.
>>>
>>> Regards,
>>> Jamshaid
>>>
>>>
>>> On Mon, Jun 10, 2013 at 4:54 PM, Sznajder ForMailingList <
>>> bs4mailinglist@gmail.com> wrote:
>>>
>>>> Hi
>>>>
>>>> Which task do you try to launch?
>>>>
>>>> Benjamin
>>>>
>>>>
>>>> On Mon, Jun 10, 2013 at 1:57 PM, Jamshaid Ashraf <jamshaid.qe@gmail.com
>>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am new to Nutch. I am trying to use Nutch with Cassandra and have
>>>>> successfully build the Nutch 2.x but shows following error when I run
>> it
>>>>> from latest eclipse.
>>>>>
>>>>>
>>>>> java.lang.NullPointerException
>>>>> at org.apache.avro.util.Utf8.<init>(Utf8.java:37)
>>>>> at
>>>> org.apache.nutch.crawl.GeneratorReducer.setup(GeneratorReducer.java:100)
>>>>> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
>>>>> at
>> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:650)
>>>>> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418)
>>>>> at
>>>>>
>> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:260).
>>>>>
>>>>> I will be grateful for any help if someone can provide.
>>>>>
>>>>>
>>>>> Thanks.
>>>>>
>>>>
>>>
>>
>> --
>> *Lewis*
>>
>