You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Bai Shen <ba...@gmail.com> on 2012/08/06 21:21:36 UTC

Nutch 2 plugins

I'm working on writing a Nutch 2 plugin.  Whenever something is configured
wrong, I don't get any valid logging info.  Instead, I always get the exact
same NPE.

SolrIndexerJob: java.lang.NullPointerException
        at
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.getOutputPath(FileOutputFormat.java:160)
        at
org.apache.nutch.indexer.solr.SolrIndexerJob.indexSolr(SolrIndexerJob.java:74)
        at
org.apache.nutch.indexer.solr.SolrIndexerJob.run(SolrIndexerJob.java:90)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at
org.apache.nutch.indexer.solr.SolrIndexerJob.main(SolrIndexerJob.java:99)

Any ideas how to get Nutch to actually give me a useful stack trace?

Thanks.

Re: Nutch 2 plugins

Posted by Bai Shen <ba...@gmail.com>.
Switching to the Nutch 2 head fixed it.

Thanks.

On Tue, Aug 7, 2012 at 3:10 AM, Ferdy Galema <fe...@kalooga.com>wrote:

> Hi,
>
> I got this one a while back. This is one of the reasons I've committed a
> patch to Nutch2 that disable the creation of tmp files during indexing.
> (They are not needed anymore.) Because: In order to clean up the tmp files,
> they are removed by a finally block. But if something goes wrong during the
> creation of the job and the output path is still null, you get this
> terrible exception that has nothing to do with the actual error.
>
> So to fix it: Apply patch in
> NUTCH-1444<https://issues.apache.org/jira/browse/NUTCH-1444> or
> update to Nutch2x head.
>
> Ferdy
>
> On Mon, Aug 6, 2012 at 9:21 PM, Bai Shen <ba...@gmail.com> wrote:
>
> > I'm working on writing a Nutch 2 plugin.  Whenever something is
> configured
> > wrong, I don't get any valid logging info.  Instead, I always get the
> exact
> > same NPE.
> >
> > SolrIndexerJob: java.lang.NullPointerException
> >         at
> >
> >
> org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.getOutputPath(FileOutputFormat.java:160)
> >         at
> >
> >
> org.apache.nutch.indexer.solr.SolrIndexerJob.indexSolr(SolrIndexerJob.java:74)
> >         at
> > org.apache.nutch.indexer.solr.SolrIndexerJob.run(SolrIndexerJob.java:90)
> >         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> >         at
> > org.apache.nutch.indexer.solr.SolrIndexerJob.main(SolrIndexerJob.java:99)
> >
> > Any ideas how to get Nutch to actually give me a useful stack trace?
> >
> > Thanks.
> >
>

Re: Nutch 2 plugins

Posted by Bai Shen <ba...@gmail.com>.
It looks like that patch requires some other changes, so I'm going to try
the head.

On Tue, Aug 7, 2012 at 3:10 AM, Ferdy Galema <fe...@kalooga.com>wrote:

> Hi,
>
> I got this one a while back. This is one of the reasons I've committed a
> patch to Nutch2 that disable the creation of tmp files during indexing.
> (They are not needed anymore.) Because: In order to clean up the tmp files,
> they are removed by a finally block. But if something goes wrong during the
> creation of the job and the output path is still null, you get this
> terrible exception that has nothing to do with the actual error.
>
> So to fix it: Apply patch in
> NUTCH-1444<https://issues.apache.org/jira/browse/NUTCH-1444> or
> update to Nutch2x head.
>
> Ferdy
>
> On Mon, Aug 6, 2012 at 9:21 PM, Bai Shen <ba...@gmail.com> wrote:
>
> > I'm working on writing a Nutch 2 plugin.  Whenever something is
> configured
> > wrong, I don't get any valid logging info.  Instead, I always get the
> exact
> > same NPE.
> >
> > SolrIndexerJob: java.lang.NullPointerException
> >         at
> >
> >
> org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.getOutputPath(FileOutputFormat.java:160)
> >         at
> >
> >
> org.apache.nutch.indexer.solr.SolrIndexerJob.indexSolr(SolrIndexerJob.java:74)
> >         at
> > org.apache.nutch.indexer.solr.SolrIndexerJob.run(SolrIndexerJob.java:90)
> >         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> >         at
> > org.apache.nutch.indexer.solr.SolrIndexerJob.main(SolrIndexerJob.java:99)
> >
> > Any ideas how to get Nutch to actually give me a useful stack trace?
> >
> > Thanks.
> >
>

Re: Nutch 2 plugins

Posted by Ferdy Galema <fe...@kalooga.com>.
Hi,

I got this one a while back. This is one of the reasons I've committed a
patch to Nutch2 that disable the creation of tmp files during indexing.
(They are not needed anymore.) Because: In order to clean up the tmp files,
they are removed by a finally block. But if something goes wrong during the
creation of the job and the output path is still null, you get this
terrible exception that has nothing to do with the actual error.

So to fix it: Apply patch in
NUTCH-1444<https://issues.apache.org/jira/browse/NUTCH-1444> or
update to Nutch2x head.

Ferdy

On Mon, Aug 6, 2012 at 9:21 PM, Bai Shen <ba...@gmail.com> wrote:

> I'm working on writing a Nutch 2 plugin.  Whenever something is configured
> wrong, I don't get any valid logging info.  Instead, I always get the exact
> same NPE.
>
> SolrIndexerJob: java.lang.NullPointerException
>         at
>
> org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.getOutputPath(FileOutputFormat.java:160)
>         at
>
> org.apache.nutch.indexer.solr.SolrIndexerJob.indexSolr(SolrIndexerJob.java:74)
>         at
> org.apache.nutch.indexer.solr.SolrIndexerJob.run(SolrIndexerJob.java:90)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at
> org.apache.nutch.indexer.solr.SolrIndexerJob.main(SolrIndexerJob.java:99)
>
> Any ideas how to get Nutch to actually give me a useful stack trace?
>
> Thanks.
>