You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@nutch.apache.org by Noah Silverman <no...@smartmediacorp.com> on 2009/12/17 07:09:01 UTC

Nutch search works, but no results in Tomcat

Hi,

Just installed Nutch 1.0 and tomcat.  Starting to play around with things.

I've managed to execute a crawl using : Nutch crawl
It appears as if the crawl worked.  I can do a test search from the 
command line with:
bin/nutch org.apache.nutch.searcher.NutchBean foobar

It returns 10 results that look correct.

Then I setup Tomcat by modifying the nutch-site.xml file to contain the 
path to the crawl directory (containing segments, index, etc.)

Starting Tomcat, I see in the logfile that the index was found: 
"2009-12-16 22:00:39,740 INFO  SearchBean - opening indexes in 
/home/noah/Documents/nutch/crawl/indexes"

BUT, HERE IS THE PROBLEM:  When I go to localhost:8080 in a browser and 
search for anything, it returns 0 results.

I'm stumped:
- Nutch crawled correctly
- I can search from the command line and it works
- the Catalina log appears to have found the nutch index
- but, no results??

Any suggestions?

Re: Nutch search works, but no results in Tomcat

Posted by Fadzi Ushewokunze <fa...@butterflycluster.net>.

put more logging statements in NutchBean;

have you checked errors in catalina.out?



On Thu, 2009-12-17 at 16:32 -0800, Noah Silverman wrote:
> Hello,
> 
> Just to summarize.
> 
> 1) Nutch crawl completes without error.
> 
> 2) I can search from command line and see results.  (Assume this means
> that index is created.)
>     bin/nutch org.apache.nutch.searcher.NutchBean foobar
> 
> 3) Tomcat configured through nutch-site file to point to nutch/crawl
> directory
> 
> 4) catalina.out logfile indicates that tomcat is opening nutch/crawl
>     2009-12-16 22:00:39,740 INFO SearchBean - opening indexes in
> /home/noah/Documents/nutch/crawl/indexes
> 
> 5) No results when searching in web front end
> 
> 6) No errors in the logs
> 
> Is there some way to debug this?  Perhaps more verbose logging?
> 
> Thanks!!!
> 
> -N
> 
> 
> 
> 
> On 12/17/09 3:08 PM, MilleBii wrote:
> > Is there anything in your index ? Use Luke to check.
> > Also make sure to point on nutch/crawl and not on nutch/crawl/indexes, the
> > searcher needs two directory crawl/index and crawl/segments to actually
> > work.
> >
> >
> >
> > 2009/12/17 Noah Silverman <no...@smartmediacorp.com>
> >
> >   
> >> Hi,
> >>
> >> Yes I did.  I can see the "Nutch" page in my borwser and input a search
> >> term.  It just returns zero results.
> >>
> >> -N
> >>
> >> On 12/17/09 5:31 AM, Peters, Vijaya wrote:
> >>     
> >>> did you put the nutch war file into your servlet container?
> >>>
> >>> The tutorial says cp nutch*.war !/local/tomcat/webapps/ROOT.war
> >>>
> >>> I had to call it ROOT.war to get it to work.
> >>>
> >>> Vijaya Peters
> >>> SRA International, Inc.
> >>> 4350 Fair Lakes Court North
> >>> Room 4004
> >>> Fairfax, VA  22033
> >>> Tel:  703-502-1184
> >>>
> >>> www.sra.com
> >>> Named to FORTUNE's "100 Best Companies to Work For" list for 10
> >>> consecutive years
> >>> P Please consider the environment before printing this e-mail
> >>> This electronic message transmission contains information from SRA
> >>> International, Inc. which may be confidential, privileged or
> >>> proprietary.  The information is intended for the use of the individual
> >>> or entity named above.  If you are not the intended recipient, be aware
> >>> that any disclosure, copying, distribution, or use of the contents of
> >>> this information is strictly prohibited.  If you have received this
> >>> electronic information in error, please notify us immediately by
> >>> telephone at 866-584-2143.
> >>> -----Original Message-----
> >>> From: Noah Silverman [mailto:noah@smartmediacorp.com]
> >>> Sent: Thursday, December 17, 2009 1:09 AM
> >>> To: nutch-user@lucene.apache.org
> >>> Subject: Nutch search works, but no results in Tomcat
> >>>
> >>> Hi,
> >>>
> >>> Just installed Nutch 1.0 and tomcat.  Starting to play around with
> >>> things.
> >>>
> >>> I've managed to execute a crawl using : Nutch crawl
> >>> It appears as if the crawl worked.  I can do a test search from the
> >>> command line with:
> >>> bin/nutch org.apache.nutch.searcher.NutchBean foobar
> >>>
> >>> It returns 10 results that look correct.
> >>>
> >>> Then I setup Tomcat by modifying the nutch-site.xml file to contain the
> >>> path to the crawl directory (containing segments, index, etc.)
> >>>
> >>> Starting Tomcat, I see in the logfile that the index was found:
> >>> "2009-12-16 22:00:39,740 INFO  SearchBean - opening indexes in
> >>> /home/noah/Documents/nutch/crawl/indexes"
> >>>
> >>> BUT, HERE IS THE PROBLEM:  When I go to localhost:8080 in a browser and
> >>> search for anything, it returns 0 results.
> >>>
> >>> I'm stumped:
> >>> - Nutch crawled correctly
> >>> - I can search from the command line and it works
> >>> - the Catalina log appears to have found the nutch index
> >>> - but, no results??
> >>>
> >>> Any suggestions?
> >>>
> >>>
> >>>
> >>>       
> >>     
> >
> >
> >

Re: Nutch search works, but no results in Tomcat

Posted by Mischa Tuffield <mi...@garlik.com>.

Hello I have found that I had to include the following property in nutch-site.xml when building the WAR file : 

  <property>
    <name>fs.default.name</name>
    <value>file:///</value>
  </property>

Which in turn tells the WAR file to look on the file system to find the index directory. 

Furthermore, I need to comment this config property out when I build the nutch binaries which I use for the crawling process.

I hope this helps,

Mischa

On 18 Dec 2009, at 12:54, MilleBii wrote:

> Wich nutch-site.xml did you update ? The one under $NUTCH_HOME/conf or
> the one under webapps/ROOT/WEB-INF
> 
> again have a look at the query plugins to see if they are enabled and present.
> 
> Also presumably you do a tomcat restart when changing things under
> ROOT othewise nothing changes really.
> 
> 2009/12/18, Noah Silverman <no...@smartmediacorp.com>:
>> No errors in the logs.
>> 
>> Everything I'm running is "stock"
>> 
>> 1) I downloaded nutch from lucene.apache.org
>> 2) I used the yum package manager to install tomcat (running a redhat
>> server)
>> 3) copied over the war file for nutch and renamed it to root
>> 4) Edited nutch-site.xml to point to my crawl index.
>> 
>> Searches with NutchBean from the command line work
>> 
>> luke opens the index, shows good data and can search and return results.
>> 
>> So, my guess is that nutch is crawling and indexing correctly.
>> 
>> Something must be wrong with the tomcat side or the war file???
>> 
>> -N
>> 
>> 
>> On 12/18/09 3:30 AM, MilleBii wrote:
>>> You can patch search.jsp to make it log things.
>>> 
>>> Is the query analysis working?
>>> 
>>> Check your query plug-in's .
>>> are they enabled and the right ones.
>>> Check the logs it should tell if has found results or not.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 2009/12/18, Fadzi Ushewokunze <fa...@butterflycluster.net>:
>>> 
>>>> forgot to mention that; the default path to the index is
>>>> 
>>>> $NUTCH_HOME/crawl/index,
>>>> 
>>>> seems you have
>>>> 
>>>> $NUTCH_HOME/crawl/indexes
>>>> 
>>>> if $NUTCH_HOME/crawl/indexes doesnt exist there will a nullpointer
>>>> exception or some runtime exception in your catalina.out or some of
>>>> tomcat's logs.
>>>> 
>>>> On Thu, 2009-12-17 at 16:32 -0800, Noah Silverman wrote:
>>>> 
>>>>> Hello,
>>>>> 
>>>>> Just to summarize.
>>>>> 
>>>>> 1) Nutch crawl completes without error.
>>>>> 
>>>>> 2) I can search from command line and see results.  (Assume this means
>>>>> that index is created.)
>>>>>    bin/nutch org.apache.nutch.searcher.NutchBean foobar
>>>>> 
>>>>> 3) Tomcat configured through nutch-site file to point to nutch/crawl
>>>>> directory
>>>>> 
>>>>> 4) catalina.out logfile indicates that tomcat is opening nutch/crawl
>>>>>    2009-12-16 22:00:39,740 INFO SearchBean - opening indexes in
>>>>> /home/noah/Documents/nutch/crawl/indexes
>>>>> 
>>>>> 5) No results when searching in web front end
>>>>> 
>>>>> 6) No errors in the logs
>>>>> 
>>>>> Is there some way to debug this?  Perhaps more verbose logging?
>>>>> 
>>>>> Thanks!!!
>>>>> 
>>>>> -N
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> On 12/17/09 3:08 PM, MilleBii wrote:
>>>>> 
>>>>>> Is there anything in your index ? Use Luke to check.
>>>>>> Also make sure to point on nutch/crawl and not on nutch/crawl/indexes,
>>>>>> the
>>>>>> searcher needs two directory crawl/index and crawl/segments to actually
>>>>>> work.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 2009/12/17 Noah Silverman <no...@smartmediacorp.com>
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> Hi,
>>>>>>> 
>>>>>>> Yes I did.  I can see the "Nutch" page in my borwser and input a
>>>>>>> search
>>>>>>> term.  It just returns zero results.
>>>>>>> 
>>>>>>> -N
>>>>>>> 
>>>>>>> On 12/17/09 5:31 AM, Peters, Vijaya wrote:
>>>>>>> 
>>>>>>> 
>>>>>>>> did you put the nutch war file into your servlet container?
>>>>>>>> 
>>>>>>>> The tutorial says cp nutch*.war !/local/tomcat/webapps/ROOT.war
>>>>>>>> 
>>>>>>>> I had to call it ROOT.war to get it to work.
>>>>>>>> 
>>>>>>>> Vijaya Peters
>>>>>>>> SRA International, Inc.
>>>>>>>> 4350 Fair Lakes Court North
>>>>>>>> Room 4004
>>>>>>>> Fairfax, VA  22033
>>>>>>>> Tel:  703-502-1184
>>>>>>>> 
>>>>>>>> www.sra.com
>>>>>>>> Named to FORTUNE's "100 Best Companies to Work For" list for 10
>>>>>>>> consecutive years
>>>>>>>> P Please consider the environment before printing this e-mail
>>>>>>>> This electronic message transmission contains information from SRA
>>>>>>>> International, Inc. which may be confidential, privileged or
>>>>>>>> proprietary.  The information is intended for the use of the
>>>>>>>> individual
>>>>>>>> or entity named above.  If you are not the intended recipient, be
>>>>>>>> aware
>>>>>>>> that any disclosure, copying, distribution, or use of the contents of
>>>>>>>> this information is strictly prohibited.  If you have received this
>>>>>>>> electronic information in error, please notify us immediately by
>>>>>>>> telephone at 866-584-2143.
>>>>>>>> -----Original Message-----
>>>>>>>> From: Noah Silverman [mailto:noah@smartmediacorp.com]
>>>>>>>> Sent: Thursday, December 17, 2009 1:09 AM
>>>>>>>> To: nutch-user@lucene.apache.org
>>>>>>>> Subject: Nutch search works, but no results in Tomcat
>>>>>>>> 
>>>>>>>> Hi,
>>>>>>>> 
>>>>>>>> Just installed Nutch 1.0 and tomcat.  Starting to play around with
>>>>>>>> things.
>>>>>>>> 
>>>>>>>> I've managed to execute a crawl using : Nutch crawl
>>>>>>>> It appears as if the crawl worked.  I can do a test search from the
>>>>>>>> command line with:
>>>>>>>> bin/nutch org.apache.nutch.searcher.NutchBean foobar
>>>>>>>> 
>>>>>>>> It returns 10 results that look correct.
>>>>>>>> 
>>>>>>>> Then I setup Tomcat by modifying the nutch-site.xml file to contain
>>>>>>>> the
>>>>>>>> path to the crawl directory (containing segments, index, etc.)
>>>>>>>> 
>>>>>>>> Starting Tomcat, I see in the logfile that the index was found:
>>>>>>>> "2009-12-16 22:00:39,740 INFO  SearchBean - opening indexes in
>>>>>>>> /home/noah/Documents/nutch/crawl/indexes"
>>>>>>>> 
>>>>>>>> BUT, HERE IS THE PROBLEM:  When I go to localhost:8080 in a browser
>>>>>>>> and
>>>>>>>> search for anything, it returns 0 results.
>>>>>>>> 
>>>>>>>> I'm stumped:
>>>>>>>> - Nutch crawled correctly
>>>>>>>> - I can search from the command line and it works
>>>>>>>> - the Catalina log appears to have found the nutch index
>>>>>>>> - but, no results??
>>>>>>>> 
>>>>>>>> Any suggestions?
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>>> 
>>> 
>> 
> 
> 
> -- 
> -MilleBii-

___________________________________
Mischa Tuffield
Email: mischa.tuffield@garlik.com
Homepage - http://mmt.me.uk/
Garlik Limited, 2 Sheen Road, Richmond, TW9 1AE, UK
+44(0)20 8973 2465  http://www.garlik.com/
Registered in England and Wales 535 7233 VAT # 849 0517 11
Registered office: Thames House, Portsmouth Road, Esher, Surrey, KT10 9AD