You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Christophe Noel (JIRA)" <ji...@apache.org> on 2005/07/15 13:27:10 UTC
[jira] Created: (NUTCH-72) Query basic filter with correction feature
Query basic filter with correction feature
------------------------------------------
Key: NUTCH-72
URL: http://issues.apache.org/jira/browse/NUTCH-72
Project: Nutch
Type: New Feature
Components: searcher
Environment: lucene
Reporter: Christophe Noel
This plugin improves query-basic plugin with a correction feature.
Lucene includes FuzzyQuery feature which consists of searching not only for matching terms, but searching for very similar terms too.
This plugin should be used instead of query-basic, for people looking for an easy solution about users query requests correction.
Correction Query Plugin can be used as follows :
Solution 1 : If you want to search for very similar terms, add autocorrectionmod as the first term of the query (example : 'nutch engine' -> 'autocorrectionmod nutch engine')
Solution 2 : Create a new search.jsp page which include a "correction" checkbox management (<input type="checkbox" name="autocorrection" value="true"> may automatically add 'autocorrectionmod' as the first term of the query)
QueryFuzzy knows a big problem : it is very slow for large index !
So Correction Query Plugin works as follows :
- it is not useful for big indexes
- it only works for 5 characters and more words
- it only look for words matching with the 2 first characters (to improve performance this should be set to 3/4)
- it only works for 65 % matching suffixes (algorithm is levenstein)
PLease give your opinion about it.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
indexed records in segments
Posted by yoursoft <yo...@freemail.hu>.
Dear Developers,
I maked a modified version of SegmentRead.java. This modification is
show indexed number of docs too.
I think this is usefully when ballancing segments between backends.
The real number of used docs is the indexed number. This is not equal
with number of records in the segment. After fetch the number of indexed
record decreassed to 80-90% of records. After 'dedup' this percentage
is more lower value. After 'prune' this value is decreassed more. In
your segments the real and used records number will be very different.
When you see your balance of backends, the load avarage will be very
different.
I think the load avarage is depending the avarage 'boost' value of the
segment too.
What you think from these thinks?
Regards,
Ferenc
Re: [Nutch-dev] Re: a silly question
Posted by "Feng (Michael) Ji" <fj...@yahoo.com>.
That works finally,
Thank you so much,
Michael,
--- Piotr Kosiorowski <pk...@gmail.com> wrote:
> Hello,
>
> I understood you have all your segments in
> /home/fji/SE/nutch-nightly/crawl.test/
> but according to log file you sent nutch is looking
> in:
> /home/fji/SE/tomcat4/segments
> Please copy your segment directory from
> /home/fji/SE/nutch-nightly/crawl.test/
> to
> /home/fji/SE/tomcat4/
> and restart tomcat.
> Regards
> Piotr
>
>
> Feng (Michael) Ji wrote:
> > thanks all the suggestions,
> >
> > I caught the catalina.out as following:
> >
> > "Jul 16, 2005 3:09:27 PM
> > org.apache.coyote.http11.Http11Protocol init
> > INFO: Initializing Coyote HTTP/1.1 on http-8888
> > Starting service Tomcat-Standalone
> > Apache Tomcat/4.1.31
> > Jul 16, 2005 3:09:29 PM
> > org.apache.struts.util.PropertyMessageResources
> <init>
> > INFO: Initializing,
> > config='org.apache.struts.util.LocalStrings',
> > returnNull=true
> > Jul 16, 2005 3:09:29 PM
> > org.apache.struts.util.PropertyMessageResources
> <init>
> > INFO: Initializing,
> > config='org.apache.struts.action.ActionResources',
> > returnNull=true
> > Jul 16, 2005 3:09:29 PM
> > org.apache.struts.util.PropertyMessageResources
> <init>
> > INFO: Initializing,
> >
>
config='org.apache.webapp.admin.ApplicationResources',
> > returnNull=true
> > Jul 16, 2005 3:09:35 PM
> > org.apache.coyote.http11.Http11Protocol start
> > INFO: Starting Coyote HTTP/1.1 on http-8888
> > Jul 16, 2005 3:09:35 PM
> > org.apache.jk.common.ChannelSocket init
> > INFO: JK2: ajp13 listening on /0.0.0.0:8009
> > Jul 16, 2005 3:09:35 PM
> org.apache.jk.server.JkMain
> > start
> > INFO: Jk running ID=0 time=0/81
> > config=/home/fji/SE/tomcat4/conf/jk2.properties
> > 050716 150937 parsing
> >
>
file:/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/nutch-default.xml
> > 050716 150937 parsing
> >
>
file:/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/nutch-site.xml
> > 050716 150937 Plugins: looking in:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins
> > 050716 150937 not including:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/clustering-carrot2
> > 050716 150937 not including:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/creativecommons
> > 050716 150937 parsing:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/index-basic/plugin.xml
> > 050716 150937 impl:
> > point=org.apache.nutch.indexer.IndexingFilter
> >
>
class=org.apache.nutch.indexer.basic.BasicIndexingFilter
> > 050716 150937 not including:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/index-more
> > 050716 150937 not including:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/language-identifier
> > 050716 150937 not including:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/ontology
> > 050716 150937 not including:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/parse-ext
> > 050716 150937 parsing:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/parse-html/plugin.xml
> > 050716 150937 impl:
> > point=org.apache.nutch.parse.Parser
> > class=org.apache.nutch.parse.html.HtmlParser
> > 050716 150937 parsing:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/parse-js/plugin.xml
> > 050716 150937 impl:
> > point=org.apache.nutch.parse.Parser
> > class=org.apache.nutch.parse.js.JSParseFilter
> > 050716 150937 impl:
> > point=org.apache.nutch.parse.HtmlParseFilter
> > class=org.apache.nutch.parse.js.JSParseFilter
> > 050716 150937 not including:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/parse-msword
> > 050716 150937 not including:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/parse-pdf
> > 050716 150937 parsing:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/parse-text/plugin.xml
> > 050716 150937 impl:
> > point=org.apache.nutch.parse.Parser
> > class=org.apache.nutch.parse.text.TextParser
> > 050716 150937 not including:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/protocol-file
> > 050716 150937 not including:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/protocol-ftp
> > 050716 150937 not including:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/protocol-http
> > 050716 150937 parsing:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/protocol-httpclient/plugin.xml
> > 050716 150937 impl:
> > point=org.apache.nutch.protocol.Protocol
> > class=org.apache.nutch.protocol.httpclient.Http
> > 050716 150937 impl:
> > point=org.apache.nutch.protocol.Protocol
> > class=org.apache.nutch.protocol.httpclient.Http
> > 050716 150937 parsing:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/query-basic/plugin.xml
> > 050716 150937 impl:
> > point=org.apache.nutch.searcher.QueryFilter
> >
>
class=org.apache.nutch.searcher.basic.BasicQueryFilter
> > 050716 150937 not including:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/query-more
> > 050716 150937 parsing:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/query-site/plugin.xml
> > 050716 150937 impl:
> > point=org.apache.nutch.searcher.QueryFilter
> >
> class=org.apache.nutch.searcher.site.SiteQueryFilter
> > 050716 150937 parsing:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/query-url/plugin.xml
> > 050716 150937 impl:
> > point=org.apache.nutch.searcher.QueryFilter
> > class=org.apache.nutch.searcher.url.URLQueryFilter
> > 050716 150937 not including:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/urlfilter-prefix
> > 050716 150937 parsing:
> >
>
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/urlfilter-regex/plugin.xml
> > 050716 150937 impl:
> > point=org.apache.nutch.net.URLFilter
> > class=org.apache.nutch.net.RegexURLFilter
> > 050716 150937 11 creating new bean
> > 050716 150937 11 opening segment indexes in
> > /home/fji/SE/tomcat4/segments
> > "
> >
> > I didn't see any complain, or I miss something
> > important?
> >
> > The browser gives me the same error message as
> before.
> >
> > Michael,
> >
> > --- yoursoft <yo...@freemail.hu> wrote:
> >
> >
> >>I think please check your log/catalina.out for
> more
> >>details of error.
> >>
> >>Howie Wang wrotte:
> >>
> >>
> >>>>hi howie:
> >>>>
> >>>>For search.dir value of both nutch-default.xml
> >>
> >>and
> >>
> >>>>nutch-site.xml;
> >>>
> >>>
> >
>
<value>/home/fji/SE/nutch-nightly/crawl.test/</value>
>
=== message truncated ===
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Re: [Nutch-dev] Re: a silly question
Posted by "Feng (Michael) Ji" <fj...@yahoo.com>.
hi there:
I copy crawled segments to tomcat root directory. Now
searching has no internal error, but I didn't get any
search result. After take look in my catalina log
file, I found researching is no result.
Any suggestions you can give out?
thanks,
Michael,
======
--- Piotr Kosiorowski <pk...@gmail.com> wrote:
> Hello,
>
> I understood you have all your segments in
> /home/fji/SE/nutch-nightly/crawl.test/
> but according to log file you sent nutch is looking
> in:
> /home/fji/SE/tomcat4/segments
> Please copy your segment directory from
> /home/fji/SE/nutch-nightly/crawl.test/
> to
> /home/fji/SE/tomcat4/
> and restart tomcat.
> Regards
> Piotr
>
>
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Re: ant setup for Cgywin
Posted by Michael Ji <fj...@yahoo.com>.
Hi Dawid:
Thanks, it works now:
----Before:
Cygwin is installed on E---secondary drive of my
system.
Ant is installed in E as well.
It doesn't work.
----Now:
Now, I change Ant directory to C:---primary directory
and I reset the ANT_HOME; Now everything works fine. A
bit weird, right? Don't know how it caused;
Just want to share my experience with people in group.
thanks again,
Michael Ji,
--- Dawid Weiss <da...@cs.put.poznan.pl> wrote:
>
> Check if ANT_HOME points in the right location (root
> of ANT
> installation; spare yourself some trouble by
> choosing a
> folder with no spaces or special characters).
>
> Dawid
>
> P.S. Ant is not a compiler. It is a build tool.
>
>
> Michael Ji wrote:
> > hi there:
> >
> > I tried to setup code compiling environment for my
> > windows XP box.
> >
> > I installed Cygwin as my running environment.
> > Subversion works fine.
> >
> > But compiler, "ant", isn't working. Gives me the
> > following error message:
> > "
> > Exception in thread "main"
> > java.lang.NoClassDefFoundError:
> org/apache/tools/ant/
> > launch/Launcher
> > "
> >
> > I do everything as ant manual and even run JWhich
> to
> > search the classpath. No result returned.
> >
> > Any suggestions will be very helpful.
> >
> > thanks,
> >
> > Michael Ji,
> >
> >
> > __________________________________________________
> > Do You Yahoo!?
> > Tired of spam? Yahoo! Mail has the best spam
> protection around
> > http://mail.yahoo.com
>
____________________________________________________
Start your day with Yahoo! - make it your home page
http://www.yahoo.com/r/hs
Re: ant setup for Cgywin
Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
Check if ANT_HOME points in the right location (root of ANT
installation; spare yourself some trouble by choosing a
folder with no spaces or special characters).
Dawid
P.S. Ant is not a compiler. It is a build tool.
Michael Ji wrote:
> hi there:
>
> I tried to setup code compiling environment for my
> windows XP box.
>
> I installed Cygwin as my running environment.
> Subversion works fine.
>
> But compiler, "ant", isn't working. Gives me the
> following error message:
> "
> Exception in thread "main"
> java.lang.NoClassDefFoundError: org/apache/tools/ant/
> launch/Launcher
> "
>
> I do everything as ant manual and even run JWhich to
> search the classpath. No result returned.
>
> Any suggestions will be very helpful.
>
> thanks,
>
> Michael Ji,
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
Re: page ranking weights
Posted by Ken Krugler <kk...@transpac.com>.
>also how does it keep track of incoming links globally on these pages, if
>the weight is determined by # of incoming links then there would have to be
>somewhere it keeps track so when you split your indexes it can still have an
>accurate value for the distributed search?
The WebDB keeps track of this info. It's not in the segments/indexes.
> > at which step does nutch figure out the weight of each page, the updatedb
> > step? or the index step?
The updatedb step.
In UpdateDatabaseTool.java's PageContentChanged() method, first all
of the outlink URLs are harvested from the fetched page. Then a score
is calculated for each of the pages referenced by these outlink URLs,
based on the score of the fetched page, multiplied by either the
internal or external link weight (from Nutch config XML data, both
1.0 by default), depending on whether the URL is in the same domain
as the fetched page.
When you inject URLs, there is no referring page, so it arbitrarily
uses the db.score.injected value (1.0 by default).
So if you leave everything set to default values, and don't perform
link analysis, I think every page will wind up with a score of 1.0.
-- Ken
--
Ken Krugler
TransPac Software, Inc.
<http://www.transpac.com>
+1 530-470-9200
Re: page ranking weights
Posted by Piotr Kosiorowski <pk...@gmail.com>.
Boost for the page maybe calculated in few different ways (and in few
different places in nutch):
1) PageRank based score
- calculated by "nutch analyze" command based on WebDB
- during fetchlist generation scores from WebDB are stored in segment
- indexing phase uses score to set the boost for a page
2) based on number of incoming links
- during fetchlist generation inlinks are stored in segment
- during indexing number of inlinks is read from segment and used in
boost calculation
There is a separate command (updatesegs) to update score and inlink
information in existing segments.
Regards
Piotr
Jay Pound wrote:
> also how does it keep track of incoming links globally on these pages, if
> the weight is determined by # of incoming links then there would have to be
> somewhere it keeps track so when you split your indexes it can still have an
> accurate value for the distributed search?
> -J
> ----- Original Message -----
> From: "Jay Pound" <we...@poundwebhosting.com>
> To: <nu...@lucene.apache.org>
> Sent: Thursday, August 11, 2005 4:49 PM
> Subject: page ranking weights
>
>
>
>>at which step does nutch figure out the weight of each page, the updatedb
>>step? or the index step?
>>Thanks,
>>-Jay
>>
>>
>>
>
>
>
>
Re: page ranking weights
Posted by Jay Pound <we...@poundwebhosting.com>.
also how does it keep track of incoming links globally on these pages, if
the weight is determined by # of incoming links then there would have to be
somewhere it keeps track so when you split your indexes it can still have an
accurate value for the distributed search?
-J
----- Original Message -----
From: "Jay Pound" <we...@poundwebhosting.com>
To: <nu...@lucene.apache.org>
Sent: Thursday, August 11, 2005 4:49 PM
Subject: page ranking weights
> at which step does nutch figure out the weight of each page, the updatedb
> step? or the index step?
> Thanks,
> -Jay
>
>
>
page ranking weights
Posted by Jay Pound <we...@poundwebhosting.com>.
at which step does nutch figure out the weight of each page, the updatedb
step? or the index step?
Thanks,
-Jay
ant setup for Cgywin
Posted by Michael Ji <fj...@yahoo.com>.
hi there:
I tried to setup code compiling environment for my
windows XP box.
I installed Cygwin as my running environment.
Subversion works fine.
But compiler, "ant", isn't working. Gives me the
following error message:
"
Exception in thread "main"
java.lang.NoClassDefFoundError: org/apache/tools/ant/
launch/Launcher
"
I do everything as ant manual and even run JWhich to
search the classpath. No result returned.
Any suggestions will be very helpful.
thanks,
Michael Ji,
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
ant setup for Cgywin
Posted by Michael Ji <fj...@yahoo.com>.
hi there:
I tried to setup code compiling environment for my
windows XP box.
I installed Cygwin as my running environment.
Subversion works fine.
But compiler, "ant", isn't working. Gives me the
following error message:
"
Exception in thread "main"
java.lang.NoClassDefFoundError: org/apache/tools/ant/
launch/Launcher
"
I do everything as ant manual and even run JWhich to
search the classpath. No result returned.
Any suggestions will be very helpful.
thanks,
Michael Ji,
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
search result NULL
Posted by "Feng (Michael) Ji" <fj...@yahoo.com>.
hi there:
I copy crawled segments to tomcat root directory. Now
searching has no internal error, but I didn't get any
search result. I can see it in my catalina log file.
Any suggestions you can give out?
thanks,
Michael,
======
____________________________________________________
Start your day with Yahoo! - make it your home page
http://www.yahoo.com/r/hs
Re: [Nutch-dev] Re: a silly question
Posted by Piotr Kosiorowski <pk...@gmail.com>.
Hello,
I understood you have all your segments in
/home/fji/SE/nutch-nightly/crawl.test/
but according to log file you sent nutch is looking in:
/home/fji/SE/tomcat4/segments
Please copy your segment directory from
/home/fji/SE/nutch-nightly/crawl.test/
to
/home/fji/SE/tomcat4/
and restart tomcat.
Regards
Piotr
Feng (Michael) Ji wrote:
> thanks all the suggestions,
>
> I caught the catalina.out as following:
>
> "Jul 16, 2005 3:09:27 PM
> org.apache.coyote.http11.Http11Protocol init
> INFO: Initializing Coyote HTTP/1.1 on http-8888
> Starting service Tomcat-Standalone
> Apache Tomcat/4.1.31
> Jul 16, 2005 3:09:29 PM
> org.apache.struts.util.PropertyMessageResources <init>
> INFO: Initializing,
> config='org.apache.struts.util.LocalStrings',
> returnNull=true
> Jul 16, 2005 3:09:29 PM
> org.apache.struts.util.PropertyMessageResources <init>
> INFO: Initializing,
> config='org.apache.struts.action.ActionResources',
> returnNull=true
> Jul 16, 2005 3:09:29 PM
> org.apache.struts.util.PropertyMessageResources <init>
> INFO: Initializing,
> config='org.apache.webapp.admin.ApplicationResources',
> returnNull=true
> Jul 16, 2005 3:09:35 PM
> org.apache.coyote.http11.Http11Protocol start
> INFO: Starting Coyote HTTP/1.1 on http-8888
> Jul 16, 2005 3:09:35 PM
> org.apache.jk.common.ChannelSocket init
> INFO: JK2: ajp13 listening on /0.0.0.0:8009
> Jul 16, 2005 3:09:35 PM org.apache.jk.server.JkMain
> start
> INFO: Jk running ID=0 time=0/81
> config=/home/fji/SE/tomcat4/conf/jk2.properties
> 050716 150937 parsing
> file:/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/nutch-default.xml
> 050716 150937 parsing
> file:/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/nutch-site.xml
> 050716 150937 Plugins: looking in:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins
> 050716 150937 not including:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/clustering-carrot2
> 050716 150937 not including:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/creativecommons
> 050716 150937 parsing:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/index-basic/plugin.xml
> 050716 150937 impl:
> point=org.apache.nutch.indexer.IndexingFilter
> class=org.apache.nutch.indexer.basic.BasicIndexingFilter
> 050716 150937 not including:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/index-more
> 050716 150937 not including:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/language-identifier
> 050716 150937 not including:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/ontology
> 050716 150937 not including:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/parse-ext
> 050716 150937 parsing:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/parse-html/plugin.xml
> 050716 150937 impl:
> point=org.apache.nutch.parse.Parser
> class=org.apache.nutch.parse.html.HtmlParser
> 050716 150937 parsing:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/parse-js/plugin.xml
> 050716 150937 impl:
> point=org.apache.nutch.parse.Parser
> class=org.apache.nutch.parse.js.JSParseFilter
> 050716 150937 impl:
> point=org.apache.nutch.parse.HtmlParseFilter
> class=org.apache.nutch.parse.js.JSParseFilter
> 050716 150937 not including:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/parse-msword
> 050716 150937 not including:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/parse-pdf
> 050716 150937 parsing:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/parse-text/plugin.xml
> 050716 150937 impl:
> point=org.apache.nutch.parse.Parser
> class=org.apache.nutch.parse.text.TextParser
> 050716 150937 not including:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/protocol-file
> 050716 150937 not including:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/protocol-ftp
> 050716 150937 not including:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/protocol-http
> 050716 150937 parsing:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/protocol-httpclient/plugin.xml
> 050716 150937 impl:
> point=org.apache.nutch.protocol.Protocol
> class=org.apache.nutch.protocol.httpclient.Http
> 050716 150937 impl:
> point=org.apache.nutch.protocol.Protocol
> class=org.apache.nutch.protocol.httpclient.Http
> 050716 150937 parsing:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/query-basic/plugin.xml
> 050716 150937 impl:
> point=org.apache.nutch.searcher.QueryFilter
> class=org.apache.nutch.searcher.basic.BasicQueryFilter
> 050716 150937 not including:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/query-more
> 050716 150937 parsing:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/query-site/plugin.xml
> 050716 150937 impl:
> point=org.apache.nutch.searcher.QueryFilter
> class=org.apache.nutch.searcher.site.SiteQueryFilter
> 050716 150937 parsing:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/query-url/plugin.xml
> 050716 150937 impl:
> point=org.apache.nutch.searcher.QueryFilter
> class=org.apache.nutch.searcher.url.URLQueryFilter
> 050716 150937 not including:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/urlfilter-prefix
> 050716 150937 parsing:
> /home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/urlfilter-regex/plugin.xml
> 050716 150937 impl:
> point=org.apache.nutch.net.URLFilter
> class=org.apache.nutch.net.RegexURLFilter
> 050716 150937 11 creating new bean
> 050716 150937 11 opening segment indexes in
> /home/fji/SE/tomcat4/segments
> "
>
> I didn't see any complain, or I miss something
> important?
>
> The browser gives me the same error message as before.
>
> Michael,
>
> --- yoursoft <yo...@freemail.hu> wrote:
>
>
>>I think please check your log/catalina.out for more
>>details of error.
>>
>>Howie Wang wrotte:
>>
>>
>>>>hi howie:
>>>>
>>>>For search.dir value of both nutch-default.xml
>>
>>and
>>
>>>>nutch-site.xml;
>>>
>>>
> <value>/home/fji/SE/nutch-nightly/crawl.test/</value>
>
>>>
>>>This should be the correct one.
>>>
>>>
>>>>Still have the same error message after click
>>
>>search
>>
>>>>submit buttom in Nutch demo page
>>>>
>>>>"
>>>>HTTP Status 500
>>>>org.apache.jasper.JasperException at
>>>>
>>
> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:207)
>
>>>>....
>>>>"
>>>
>>>
>>>Hmm... does the Tomcat console or logs provide
>>
>>more information?
>>
>>>Some other things to check are JAVA_HOME and
>>>CLASSPATH. I believe nutch requires JDK 1.4. The
>>>CLASSPATH should probably be handled correctly by
>>
>>the
>>
>>>tomcat startup script. You can modify the
>>
>>catalina.sh to
>>
>>>print out the environment variables that it's
>>
>>using to check
>>
>>>if anything funny is going on.
>>>
>>>If you still can't figure things out, try playing
>>
>>with
>>
>>>strace/trace/truss to see if you're having missing
>>
>>file problems.
>>
>>>Do a 'man' for info on how to use each. You should
>>
>>be able
>>
>>>to attach them to running processes.
>>>
>>>Howie
>>>
>>>
>>>
>>>
>>>
>>
> -------------------------------------------------------
>
>>>SF.Net email is sponsored by: Discover Easy Linux
>>
>>Migration Strategies
>>
>>>from IBM. Find simple to follow Roadmaps,
>>
>>straightforward articles,
>>
>>>informative Webcasts and more! Get everything you
>>
>>need to get up to
>>
>>>speed, fast.
>>
> http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
>
>>>_______________________________________________
>>>Nutch-developers mailing list
>>>Nutch-developers@lists.sourceforge.net
>>>
>>
> https://lists.sourceforge.net/lists/listinfo/nutch-developers
>
>>>
>>
>
>
>
>
>
> __________________________________
> Do you Yahoo!?
> Yahoo! Mail - You care about security. So do we.
> http://promotions.yahoo.com/new_mail
>
Re: [Nutch-dev] turn on Log
Posted by Hasan Diwan <ha...@gmail.com>.
Michael:
On Aug 14, 2005, at 3:54 PM, Michael Ji wrote:
> I saw several Log to debug the code, such as
> LOG.fine and LOG.warning;
>
> I wonder if the debugging text is output a particular
> log file. If so, which command that I could use to
> turn it on?
Though nutch uses the Jakarta commons logging system, configuration
is handled by the underlying logging system. On JDK 1.4,
java.util.logging is said logging system. A sample configuration
file is provided in $JAVA_HOME/jre/lib/logging.properties.
Cheers,
Hasan Diwan <ha...@gmail.com>
turn on Log
Posted by Michael Ji <fj...@yahoo.com>.
hi there,
I saw several Log to debug the code, such as
LOG.fine and LOG.warning;
I wonder if the debugging text is output a particular
log file. If so, which command that I could use to
turn it on?
thanks,
Michael Ji
__________________________________
Yahoo! Mail
Stay connected, organized, and protected. Take the tour:
http://tour.mail.yahoo.com/mailtour.html
Re: Nutch Compiling
Posted by Piotr Kosiorowski <pk...@gmail.com>.
Hello,
Nutch is using ant for compilation. You have to install it and run
"ant tar" in nutch directory containing build.xml file - to generate tar
file with full distribution.
Regards
Piotr
Feng (Michael) Ji wrote:
> hi there,
>
> I installed subversion in my linux box and checkout
> source code from http://svn.apache.org/repos/asf/.
>
> Any instruction I can follow to compile the source
> code? I saw MakeFile.PL in the repository I
> downloaded. Is it the makefile I need to use?
>
> I guess there must be some instructions in nutch
> website, but I didn't find yet.
>
> thanks,
>
> Michael,
>
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>
Nutch Compiling
Posted by "Feng (Michael) Ji" <fj...@yahoo.com>.
hi there,
I installed subversion in my linux box and checkout
source code from http://svn.apache.org/repos/asf/.
Any instruction I can follow to compile the source
code? I saw MakeFile.PL in the repository I
downloaded. Is it the makefile I need to use?
I guess there must be some instructions in nutch
website, but I didn't find yet.
thanks,
Michael,
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Re: [Nutch-dev] Re: a silly question
Posted by "Feng (Michael) Ji" <fj...@yahoo.com>.
thanks all the suggestions,
I caught the catalina.out as following:
"Jul 16, 2005 3:09:27 PM
org.apache.coyote.http11.Http11Protocol init
INFO: Initializing Coyote HTTP/1.1 on http-8888
Starting service Tomcat-Standalone
Apache Tomcat/4.1.31
Jul 16, 2005 3:09:29 PM
org.apache.struts.util.PropertyMessageResources <init>
INFO: Initializing,
config='org.apache.struts.util.LocalStrings',
returnNull=true
Jul 16, 2005 3:09:29 PM
org.apache.struts.util.PropertyMessageResources <init>
INFO: Initializing,
config='org.apache.struts.action.ActionResources',
returnNull=true
Jul 16, 2005 3:09:29 PM
org.apache.struts.util.PropertyMessageResources <init>
INFO: Initializing,
config='org.apache.webapp.admin.ApplicationResources',
returnNull=true
Jul 16, 2005 3:09:35 PM
org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-8888
Jul 16, 2005 3:09:35 PM
org.apache.jk.common.ChannelSocket init
INFO: JK2: ajp13 listening on /0.0.0.0:8009
Jul 16, 2005 3:09:35 PM org.apache.jk.server.JkMain
start
INFO: Jk running ID=0 time=0/81
config=/home/fji/SE/tomcat4/conf/jk2.properties
050716 150937 parsing
file:/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/nutch-default.xml
050716 150937 parsing
file:/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/nutch-site.xml
050716 150937 Plugins: looking in:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins
050716 150937 not including:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/clustering-carrot2
050716 150937 not including:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/creativecommons
050716 150937 parsing:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/index-basic/plugin.xml
050716 150937 impl:
point=org.apache.nutch.indexer.IndexingFilter
class=org.apache.nutch.indexer.basic.BasicIndexingFilter
050716 150937 not including:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/index-more
050716 150937 not including:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/language-identifier
050716 150937 not including:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/ontology
050716 150937 not including:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/parse-ext
050716 150937 parsing:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/parse-html/plugin.xml
050716 150937 impl:
point=org.apache.nutch.parse.Parser
class=org.apache.nutch.parse.html.HtmlParser
050716 150937 parsing:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/parse-js/plugin.xml
050716 150937 impl:
point=org.apache.nutch.parse.Parser
class=org.apache.nutch.parse.js.JSParseFilter
050716 150937 impl:
point=org.apache.nutch.parse.HtmlParseFilter
class=org.apache.nutch.parse.js.JSParseFilter
050716 150937 not including:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/parse-msword
050716 150937 not including:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/parse-pdf
050716 150937 parsing:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/parse-text/plugin.xml
050716 150937 impl:
point=org.apache.nutch.parse.Parser
class=org.apache.nutch.parse.text.TextParser
050716 150937 not including:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/protocol-file
050716 150937 not including:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/protocol-ftp
050716 150937 not including:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/protocol-http
050716 150937 parsing:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/protocol-httpclient/plugin.xml
050716 150937 impl:
point=org.apache.nutch.protocol.Protocol
class=org.apache.nutch.protocol.httpclient.Http
050716 150937 impl:
point=org.apache.nutch.protocol.Protocol
class=org.apache.nutch.protocol.httpclient.Http
050716 150937 parsing:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/query-basic/plugin.xml
050716 150937 impl:
point=org.apache.nutch.searcher.QueryFilter
class=org.apache.nutch.searcher.basic.BasicQueryFilter
050716 150937 not including:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/query-more
050716 150937 parsing:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/query-site/plugin.xml
050716 150937 impl:
point=org.apache.nutch.searcher.QueryFilter
class=org.apache.nutch.searcher.site.SiteQueryFilter
050716 150937 parsing:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/query-url/plugin.xml
050716 150937 impl:
point=org.apache.nutch.searcher.QueryFilter
class=org.apache.nutch.searcher.url.URLQueryFilter
050716 150937 not including:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/urlfilter-prefix
050716 150937 parsing:
/home/fji/SE/tomcat4/webapps/ROOT/WEB-INF/classes/plugins/urlfilter-regex/plugin.xml
050716 150937 impl:
point=org.apache.nutch.net.URLFilter
class=org.apache.nutch.net.RegexURLFilter
050716 150937 11 creating new bean
050716 150937 11 opening segment indexes in
/home/fji/SE/tomcat4/segments
"
I didn't see any complain, or I miss something
important?
The browser gives me the same error message as before.
Michael,
--- yoursoft <yo...@freemail.hu> wrote:
> I think please check your log/catalina.out for more
> details of error.
>
> Howie Wang wrotte:
>
> >> hi howie:
> >>
> >> For search.dir value of both nutch-default.xml
> and
> >> nutch-site.xml;
> >
> >
> >>
>
<value>/home/fji/SE/nutch-nightly/crawl.test/</value>
> >
> >
> > This should be the correct one.
> >
> >> Still have the same error message after click
> search
> >> submit buttom in Nutch demo page
> >>
> >> "
> >> HTTP Status 500
> >> org.apache.jasper.JasperException at
> >>
>
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:207)
>
> >>
> >> ....
> >> "
> >
> >
> > Hmm... does the Tomcat console or logs provide
> more information?
> >
> > Some other things to check are JAVA_HOME and
> > CLASSPATH. I believe nutch requires JDK 1.4. The
> > CLASSPATH should probably be handled correctly by
> the
> > tomcat startup script. You can modify the
> catalina.sh to
> > print out the environment variables that it's
> using to check
> > if anything funny is going on.
> >
> > If you still can't figure things out, try playing
> with
> > strace/trace/truss to see if you're having missing
> file problems.
> > Do a 'man' for info on how to use each. You should
> be able
> > to attach them to running processes.
> >
> > Howie
> >
> >
> >
> >
> >
>
-------------------------------------------------------
> > SF.Net email is sponsored by: Discover Easy Linux
> Migration Strategies
> > from IBM. Find simple to follow Roadmaps,
> straightforward articles,
> > informative Webcasts and more! Get everything you
> need to get up to
> > speed, fast.
>
http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
> > _______________________________________________
> > Nutch-developers mailing list
> > Nutch-developers@lists.sourceforge.net
> >
>
https://lists.sourceforge.net/lists/listinfo/nutch-developers
> >
> >
>
>
__________________________________
Do you Yahoo!?
Yahoo! Mail - You care about security. So do we.
http://promotions.yahoo.com/new_mail
Re: [Nutch-dev] Re: a silly question
Posted by yoursoft <yo...@freemail.hu>.
I think please check your log/catalina.out for more details of error.
Howie Wang wrotte:
>> hi howie:
>>
>> For search.dir value of both nutch-default.xml and
>> nutch-site.xml;
>
>
>> <value>/home/fji/SE/nutch-nightly/crawl.test/</value>
>
>
> This should be the correct one.
>
>> Still have the same error message after click search
>> submit buttom in Nutch demo page
>>
>> "
>> HTTP Status 500
>> org.apache.jasper.JasperException at
>> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:207)
>>
>> ....
>> "
>
>
> Hmm... does the Tomcat console or logs provide more information?
>
> Some other things to check are JAVA_HOME and
> CLASSPATH. I believe nutch requires JDK 1.4. The
> CLASSPATH should probably be handled correctly by the
> tomcat startup script. You can modify the catalina.sh to
> print out the environment variables that it's using to check
> if anything funny is going on.
>
> If you still can't figure things out, try playing with
> strace/trace/truss to see if you're having missing file problems.
> Do a 'man' for info on how to use each. You should be able
> to attach them to running processes.
>
> Howie
>
>
>
>
> -------------------------------------------------------
> SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
> from IBM. Find simple to follow Roadmaps, straightforward articles,
> informative Webcasts and more! Get everything you need to get up to
> speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
> _______________________________________________
> Nutch-developers mailing list
> Nutch-developers@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nutch-developers
>
>
Re: a silly question
Posted by Howie Wang <ho...@hotmail.com>.
>hi howie:
>
>For search.dir value of both nutch-default.xml and
>nutch-site.xml;
><value>/home/fji/SE/nutch-nightly/crawl.test/</value>
This should be the correct one.
>Still have the same error message after click search
>submit buttom in Nutch demo page
>
>"
>HTTP Status 500
>org.apache.jasper.JasperException at
>org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:207)
>....
>"
Hmm... does the Tomcat console or logs provide more information?
Some other things to check are JAVA_HOME and
CLASSPATH. I believe nutch requires JDK 1.4. The
CLASSPATH should probably be handled correctly by the
tomcat startup script. You can modify the catalina.sh to
print out the environment variables that it's using to check
if anything funny is going on.
If you still can't figure things out, try playing with
strace/trace/truss to see if you're having missing file problems.
Do a 'man' for info on how to use each. You should be able
to attach them to running processes.
Howie
Re: a silly question
Posted by "Feng (Michael) Ji" <fj...@yahoo.com>.
hi howie:
For search.dir value of both nutch-default.xml and
nutch-site.xml;
I try all the combination for index dir as:
<value>/home/fji/SE/nutch-nightly/crawl.test/index/</value>
or
<value>/home/fji/SE/nutch-nightly/crawl.test/db/</value>
or
<value>/home/fji/SE/nutch-nightly/crawl.test/segments/</value>
or
<value>/home/fji/SE/nutch-nightly/crawl.test/</value>
Still have the same error message after click search
submit buttom in Nutch demo page
"
HTTP Status 500
org.apache.jasper.JasperException at
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:207)
....
"
Where can I narrow down my error?
thanks,
Michael,
--- Howie Wang <ho...@hotmail.com> wrote:
> >I found search engine actually running
> "search.jsp".
> >Where is this JSP? In nutch somewhere I guess.
>
> search.jsp should be under webapps/ROOT/
>
> I'm guessing it found it, because I think the error
> message
> would be 404 not 500.
>
> Just a hunch, but did you go through and edit the
> nutch-default.xml and nutch-site.xml. I had to edit
> nutch-site.xml and change the directory to point to
> my database:
>
> <property>
> <name>searcher.dir</name>
> <value>c:/my_nutch_dir/my_nutch_db</value>
> <description>
> Path to root of index directories. This directory
> is searched (in
> order) for either the file search-servers.txt,
> containing a list of
> distributed search servers, or the directory
> "index" containing
> merged indexes, or the directory "segments"
> containing segment
> indexes.
> </description>
> </property>
>
>
>
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Re: a silly question
Posted by Howie Wang <ho...@hotmail.com>.
>>I found search engine actually running "search.jsp".
>>Where is this JSP? In nutch somewhere I guess.
By the way, I find that using strace/trace/truss for unix
platforms very useful in tracking down permissions/file
location problems. On Windows, you would use FileMon
from SysInternals.
Howie
Re: a silly question
Posted by Howie Wang <ho...@hotmail.com>.
>I found search engine actually running "search.jsp".
>Where is this JSP? In nutch somewhere I guess.
search.jsp should be under webapps/ROOT/
I'm guessing it found it, because I think the error message
would be 404 not 500.
Just a hunch, but did you go through and edit the
nutch-default.xml and nutch-site.xml. I had to edit
nutch-site.xml and change the directory to point to
my database:
<property>
<name>searcher.dir</name>
<value>c:/my_nutch_dir/my_nutch_db</value>
<description>
Path to root of index directories. This directory is searched (in
order) for either the file search-servers.txt, containing a list of
distributed search servers, or the directory "index" containing
merged indexes, or the directory "segments" containing segment
indexes.
</description>
</property>
Re: a silly question
Posted by "Feng (Michael) Ji" <fj...@yahoo.com>.
hi there:
Actually, this is first time I setup Nutch;
1) I crawl a website in nutch directory
2) delete ROOT in tomcat and copy Nutch*.war to
tomcat/webapp
3) I can run tomcat successfully before and after copy
*.war file; by means I can see default tomcat homepage
and nutch search page after.
4) I change file permission to 777 by using chmod
command for both nutch and tomcat folder. Any
particular file and fold I need to take care
permission issue?
Then, when I type in a text in search text box, and
hit search button, it gives me "http status 500 ....."
error message. Some more suggestons?
I found search engine actually running "search.jsp".
Where is this JSP? In nutch somewhere I guess.
thanks,
Michale,
--- Fredrik Andersson <fi...@gmail.com>
wrote:
> Have you deleted your old nutch.war (and the
> directory named nutch) in
> the Tomcat webapps directory, prior to inserting a
> new war-file?
> Tomcat can act pretty strange if you don delete your
> old x.war and the
> webapps/x dir.
>
> Using port 8080? Permissions on the .jsp and
> war-file ok? Can you
> access the standard Tomcat site by browsing to
> localhost:8080
> (Congratulations, you have successfully installed
> Tomcat blabla...)?
>
> On 7/16/05, Feng (Michael) Ji <fj...@yahoo.com>
> wrote:
> >
> > Hi there,
> >
> > I know this questions might be related to nutch's
> user
> > group, but if any one could help me a bit, I will
> > really appreciate it.
> >
> > --------------------------------------------------
> > I try to run Nutch in my Linux server and followed
> > each step in the tutorial of
> > http://lucene.apache.org/nutch/tutorial.html
> >
> > I can run Nutch successful, such as, crawling and
> save
> > db to my local server.
> >
> > I can launch tomcat web page with Nutch search
> home
> > page successful, but after I hit search button,
> tomcat
> > give me
> > "http status 500 error..."
> >
> > Looks like JSP (java) doesn't compile properly in
> > Tomcat container.
> >
> > I think it is not Tomcat problem, but somehow,
> Nutch
> > doesn't compile well, or maybe some nutch library
> > isn't being accesses properly.
> >
> > Anyone has the similar experience? Any suggestion
> will
> > be very helpful,
> >
> > thanks ahead,
> >
> > Michael,
> >
> >
> > __________________________________________________
> > Do You Yahoo!?
> > Tired of spam? Yahoo! Mail has the best spam
> protection around
> > http://mail.yahoo.com
> >
>
____________________________________________________
Start your day with Yahoo! - make it your home page
http://www.yahoo.com/r/hs
Re: a silly question
Posted by Fredrik Andersson <fi...@gmail.com>.
Have you deleted your old nutch.war (and the directory named nutch) in
the Tomcat webapps directory, prior to inserting a new war-file?
Tomcat can act pretty strange if you don delete your old x.war and the
webapps/x dir.
Using port 8080? Permissions on the .jsp and war-file ok? Can you
access the standard Tomcat site by browsing to localhost:8080
(Congratulations, you have successfully installed Tomcat blabla...)?
On 7/16/05, Feng (Michael) Ji <fj...@yahoo.com> wrote:
>
> Hi there,
>
> I know this questions might be related to nutch's user
> group, but if any one could help me a bit, I will
> really appreciate it.
>
> --------------------------------------------------
> I try to run Nutch in my Linux server and followed
> each step in the tutorial of
> http://lucene.apache.org/nutch/tutorial.html
>
> I can run Nutch successful, such as, crawling and save
> db to my local server.
>
> I can launch tomcat web page with Nutch search home
> page successful, but after I hit search button, tomcat
> give me
> "http status 500 error..."
>
> Looks like JSP (java) doesn't compile properly in
> Tomcat container.
>
> I think it is not Tomcat problem, but somehow, Nutch
> doesn't compile well, or maybe some nutch library
> isn't being accesses properly.
>
> Anyone has the similar experience? Any suggestion will
> be very helpful,
>
> thanks ahead,
>
> Michael,
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>
a silly question
Posted by "Feng (Michael) Ji" <fj...@yahoo.com>.
Hi there,
I know this questions might be related to nutch's user
group, but if any one could help me a bit, I will
really appreciate it.
--------------------------------------------------
I try to run Nutch in my Linux server and followed
each step in the tutorial of
http://lucene.apache.org/nutch/tutorial.html
I can run Nutch successful, such as, crawling and save
db to my local server.
I can launch tomcat web page with Nutch search home
page successful, but after I hit search button, tomcat
give me
"http status 500 error..."
Looks like JSP (java) doesn't compile properly in
Tomcat container.
I think it is not Tomcat problem, but somehow, Nutch
doesn't compile well, or maybe some nutch library
isn't being accesses properly.
Anyone has the similar experience? Any suggestion will
be very helpful,
thanks ahead,
Michael,
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
[jira] Updated: (NUTCH-72) Query basic filter with correction feature
Posted by "Christophe Noel (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/NUTCH-72?page=all ]
Christophe Noel updated NUTCH-72:
---------------------------------
Attachment: querycorrectionplugin.zip
Here is the CVS diff patch.
I can send the search.jsp if requested.
> Query basic filter with correction feature
> ------------------------------------------
>
> Key: NUTCH-72
> URL: http://issues.apache.org/jira/browse/NUTCH-72
> Project: Nutch
> Type: New Feature
> Components: searcher
> Environment: lucene
> Reporter: Christophe Noel
> Attachments: querycorrectionplugin.zip
>
> This plugin improves query-basic plugin with a correction feature.
> Lucene includes FuzzyQuery feature which consists of searching not only for matching terms, but searching for very similar terms too.
> This plugin should be used instead of query-basic, for people looking for an easy solution about users query requests correction.
> Correction Query Plugin can be used as follows :
> Solution 1 : If you want to search for very similar terms, add autocorrectionmod as the first term of the query (example : 'nutch engine' -> 'autocorrectionmod nutch engine')
> Solution 2 : Create a new search.jsp page which include a "correction" checkbox management (<input type="checkbox" name="autocorrection" value="true"> may automatically add 'autocorrectionmod' as the first term of the query)
> QueryFuzzy knows a big problem : it is very slow for large index !
> So Correction Query Plugin works as follows :
> - it is not useful for big indexes
> - it only works for 5 characters and more words
> - it only look for words matching with the 2 first characters (to improve performance this should be set to 3/4)
> - it only works for 65 % matching suffixes (algorithm is levenstein)
> PLease give your opinion about it.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira