You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by Fredrik Andersson <fi...@gmail.com> on 2005/07/28 14:28:59 UTC
0.7-dev, the search scoring
Hey guys!
I just ported a lot of old 0.6 code to 0.7-dev/mapred. Lots of stuff
has changed I see! One thing I can't quite grasp though, is why the
Hit.getScore() has been removed in favour for the TopDocs-thingie
instead? I wrote a quick add-on to support getting the score straight
from the hit, which worked fine, but it would be nice to hear a reason
as to why the method was removed in the first place!
Also, are there any "secret" WIKIs, mailinglists, forums or similar
for the 0.7 development? Would be very interesting to see what's
cooking!
Greetings,
Fredrik
Re: recursion: see recursion
Posted by yoursoft <yo...@freemail.hu>.
Insert in top of the regex-urlfilter.txt:
-http://.*(/.+?)/.*?\1/.*?\1.*?/
EM wrotte:
>What to do when encountering sites where nutch falls into recursion mode?
>
>Currently I'm solving this by removing these sites with the regex filter,
>but, is anything under development currently?
>
>By recursion I mean nutch fetching
><sfdsdf>.com/<sth>/<sth>/<sth>/<sth>/<sth>/<sth> and on and on....
>
>Any tricks to limis the folder depth in the fetch mode?
>
>E.
>
>
>
>
>
recursion: see recursion
Posted by EM <em...@cpuedge.com>.
What to do when encountering sites where nutch falls into recursion mode?
Currently I'm solving this by removing these sites with the regex filter,
but, is anything under development currently?
By recursion I mean nutch fetching
<sfdsdf>.com/<sth>/<sth>/<sth>/<sth>/<sth>/<sth> and on and on....
Any tricks to limis the folder depth in the fetch mode?
E.
Re: 0.7-dev, the search scoring
Posted by Fredrik Andersson <fi...@gmail.com>.
Ah God, I am stupid ... thanks for that, Doug! I must have a bad
coding day today : )
Fredrik
On 7/28/05, Doug Cutting <cu...@nutch.org> wrote:
> Fredrik Andersson wrote:
> > I just ported a lot of old 0.6 code to 0.7-dev/mapred. Lots of stuff
> > has changed I see! One thing I can't quite grasp though, is why the
> > Hit.getScore() has been removed in favour for the TopDocs-thingie
> > instead?
>
> Hit.getScore() was generalized to Hit.getSortValue() in order to support
> sorting results by things other than score. If you sort by score, as is
> the default, then ((FloatWritable)Hit.getSortValue()).get() is the
> score. But if you sort by, e.g., a date string, then
> ((UTF8)Hit.getSortValue()).toString() is the date string sorted on, and
> the score is unavailable. Perhaps the score should be made available
> regardless?
>
> Doug
>
Re: 0.7-dev, the search scoring
Posted by Doug Cutting <cu...@nutch.org>.
Fredrik Andersson wrote:
> I just ported a lot of old 0.6 code to 0.7-dev/mapred. Lots of stuff
> has changed I see! One thing I can't quite grasp though, is why the
> Hit.getScore() has been removed in favour for the TopDocs-thingie
> instead?
Hit.getScore() was generalized to Hit.getSortValue() in order to support
sorting results by things other than score. If you sort by score, as is
the default, then ((FloatWritable)Hit.getSortValue()).get() is the
score. But if you sort by, e.g., a date string, then
((UTF8)Hit.getSortValue()).toString() is the date string sorted on, and
the score is unavailable. Perhaps the score should be made available
regardless?
Doug
Re: 0.7-dev, the search scoring
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On Jul 28, 2005, at 8:28 AM, Fredrik Andersson wrote:
> Also, are there any "secret" WIKIs, mailinglists, forums or similar
> for the 0.7 development?
No. All discussions are in the open right here on nutch-dev and
nutch-user.
Erik