You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Markus Jelsma <ma...@openindex.io> on 2010/11/04 14:26:17 UTC

Boosts on NutchDocuments

Hi,

Quick question: does Nutch set document boosts on documents that i send to 
Solr? I've got some trouble with fieldNorms which are calculated from 
document/field boosts and  need to make sure Nutch doesn't boost any documents. 
Using luke it's clear that no fields receive an index-time boost but what about 
the document itself? I believe not, but need to make sure.

Cheers,

-- 
Markus Jelsma - CTO - Openindex
http://www.linkedin.com/in/markus17
050-8536600 / 06-50258350

Re: Boosts on NutchDocuments

Posted by Markus Jelsma <ma...@openindex.io>.
I see. Then there might be an issue here. I get documents with a boost of zero 
in my Solr index, messing up my fieldNorm values. I'll add some debug output to 
the indexer to see what it's actually sending for some documents.

Thanks, i know where to investigate.

> Before sending a document to solr nutch calculates document score by
> running scoring filters. After calculating total score it gets assigned to
> the NutchDocuement and which then gets sent to Solr.
> 
> So if you want influence boosting then you need to play with scoring
> filters.
> 
> Best Regards
> Alexander Aristov
> 
> On 4 November 2010 16:31, Markus Jelsma <ma...@openindex.io> wrote:
> > Well, it clearly can set a document boost depending on the weight value
> > of a
> > NutchDocument. How can i find the value? I've looked into the segment
> > reader
> > for a specific document but haven't seen a boost. When is it
> > calculated/set?
> > 
> > There's either an issue with Solr's fieldNorm handling, or Nutch sets
> > 0.0f boosts on some documents or something else it going wrnog.
> > 
> > On Thursday 04 November 2010 14:26:17 Markus Jelsma wrote:
> > > Hi,
> > > 
> > > Quick question: does Nutch set document boosts on documents that i send
> > 
> > to
> > 
> > > Solr? I've got some trouble with fieldNorms which are calculated from
> > > document/field boosts and  need to make sure Nutch doesn't boost any
> > > documents. Using luke it's clear that no fields receive an index-time
> > > boost but what about the document itself? I believe not, but need to
> > > make sure.
> > > 
> > > Cheers,
> > 
> > --
> > Markus Jelsma - CTO - Openindex
> > http://www.linkedin.com/in/markus17
> > 050-8536600 / 06-50258350

Re: Boosts on NutchDocuments

Posted by Alexander Aristov <al...@gmail.com>.
Before sending a document to solr nutch calculates document score by running
scoring filters. After calculating total score it gets assigned to the
NutchDocuement and which then gets sent to Solr.

So if you want influence boosting then you need to play with scoring
filters.

Best Regards
Alexander Aristov


On 4 November 2010 16:31, Markus Jelsma <ma...@openindex.io> wrote:

> Well, it clearly can set a document boost depending on the weight value of
> a
> NutchDocument. How can i find the value? I've looked into the segment
> reader
> for a specific document but haven't seen a boost. When is it
> calculated/set?
>
> There's either an issue with Solr's fieldNorm handling, or Nutch sets 0.0f
> boosts on some documents or something else it going wrnog.
>
> On Thursday 04 November 2010 14:26:17 Markus Jelsma wrote:
> > Hi,
> >
> > Quick question: does Nutch set document boosts on documents that i send
> to
> > Solr? I've got some trouble with fieldNorms which are calculated from
> > document/field boosts and  need to make sure Nutch doesn't boost any
> > documents. Using luke it's clear that no fields receive an index-time
> > boost but what about the document itself? I believe not, but need to make
> > sure.
> >
> > Cheers,
>
> --
> Markus Jelsma - CTO - Openindex
> http://www.linkedin.com/in/markus17
> 050-8536600 / 06-50258350
>

Re: Boosts on NutchDocuments

Posted by Markus Jelsma <ma...@openindex.io>.
Well, it clearly can set a document boost depending on the weight value of a 
NutchDocument. How can i find the value? I've looked into the segment reader 
for a specific document but haven't seen a boost. When is it calculated/set?

There's either an issue with Solr's fieldNorm handling, or Nutch sets 0.0f 
boosts on some documents or something else it going wrnog.

On Thursday 04 November 2010 14:26:17 Markus Jelsma wrote:
> Hi,
> 
> Quick question: does Nutch set document boosts on documents that i send to
> Solr? I've got some trouble with fieldNorms which are calculated from
> document/field boosts and  need to make sure Nutch doesn't boost any
> documents. Using luke it's clear that no fields receive an index-time
> boost but what about the document itself? I believe not, but need to make
> sure.
> 
> Cheers,

-- 
Markus Jelsma - CTO - Openindex
http://www.linkedin.com/in/markus17
050-8536600 / 06-50258350