You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Pierre-Yves LANDRON <pl...@hotmail.com> on 2007/02/23 15:54:52 UTC

index browsing with solr

Hello everybody,

I'm new to this mailing list, so excuse me if my question has already been 
debated here (I've searched on the web and found nothing about it).

I've used solr for two weeks now, and so far it's a really neat solution. 
I've replaced my previous index searcher app by solr in my current project, 
but can not find a way to substitute the browseIndex(field, startterm, 
numberoftermsretuened) function i've used so far. It's a very usefull method 
and I'm sure it can be accomplished with solr, but I can't figure how.
Lucene does it throught the terms method from the class IndexReader, I think 
:
abstract  TermEnum	terms(Term t) : Returns an enumeration of all terms after 
a given term.

Does an implementation of this method exists in solr ?

If not, is it difficult to develop new instructions for solr ? where I must 
start to do so ?

Thanks !
Pierre-Yves Landron

_________________________________________________________________
FREE pop-up blocking with the new MSN Toolbar - get it now! 
http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/


Re: index browsing with solr

Posted by Pierre-Yves LANDRON <pl...@hotmail.com>.
>>Lucene does it throught the terms method from the class IndexReader, I 
>>think
>>:
>>abstract  TermEnum      terms(Term t) : Returns an enumeration of all 
>>terms after
>>a given term.
>>
>>Does an implementation of this method exists in solr ?
>
>You can get this functionality from the current faceting implementation,
>the downside is that it will be slower.
>
>-Yonik

ok. it seems great. time for me to look at faceted search more closely. Have 
you an idea of how much slower it will be ? does the implementation of an 
indexbrowsing method worth it (it will take me to learn how to integrate the 
new java method to the solr package - and I'm not a coding freak :(( )
anyway, at least i know now i can make it the easy way using faceted search, 
if necessary. thanks !

Pierre-Yves Landron

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/


Re: index browsing with solr

Posted by Yonik Seeley <yo...@apache.org>.
On 2/23/07, Pierre-Yves LANDRON <pl...@hotmail.com> wrote:
> I've used solr for two weeks now, and so far it's a really neat solution.
> I've replaced my previous index searcher app by solr in my current project,
> but can not find a way to substitute the browseIndex(field, startterm,
> numberoftermsretuened) function i've used so far. It's a very usefull method
> and I'm sure it can be accomplished with solr, but I can't figure how.
> Lucene does it throught the terms method from the class IndexReader, I think
> :
> abstract  TermEnum      terms(Term t) : Returns an enumeration of all terms after
> a given term.
>
> Does an implementation of this method exists in solr ?

You can get this functionality from the current faceting implementation,
the downside is that it will be slower.

-Yonik

Re: index browsing with solr

Posted by Ryan McKinley <ry...@gmail.com>.
>
> I'm not sure I understand...
> Ryan, how can I have access to your contribution ? (is it a contribution to
> Luke or to the solr rest interface ?)
> Is this implemented yet, and if so, how can I use it ?
> Thanks.
>

Solr has a pluggable request handling framework that lets you easily
write custom logic and takes care of the xml/json/etc writing for you.
 Check:

http://wiki.apache.org/solr/SolrPlugins#head-7c0d03515c496017f6c0116ebb096e34a872cb61
http://wiki.apache.org/solr/SolrRequestHandler

Since the exact term browsing mechanism you asked for is not
supported, I suggested writing your own and looking to the
IndexInfoRequestHandler as a simple starting place.

After more thought (and Yonik pointing it out), you are probably best
off if you can use faced browsing to do what you need.
http://wiki.apache.org/solr/SolrFacetingOverview

The 'luke' comments are unrelated to your direct question, but you can
check that out here: http://issues.apache.org/jira/browse/SOLR-162

ryan

Re: No segments file after optimizing using Luke

Posted by Yonik Seeley <yo...@apache.org>.
On 4/11/07, Michael Levy <Lu...@gmail.com> wrote:
> After using Luke v0.7 to Optimize Index, with either "Using Standard
> Format" or "Using Compound Format", and restarting Solr using jetty, I
> get a java.io.FileNotFoundException indicating that it can't find the
> segments file.
>
> There's no segments file in the index directory, but Luke optimization
> created files named segments_n (n being 3 for standard format and 6 for
> compound format) and segments.gen.
>
> How can I resolve this?  Thanks in advance!

If you are going to use another program to modify a Solr-Lucene index,
make sure that program is using the same version of Lucene that Solr
is :-)

The very latest version of Luke uses Lucene 2.1, which is later than
Solr1.1 uses.
Try using a nightly build of Solr.

-Yonik

No segments file after optimizing using Luke

Posted by Michael Levy <Lu...@gmail.com>.
After using Luke v0.7 to Optimize Index, with either "Using Standard 
Format" or "Using Compound Format", and restarting Solr using jetty, I 
get a java.io.FileNotFoundException indicating that it can't find the 
segments file.

There's no segments file in the index directory, but Luke optimization 
created files named segments_n (n being 3 for standard format and 6 for 
compound format) and segments.gen.

How can I resolve this?  Thanks in advance!







Re: how to interpret the Solr score ?

Posted by Yonik Seeley <yo...@apache.org>.
On 3/1/07, Pierre-Yves LANDRON <pl...@hotmail.com> wrote:
> I haven't seen that maxScore you're speaking about. It's exactly what I
> needed.

It's an attribute in <result> when you elect to return scores.

<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader">
 <int name="status">0</int>
 <int name="QTime">0</int>
 <lst name="params">
  <str name="fl">score,*</str>
  <str name="q">ipod</str>
  <str name="indent">on</str>
 </lst>
</lst>
<result name="response" numFound="3" start="0" maxScore="2.4851787">
 <doc>
  <float name="score">2.4851787</float>
  <arr name="cat"><str>electronics</str><str>connector</str></arr>
  <arr name="features"><str>car power adapter for iPod, white</str></arr>
  <str name="id">IW-02</str>
  <bool name="inStock">false</bool>
  <str name="manu">Belkin</str>
  <str name="name">iPod &amp; iPod Mini USB 2.0 Cable</str>
  <int name="popularity">1</int>
  <float name="price">11.5</float>
  <str name="sku">IW-02</str>
  <date name="timestamp">2007-01-31T05:12:44.484Z</date>
  <float name="weight">2.0</float>
 </doc>
 <doc>
[...]

Re: how to interpret the Solr score ?

Posted by Pierre-Yves LANDRON <pl...@hotmail.com>.
Thank you Yonik,

I haven't seen that maxScore you're speaking about. It's exactly what I 
needed.

Cheers,
P-Y L


>From: "Yonik Seeley" <yo...@apache.org>
>Reply-To: solr-user@lucene.apache.org
>To: solr-user@lucene.apache.org
>Subject: Re: how to interpret the Solr score ?
>Date: Wed, 28 Feb 2007 12:31:34 -0500
>
>On 2/28/07, Pierre-Yves LANDRON <pl...@hotmail.com> wrote:
>>A silly question, but I think the subject isn't cover on the Solr wiki :
>>I'd like to use the score returned by Solr to give the user an estimation 
>>of
>>the response adequacy to his request (ok, that's what the score is 
>>for...).
>>But I cannot figure the meaning of this score : is it a precentage of the
>>response relevancy ? is it absolute, or relative to others given response 
>>?
>>Is it the raw Lucene score (perhaps I'm wrong but it doesn't seems it is 
>>in
>>my case) ? Is there a way to obtain a meaningful score ( ie: human 
>>readable
>>) ?
>
>It's the raw lucene score.  Some lucene functions normalize the score
>by dividing all scores by the top score in matching documents, if that
>top score is greater than 1.  Solr also returns maxScore so you can do
>the same if you desire.
>
>-Yonik

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/


Re: how to interpret the Solr score ?

Posted by Yonik Seeley <yo...@apache.org>.
On 2/28/07, Pierre-Yves LANDRON <pl...@hotmail.com> wrote:
> A silly question, but I think the subject isn't cover on the Solr wiki :
> I'd like to use the score returned by Solr to give the user an estimation of
> the response adequacy to his request (ok, that's what the score is for...).
> But I cannot figure the meaning of this score : is it a precentage of the
> response relevancy ? is it absolute, or relative to others given response ?
> Is it the raw Lucene score (perhaps I'm wrong but it doesn't seems it is in
> my case) ? Is there a way to obtain a meaningful score ( ie: human readable
> ) ?

It's the raw lucene score.  Some lucene functions normalize the score
by dividing all scores by the top score in matching documents, if that
top score is greater than 1.  Solr also returns maxScore so you can do
the same if you desire.

-Yonik

how to interpret the Solr score ?

Posted by Pierre-Yves LANDRON <pl...@hotmail.com>.
Hello,

A silly question, but I think the subject isn't cover on the Solr wiki :
I'd like to use the score returned by Solr to give the user an estimation of 
the response adequacy to his request (ok, that's what the score is for...). 
But I cannot figure the meaning of this score : is it a precentage of the 
response relevancy ? is it absolute, or relative to others given response ? 
Is it the raw Lucene score (perhaps I'm wrong but it doesn't seems it is in 
my case) ? Is there a way to obtain a meaningful score ( ie: human readable 
) ?

Thanks for reading !
Pierre-Yves Landron

_________________________________________________________________
Play Flexicon: the crossword game that feeds your brain. PLAY now for FREE.  
  http://zone.msn.com/en/flexicon/default.htm?icid=flexicon_hmtagline


Re: index browsing with solr

Posted by Pierre-Yves LANDRON <pl...@hotmail.com>.
>>http://localhost:8983/solr/select?qt=indexinfo&wt=ruby&indent=on
>>
>>Though IndexInfoRequestHandler is practically obsolete with Ryan's
>>"Luke" contribution... isn't that so, Ryan?
>>
>
>functionality-wise, yes.
>
>I pointed to the IndexInfoRequestHandler because it is the simplest
>SolrRequestHandler that gets into lucene internals.  Adding term
>browsing to it is really straight forward - thats how i stumbled into
>writing the "luke" thing!

I'm not sure I understand...
Ryan, how can I have access to your contribution ? (is it a contribution to 
Luke or to the solr rest interface ?)
Is this implemented yet, and if so, how can I use it ?
Thanks.

Pierre-Yves Landron

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/


Re: index browsing with solr

Posted by Ryan McKinley <ry...@gmail.com>.
On 2/24/07, Erik Hatcher <er...@ehatchersolutions.com> wrote:
>
> On Feb 24, 2007, at 6:26 AM, Erik Hatcher wrote:
>
> >
> > On Feb 24, 2007, at 3:36 AM, Pierre-Yves LANDRON wrote:
> >
> >>> it will be easy to add.  take a look at a simple SolrRequestHandler:
> >>>
> >>> http://svn.apache.org/repos/asf/lucene/solr/trunk/src/java/org/
> >>> apache/solr/handler/IndexInfoRequestHandler.java
> >>>
> >>> this gets the IndexReader and writes out some stuff.
> >>
> >> thanks ! i will look at it. This handler is not accessible from
> >> the "rest" interface ? if so, why does it exists : is it used for
> >> the implementation of other rest instruction ?
> >
> > Sure, this handler is available (as long as you have it configured
> > in your solrconfig.xml; it is configured in the example one):
> >
> >       http://localhost:8983/solr/select?qt=indexinfo&wt=ruby&indent=on
>
> Though IndexInfoRequestHandler is practically obsolete with Ryan's
> "Luke" contribution... isn't that so, Ryan?
>

functionality-wise, yes.

I pointed to the IndexInfoRequestHandler because it is the simplest
SolrRequestHandler that gets into lucene internals.  Adding term
browsing to it is really straight forward - thats how i stumbled into
writing the "luke" thing!


> I can't wait for Flare to start getting its hands on the Luke
> handlers!  wow.
>

i can't wait to see it

Re: index browsing with solr

Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On Feb 24, 2007, at 6:26 AM, Erik Hatcher wrote:

>
> On Feb 24, 2007, at 3:36 AM, Pierre-Yves LANDRON wrote:
>
>>> it will be easy to add.  take a look at a simple SolrRequestHandler:
>>>
>>> http://svn.apache.org/repos/asf/lucene/solr/trunk/src/java/org/ 
>>> apache/solr/handler/IndexInfoRequestHandler.java
>>>
>>> this gets the IndexReader and writes out some stuff.
>>
>> thanks ! i will look at it. This handler is not accessible from  
>> the "rest" interface ? if so, why does it exists : is it used for  
>> the implementation of other rest instruction ?
>
> Sure, this handler is available (as long as you have it configured  
> in your solrconfig.xml; it is configured in the example one):
>
> 	http://localhost:8983/solr/select?qt=indexinfo&wt=ruby&indent=on

Though IndexInfoRequestHandler is practically obsolete with Ryan's  
"Luke" contribution... isn't that so, Ryan?

I can't wait for Flare to start getting its hands on the Luke  
handlers!  wow.

	Erik


Re: index browsing with solr

Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On Feb 24, 2007, at 3:36 AM, Pierre-Yves LANDRON wrote:

>> it will be easy to add.  take a look at a simple SolrRequestHandler:
>>
>> http://svn.apache.org/repos/asf/lucene/solr/trunk/src/java/org/ 
>> apache/solr/handler/IndexInfoRequestHandler.java
>>
>> this gets the IndexReader and writes out some stuff.
>
> thanks ! i will look at it. This handler is not accessible from the  
> "rest" interface ? if so, why does it exists : is it used for the  
> implementation of other rest instruction ?

Sure, this handler is available (as long as you have it configured in  
your solrconfig.xml; it is configured in the example one):

	http://localhost:8983/solr/select?qt=indexinfo&wt=ruby&indent=on

Erik


Re: index browsing with solr

Posted by Pierre-Yves LANDRON <pl...@hotmail.com>.
>it will be easy to add.  take a look at a simple SolrRequestHandler:
>
>http://svn.apache.org/repos/asf/lucene/solr/trunk/src/java/org/apache/solr/handler/IndexInfoRequestHandler.java
>
>this gets the IndexReader and writes out some stuff.

thanks ! i will look at it. This handler is not accessible from the "rest" 
interface ? if so, why does it exists : is it used for the implementation of 
other rest instruction ?

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! 
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/


Re: index browsing with solr

Posted by Ryan McKinley <ry...@gmail.com>.
>
> Does an implementation of this method exists in solr ?
>

i don;t think so.


> If not, is it difficult to develop new instructions for solr ? where I must
> start to do so ?
>

it will be easy to add.  take a look at a simple SolrRequestHandler:

http://svn.apache.org/repos/asf/lucene/solr/trunk/src/java/org/apache/solr/handler/IndexInfoRequestHandler.java

this gets the IndexReader and writes out some stuff.