You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Mark Bennett <ma...@lucidworks.com> on 2013/03/04 22:52:47 UTC
Re: [ANN] vifun: tool to help visually tweak Solr boosting
Hello Xavier,
Thanks for uploading this and sharing. I also read the other messages in the thread.
I'm able to get part way through your Getting Started section, I get results, but I get stuck on the editing values. I've tried with Java 6 and 7, with both the 0.5 binary and from the source distribution.
What's working:
* Default Solr 4.1 install (plus a couple extra fields in schema)
* Able to connect to Solr (/collection1)
* Able to select handler (/select)
* Able to run a search:
q=bandwidth
rows=10
fl=title
rest: pt=45.15,-93.85 (per your example)
* Get 2 search results with titles
* Able to select a result, mouse over, highlight score, etc.
However, what I'm stuck on:
* Below the Run Query button, I only see the grayed out Scoring slider.
* The instructions say to highlight some numbers
- I tried highlighting the 10 in rows paramour
- I also tried the 45.15 in "rest", and some of the scores in the results list
I never see the extra parameters you show in this screen shot:
https://raw.github.com/jmlucjav/vifun/master/img/screenshot-selecttarget.jpg
I see the word "Scoring:"
I don't see the blue text "Select a number as a target to tweak"
I don't see the parameters qf, bf_0, 1, 2, bq_0, etc.
I'm not sure how to get those extra fields to appear in the UI.
I also tried adding defType=edismax, no luck
The Handlers it sees:
/select, /query, /browse, /spell, /tvrh, /clustering, /terms, /elevate
(from default Solr 4.1 solrconfig.xml)
I'm using /select
--
Mark Bennett / LucidWorks: Search & Big Data / mark.bennett@lucidworks.com
Office: 408-898-4201 / Telecommute: 408-733-0387 / Cell: 408-829-6513
On Feb 23, 2013, at 6:12 AM, jmlucjav <jm...@gmail.com> wrote:
> Hi,
>
> I have built a small tool to help me tweak some params in Solr (typically
> qf, bf in edismax). As maybe others find it useful, I am open sourcing it
> on github: https://github.com/jmlucjav/vifun
>
> Check github for some more info and screenshots. I include part of the
> github page below.
> regards
>
> Description
>
> Did you ever spend lots of time trying to tweak all numbers in a *edismax*
> handler *qf*, *bf*, etc params so docs get scored to your liking? Imagine
> you have the params below, is 20 the right boosting for *name* or is it too
> much? Is *population* being boosted too much versus distance? What about
> new documents?
>
> <!-- fields, boost some -->
> <str name="qf">name^20 textsuggest^10 edge^5 ngram^2 phonetic^1</str>
> <str name="mm">33%</str>
> <!-- boost closest hits -->
> <str name="bf">recip(geodist(),1,500,0)</str>
> <!-- boost by population -->
> <str name="bf">product(log(sum(population,1)),100)</str>
> <!-- boost newest docs -->
> <str name="bf">recip(rord(moddate),1,1000,1000)</str>
>
> This tool was developed in order to help me tweak the values of boosting
> functions etc in Solr, typically when using edismax handler. If you are fed
> up of: change a number a bit, restart Solr, run the same query to see how
> documents are scored now...then this tool is for you.
> <https://github.com/jmlucjav/vifun#features>Features
>
> - Can tweak numeric values in the following params: *qf, pf, bf, bq,
> boost, mm* (others can be easily added) even in *<appends> or
> <invariants>*
> - View side by side a Baseline query result and how it changes when you
> gradually change each value in the params
> - Colorized values, color depends on how the document does related to
> baseline query
> - Tooltips give you Explain info
> - Works on remote Solr installations
> - Tested with Solr 3.6, 4.0 and 4.1 (other versions would work too, as
> long as wt=javabin format is compatible)
> - Developed using Groovy/Griffon
>
> <https://github.com/jmlucjav/vifun#requirements>Requirements
>
> - */select* handler should be available, and not have any *<appends> or
> <invariants>*, as it could interfere with how vifun works.
> - Java6 is needed (maybe it runs on Java5 too). A JRE should be enough.
>
> <https://github.com/jmlucjav/vifun#getting-started>Getting started
> <https://github.com/jmlucjav/vifun#click-here-to-download-latest-version-and-unzip>Click
> here to download latest
> version<http://code.google.com/p/vifun/downloads/detail?name=vifun-0.4.zip>
> and
> unzip
>
> - Run vifun-0.4\bin\vifun.bat or vifun-04\bin\vifun if on linux/OSX
> - Edit *Solr URL* to match yours (in Sol4.1 default is
> http://localhost:8983/solr/collection1 for example) [image: hander
> selection]<https://github.com/jmlucjav/vifun/raw/master/img/screenshot-handlers.jpg>
> - *Show Handerls*, and select the handler you wish to tweak from *
> Handerls* dropdown. The text area below shows the parameters of the
> handler.
> - Modify the values to run a baseline query:
> - *q*: query string you want to use
> - *rows*: as in Solr, don't choose a number too small, so you can see
> more documents, I typically use 500
> - *fl*: comma separated list of fields you want to show for each doc,
> keep it short (other fields needed will be added, like the id, score)
> - *rest*: in case you need to add more params, for example: sfield,
> fq etc) [image: query
> params]<https://github.com/jmlucjav/vifun/raw/master/img/screenshot-qparams.jpg>
> - *Run Query*. The two panels on the right will show the same result,
> sorted by score.[image:
> results]<https://github.com/jmlucjav/vifun/raw/master/img/screenshot-results.jpg>
> - Use the mouse to select the number you want to tweak in Score params
> (select all the digits). Note the label of the field is highlighted with
> current value. [image: target
> selection]<https://github.com/jmlucjav/vifun/raw/master/img/screenshot-selecttarget.jpg>
> - Move the slider, release and see how a new query is run, and you can
> compare how result changes with the current value. In the Current table,
> you can see current position/score and also delta relative to the baseline.
> The colour of the row reflects how much the doc gained/lost. [image:
> tweaking a value]<https://github.com/jmlucjav/vifun/raw/master/img/screenshot-baseline.jpg>
> - You can increase the limits of the slider, and if you are satisfied
> with a value, set it, so it will be set to current value.
> - Tweak another number...
> - Tooltips exist in Current (Explain info) and Baseline (pos/score of
> that doc in Current, and Explain in Score column)
Re: [ANN] vifun: tool to help visually tweak Solr boosting
Posted by xavier jmlucjav <jm...@gmail.com>.
Hi Mark,
Thanks for trying it out.
Let me see if I explain it better: the number you have to select (in order
to later being able to tweak it with the slider), is any number that must
be in one of the parameters in the Scoring section.
The issue you have, is that you are using /select handler from the example
distribution, and that handler does not have any of these parameters (qf,
pf, pf2, pf3, ps, ps2, ps3, bf, bq, boost, mm, tie), so it's normal they
don't show up, there is nothing to tweak...
In the example configuration from 4.1, you can select /browse handler, as
it uses qf and mm, and you should be able to tweak them. Of course If you
were using a real Solr installation with a sizable number of documents and
some complex usage of edismax, you would be able to see much better what
the tool can do.
xavier
On Mon, Mar 4, 2013 at 10:52 PM, Mark Bennett
<ma...@lucidworks.com>wrote:
> Hello Xavier,
>
> Thanks for uploading this and sharing. I also read the other messages in
> the thread.
>
> I'm able to get part way through your Getting Started section, I get
> results, but I get stuck on the editing values. I've tried with Java 6 and
> 7, with both the 0.5 binary and from the source distribution.
>
> What's working:
> * Default Solr 4.1 install (plus a couple extra fields in schema)
> * Able to connect to Solr (/collection1)
> * Able to select handler (/select)
> * Able to run a search:
> q=bandwidth
> rows=10
> fl=title
> rest: pt=45.15,-93.85 (per your example)
> * Get 2 search results with titles
> * Able to select a result, mouse over, highlight score, etc.
>
> However, what I'm stuck on:
> * Below the Run Query button, I only see the grayed out Scoring slider.
> * The instructions say to highlight some numbers
> - I tried highlighting the 10 in rows paramour
> - I also tried the 45.15 in "rest", and some of the scores in the
> results list
>
> I never see the extra parameters you show in this screen shot:
>
> https://raw.github.com/jmlucjav/vifun/master/img/screenshot-selecttarget.jpg
> I see the word "Scoring:"
> I don't see the blue text "Select a number as a target to tweak"
> I don't see the parameters qf, bf_0, 1, 2, bq_0, etc.
>
> I'm not sure how to get those extra fields to appear in the UI.
>
> I also tried adding defType=edismax, no luck
>
> The Handlers it sees:
> /select, /query, /browse, /spell, /tvrh, /clustering, /terms,
> /elevate
> (from default Solr 4.1 solrconfig.xml)
> I'm using /select
>
>
> --
> Mark Bennett / LucidWorks: Search & Big Data / mark.bennett@lucidworks.com
> Office: 408-898-4201 / Telecommute: 408-733-0387 / Cell: 408-829-6513
>
>
>
>
>
>
>
> On Feb 23, 2013, at 6:12 AM, jmlucjav <jm...@gmail.com> wrote:
>
> > Hi,
> >
> > I have built a small tool to help me tweak some params in Solr (typically
> > qf, bf in edismax). As maybe others find it useful, I am open sourcing it
> > on github: https://github.com/jmlucjav/vifun
> >
> > Check github for some more info and screenshots. I include part of the
> > github page below.
> > regards
> >
> > Description
> >
> > Did you ever spend lots of time trying to tweak all numbers in a
> *edismax*
> > handler *qf*, *bf*, etc params so docs get scored to your liking? Imagine
> > you have the params below, is 20 the right boosting for *name* or is it
> too
> > much? Is *population* being boosted too much versus distance? What about
> > new documents?
> >
> > <!-- fields, boost some -->
> > <str name="qf">name^20 textsuggest^10 edge^5 ngram^2
> phonetic^1</str>
> > <str name="mm">33%</str>
> > <!-- boost closest hits -->
> > <str name="bf">recip(geodist(),1,500,0)</str>
> > <!-- boost by population -->
> > <str name="bf">product(log(sum(population,1)),100)</str>
> > <!-- boost newest docs -->
> > <str name="bf">recip(rord(moddate),1,1000,1000)</str>
> >
> > This tool was developed in order to help me tweak the values of boosting
> > functions etc in Solr, typically when using edismax handler. If you are
> fed
> > up of: change a number a bit, restart Solr, run the same query to see how
> > documents are scored now...then this tool is for you.
> > <https://github.com/jmlucjav/vifun#features>Features
> >
> > - Can tweak numeric values in the following params: *qf, pf, bf, bq,
> > boost, mm* (others can be easily added) even in *<appends> or
> > <invariants>*
> > - View side by side a Baseline query result and how it changes when you
> > gradually change each value in the params
> > - Colorized values, color depends on how the document does related to
> > baseline query
> > - Tooltips give you Explain info
> > - Works on remote Solr installations
> > - Tested with Solr 3.6, 4.0 and 4.1 (other versions would work too, as
> > long as wt=javabin format is compatible)
> > - Developed using Groovy/Griffon
> >
> > <https://github.com/jmlucjav/vifun#requirements>Requirements
> >
> > - */select* handler should be available, and not have any *<appends> or
> > <invariants>*, as it could interfere with how vifun works.
> > - Java6 is needed (maybe it runs on Java5 too). A JRE should be enough.
> >
> > <https://github.com/jmlucjav/vifun#getting-started>Getting started
> > <
> https://github.com/jmlucjav/vifun#click-here-to-download-latest-version-and-unzip
> >Click
> > here to download latest
> > version<
> http://code.google.com/p/vifun/downloads/detail?name=vifun-0.4.zip>
> > and
> > unzip
> >
> > - Run vifun-0.4\bin\vifun.bat or vifun-04\bin\vifun if on linux/OSX
> > - Edit *Solr URL* to match yours (in Sol4.1 default is
> > http://localhost:8983/solr/collection1 for example) [image: hander
> > selection]<
> https://github.com/jmlucjav/vifun/raw/master/img/screenshot-handlers.jpg>
> > - *Show Handerls*, and select the handler you wish to tweak from *
> > Handerls* dropdown. The text area below shows the parameters of the
> > handler.
> > - Modify the values to run a baseline query:
> > - *q*: query string you want to use
> > - *rows*: as in Solr, don't choose a number too small, so you can
> see
> > more documents, I typically use 500
> > - *fl*: comma separated list of fields you want to show for each
> doc,
> > keep it short (other fields needed will be added, like the id,
> score)
> > - *rest*: in case you need to add more params, for example: sfield,
> > fq etc) [image: query
> > params]<
> https://github.com/jmlucjav/vifun/raw/master/img/screenshot-qparams.jpg>
> > - *Run Query*. The two panels on the right will show the same result,
> > sorted by score.[image:
> > results]<
> https://github.com/jmlucjav/vifun/raw/master/img/screenshot-results.jpg>
> > - Use the mouse to select the number you want to tweak in Score params
> > (select all the digits). Note the label of the field is highlighted
> with
> > current value. [image: target
> > selection]<
> https://github.com/jmlucjav/vifun/raw/master/img/screenshot-selecttarget.jpg
> >
> > - Move the slider, release and see how a new query is run, and you can
> > compare how result changes with the current value. In the Current
> table,
> > you can see current position/score and also delta relative to the
> baseline.
> > The colour of the row reflects how much the doc gained/lost. [image:
> > tweaking a value]<
> https://github.com/jmlucjav/vifun/raw/master/img/screenshot-baseline.jpg>
> > - You can increase the limits of the slider, and if you are satisfied
> > with a value, set it, so it will be set to current value.
> > - Tweak another number...
> > - Tooltips exist in Current (Explain info) and Baseline (pos/score of
> > that doc in Current, and Explain in Score column)
>
>