You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Alessandro Benedetti <ab...@apache.org> on 2015/12/30 14:37:29 UTC
Parameter Substitution / Macro Expansion
I was taking a look to this article from Yonik [1] .
I like the feature and actually it could useful for some use cases of mine.
Started to play with it in Solr 5.4 .
I verified it is working if you pass both the query and the parameters to
substitute as request parameters.
I verified it is working if you pass both the query and the parameters to
substitute as defaults or appends for the request handler.
Currently I am not able to make it working the following scenario :
solrconfig.xml
<requestHandler name="/getSimilar" class="solr.SearchHandler">
> <lst name="defaults">
> ...
> <str name="q">{!boost b=recip(dist(2,0,star_rating,0,
> *${star_rating:3}*),1,10,10)}{!boost b=recip(geodist(lat_lon,*${lat:0}*,
> *${lon:0}*),2,200,20)}{!mlt
> qf=name,description,facilities,resort,region,dest_level_2 mintf=1 mindf=5
> maxqt=50}*${master_id:0}*</str>
> ...
> </lst>
> </requestHandler>
Then I pass in the request the parameter to substitute , like :
master_id=1 .
Unfortunately is always getting the default one.
Am i doing something wrong ?
If it is known limitation I would be happy to contribute a fix, what are
the components involved ?
Cheers
[1] http://yonik.com/solr-query-parameter-substitution/
--
--------------------------
Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti
"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"
William Blake - Songs of Experience -1794 England
Re: Parameter Substitution / Macro Expansion
Posted by Erik Hatcher <er...@gmail.com>.
I dunno why that doesn't work but try {!mlt ... v=$master_id} and set a default value in your config for master_id and that should at least work.
Erik
> On Dec 30, 2015, at 08:37, Alessandro Benedetti <ab...@apache.org> wrote:
>
> I was taking a look to this article from Yonik [1] .
> I like the feature and actually it could useful for some use cases of mine.
> Started to play with it in Solr 5.4 .
>
> I verified it is working if you pass both the query and the parameters to
> substitute as request parameters.
>
> I verified it is working if you pass both the query and the parameters to
> substitute as defaults or appends for the request handler.
>
> Currently I am not able to make it working the following scenario :
> solrconfig.xml
>
> <requestHandler name="/getSimilar" class="solr.SearchHandler">
>> <lst name="defaults">
>> ...
>> <str name="q">{!boost b=recip(dist(2,0,star_rating,0,
>> *${star_rating:3}*),1,10,10)}{!boost b=recip(geodist(lat_lon,*${lat:0}*,
>> *${lon:0}*),2,200,20)}{!mlt
>> qf=name,description,facilities,resort,region,dest_level_2 mintf=1 mindf=5
>> maxqt=50}*${master_id:0}*</str>
>> ...
>> </lst>
>> </requestHandler>
>
>
>
> Then I pass in the request the parameter to substitute , like :
> master_id=1 .
> Unfortunately is always getting the default one.
> Am i doing something wrong ?
> If it is known limitation I would be happy to contribute a fix, what are
> the components involved ?
>
> Cheers
>
> [1] http://yonik.com/solr-query-parameter-substitution/
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card : http://about.me/alessandro_benedetti
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
Re: Parameter Substitution / Macro Expansion
Posted by Alessandro Benedetti <ab...@apache.org>.
Reference :
https://wiki.apache.org/solr/SolrConfigXml#System_property_substitution
On 30 December 2015 at 15:30, Alessandro Benedetti <ab...@apache.org>
wrote:
> mmmm debugging I confirm you are right Yonik, but also that the defaults
> are already applied at this point :
> org/apache/solr/request/json/RequestUtil.java:158
>
> Could be a resolution of properties that happens at solrconfig, xml level ?
> It verifies if any java properties is in there when running solr, and
> because of that the defaults are already applied ?
>
> Cheers
>
> On 30 December 2015 at 15:20, Alessandro Benedetti <ab...@apache.org>
> wrote:
>
>> Erick,
>> your suggestion worked :
>>
>> <requestHandler name="/getSimilar" class="solr.SearchHandler">
>> <lst name="defaults">
>> <!-- Query settings -->
>> <str name="title">Hotels Similarity</str>
>> <str name="defType">edismax</str>
>> <str name="rows">5</str>
>> <str name="fl">*,score</str>
>> <str name="fq">active:true</str>
>> <str name="echoParams">explicit</str>
>> <str name="wt">json</str>
>> <str name="q">{!boost b=recip(dist(2,0,star_rating,0,$star_rating),1,10,10)}{!boost b=recip(geodist(lat_lon,$lat,$lon),2,200,20)}{!mlt qf=name,description,facilities,resort,region,dest_level_2 mintf=1 mindf=5 maxqt=50 v=$master_id}</str>
>> <!-- Default Similarity Parameters-->
>> <str name="master_id">0</str>
>> <str name="star_rating">3</str>
>> <str name="lat">0</str>
>> <str name="lon">0</str>
>> </lst>
>> </requestHandler>
>>
>> Yonik, running debugQuery=on i verified that the default were added
>> before, so basically I can see the default replaced but the one I am
>> sending in the request param take no effect ( as actually the macro
>> disappeared , replaced by the default) .
>> Is maybe because I am inside a block with different query parsers ?
>>
>> Cheers
>>
>>
>> On 30 December 2015 at 15:14, Yonik Seeley <ys...@gmail.com> wrote:
>>
>>> On Wed, Dec 30, 2015 at 8:37 AM, Alessandro Benedetti
>>> <ab...@apache.org> wrote:
>>> > I was taking a look to this article from Yonik [1] .
>>> > I like the feature and actually it could useful for some use cases of
>>> mine.
>>> > Started to play with it in Solr 5.4 .
>>> >
>>> > I verified it is working if you pass both the query and the parameters
>>> to
>>> > substitute as request parameters.
>>> >
>>> > I verified it is working if you pass both the query and the parameters
>>> to
>>> > substitute as defaults or appends for the request handler.
>>> >
>>> > Currently I am not able to make it working the following scenario :
>>> > solrconfig.xml
>>> >
>>> > <requestHandler name="/getSimilar" class="solr.SearchHandler">
>>> >> <lst name="defaults">
>>> >> ...
>>> >> <str name="q">{!boost b=recip(dist(2,0,star_rating,0,
>>> >> *${star_rating:3}*),1,10,10)}{!boost
>>> b=recip(geodist(lat_lon,*${lat:0}*,
>>> >> *${lon:0}*),2,200,20)}{!mlt
>>> >> qf=name,description,facilities,resort,region,dest_level_2 mintf=1
>>> mindf=5
>>> >> maxqt=50}*${master_id:0}*</str>
>>> >> ...
>>> >> </lst>
>>> >> </requestHandler>
>>> >
>>> >
>>> >
>>> > Then I pass in the request the parameter to substitute , like :
>>> > master_id=1 .
>>> > Unfortunately is always getting the default one.
>>>
>>> Hmmm, that's odd.
>>> Macro expansion is done after defaults/appends/invariants are added to
>>> the params, so it should work (although we lack a test for this).
>>>
>>> See RequestUtil.java:
>>> newMap = MacroExpander.expand(newMap);
>>>
>>> If you add echoParams=all to your request, that currently shows the
>>> params post-expansion to verify you're actually getting the default.
>>>
>>> -Yonik
>>>
>>
>>
>>
>> --
>> --------------------------
>>
>> Benedetti Alessandro
>> Visiting card : http://about.me/alessandro_benedetti
>>
>> "Tyger, tyger burning bright
>> In the forests of the night,
>> What immortal hand or eye
>> Could frame thy fearful symmetry?"
>>
>> William Blake - Songs of Experience -1794 England
>>
>
>
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card : http://about.me/alessandro_benedetti
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
>
--
--------------------------
Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti
"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"
William Blake - Songs of Experience -1794 England
Re: Parameter Substitution / Macro Expansion
Posted by Yonik Seeley <ys...@gmail.com>.
On Wed, Dec 30, 2015 at 10:30 AM, Alessandro Benedetti
<ab...@apache.org> wrote:
> mmmm debugging I confirm you are right Yonik, but also that the defaults
> are already applied at this point :
> org/apache/solr/request/json/RequestUtil.java:158
>
> Could be a resolution of properties that happens at solrconfig, xml level ?
Ah, good call - that must be it!
I wonder if there is a way to escape it at that level (config parsing)
so that it binds later (request param substitution)?
-Yonik
Re: Parameter Substitution / Macro Expansion
Posted by Alessandro Benedetti <ab...@apache.org>.
mmmm debugging I confirm you are right Yonik, but also that the defaults
are already applied at this point :
org/apache/solr/request/json/RequestUtil.java:158
Could be a resolution of properties that happens at solrconfig, xml level ?
It verifies if any java properties is in there when running solr, and
because of that the defaults are already applied ?
Cheers
On 30 December 2015 at 15:20, Alessandro Benedetti <ab...@apache.org>
wrote:
> Erick,
> your suggestion worked :
>
> <requestHandler name="/getSimilar" class="solr.SearchHandler">
> <lst name="defaults">
> <!-- Query settings -->
> <str name="title">Hotels Similarity</str>
> <str name="defType">edismax</str>
> <str name="rows">5</str>
> <str name="fl">*,score</str>
> <str name="fq">active:true</str>
> <str name="echoParams">explicit</str>
> <str name="wt">json</str>
> <str name="q">{!boost b=recip(dist(2,0,star_rating,0,$star_rating),1,10,10)}{!boost b=recip(geodist(lat_lon,$lat,$lon),2,200,20)}{!mlt qf=name,description,facilities,resort,region,dest_level_2 mintf=1 mindf=5 maxqt=50 v=$master_id}</str>
> <!-- Default Similarity Parameters-->
> <str name="master_id">0</str>
> <str name="star_rating">3</str>
> <str name="lat">0</str>
> <str name="lon">0</str>
> </lst>
> </requestHandler>
>
> Yonik, running debugQuery=on i verified that the default were added
> before, so basically I can see the default replaced but the one I am
> sending in the request param take no effect ( as actually the macro
> disappeared , replaced by the default) .
> Is maybe because I am inside a block with different query parsers ?
>
> Cheers
>
>
> On 30 December 2015 at 15:14, Yonik Seeley <ys...@gmail.com> wrote:
>
>> On Wed, Dec 30, 2015 at 8:37 AM, Alessandro Benedetti
>> <ab...@apache.org> wrote:
>> > I was taking a look to this article from Yonik [1] .
>> > I like the feature and actually it could useful for some use cases of
>> mine.
>> > Started to play with it in Solr 5.4 .
>> >
>> > I verified it is working if you pass both the query and the parameters
>> to
>> > substitute as request parameters.
>> >
>> > I verified it is working if you pass both the query and the parameters
>> to
>> > substitute as defaults or appends for the request handler.
>> >
>> > Currently I am not able to make it working the following scenario :
>> > solrconfig.xml
>> >
>> > <requestHandler name="/getSimilar" class="solr.SearchHandler">
>> >> <lst name="defaults">
>> >> ...
>> >> <str name="q">{!boost b=recip(dist(2,0,star_rating,0,
>> >> *${star_rating:3}*),1,10,10)}{!boost
>> b=recip(geodist(lat_lon,*${lat:0}*,
>> >> *${lon:0}*),2,200,20)}{!mlt
>> >> qf=name,description,facilities,resort,region,dest_level_2 mintf=1
>> mindf=5
>> >> maxqt=50}*${master_id:0}*</str>
>> >> ...
>> >> </lst>
>> >> </requestHandler>
>> >
>> >
>> >
>> > Then I pass in the request the parameter to substitute , like :
>> > master_id=1 .
>> > Unfortunately is always getting the default one.
>>
>> Hmmm, that's odd.
>> Macro expansion is done after defaults/appends/invariants are added to
>> the params, so it should work (although we lack a test for this).
>>
>> See RequestUtil.java:
>> newMap = MacroExpander.expand(newMap);
>>
>> If you add echoParams=all to your request, that currently shows the
>> params post-expansion to verify you're actually getting the default.
>>
>> -Yonik
>>
>
>
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card : http://about.me/alessandro_benedetti
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
>
--
--------------------------
Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti
"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"
William Blake - Songs of Experience -1794 England
Re: Parameter Substitution / Macro Expansion
Posted by Alessandro Benedetti <ab...@apache.org>.
Erick,
your suggestion worked :
<requestHandler name="/getSimilar" class="solr.SearchHandler">
<lst name="defaults">
<!-- Query settings -->
<str name="title">Hotels Similarity</str>
<str name="defType">edismax</str>
<str name="rows">5</str>
<str name="fl">*,score</str>
<str name="fq">active:true</str>
<str name="echoParams">explicit</str>
<str name="wt">json</str>
<str name="q">{!boost
b=recip(dist(2,0,star_rating,0,$star_rating),1,10,10)}{!boost
b=recip(geodist(lat_lon,$lat,$lon),2,200,20)}{!mlt
qf=name,description,facilities,resort,region,dest_level_2 mintf=1
mindf=5 maxqt=50 v=$master_id}</str>
<!-- Default Similarity Parameters-->
<str name="master_id">0</str>
<str name="star_rating">3</str>
<str name="lat">0</str>
<str name="lon">0</str>
</lst>
</requestHandler>
Yonik, running debugQuery=on i verified that the default were added before,
so basically I can see the default replaced but the one I am sending in the
request param take no effect ( as actually the macro disappeared , replaced
by the default) .
Is maybe because I am inside a block with different query parsers ?
Cheers
On 30 December 2015 at 15:14, Yonik Seeley <ys...@gmail.com> wrote:
> On Wed, Dec 30, 2015 at 8:37 AM, Alessandro Benedetti
> <ab...@apache.org> wrote:
> > I was taking a look to this article from Yonik [1] .
> > I like the feature and actually it could useful for some use cases of
> mine.
> > Started to play with it in Solr 5.4 .
> >
> > I verified it is working if you pass both the query and the parameters to
> > substitute as request parameters.
> >
> > I verified it is working if you pass both the query and the parameters to
> > substitute as defaults or appends for the request handler.
> >
> > Currently I am not able to make it working the following scenario :
> > solrconfig.xml
> >
> > <requestHandler name="/getSimilar" class="solr.SearchHandler">
> >> <lst name="defaults">
> >> ...
> >> <str name="q">{!boost b=recip(dist(2,0,star_rating,0,
> >> *${star_rating:3}*),1,10,10)}{!boost b=recip(geodist(lat_lon,*${lat:0}*,
> >> *${lon:0}*),2,200,20)}{!mlt
> >> qf=name,description,facilities,resort,region,dest_level_2 mintf=1
> mindf=5
> >> maxqt=50}*${master_id:0}*</str>
> >> ...
> >> </lst>
> >> </requestHandler>
> >
> >
> >
> > Then I pass in the request the parameter to substitute , like :
> > master_id=1 .
> > Unfortunately is always getting the default one.
>
> Hmmm, that's odd.
> Macro expansion is done after defaults/appends/invariants are added to
> the params, so it should work (although we lack a test for this).
>
> See RequestUtil.java:
> newMap = MacroExpander.expand(newMap);
>
> If you add echoParams=all to your request, that currently shows the
> params post-expansion to verify you're actually getting the default.
>
> -Yonik
>
--
--------------------------
Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti
"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"
William Blake - Songs of Experience -1794 England
Re: Parameter Substitution / Macro Expansion
Posted by Yonik Seeley <ys...@gmail.com>.
On Wed, Dec 30, 2015 at 8:37 AM, Alessandro Benedetti
<ab...@apache.org> wrote:
> I was taking a look to this article from Yonik [1] .
> I like the feature and actually it could useful for some use cases of mine.
> Started to play with it in Solr 5.4 .
>
> I verified it is working if you pass both the query and the parameters to
> substitute as request parameters.
>
> I verified it is working if you pass both the query and the parameters to
> substitute as defaults or appends for the request handler.
>
> Currently I am not able to make it working the following scenario :
> solrconfig.xml
>
> <requestHandler name="/getSimilar" class="solr.SearchHandler">
>> <lst name="defaults">
>> ...
>> <str name="q">{!boost b=recip(dist(2,0,star_rating,0,
>> *${star_rating:3}*),1,10,10)}{!boost b=recip(geodist(lat_lon,*${lat:0}*,
>> *${lon:0}*),2,200,20)}{!mlt
>> qf=name,description,facilities,resort,region,dest_level_2 mintf=1 mindf=5
>> maxqt=50}*${master_id:0}*</str>
>> ...
>> </lst>
>> </requestHandler>
>
>
>
> Then I pass in the request the parameter to substitute , like :
> master_id=1 .
> Unfortunately is always getting the default one.
Hmmm, that's odd.
Macro expansion is done after defaults/appends/invariants are added to
the params, so it should work (although we lack a test for this).
See RequestUtil.java:
newMap = MacroExpander.expand(newMap);
If you add echoParams=all to your request, that currently shows the
params post-expansion to verify you're actually getting the default.
-Yonik