You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Tom Burton-West <tb...@umich.edu> on 2018/07/26 21:52:14 UTC
Can the export handler be used with the edismax or dismax query handler
Hello all,
I am completely new to the export handler.
Can the export handler be used with the edismax or dismax query handler?
I tried using local params :
q= _query_:"{!edismax qf='ocr^50000+allfields^1+titleProper^50' mm='100%25'
tie='0.9' } art"
which does not seem to be working.
Tom
Re: Can the export handler be used with the edismax or dismax query handler
Posted by Tom Burton-West <tb...@umich.edu>.
Thanks Mikhail and Erick,
I don't need ranks or score. I just need the full set of results. Will
the export handler work with a fq that uses edismax? (I'm not at work
today, but I can try it out tomorrow.)
I compared a simple (not edismax) query and the export handler with
cursormark with rows = 50K to 200K. The export handler took about 8 ms to
export all 1.9 million results and had minimal impact on server CPU and
memory. With the cursormark it took about 1 minute 20 seconds, CPU use
increased by about 25% and there were many more garbage collections
although the time for GC totaled only a few seconds.
Tom
On Sat, Jul 28, 2018 at 4:25 AM, Mikhail Khludnev <mk...@apache.org> wrote:
> Tom,
> Do you say you don't need rank results or you don't need to export score?
> If the former is true, you can just put edismax to fq.
> Just a note: using cursor mark with the score may cause some kind of hit
> dupes and probably missing some ones.
>
> On Sat, Jul 28, 2018 at 5:20 AM Erick Erickson <er...@gmail.com>
> wrote:
>
> > What about cursorMark? That's designed to handle repeated calls with
> > increasing "start" parameters without bogging down.
> >
> > https://lucene.apache.org/solr/guide/6_6/pagination-of-results.html
> >
> > Best,
> > Erick
> >
> > On Fri, Jul 27, 2018 at 9:47 AM, Tom Burton-West <tb...@umich.edu>
> > wrote:
> > > Thanks Joel,
> > >
> > > My use case is that I have a complex edismax query (example below) and
> > the
> > > user wants to download the set of *all* search results (ids and some
> > small
> > > metadata fields). So they don't need the relevance ranking.
> However, I
> > > need to somehow get the exact set that the complex edismax query
> matched.
> > >
> > > Should I try to write some code to rewrite the logic of the edismax
> > query
> > > with a complex boolean query or would it make sense for me to look at
> > > possibly modifying the export handler for my use case?
> > >
> > > Tom
> > >
> > > "q= _query_:"{!edismax
> > >
> > qf='ocr^50000+allfieldsProper^2+allfields^1+titleProper^50+
> title_topProper^30+title_restProper^15+title^10+title_
> top^5+title_rest^2+series^5+series2^5+author^80+author2^
> 50+issn^1+isbn^1+oclc^1+sdrnum^1+ctrlnum^1+id^1+
> rptnum^1+topicProper^2+topic^1+hlb3^1+fullgeographic^1+fullgenre^1+era^1+'
> > >
> > pf='title_ab^10000+titleProper^1500+title_topProper^1000+title_
> restProper^800+series^100+series2^100+author^1600+
> author2^800+topicProper^200+fullgenre^200+hlb3^200+allfieldsProper^100+'
> > > mm='100%25' tie='0.9' } European Art History"
> > >
> > >
> > > On Thu, Jul 26, 2018 at 6:02 PM, Joel Bernstein <jo...@gmail.com>
> > wrote:
> > >
> > >> The export handler doesn't allow sorting by score at this time. It
> only
> > >> supports sorting on fields. So the edismax qparser won't cxcurrently
> > work
> > >> with the export handler.
> > >>
> > >> Joel Bernstein
> > >> http://joelsolr.blogspot.com/
> > >>
> > >> On Thu, Jul 26, 2018 at 5:52 PM, Tom Burton-West <tb...@umich.edu>
> > >> wrote:
> > >>
> > >> > Hello all,
> > >> >
> > >> > I am completely new to the export handler.
> > >> >
> > >> > Can the export handler be used with the edismax or dismax query
> > handler?
> > >> >
> > >> > I tried using local params :
> > >> >
> > >> > q= _query_:"{!edismax qf='ocr^50000+allfields^1+titleProper^50'
> > >> > mm='100%25'
> > >> > tie='0.9' } art"
> > >> >
> > >> > which does not seem to be working.
> > >> >
> > >> > Tom
> > >> >
> > >>
> >
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>
Re: Can the export handler be used with the edismax or dismax query handler
Posted by Mikhail Khludnev <mk...@apache.org>.
Tom,
Do you say you don't need rank results or you don't need to export score?
If the former is true, you can just put edismax to fq.
Just a note: using cursor mark with the score may cause some kind of hit
dupes and probably missing some ones.
On Sat, Jul 28, 2018 at 5:20 AM Erick Erickson <er...@gmail.com>
wrote:
> What about cursorMark? That's designed to handle repeated calls with
> increasing "start" parameters without bogging down.
>
> https://lucene.apache.org/solr/guide/6_6/pagination-of-results.html
>
> Best,
> Erick
>
> On Fri, Jul 27, 2018 at 9:47 AM, Tom Burton-West <tb...@umich.edu>
> wrote:
> > Thanks Joel,
> >
> > My use case is that I have a complex edismax query (example below) and
> the
> > user wants to download the set of *all* search results (ids and some
> small
> > metadata fields). So they don't need the relevance ranking. However, I
> > need to somehow get the exact set that the complex edismax query matched.
> >
> > Should I try to write some code to rewrite the logic of the edismax
> query
> > with a complex boolean query or would it make sense for me to look at
> > possibly modifying the export handler for my use case?
> >
> > Tom
> >
> > "q= _query_:"{!edismax
> >
> qf='ocr^50000+allfieldsProper^2+allfields^1+titleProper^50+title_topProper^30+title_restProper^15+title^10+title_top^5+title_rest^2+series^5+series2^5+author^80+author2^50+issn^1+isbn^1+oclc^1+sdrnum^1+ctrlnum^1+id^1+rptnum^1+topicProper^2+topic^1+hlb3^1+fullgeographic^1+fullgenre^1+era^1+'
> >
> pf='title_ab^10000+titleProper^1500+title_topProper^1000+title_restProper^800+series^100+series2^100+author^1600+author2^800+topicProper^200+fullgenre^200+hlb3^200+allfieldsProper^100+'
> > mm='100%25' tie='0.9' } European Art History"
> >
> >
> > On Thu, Jul 26, 2018 at 6:02 PM, Joel Bernstein <jo...@gmail.com>
> wrote:
> >
> >> The export handler doesn't allow sorting by score at this time. It only
> >> supports sorting on fields. So the edismax qparser won't cxcurrently
> work
> >> with the export handler.
> >>
> >> Joel Bernstein
> >> http://joelsolr.blogspot.com/
> >>
> >> On Thu, Jul 26, 2018 at 5:52 PM, Tom Burton-West <tb...@umich.edu>
> >> wrote:
> >>
> >> > Hello all,
> >> >
> >> > I am completely new to the export handler.
> >> >
> >> > Can the export handler be used with the edismax or dismax query
> handler?
> >> >
> >> > I tried using local params :
> >> >
> >> > q= _query_:"{!edismax qf='ocr^50000+allfields^1+titleProper^50'
> >> > mm='100%25'
> >> > tie='0.9' } art"
> >> >
> >> > which does not seem to be working.
> >> >
> >> > Tom
> >> >
> >>
>
--
Sincerely yours
Mikhail Khludnev
Re: Can the export handler be used with the edismax or dismax query handler
Posted by Erick Erickson <er...@gmail.com>.
What about cursorMark? That's designed to handle repeated calls with
increasing "start" parameters without bogging down.
https://lucene.apache.org/solr/guide/6_6/pagination-of-results.html
Best,
Erick
On Fri, Jul 27, 2018 at 9:47 AM, Tom Burton-West <tb...@umich.edu> wrote:
> Thanks Joel,
>
> My use case is that I have a complex edismax query (example below) and the
> user wants to download the set of *all* search results (ids and some small
> metadata fields). So they don't need the relevance ranking. However, I
> need to somehow get the exact set that the complex edismax query matched.
>
> Should I try to write some code to rewrite the logic of the edismax query
> with a complex boolean query or would it make sense for me to look at
> possibly modifying the export handler for my use case?
>
> Tom
>
> "q= _query_:"{!edismax
> qf='ocr^50000+allfieldsProper^2+allfields^1+titleProper^50+title_topProper^30+title_restProper^15+title^10+title_top^5+title_rest^2+series^5+series2^5+author^80+author2^50+issn^1+isbn^1+oclc^1+sdrnum^1+ctrlnum^1+id^1+rptnum^1+topicProper^2+topic^1+hlb3^1+fullgeographic^1+fullgenre^1+era^1+'
> pf='title_ab^10000+titleProper^1500+title_topProper^1000+title_restProper^800+series^100+series2^100+author^1600+author2^800+topicProper^200+fullgenre^200+hlb3^200+allfieldsProper^100+'
> mm='100%25' tie='0.9' } European Art History"
>
>
> On Thu, Jul 26, 2018 at 6:02 PM, Joel Bernstein <jo...@gmail.com> wrote:
>
>> The export handler doesn't allow sorting by score at this time. It only
>> supports sorting on fields. So the edismax qparser won't cxcurrently work
>> with the export handler.
>>
>> Joel Bernstein
>> http://joelsolr.blogspot.com/
>>
>> On Thu, Jul 26, 2018 at 5:52 PM, Tom Burton-West <tb...@umich.edu>
>> wrote:
>>
>> > Hello all,
>> >
>> > I am completely new to the export handler.
>> >
>> > Can the export handler be used with the edismax or dismax query handler?
>> >
>> > I tried using local params :
>> >
>> > q= _query_:"{!edismax qf='ocr^50000+allfields^1+titleProper^50'
>> > mm='100%25'
>> > tie='0.9' } art"
>> >
>> > which does not seem to be working.
>> >
>> > Tom
>> >
>>
Re: Can the export handler be used with the edismax or dismax query handler
Posted by Tom Burton-West <tb...@umich.edu>.
Thanks Joel,
My use case is that I have a complex edismax query (example below) and the
user wants to download the set of *all* search results (ids and some small
metadata fields). So they don't need the relevance ranking. However, I
need to somehow get the exact set that the complex edismax query matched.
Should I try to write some code to rewrite the logic of the edismax query
with a complex boolean query or would it make sense for me to look at
possibly modifying the export handler for my use case?
Tom
"q= _query_:"{!edismax
qf='ocr^50000+allfieldsProper^2+allfields^1+titleProper^50+title_topProper^30+title_restProper^15+title^10+title_top^5+title_rest^2+series^5+series2^5+author^80+author2^50+issn^1+isbn^1+oclc^1+sdrnum^1+ctrlnum^1+id^1+rptnum^1+topicProper^2+topic^1+hlb3^1+fullgeographic^1+fullgenre^1+era^1+'
pf='title_ab^10000+titleProper^1500+title_topProper^1000+title_restProper^800+series^100+series2^100+author^1600+author2^800+topicProper^200+fullgenre^200+hlb3^200+allfieldsProper^100+'
mm='100%25' tie='0.9' } European Art History"
On Thu, Jul 26, 2018 at 6:02 PM, Joel Bernstein <jo...@gmail.com> wrote:
> The export handler doesn't allow sorting by score at this time. It only
> supports sorting on fields. So the edismax qparser won't cxcurrently work
> with the export handler.
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Thu, Jul 26, 2018 at 5:52 PM, Tom Burton-West <tb...@umich.edu>
> wrote:
>
> > Hello all,
> >
> > I am completely new to the export handler.
> >
> > Can the export handler be used with the edismax or dismax query handler?
> >
> > I tried using local params :
> >
> > q= _query_:"{!edismax qf='ocr^50000+allfields^1+titleProper^50'
> > mm='100%25'
> > tie='0.9' } art"
> >
> > which does not seem to be working.
> >
> > Tom
> >
>
Re: Can the export handler be used with the edismax or dismax query handler
Posted by Joel Bernstein <jo...@gmail.com>.
The export handler doesn't allow sorting by score at this time. It only
supports sorting on fields. So the edismax qparser won't cxcurrently work
with the export handler.
Joel Bernstein
http://joelsolr.blogspot.com/
On Thu, Jul 26, 2018 at 5:52 PM, Tom Burton-West <tb...@umich.edu> wrote:
> Hello all,
>
> I am completely new to the export handler.
>
> Can the export handler be used with the edismax or dismax query handler?
>
> I tried using local params :
>
> q= _query_:"{!edismax qf='ocr^50000+allfields^1+titleProper^50'
> mm='100%25'
> tie='0.9' } art"
>
> which does not seem to be working.
>
> Tom
>