You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Midas A <te...@gmail.com> on 2016/07/13 07:24:27 UTC
Query optimization
http://
#####:8983/solr/prod/select?q=id_path_ids:166&fq=sort_price:[0%20TO%20*]&fq=status:A&fq=company_status:A&facet=true&facet.mincount=1&facet.field=show_meta_id&facet.field=show_brand&facet.field=product_amount_available&facet.field=by_processor&facet.field=by_system_memory&facet.field=by_screen_size&facet.field=by_operating_system&facet.field=by_laptop_type&facet.field=by_processor_brand&facet.field=by_hard_drive_capacity&facet.field=by_touchscreen&facet.field=by_warranty&facet.field=by_graphic_memory&facet.field=is_trm&facet.field=show_merchant&facet.field=is_cod&facet.field=show_market&facet.query={!ex=p_r%20key=product_rating:[4-5]}product_rating:[4%20TO%205]&facet.query={!ex=p_r%20key=product_rating:[3-5]}product_rating:[3%20TO%205]&facet.query={!ex=p_r%20key=product_rating:[2-5]}product_rating:[2%20TO%205]&facet.query={!ex=p_r%20key=product_rating:[1-5]}product_rating:[1%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[4-5]}merchant_rating:[4%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[3-5]}merchant_rating:[3%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[2-5]}merchant_rating:[2%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[1-5]}merchant_rating:[1%20TO%205]&facet.limit=500&stats=true&stats.field=sort_price&start=0&rows=10&sort=product_amount_available%20desc,boost_index%20asc,popularity%20desc,is_cod%20desc
What kind of optimization we can do in above query . it is taking 2400 ms .
Re: Query optimization
Posted by Midas A <te...@gmail.com>.
Hi ,
One more thing i would like to add here is we build facet queries over
dynamic fields so my question is
a) Is there any harm of using docValues true with dynamic fields.
b) Other suggestion that we can implement to optimize this query my index
size is 8GB and query is taking more tha 3 seconds.
Regards,
Abhishek Tiwari
On Thu, Jul 14, 2016 at 6:42 AM, Erick Erickson <er...@gmail.com>
wrote:
> DocValues are now the preferred mechanism
> whenever you need to sort, facet or group. It'll
> make your on-disk index bigger, but.... the on-disk
> structure would have been built in Java's memory
> if you didn't use DocValues whereas if you do
> it's MMap'd.
>
> So overall, use DocValues by preference.
>
> Best,
> Erick
>
> On Wed, Jul 13, 2016 at 5:36 AM, sara hajili <ha...@gmail.com>
> wrote:
> > as i know when you use docValue=true
> > solr when indexing doc,
> > solr although store doc and docValue=true field in memory.to use that in
> > facet query and sort query result.
> > so maybe use a lot docvalue=true use a lot memory of ur system.
> > but use it in logical way.can make better query response time
> >
> > On Wed, Jul 13, 2016 at 5:11 AM, Midas A <te...@gmail.com> wrote:
> >
> >> Is there any draw back of using docValues=true ?
> >>
> >> On Wed, Jul 13, 2016 at 2:28 PM, sara hajili <ha...@gmail.com>
> >> wrote:
> >>
> >> > Hi.
> >> > Facet query take a long time.you vcan use group query.
> >> > Or in fileds in schema that you run facet query on that filed.
> >> > Set doc value=true.
> >> > To get better answer.in quick time.
> >> > On Jul 13, 2016 11:54 AM, "Midas A" <te...@gmail.com> wrote:
> >> >
> >> > > http://
> >> > >
> >> > >
> >> >
> >>
> #####:8983/solr/prod/select?q=id_path_ids:166&fq=sort_price:[0%20TO%20*]&fq=status:A&fq=company_status:A&facet=true&facet.mincount=1&facet.field=show_meta_id&facet.field=show_brand&facet.field=product_amount_available&facet.field=by_processor&facet.field=by_system_memory&facet.field=by_screen_size&facet.field=by_operating_system&facet.field=by_laptop_type&facet.field=by_processor_brand&facet.field=by_hard_drive_capacity&facet.field=by_touchscreen&facet.field=by_warranty&facet.field=by_graphic_memory&facet.field=is_trm&facet.field=show_merchant&facet.field=is_cod&facet.field=show_market&facet.query={!ex=p_r%20key=product_rating:[4-5]}product_rating:[4%20TO%205]&facet.query={!ex=p_r%20key=product_rating:[3-5]}product_rating:[3%20TO%205]&facet.query={!ex=p_r%20key=product_rating:[2-5]}product_rating:[2%20TO%205]&facet.query={!ex=p_r%20key=product_rating:[1-5]}product_rating:[1%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[4-5]}merchant_rating:[4%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[3-5]}merchant_rating:[3%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[2-5]}merchant_rating:[2%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[1-5]}merchant_rating:[1%20TO%205]&facet.limit=500&stats=true&stats.field=sort_price&start=0&rows=10&sort=product_amount_available%20desc,boost_index%20asc,popularity%20desc,is_cod%20desc
> >> > >
> >> > >
> >> > > What kind of optimization we can do in above query . it is taking
> 2400
> >> > ms .
> >> > >
> >> >
> >>
>
Re: Query optimization
Posted by Erick Erickson <er...@gmail.com>.
DocValues are now the preferred mechanism
whenever you need to sort, facet or group. It'll
make your on-disk index bigger, but.... the on-disk
structure would have been built in Java's memory
if you didn't use DocValues whereas if you do
it's MMap'd.
So overall, use DocValues by preference.
Best,
Erick
On Wed, Jul 13, 2016 at 5:36 AM, sara hajili <ha...@gmail.com> wrote:
> as i know when you use docValue=true
> solr when indexing doc,
> solr although store doc and docValue=true field in memory.to use that in
> facet query and sort query result.
> so maybe use a lot docvalue=true use a lot memory of ur system.
> but use it in logical way.can make better query response time
>
> On Wed, Jul 13, 2016 at 5:11 AM, Midas A <te...@gmail.com> wrote:
>
>> Is there any draw back of using docValues=true ?
>>
>> On Wed, Jul 13, 2016 at 2:28 PM, sara hajili <ha...@gmail.com>
>> wrote:
>>
>> > Hi.
>> > Facet query take a long time.you vcan use group query.
>> > Or in fileds in schema that you run facet query on that filed.
>> > Set doc value=true.
>> > To get better answer.in quick time.
>> > On Jul 13, 2016 11:54 AM, "Midas A" <te...@gmail.com> wrote:
>> >
>> > > http://
>> > >
>> > >
>> >
>> #####:8983/solr/prod/select?q=id_path_ids:166&fq=sort_price:[0%20TO%20*]&fq=status:A&fq=company_status:A&facet=true&facet.mincount=1&facet.field=show_meta_id&facet.field=show_brand&facet.field=product_amount_available&facet.field=by_processor&facet.field=by_system_memory&facet.field=by_screen_size&facet.field=by_operating_system&facet.field=by_laptop_type&facet.field=by_processor_brand&facet.field=by_hard_drive_capacity&facet.field=by_touchscreen&facet.field=by_warranty&facet.field=by_graphic_memory&facet.field=is_trm&facet.field=show_merchant&facet.field=is_cod&facet.field=show_market&facet.query={!ex=p_r%20key=product_rating:[4-5]}product_rating:[4%20TO%205]&facet.query={!ex=p_r%20key=product_rating:[3-5]}product_rating:[3%20TO%205]&facet.query={!ex=p_r%20key=product_rating:[2-5]}product_rating:[2%20TO%205]&facet.query={!ex=p_r%20key=product_rating:[1-5]}product_rating:[1%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[4-5]}merchant_rating:[4%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[3-5]}merchant_rating:[3%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[2-5]}merchant_rating:[2%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[1-5]}merchant_rating:[1%20TO%205]&facet.limit=500&stats=true&stats.field=sort_price&start=0&rows=10&sort=product_amount_available%20desc,boost_index%20asc,popularity%20desc,is_cod%20desc
>> > >
>> > >
>> > > What kind of optimization we can do in above query . it is taking 2400
>> > ms .
>> > >
>> >
>>
Re: Query optimization
Posted by sara hajili <ha...@gmail.com>.
as i know when you use docValue=true
solr when indexing doc,
solr although store doc and docValue=true field in memory.to use that in
facet query and sort query result.
so maybe use a lot docvalue=true use a lot memory of ur system.
but use it in logical way.can make better query response time
On Wed, Jul 13, 2016 at 5:11 AM, Midas A <te...@gmail.com> wrote:
> Is there any draw back of using docValues=true ?
>
> On Wed, Jul 13, 2016 at 2:28 PM, sara hajili <ha...@gmail.com>
> wrote:
>
> > Hi.
> > Facet query take a long time.you vcan use group query.
> > Or in fileds in schema that you run facet query on that filed.
> > Set doc value=true.
> > To get better answer.in quick time.
> > On Jul 13, 2016 11:54 AM, "Midas A" <te...@gmail.com> wrote:
> >
> > > http://
> > >
> > >
> >
> #####:8983/solr/prod/select?q=id_path_ids:166&fq=sort_price:[0%20TO%20*]&fq=status:A&fq=company_status:A&facet=true&facet.mincount=1&facet.field=show_meta_id&facet.field=show_brand&facet.field=product_amount_available&facet.field=by_processor&facet.field=by_system_memory&facet.field=by_screen_size&facet.field=by_operating_system&facet.field=by_laptop_type&facet.field=by_processor_brand&facet.field=by_hard_drive_capacity&facet.field=by_touchscreen&facet.field=by_warranty&facet.field=by_graphic_memory&facet.field=is_trm&facet.field=show_merchant&facet.field=is_cod&facet.field=show_market&facet.query={!ex=p_r%20key=product_rating:[4-5]}product_rating:[4%20TO%205]&facet.query={!ex=p_r%20key=product_rating:[3-5]}product_rating:[3%20TO%205]&facet.query={!ex=p_r%20key=product_rating:[2-5]}product_rating:[2%20TO%205]&facet.query={!ex=p_r%20key=product_rating:[1-5]}product_rating:[1%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[4-5]}merchant_rating:[4%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[3-5]}merchant_rating:[3%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[2-5]}merchant_rating:[2%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[1-5]}merchant_rating:[1%20TO%205]&facet.limit=500&stats=true&stats.field=sort_price&start=0&rows=10&sort=product_amount_available%20desc,boost_index%20asc,popularity%20desc,is_cod%20desc
> > >
> > >
> > > What kind of optimization we can do in above query . it is taking 2400
> > ms .
> > >
> >
>
Re: Query optimization
Posted by Midas A <te...@gmail.com>.
Is there any draw back of using docValues=true ?
On Wed, Jul 13, 2016 at 2:28 PM, sara hajili <ha...@gmail.com> wrote:
> Hi.
> Facet query take a long time.you vcan use group query.
> Or in fileds in schema that you run facet query on that filed.
> Set doc value=true.
> To get better answer.in quick time.
> On Jul 13, 2016 11:54 AM, "Midas A" <te...@gmail.com> wrote:
>
> > http://
> >
> >
> #####:8983/solr/prod/select?q=id_path_ids:166&fq=sort_price:[0%20TO%20*]&fq=status:A&fq=company_status:A&facet=true&facet.mincount=1&facet.field=show_meta_id&facet.field=show_brand&facet.field=product_amount_available&facet.field=by_processor&facet.field=by_system_memory&facet.field=by_screen_size&facet.field=by_operating_system&facet.field=by_laptop_type&facet.field=by_processor_brand&facet.field=by_hard_drive_capacity&facet.field=by_touchscreen&facet.field=by_warranty&facet.field=by_graphic_memory&facet.field=is_trm&facet.field=show_merchant&facet.field=is_cod&facet.field=show_market&facet.query={!ex=p_r%20key=product_rating:[4-5]}product_rating:[4%20TO%205]&facet.query={!ex=p_r%20key=product_rating:[3-5]}product_rating:[3%20TO%205]&facet.query={!ex=p_r%20key=product_rating:[2-5]}product_rating:[2%20TO%205]&facet.query={!ex=p_r%20key=product_rating:[1-5]}product_rating:[1%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[4-5]}merchant_rating:[4%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[3-5]}merchant_rating:[3%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[2-5]}merchant_rating:[2%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[1-5]}merchant_rating:[1%20TO%205]&facet.limit=500&stats=true&stats.field=sort_price&start=0&rows=10&sort=product_amount_available%20desc,boost_index%20asc,popularity%20desc,is_cod%20desc
> >
> >
> > What kind of optimization we can do in above query . it is taking 2400
> ms .
> >
>
Re: Query optimization
Posted by sara hajili <ha...@gmail.com>.
Hi.
Facet query take a long time.you vcan use group query.
Or in fileds in schema that you run facet query on that filed.
Set doc value=true.
To get better answer.in quick time.
On Jul 13, 2016 11:54 AM, "Midas A" <te...@gmail.com> wrote:
> http://
>
> #####:8983/solr/prod/select?q=id_path_ids:166&fq=sort_price:[0%20TO%20*]&fq=status:A&fq=company_status:A&facet=true&facet.mincount=1&facet.field=show_meta_id&facet.field=show_brand&facet.field=product_amount_available&facet.field=by_processor&facet.field=by_system_memory&facet.field=by_screen_size&facet.field=by_operating_system&facet.field=by_laptop_type&facet.field=by_processor_brand&facet.field=by_hard_drive_capacity&facet.field=by_touchscreen&facet.field=by_warranty&facet.field=by_graphic_memory&facet.field=is_trm&facet.field=show_merchant&facet.field=is_cod&facet.field=show_market&facet.query={!ex=p_r%20key=product_rating:[4-5]}product_rating:[4%20TO%205]&facet.query={!ex=p_r%20key=product_rating:[3-5]}product_rating:[3%20TO%205]&facet.query={!ex=p_r%20key=product_rating:[2-5]}product_rating:[2%20TO%205]&facet.query={!ex=p_r%20key=product_rating:[1-5]}product_rating:[1%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[4-5]}merchant_rating:[4%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[3-5]}merchant_rating:[3%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[2-5]}merchant_rating:[2%20TO%205]&facet.query={!ex=m_r%20key=merchant_rating:[1-5]}merchant_rating:[1%20TO%205]&facet.limit=500&stats=true&stats.field=sort_price&start=0&rows=10&sort=product_amount_available%20desc,boost_index%20asc,popularity%20desc,is_cod%20desc
>
>
> What kind of optimization we can do in above query . it is taking 2400 ms .
>