Increase in Response time when solr fields are merged

[Ajay Sharma <aj...@indiamart.com.INVALID>]

Hi All,

Earlier we were searching in 6 fields i.e qf is applied on 6 fields like
below
<lst name="defaults">
  <str name="qf">A</str>
  <str name="qf">B</str>
  <str name="qf">C</str>
  <str name="qf">D</str>
  <str name="qf">E</str>
  <str name="qf">F</str>
</lst>

We had assumed if we reduced the number of fields being used to search then
the index size and response time both will decrease.

We merged all these 6 fields into one field X and now while searching we
using this single filed X
By merge i mean i index all the 6 field data into single field X
<lst name="defaults">
  <str name="qf">X</str>
</lst>

We are able to see a decrease in index size but the response time has
increased.
*Are we missing something? Is our assumption correct?*

Any help will be highly appreciated.


-- 
Thanks & Regards,
Ajay Sharma
Software Engineer, Product-Search,
IndiaMART InterMESH Ltd

-- 

Re: Increase in Response time when solr fields are merged

[Ajay Sharma <aj...@indiamart.com.INVALID>]

Thank you Shawn for the valuable inputs.

I am assuming, please correct me if I am wrong =>
If we have two fields one has a large amount of text wrt to a field with
shorter text like description and title.
So, the Number of the tokens created for the description field will be much
high w.r.t. the title.
So if I search in the title field, it will be comparatively fast w.r.t. to
the description field

If the above listed is true then in the case of merged fields, no of tokens
have increased exponentially in a single merged field and could be a
possible reason?


On Thu, Nov 19, 2020 at 5:06 PM Shawn Heisey <ap...@elyograg.org> wrote:

> On 11/19/2020 2:12 AM, Ajay Sharma wrote:
> > Earlier we were searching in 6 fields i.e qf is applied on 6 fields like
> > below
>
> <snip>
>
> > We merged all these 6 fields into one field X and now while searching we
> > using this single filed X
>
> <snip>
>
> > We are able to see a decrease in index size but the response time has
> > increased.
>
> I can't say for sure, but I would imagine that when querying multiple
> fields using edismax, Solr can manage to do some of that work in
> parallel.  But with only one field, any parallel processing is lost.  If
> I have the right idea, that could explain what you are seeing.
>
> Somebody with far more intimate knowledge of edismax will need to
> confirm or refute my thoughts.
>
> Thanks,
> Shawn
>


-- 
Thanks & Regards,
Ajay Sharma
Product Search
+91-8954492245

-- 

Re: Increase in Response time when solr fields are merged

[Shawn Heisey <ap...@elyograg.org>]

On 11/19/2020 2:12 AM, Ajay Sharma wrote:
> Earlier we were searching in 6 fields i.e qf is applied on 6 fields like
> below

<snip>

> We merged all these 6 fields into one field X and now while searching we
> using this single filed X

<snip>

> We are able to see a decrease in index size but the response time has
> increased.

I can't say for sure, but I would imagine that when querying multiple 
fields using edismax, Solr can manage to do some of that work in 
parallel.  But with only one field, any parallel processing is lost.  If 
I have the right idea, that could explain what you are seeing.

Somebody with far more intimate knowledge of edismax will need to 
confirm or refute my thoughts.

Thanks,
Shawn