[jira] [Commented] (SOLR-5743) Faceting with BlockJoin support

["Vijay Sekhri (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15128887#comment-15128887 ] 

Vijay Sekhri commented on SOLR-5743:
------------------------------------

Hi Mikhail,
I did benchmark testing of this feature to determine the efficiency and performance .
In our stress environment I have roughly 57 Mil documents in solr index. 10 shards and each shard hosting around 5.7 Mil documents . Each shard has one replica and one leader .
Like in this figure.
!cluster.jpg!

There is solrj service that connects to solr cluster hosted on 8 hosts and each having 3 JVM instances. So in total 24 round robin instances of solrj service running and issuing queries to solr cluster. 
Solr version is 5.3.1

Here is the baseline 
With a load of 50 requests per seconds to the solrj service the average response times in service is 290 milliseconds. Same translated into solr cluster results in average response Qtimes of 22 milliseconds.
Here is the picture of average response times at service 

!service_baseline.png!

Here is the picture of average response Qtime of the solr
!solr_baseline.jpg!


Now I converted most of the documents with parent child relationship . In total there were 27 Mil new child documents  . So the total count of the documents increased from 57 Mil to 83 Mil documents.  I converted all the queries into the format of parent child in the solrj service layer .   Now with the same load the average response times in service increased to 1.3 seconds and average response Qtimes increased to 500 milliseconds. 
The solr version is 5.4. trunk with your code in it . 

Here is the picture of average response times at service with parent child
!service_new_baseline.jpg!

Here is the picture of average response Qtime of the solr with parent child 
!solr_new_baseline.jpg!

The overall performance was 10 times slower in solr layer and 3 times slower in solrj service layer with the same load . 

BTW I only tested with org.apache.solr.search.join.BlockJoinFacetComponent . Do you think that org.apache.solr.search.join.BlockJoinDocSetFacetComponent would be faster? 

Vijay

> Faceting with BlockJoin support
> -------------------------------
>
>                 Key: SOLR-5743
>                 URL: https://issues.apache.org/jira/browse/SOLR-5743
>             Project: Solr
>          Issue Type: New Feature
>          Components: faceting
>            Reporter: abipc
>            Assignee: Mikhail Khludnev
>              Labels: features
>             Fix For: 5.5, Trunk
>
>         Attachments: SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, SOLR-5743.patch, cluster.jpg, service_baseline.png, service_new_baseline.jpg, solr_baseline.jpg, solr_new_baseline.jpg
>
>
> For a sample inventory(note - nested documents) like this -   
>  <doc>
> <field name="id">10</field>
> <field name="type_s">parent</field>
> <field name="BRAND_s">Nike</field>
> <doc>
> <field name="id">11</field>
> <field name="COLOR_s">Red</field>
> <field name="SIZE_s">XL</field>
> </doc>
> <doc>
> <field name="id">12</field>
> <field name="COLOR_s">Blue</field>
> <field name="SIZE_s">XL</field>
> </doc>
> </doc>
> Faceting results must contain - 
> Red(1)
> XL(1) 
> Blue(1) 
> for a "q=*" query. 
> PS : The inventory example has been taken from this blog - http://blog.griddynamics.com/2013/09/solr-block-join-support.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org