[jira] [Commented] (SOLR-9636) Add support for javabin for /stream, /sql internode communication

["Joel Bernstein (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-9636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15781496#comment-15781496 ] 

Joel Bernstein commented on SOLR-9636:
--------------------------------------

I finally had the time to test out the javabin writer with the /export handler and streaming stack. My initial findings are really good. Here is a summary:

1) Currently testing must be done on branch_6x. There is a bug in master which breaks the /export handler. I haven't gotten to the bottom yet but I'm pretty sure it was introduced with the new docValues iterator API which is only in master. I will open a ticket for this bug shortly and see if I can fix the problem.

But testing in branch_6x is better anyway as it won't be testing both the docValues iterator API performance at the same time as the javabin /export performance.

2) For my test I worked on a single Solr instance with a single data shard loaded with 10,000,000 small documents. I also created a worker collection with 5 shards. The I ran the following expression with and without the javabin writer.
{code}
parallel(collection2, workers=5, sort="test_s desc", 
            rollup(over="test_s", sum(price_f),
                      search(collection1, q=*:*, fl="test_s, price_f", sort="test_s desc", qt="/export", wt="javabin", partitionKeys=test_s)))
{code}




> Add support for javabin for /stream, /sql internode communication
> -----------------------------------------------------------------
>
>                 Key: SOLR-9636
>                 URL: https://issues.apache.org/jira/browse/SOLR-9636
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Noble Paul
>            Assignee: Noble Paul
>             Fix For: master (7.0), 6.4
>
>         Attachments: SOLR-9636.patch
>
>
> currently it uses json, which is verbose and slow



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org