[jira] [Commented] (SOLR-9905) Add NullStream to isolate the performance of the ExportWriter

["Joel Bernstein (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-9905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15785517#comment-15785517 ] 

Joel Bernstein commented on SOLR-9905:
--------------------------------------

The NullStream is a utility function to test the raw performance of the ExportWriter. This is a nice utility to have to diagnose bottlenecks in slow running MapReduce operations. The NullStream will allow developers to test the performance of the shuffling (Sorting, Partitioning, Exporting) in isolation from the reduce operation (Rollup, Join, Group, etc..). 

{code}
parallel(collection2, workers=7, sort="count desc", 
      null(search(collection1, 
                   q=*:*, 
                   fl="id", 
                   sort="id desc", 
                   qt="/export", 
                   wt="javabin", 
                   partitionKeys=id)))
{code}


> Add NullStream to isolate the performance of the ExportWriter
> -------------------------------------------------------------
>
>                 Key: SOLR-9905
>                 URL: https://issues.apache.org/jira/browse/SOLR-9905
>             Project: Solr
>          Issue Type: New Feature
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Joel Bernstein
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org