[jira] Commented: (DIRSERVER-1161) search results are not streamed to the client until final done response is queued

["Emmanuel Lecharny (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DIRSERVER-1161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12649527#action_12649527 ] 

Emmanuel Lecharny commented on DIRSERVER-1161:
----------------------------------------------

I think I have all the responses flushed one by one by adding an ExecutorFilter on the WRITE EventType :

...
        // Inject the codec into the chain
        ((DefaultIoFilterChainBuilder)chain).addLast( "codec", 
                new ProtocolCodecFilter( this.getProtocolCodecFactory() ) );
        
        // Now inject an ExecutorFilter for the write operations
        ((DefaultIoFilterChainBuilder)chain).addLast( "executor", 
        		new ExecutorFilter( new OrderedThreadPoolExecutor( 10 ), 
        			IoEventType.WRITE ) );
...

I have no idea about what should be the number of threads in the pool, so I just set it to 10 atm. Integration tests are passing

(This is on the ads-mina2.0 branch)
 

> search results are not streamed to the client until final done response is queued
> ---------------------------------------------------------------------------------
>
>                 Key: DIRSERVER-1161
>                 URL: https://issues.apache.org/jira/browse/DIRSERVER-1161
>             Project: Directory ApacheDS
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 1.0
>         Environment: JDK 1.5.0_11 
>            Reporter: Norval Hope
>            Assignee: Emmanuel Lecharny
>            Priority: Blocker
>             Fix For: 1.5.5
>
>         Attachments: apacheds_1.5.1_streaming.patch, apacheds_1.5.1_streaming.patch, apacheds_1.5.1_streaming_log_output.txt, installers_1.5.1_streaming.patch, installers_1.5.1_streaming.patch, mina_1.1.2_streaming.patch, mina_1.1_trunk_streaming.patch, pom.xml, streaming_log_output.txt, streaming_logging.patch
>
>   Original Estimate: 120h
>  Remaining Estimate: 120h
>
> Search results accumulate in Events on the SessionBuffer.eventQueue within ExecutorFilter.fireEvent() until final done response for the search is written to the session and then all results for the search (possibly millions depending on the search and state of the directory) are written out at once. This is a big problem for scalability and I gather from previous correspondence with Alex that this behaviour is unexpected.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.