[jira] [Commented] (IGNITE-12291) Create controllable paged query requests / responses for TextQuery similar to current SQL result processing

["Maksim Timonin (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IGNITE-12291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17340331#comment-17340331 ] 

Maksim Timonin commented on IGNITE-12291:
-----------------------------------------

[~amashenkov], [~Yuriy_Shuliha] hi!

I investigated this issue. I found some issues there:
 # Text queries ignore pageSize set in a query.
 # Lucene returns non-sorted result from every node, so we can't use MergeSort like SQL queries do that. 

So, I think, that number of pages to load can be limited with different approach that described in the ticket description. We can move check of limit to the `next()` call instead of `enqueue()`. After `next()` returns NULL iterator automatically closes and send cancel requests to all nodes. 

Please, check this approach, in this PR: [https://github.com/apache/ignite/pull/9086/files]. There are some new tests that cover page loading. 

Also, I've prepared a draft PR ([https://github.com/apache/ignite/pull/9081/files|https://github.com/apache/ignite/pull/9081/files]) that introduces the MergeSort reducer. Also it loads pages more carefully, one by one from every node. It'll be used in index queries, but for TextQueries it's useless due to our lucene index doesn't return sorted result from every node, so we need to decide what to do with this issue.

I think that we should follow both of this approaches but sequentially. Firstly fix useless page loadings, then provide sorting for text queries. WDYT? 

 

 

 

> Create controllable paged query requests / responses for TextQuery similar to current SQL result processing
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: IGNITE-12291
>                 URL: https://issues.apache.org/jira/browse/IGNITE-12291
>             Project: Ignite
>          Issue Type: Improvement
>            Reporter: Yuriy Shuliha 
>            Assignee: Maksim Timonin
>            Priority: Major
>
> For now query initiator node sends 1 _GridCacheQueryRequest_ and can get multiple _GridCacheQueryResponse_-s per remote.
> TextQuery processing finishes when all remotes send _GridCacheQueryResponse_ with 'finished' flag.
> _TextQuery_ processing  should be reworked like it was done for SQL queries.
> Ignite has _GridQueryNextPageRequest \ Response_ classes for SQL result processing.
> Similar processing should be implemented for _TextQueries_:
> *GridTextQueryNextPageRequest*
> *GridTextQueryNextPageResponse* 
> Proper _TextQuery_ response processing should be implemented in _GridCacheQueryFutureAdapter._
> This will allow us to add correct sorting and apply limit correctly on reduce.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)