[jira] [Created] (UNOMI-776) Avoid the PersistenceService auto scroll when doing queries

["Kevan Jahanshahi (Jira)" <ji...@apache.org>]

Kevan Jahanshahi created UNOMI-776:
--------------------------------------

             Summary: Avoid the PersistenceService auto scroll when doing queries
                 Key: UNOMI-776
                 URL: https://issues.apache.org/jira/browse/UNOMI-776
             Project: Apache Unomi
          Issue Type: Bug
    Affects Versions: unomi-1.8.0, unomi-2.2.0
            Reporter: Kevan Jahanshahi


recently discovered in: UNOMI-764

The persistence service is doing an automatic scroll queries when doing queries with limit {*}-1{*}:

 
{code:java}
persistenceService.query(propertyCondition, null, Profile.class); {code}
Nothing in the Java Doc is mentioning this behavior.

 

And it's dangerous because if this query is done a lot of time it will create a lot of scroll id in RAM and kill the memory.

ch scroll query creates a cursor on the Elasticsearch server, which consumes system resources, including memory, CPU, and disk I/O.If you have a high number of concurrent scroll queries running simultaneously, it can lead to increased resource utilization and potentially cause performance issues or even out-of-memory errors if the system is not adequately configured to handle the load.

This is especially true when dealing with large indices with a high volume of data. (lot of profiles, lot of sessions in case of the merge)

*To me we should remove this auto-scroll done in the persistence service, and create a dedicated method in the persistence service to let the implementation choose between scrolling or not.*

*TODO:*
 * *remove auto scroll*
 * *create a new dedicated method in the service to do scroll queries.*



--
This message was sent by Atlassian Jira
(v8.20.10#820010)