[jira] [Created] (UNOMI-739) Implemenation of purge of sessions / events following rollover usage.

["David Griffon (Jira)" <ji...@apache.org>]

David Griffon created UNOMI-739:
-----------------------------------

             Summary: Implemenation of purge of sessions / events following rollover usage.
                 Key: UNOMI-739
                 URL: https://issues.apache.org/jira/browse/UNOMI-739
             Project: Apache Unomi
          Issue Type: New Feature
            Reporter: David Griffon


Currently the configuration is set by:

{code}
# Purge all monthly indexes (sessions/events) that have been created for a specific number of months
org.apache.unomi.monthly.index.purge.existTime=${env:UNOMI_MONTHLY_INDEX_PURGE_EXISTTIME:-12}
{code}

it has to be changed to allow a purge by type: session and event
{code}
org.apache.unomi.sessions.purge.existTime=${env:UNOMI_SESSIONS_PURGE_EXISTTIME:-}
org.apache.unomi.events.purge.existTime=${env:UNOMI_EVENTS_PURGE_EXISTTIME:-}
{code}

Old value should be deprecated and use as default if the new one are not set. 

h2. Implementation
2 mechanisms are involved in the purge of sessions / events:

h3. Deleting old documents:
There is a job that currently purge monthly indices and profiles, the goal is to replace the [monthly indices purge|https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L402] by a dedicated method to remove events and sessions. 

This method will work as the [profile purge|https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L333] calling {{persistenceService.removeByQuery(..)}}.
Like in here:
https://github.com/apache/unomi/blob/7feff5c5067ff679a079211d9f2eb989898efc75/services/src/main/java/org/apache/unomi/services/impl/profiles/ProfileServiceImpl.java#L365

*A log need to be added to mention the number of document removed.*

h3. Removing empty indices. 
When documents are deleted, it will happen that some indices contain no document, we need to have a dedicated execution to remove the empty indices. 

*A log need to be added to mention the deleted indices and the remaining indices with the number of documents in each.*

h2. Handle TODOs
- Remove index deletion from the rollup configuration.
- Enable back and adapt integration test related to the purge.  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)