[jira] [Assigned] (IOTDB-6036) [delete timeseries] the mods file is too large, causing Query very slow even OOM problem

["Zhijia Cao (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IOTDB-6036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhijia Cao reassigned IOTDB-6036:
---------------------------------

    Assignee: Zhijia Cao

> [delete timeseries] the mods file is too large, causing Query very slow even OOM problem
> ----------------------------------------------------------------------------------------
>
>                 Key: IOTDB-6036
>                 URL: https://issues.apache.org/jira/browse/IOTDB-6036
>             Project: Apache IoTDB
>          Issue Type: Bug
>          Components: Core/Server
>            Reporter: Zhijia Cao
>            Assignee: Zhijia Cao
>            Priority: Blocker
>             Fix For: master branch
>
>         Attachments: Snipaste_2023-06-29_17-52-27.png, mods.png
>
>
> In some scenarios, users perform ttl operations by deleting the time series.
> There is a user every 5 seconds to delete 14 days old data, pseudo-SQL is as follows:
> {code:java}
> delete timeseres root.cisdi.im.steel.sentry.** where time < systemTime - 14d {code}
> Each execution of this SQL will append a line of records to the mods file, and after a while, the file will become very large, as shown in the attachment, the mods file has reached 200 of megabytes, and the deleted records in the file have been about 3 million lines.
>  
> When you query the contents related to this sequence, the query engine filters out the invalid contents in the mods file. Therefore, all the contents of the mods file are loaded into the memory. This operation may cause that the query result cannot be loaded for a long time or even the OOM result is displayed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)