[jira] [Assigned] (IOTDB-2101) Reduce the memory footprint of QueryDataSource

["Yuan Tian (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IOTDB-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yuan Tian reassigned IOTDB-2101:
--------------------------------

              Assignee: Minghui Liu
    Remaining Estimate: 40h
     Original Estimate: 40h

> Reduce the memory footprint of QueryDataSource
> ----------------------------------------------
>
>                 Key: IOTDB-2101
>                 URL: https://issues.apache.org/jira/browse/IOTDB-2101
>             Project: Apache IoTDB
>          Issue Type: Improvement
>          Components: Core/Query
>            Reporter: Yuan Tian
>            Assignee: Minghui Liu
>            Priority: Major
>   Original Estimate: 40h
>  Remaining Estimate: 40h
>
> Now, we will create QueryDataSource for each sensor in one query. QueryDataSource contains two file lists, one for seq files and another one for unseq files. This means, if we have 90,000 files and 1,000 sensors in one query, we will occupy 90,000 * 1,000 * 24B(about 2GB memory footprint) which will cause OOM problems.
> However, we can let all sensors in one virtual sg share same QueryDataSource, and save its own seq and unseq cursor in SeriesReader.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)