You are viewing a plain text version of this content. The canonical link for it is here.

Posted to notifications@iotdb.apache.org by "Jialin Qiao (Jira)" <ji...@apache.org> on 2021/04/13 03:47:00 UTC

[jira] [Created] (IOTDB-1297) Refactor the memory control when enabling time partitions

Jialin Qiao created IOTDB-1297:
----------------------------------

             Summary: Refactor the memory control when enabling time partitions
                 Key: IOTDB-1297
                 URL: https://issues.apache.org/jira/browse/IOTDB-1297
             Project: Apache IoTDB
          Issue Type: Task
          Components: Core/Engine
            Reporter: Jialin Qiao


Currently, the DirectByteBuffer allocated for WAL could be calculated as the number of working TsFileProcessors * wal_buffer_size, where the max number of working TsFileProcessors depends on the concurrentWritingTimePartition(has an upper bound but depends on the write workload) and the number of storage groups. 

 

So, when there are too many storage groups, OOM may happen.

 

We need to redesign the memory control mechanism, some questions need to consider:

 

(1) Whether to reserve the concurrentWritingTimePartition parameter (currently this is an internal parameter, users can not see this)? Do we need to restrict this?

(2) Do we need to restrict the number of created storage groups or adjust the wal_buffer_size dynamically?

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)