You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@iotdb.apache.org by "yanze chen (Jira)" <ji...@apache.org> on 2022/10/13 08:44:00 UTC
[jira] [Commented] (IOTDB-4022) Adjust the storage_group_report_threshold according to memory for writing
[ https://issues.apache.org/jira/browse/IOTDB-4022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616878#comment-17616878 ]
yanze chen commented on IOTDB-4022:
-----------------------------------
更新测试：
吞吐(point/s)，写入内存分配=8980481489
|| ||aligned_seq_config.properties||seq_config.properties||unseq_config.properties||aligned_unseq_config.properties||
|storage_group_report_threshold=16777216|8912483.19|17433312.42/ 16564385.56|3345125.65 |半个小时只跑了10%|
|write_memory_vriation_report_proportion=0.001|13742844.78|16521936.57 /13984814.52|5082498.77 | |
吞吐(point/s)，总内存分配=100M
|| ||seq_config.properties||
|storage_group_report_threshold=16777216|写入一部分后OOM|
|write_memory_vriation_report_proportion=0.001|写入失败，reject|
> Adjust the storage_group_report_threshold according to memory for writing
> -------------------------------------------------------------------------
>
> Key: IOTDB-4022
> URL: https://issues.apache.org/jira/browse/IOTDB-4022
> Project: Apache IoTDB
> Issue Type: Improvement
> Reporter: Jialin Qiao
> Assignee: yanze chen
> Priority: Major
> Labels: pull-request-available
> Attachments: 完整实验结果.md
>
>
> Currently, we have a parameter called storage_group_report_threshold, which means when a storage group report its memory usage, it should accumulate a memory change of this number.
> If this parameter is too small, may report frequently, which may decrease the insertion performance.
> If this parameter is too large, may report not in time, which may cause OOM.
>
> We need to change this parameter according to the memory allocated for writing.
>
> (1) remove storage_group_report_threshold
> (2) Add a new parameter write_memory_vriation_report_proportion
>
> To rel/0.13 and master
--
This message was sent by Atlassian Jira
(v8.20.10#820010)