[jira] [Created] (IOTDB-5623) [compaction] The optimized version compacts slowly

["刘珍 (Jira)" <ji...@apache.org>]

刘珍 created IOTDB-5623:
-------------------------

             Summary: [compaction] The optimized version compacts slowly
                 Key: IOTDB-5623
                 URL: https://issues.apache.org/jira/browse/IOTDB-5623
             Project: Apache IoTDB
          Issue Type: Improvement
          Components: Core/Compaction, mpp-cluster
            Reporter: 刘珍
            Assignee: 周沛辰
         Attachments: comp1.conf, comp2.conf, image-2023-03-06-11-50-02-812.png, run_2_comp.sh, test.sh

对比balance合并调度策略优化后版本与优化前版本的合并性能，优化后版本慢80%：
 !image-2023-03-06-11-50-02-812.png! 
只有合并，无其他操作，数据50%乱序。
测试环境：192.168.130.1 1C1D   16CPU 32G内存
1.关合并，生成数据备份
/data1/iotdb/m_0227_74c950d/data_close_comp_1partition
CN ：MAX_HEAP_SIZE="2G"
DN：MAX_HEAP_SIZE="20G"
MAX_DIRECT_MEMORY_SIZE="6G"
COMMON ：
 time_partition_interval=6048000000
 max_waiting_time_when_insert_blocked=3600000
 enable_seq_space_compaction=false
 enable_unseq_space_compaction=false
 enable_cross_space_compaction=false
benchmark配置见附件

2. copy 同一份数据，对比2个版本的合并性能
2个版本的显式设置的参数：
CN ：
MAX_HEAP_SIZE="2G"
DN ：
MAX_HEAP_SIZE="20G"
MAX_DIRECT_MEMORY_SIZE="6G"
COMMON:
 time_partition_interval=6048000000
 query_timeout_threshold=36000000
 max_waiting_time_when_insert_blocked=3600000
 enable_seq_space_compaction=true
 enable_unseq_space_compaction=true
 enable_cross_space_compaction=true
compaction_write_throughput_mb_per_sec=1024

2个版本的参数对比
优化版多了一个参数：
opt_0302_33cd749/conf/iotdb-common.properties
优化版
max_number_of_points_in_page=10000
优化前的版本此参数
max_number_of_points_in_page=1048576

3.两个 版本日志路径
/data1/iotdb/bef_opt_854e4c8/logs
/data1/iotdb/opt_0302_33cd749/logs




--
This message was sent by Atlassian Jira
(v8.20.10#820010)