You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@iotdb.apache.org by "刘珍 (Jira)" <ji...@apache.org> on 2023/03/06 02:05:00 UTC
[jira] [Commented] (IOTDB-5192) [CrossSpaceCompaction] The level of compaction file is too large
[ https://issues.apache.org/jira/browse/IOTDB-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17696649#comment-17696649 ]
刘珍 commented on IOTDB-5192:
---------------------------
master_0302_33cd749 : balance合并任务调度优化，跨空间合并合出40+层级tsfile
!screenshot-1.png!
测试环境：3副本3C3D
192.168.10.62/66/68
ConfigNode ENV:
MAX_HEAP_SIZE="8G"
DataNode ENV:
MAX_HEAP_SIZE="192G"
MAX_DIRECT_MEMORY_SIZE="32G"
COMMON PROP:
schema_replication_factor=3
data_replication_factor=3
disk_space_warning_threshold=0.01
compaction_write_throughput_mb_per_sec=64
iot_consensus_throttle_threshold_in_byte=536870912000
benchmark配置文件见附件comp_2023.conf
> [CrossSpaceCompaction] The level of compaction file is too large
> -----------------------------------------------------------------
>
> Key: IOTDB-5192
> URL: https://issues.apache.org/jira/browse/IOTDB-5192
> Project: Apache IoTDB
> Issue Type: Improvement
> Components: mpp-cluster
> Reporter: 刘珍
> Assignee: 周沛辰
> Priority: Major
> Attachments: fast.conf, image-2022-12-13-16-06-23-875.png, screenshot-1.png
>
>
> master 1208 7f2218b
> 跨空间合并，合并文件层级过大（32层）：
> !image-2022-12-13-16-06-23-875.png!
> 日志：
> 2022-12-13 14:37:09,832 [pool-64-IoTDB-Compaction-1] INFO o.a.i.d.e.c.c.CrossSpaceCompactionTask:219 - root.test.g_1-10 [Compaction] CrossSpaceCompaction task finishes successfully, time cost is {color:#DE350B}*18029 *{color}s, compaction speed is 3.2967052791379006 MB/s
> 备份数据：
> fit 机器 192.168.10.72
> /data/mpp_test/m_1207_a0b2c8c/data
> 数据库配置：
> ConfigNode配置
> MAX_HEAP_SIZE="8G"
> DataNode配置
> MAX_HEAP_SIZE="256G"
> MAX_DIRECT_MEMORY_SIZE="32G"
> Common配置（此版本合并全失败，可以设置为关合并）
> 备份数据生成方式：
> BM配置文件见附件。
> master 1208 7f2218b Common配置 （confignode和datanode配置同上）：
> cross_performer=fast
> inner_unseq_performer=fast
--
This message was sent by Atlassian Jira
(v8.20.10#820010)