You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@iotdb.apache.org by "周沛辰 (Jira)" <ji...@apache.org> on 2023/02/08 05:15:00 UTC
[jira] [Created] (IOTDB-5500) Fast compaction is slow with template
周沛辰 created IOTDB-5500:
--------------------------
Summary: Fast compaction is slow with template
Key: IOTDB-5500
URL: https://issues.apache.org/jira/browse/IOTDB-5500
Project: Apache IoTDB
Issue Type: Bug
Reporter: 周沛辰
Assignee: 周沛辰
h4. 对齐序列+template
FIT物理机 192.168.10.75
数据备份 /data/mpp_test/r_1214_b7516d4/data 顺序tsfile49（9.9GB），乱序tsfile186（39GB）。共48.9GB。
但是测试版本是 master 1214_2de2096
|序号|配置参数|合并耗时|数据一致性|合出的tsfile数量|合出的tsfile总大小|cpu/mem使用率率|
|1|cross_performer=fast|2022-12-14 13:51:55
2022-12-14 15:56:02
2小时4分钟7秒|/
|48
|30GB
|!https://apache-iotdb.feishu.cn/space/api/box/stream/download/asynccode/?code=MGY1Y2Y0ZTJlYjZmM2QxMGVkYjZkMDg2NDEwMTZhMzJfVFRncEVmVkZUTHYweFhaeU5FYjRlM1BmQ3E5cHp3ZkpfVG9rZW46Ym94Y243RW1Cd05LOWZvR0t4Uml3V1paaUVnXzE2NzU4MzMxNjM6MTY3NTgzNjc2M19WNA!
序号1：fast1内存使用最高，序号2和3差不多。
默认参数：cpu使用率最高，io最高。|
|2
|cross_performer=fast
inner_unseq_performer=fast
|2022-12-14 16:26:59
2022-12-14 18:27:46
2小时47秒|/
|49
|30GB
|
|3|cross_performer=fast
inner_unseq_performer=fast
inner_seq_performer=fast|2022-12-14 19:01:59
2022-12-14 21:07:28
2小时5分钟29秒|/|48
|30GB
|
|4|默认|2022-12-14 21:36:58
2022-12-14 23:43:14
2小时6分钟16秒|/
|47
|30GB
|
|5|结果描述
|差不多
|select min_time(s_477),max_time(s_477) ,count(s_477) from root.** align by device;|/
|/
| |
|6|测试结果|待定|通过|通过|通过|序号1的内存使用略高。|
h4. 普通序列+template 合并不限流
测试机器 fit 192.168.10.75 48CPU384GB
数据备份 /data/mpp_test/m_1215_831a45e/data
顺序数据：2731 （267G）
乱序数据：5276 (511G)
序号1 /data/mpp_test/m_1215_831a45e_fast1
序号2 /data/mpp_test/m_1215_831a45e_fast2
序号3 /data/mpp_test/m_1215_831a45e_fast3
序号4 /data/mpp_test/m_1215_831a45e_comm
compaction_write_throughput_mb_per_sec=1048576
|序号|配置参数|合并耗时|数据一致性|合出的tsfile数量|合出的tsfile总大小|cpu/mem使用率率|
|1|cross_performer=fast|2022-12-16 01:41:10
2022-12-16 10:51:20
33010秒|/
|494
|139GB
|CPU
!https://apache-iotdb.feishu.cn/space/api/box/stream/download/asynccode/?code=Y2I2MWRjZTRjOTc3Y2NlMTBiODE2NGU4NzM0NTA2YmRfM01pODUxNGdvZVhtSjRNT3pzZmVFdEJIN0NwTXBvellfVG9rZW46Ym94Y25RRUR0V21KTmlQM0NGaDVlYllBZEtlXzE2NzU4MzMxNjM6MTY3NTgzNjc2M19WNA!
!https://apache-iotdb.feishu.cn/space/api/box/stream/download/asynccode/?code=M2FjOTliZDZmNjYwMTJjYTlhZWU2YmU2YzdjZTMyODNfdDZoeEIyT3BKR25McGxhWU9weFJsWGZqMzNLNjB2d2RfVG9rZW46Ym94Y25sdzNoU3UybjJTemNqVFlzU0w2VThmXzE2NzU4MzMxNjM6MTY3NTgzNjc2M19WNA!
!https://apache-iotdb.feishu.cn/space/api/box/stream/download/asynccode/?code=ZGQwYTg4OTc4MDZlYjBhMzMxNDFmOTRhOGI2MTc5MDVfSWJ6NFlMV0loaGRNVnB2WnZacksxbTloUzROWlVid1ZfVG9rZW46Ym94Y25lUTI1N2NqTUF3ZDFCeUE2VmJmVGxlXzE2NzU4MzMxNjM6MTY3NTgzNjc2M19WNA!
!https://apache-iotdb.feishu.cn/space/api/box/stream/download/asynccode/?code=ZjY3MTM3NjRkNWFlNGJmZWI4MGJkMmM4NDgwYjQ3OTlfZGJLS3M5YkZ1VnB3b0xhWUNHRVdpRmVVYVNTdHNZSndfVG9rZW46Ym94Y250TnkxdjA5Szl4Z2ZQZkRxQ3hDZHpiXzE2NzU4MzMxNjM6MTY3NTgzNjc2M19WNA!
|
|2
|cross_performer=fast
inner_unseq_performer=fast
|2022-12-16 11:18:32
2022-12-16 19:09:33
28261秒|/
|453
|138GB
|
|3|cross_performer=fast
inner_unseq_performer=fast
inner_seq_performer=fast|2022-12-16 19:35:36
2022-12-17 05:21:06
35130秒|/|546
|270GB
|
|4|默认|2022-12-17 05:35:44
2022-12-17 15:57:24
37300秒|/
|495
|138GB
|
|5|结果描述
|!https://apache-iotdb.feishu.cn/space/api/box/stream/download/asynccode/?code=OGQ0YjU0NjljYjBmZmRlMzA3NTMwZWRmMDk5NDRlNmNfVGVQQ2dCaFN5eHV4ZzVvN1FSRGFKUHc4dmJlUUs5VWRfVG9rZW46Ym94Y25vWHFwbGJiejRSekoyWnI3cGJyMkpjXzE2NzU4MzMxNjM6MTY3NTgzNjc2M19WNA!
纵轴越高，合并越慢
默认/fast1=1.13
默认/fast2=1.32
默认/fast3=1.06|select min_time(s_433),max_time(s_433),count(s_433) from root.** align by device|3个参数设置fast合并完成，tsfile数多。
|3个参数设置fast合并完成，tsfile占用空间大。
|序号1 和序号2的使用内存比默认略高。
序号3和默认的使用内存差不多。
|
|6|测试结果|fast总体性能优势不明显。|通过|通过|不通过| |
--
This message was sent by Atlassian Jira
(v8.20.10#820010)