You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@iotdb.apache.org by "huxiangpeng (Jira)" <ji...@apache.org> on 2023/02/16 06:42:00 UTC
[jira] [Commented] (IOTDB-5512) Fixed IoTConsensus may repeatedly send some log when restarting
[ https://issues.apache.org/jira/browse/IOTDB-5512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17689561#comment-17689561 ]
huxiangpeng commented on IOTDB-5512:
------------------------------------
Please see details in [feishu cloud doc|https://apache-iotdb.feishu.cn/docx/DddIdK9bnojYbsxQHFbcmkx7nKc].
> Fixed IoTConsensus may repeatedly send some log when restarting
> ---------------------------------------------------------------
>
> Key: IOTDB-5512
> URL: https://issues.apache.org/jira/browse/IOTDB-5512
> Project: Apache IoTDB
> Issue Type: Improvement
> Components: mpp-cluster
> Reporter: 刘珍
> Assignee: huxiangpeng
> Priority: Major
> Attachments: image-2023-02-09-15-45-05-253.png, image-2023-02-09-15-52-51-085.png, insert_no_overflow.config.properties
>
>
> 测试版本：rc/1.0.1 20230202 63b16f2
> 问题描述：
> 3副本3节点集群，benchmark写入顺序数据，重启前检查data 为全顺序，重启集群后，会生成部分乱序tsfile。
> 重启集群前查看所有dataregion的log同步情况，已同步完成：
> !image-2023-02-09-15-45-05-253.png!
> consensus 文件记录的已同步的log index是100000
> !image-2023-02-09-15-52-51-085.png!
> 重启集群后，这90条log会重发，导致有乱序tsfile生成，可以优化一下，解决这个问题。
> 测试流程：
> 1. 私有云3期
> 1ConfigNode 172.20.70.5
> 3DataNode 172.20.70.2/4/14
> benchmark 在172.20.70.13 （配置见附件）
> 集群配置参数
> ConfigNode
> MAX_HEAP_SIZE="20G"
> MAX_DIRECT_MEMORY_SIZE="6G"
> DataNode
> MAX_HEAP_SIZE="20G"
> MAX_DIRECT_MEMORY_SIZE="6G"
> dn_max_connection_for_internal_service=300
> common文件
> schema_replication_factor=3
> data_replication_factor=3
> enable_seq_space_compaction=false
> enable_unseq_space_compaction=false
> enable_cross_space_compaction=false
> config_node_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus
> schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus
> data_region_consensus_protocol_class=org.apache.iotdb.consensus.iot.IoTConsensus
> 2. 运行benchmark
> 3. 写入完成，检查data ，无乱序tsfile，log同步完成，重启集群。
> 预期结果，重启后，无乱序tsfile。
--
This message was sent by Atlassian Jira
(v8.20.10#820010)