You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@iotdb.apache.org by "刘珍 (Jira)" <ji...@apache.org> on 2023/01/13 10:06:00 UTC

[jira] [Commented] (IOTDB-5324) Fix wal cann't be deleted in destDataNode after region migration when data_replication_factor is 1 In IoTConsensus

    [ https://issues.apache.org/jira/browse/IOTDB-5324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676553#comment-17676553 ] 

刘珍 commented on IOTDB-5324:
---------------------------

master_0113_a5167ac ,已修复。

> Fix wal cann't be deleted in destDataNode  after region migration when data_replication_factor is 1 In IoTConsensus
> -------------------------------------------------------------------------------------------------------------------
>
>                 Key: IOTDB-5324
>                 URL: https://issues.apache.org/jira/browse/IOTDB-5324
>             Project: Apache IoTDB
>          Issue Type: Bug
>          Components: mpp-cluster
>    Affects Versions: master branch
>            Reporter: 刘珍
>            Assignee: Xinyu Tan
>            Priority: Major
>              Labels: pull-request-available
>         Attachments: 40971672369689_.pic.jpg, mig.conf, screenshot-1.png, screenshot-2.png
>
>
> m_1229_0fedffd
> 问题描述
> 1副本1C4D集群,写入数据过程中,迁移region(Id=1 from ip4 to ip14)成功,但是目的节点的wal删不掉。
> 1.启动1副本1C4D集群
> config/schema/data 是ratis/ratis/IoT协议
> 2.BM 写入数据(配置见附件)
> 9分钟后,迁移region
> ./sbin/start-cli.sh -h 172.20.70.4 -e "migrate region 1 from 2 to 3"
> 迁移成功,耗时20秒(2022-12-29 18:25:17,621-2022-12-29 18:25:37,676)
> 但是ip14的datanode 的regionId=1的wal删除不掉,导致大小为50GB,一直有限流的WARN日志,BM16个多小时不结束,理论上BM1个多小时就应该执行完成:
> 2022-12-30 10:14:07,669 [pool-25-IoTDB-ClientRPC-Processor-59$20221230_021337_10719_3.1.0] WARN  o.a.i.d.m.p.s.FragmentInstanceDispatcherImpl:243 - write locally failed. TSStatus: TSStatus(code:606, message:Reject write because there are too many requests need to process), message: Reject write because there are too many requests need to process
> 测试环境:私有云3期
> DataNode配置
> MAX_HEAP_SIZE="20G"
> MAX_DIRECT_MEMORY_SIZE="6G"
> dn_max_connection_for_internal_service=300
> ConfigNode配置
> MAX_HEAP_SIZE="20G"
> MAX_DIRECT_MEMORY_SIZE="6G"
> 迁移region前的region信息
>  !screenshot-1.png! 
> 迁移region成功后的region信息
>  !screenshot-2.png! 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)