You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@iotdb.apache.org by "刘珍 (Jira)" <ji...@apache.org> on 2022/10/10 05:48:00 UTC
[jira] [Created] (IOTDB-4593) [ multiLeader remove datanode ] Removing nodes writes data
刘珍 created IOTDB-4593:
-------------------------
Summary: [ multiLeader remove datanode ] Removing nodes writes data
Key: IOTDB-4593
URL: https://issues.apache.org/jira/browse/IOTDB-4593
Project: Apache IoTDB
Issue Type: Bug
Affects Versions: 0.14.0-SNAPSHOT
Reporter: 刘珍
Assignee: Jinrui Zhang
Attachments: image-2022-10-10-13-36-14-475.png
m_0930_2a30316
问题描述：
缩容datanode，节点置为Removing状态，但是在继续接受写入：
2022-10-08 13:23:54,686 [pool-20-IoTDB-DataNodeInternalRPC-Processor-148] INFO o.a.i.c.conf.CommonConfig:305 - *Set system mode from Running to Removing*.
Removing状态后（create 207个tsfile），
!image-2022-10-10-13-36-14-475.png!
测试环境
1. 192.168.10.71-76 6台物理机 48cpu 384GB
3C : 192.168.10.72 , 73,74
6D : 192.168.10.72 , 73,74 , 75 , 76
benchmark：192.168.10.71
ConfigNode配置参数
MAX_HEAP_SIZE="8G"
schema_region_consensus_protocol_class=org.apache.iotdb.consensus.ratis.RatisConsensus
data_region_consensus_protocol_class=org.apache.iotdb.consensus.multileader.MultiLeaderConsensus
schema_replication_factor=3
data_replication_factor=3
connection_timeout_ms=120000
DataNode配置参数
MAX_HEAP_SIZE="256G"
MAX_DIRECT_MEMORY_SIZE="32G"
connection_timeout_ms=120000
max_connection_for_internal_service=200
max_waiting_time_when_insert_blocked=600000
query_timeout_threshold=36000000
2. benchmark 配置文件见附件
GROUP_NUMBER=10
DEVICE_NUMBER=50000
SENSOR_NUMBER=600
IS_OUT_OF_ORDER=false
OPERATION_PROPORTION=1:0:0:0:0:0:0:0:0:0:0
CLIENT_NUMBER=100
LOOP=100
BATCH_SIZE_PER_WRITE=100
3. 运行1小时，ip72缩容
liuzhen@fit-72:/data/mpp_test/m_0930_2a30316/datanode$ cat 1008_test_remove_1h.sh
sleep 1h
/data/mpp_test/m_0930_2a30316/datanode/sbin/start-cli.sh -h 192.168.10.72 -e "show cluster" > 1008_3c5d_bef_remove.out
/data/mpp_test/m_0930_2a30316/datanode/sbin/start-cli.sh -h 192.168.10.72 -e "show regions" >> 1008_3c5d_bef_remove.out
/data/mpp_test/m_0930_2a30316/datanode/sbin/remove-datanode.sh "192.168.10.72:6667" >> 1008_3c5d_1hour_remove_ip72.out
4. ip72 的日志见机器上的备份
/data/mpp_test/m_0930_2a30316/datanode/logs_bm_1h_remove_ip72
--
This message was sent by Atlassian Jira
(v8.20.10#820010)