You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2021/11/02 16:43:37 UTC
[GitHub] [iotdb] LebronAl commented on issue #4259: [Refactor Cluster] query performance improvement in cluster- branch analysis
LebronAl commented on issue #4259:
URL: https://github.com/apache/iotdb/issues/4259#issuecomment-957932611
## Environment:
- 16 Core
- 32G memory
- 300G HDD
- 10000 Mb/s
- Ubuntu 18.04.5 LTS
## Version:
- IoTDB master: 0013531610fea6b86ca53c259081abe4b0b78b68
- IoTDB cluster-: 886d06b5ba97407d0a45a58fe843caecf1a18f4e
- Benchmark master: 3ef6dca56ac8f8d197cd180fbfbe8cabdd6cc60d
## Cluster Configuration:
- default_replica_num = 1
- seed_nodes = "172.20.70.66:9003,172.20.70.67:9003,172.20.70.68:9003"
## Result
### master:
- write:
```
----------------------Main Configurations----------------------
CREATE_SCHEMA=true
START_TIME=2018-9-20T00:00:00+08:00
INSERT_DATATYPE_PROPORTION=1:1:1:1:1:1
BATCH_SIZE_PER_WRITE=10
IS_CLIENT_BIND=true
LOOP=300
IS_OUT_OF_ORDER=false
IS_REGULAR_FREQUENCY=true
GROUP_NUMBER=20
IS_RECENT_QUERY=false
QUERY_INTERVAL=250000
SENSOR_NUMBER=10
RESULT_PRECISION=0.1%
POINT_STEP=5000
CLIENT_NUMBER=20
SG_STRATEGY=mod
REAL_INSERT_RATE=1.0
OUT_OF_ORDER_MODE=0
DBConfig=
DB_SWITCH=IoTDB-012-SESSION_BY_TABLET
HOST=[172.20.70.66]
PORT=[6667]
USERNAME=root
PASSWORD=root
DB_NAME=test
TOKEN=token
DOUBLE_WRITE=false
BENCHMARK_WORK_MODE=testWithDefaultPath
OP_INTERVAL=0
OPERATION_PROPORTION=1:0:0:0:0:0:0:0:0:0:0
DEVICE_NUMBER=100000
OUT_OF_ORDER_RATIO=0.5
BENCHMARK_CLUSTER=false
IS_DELETE_DATA=true
IS_SENSOR_TS_ALIGNMENT=true
---------------------------------------------------------------
```
- read:
```
----------------------Main Configurations----------------------
CREATE_SCHEMA=false
START_TIME=2018-9-20T00:00:00+08:00
INSERT_DATATYPE_PROPORTION=1:1:1:1:1:1
BATCH_SIZE_PER_WRITE=10
IS_CLIENT_BIND=true
LOOP=3000
IS_OUT_OF_ORDER=false
IS_REGULAR_FREQUENCY=true
GROUP_NUMBER=20
IS_RECENT_QUERY=false
QUERY_INTERVAL=250000
SENSOR_NUMBER=10
RESULT_PRECISION=0.1%
POINT_STEP=5000
CLIENT_NUMBER=20
SG_STRATEGY=mod
REAL_INSERT_RATE=1.0
OUT_OF_ORDER_MODE=0
DBConfig=
DB_SWITCH=IoTDB-012-SESSION_BY_TABLET
HOST=[172.20.70.66]
PORT=[6667]
USERNAME=root
PASSWORD=root
DB_NAME=test
TOKEN=token
DOUBLE_WRITE=false
BENCHMARK_WORK_MODE=testWithDefaultPath
OP_INTERVAL=0
OPERATION_PROPORTION=0:1:1:1:1:1:1:1:1:1:1
DEVICE_NUMBER=100000
OUT_OF_ORDER_RATIO=0.5
BENCHMARK_CLUSTER=false
IS_DELETE_DATA=false
IS_SENSOR_TS_ALIGNMENT=true
---------------------------------------------------------------
```
- read twice(with cache):
### master:
- write:
```
----------------------Main Configurations----------------------
CREATE_SCHEMA=true
START_TIME=2018-9-20T00:00:00+08:00
INSERT_DATATYPE_PROPORTION=1:1:1:1:1:1
BATCH_SIZE_PER_WRITE=10
IS_CLIENT_BIND=true
LOOP=300
IS_OUT_OF_ORDER=false
IS_REGULAR_FREQUENCY=true
GROUP_NUMBER=20
IS_RECENT_QUERY=false
QUERY_INTERVAL=250000
SENSOR_NUMBER=10
RESULT_PRECISION=0.1%
POINT_STEP=5000
CLIENT_NUMBER=20
SG_STRATEGY=mod
REAL_INSERT_RATE=1.0
OUT_OF_ORDER_MODE=0
DBConfig=
DB_SWITCH=IoTDB-012-SESSION_BY_TABLET
HOST=[172.20.70.66]
PORT=[6667]
USERNAME=root
PASSWORD=root
DB_NAME=test
TOKEN=token
DOUBLE_WRITE=false
BENCHMARK_WORK_MODE=testWithDefaultPath
OP_INTERVAL=0
OPERATION_PROPORTION=1:0:0:0:0:0:0:0:0:0:0
DEVICE_NUMBER=100000
OUT_OF_ORDER_RATIO=0.5
BENCHMARK_CLUSTER=false
IS_DELETE_DATA=true
IS_SENSOR_TS_ALIGNMENT=true
---------------------------------------------------------------
```
- read:
```
----------------------Main Configurations----------------------
CREATE_SCHEMA=false
START_TIME=2018-9-20T00:00:00+08:00
INSERT_DATATYPE_PROPORTION=1:1:1:1:1:1
BATCH_SIZE_PER_WRITE=10
IS_CLIENT_BIND=true
LOOP=3000
IS_OUT_OF_ORDER=false
IS_REGULAR_FREQUENCY=true
GROUP_NUMBER=20
IS_RECENT_QUERY=false
QUERY_INTERVAL=250000
SENSOR_NUMBER=10
RESULT_PRECISION=0.1%
POINT_STEP=5000
CLIENT_NUMBER=20
SG_STRATEGY=mod
REAL_INSERT_RATE=1.0
OUT_OF_ORDER_MODE=0
DBConfig=
DB_SWITCH=IoTDB-012-SESSION_BY_TABLET
HOST=[172.20.70.66]
PORT=[6667]
USERNAME=root
PASSWORD=root
DB_NAME=test
TOKEN=token
DOUBLE_WRITE=false
BENCHMARK_WORK_MODE=testWithDefaultPath
OP_INTERVAL=0
OPERATION_PROPORTION=0:1:1:1:1:1:1:1:1:1:1
DEVICE_NUMBER=100000
OUT_OF_ORDER_RATIO=0.5
BENCHMARK_CLUSTER=false
IS_DELETE_DATA=false
IS_SENSOR_TS_ALIGNMENT=true
---------------------------------------------------------------
```
- read twice:
## Conclusion
In order to more objectively measure the performance comparison between `master` and `cluster-`, I conducted the above test. The test results show that the read and write performance of `cluster-` and `master` branches is basically the same, which is basically in line with our changes.
As for why @chengjianyun tests for a huge performance difference, I suspect there are two reasons:
1. The query time is too short, only a few seconds of the query, such a short time test results are often inaccurate. So I made the query loop larger during above test.
2. Leader distribution in the case of `default_replica_num=3` may also affect read and write performance. So I chose `default_replica_num=1` for comparison
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org