You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2021/02/15 10:27:46 UTC
[GitHub] [iotdb] proven-mozdemir opened a new issue #2683: Cluster Data Replication Problem v.0.12.0-SNAPSHOT
proven-mozdemir opened a new issue #2683:
URL: https://github.com/apache/iotdb/issues/2683
**Describe the bug**
Cannot replicate data with IoTDB 0.12.0-SNAPSHOT. Lots of "TTransportException: null" exceptions seen on logs.
**To Reproduce**
Steps to reproduce the behavior:
1. Download source code
```
git clone https://github.com/apache/iotdb.git
mv iotdb iotdb-github
```
2. Compile with maven
```
cd iotdb-github/
./mvnw.sh clean package -DskipTests
cp cluster/target/cluster-0.12.0-SNAPSHOT.zip .
```
3. Extract zip from target into node1
```
unzip cluster-0.12.0-SNAPSHOT.zip
mkdir node1
mv lib node1/
mv sbin/ node1/
mv tools/ node1/
mv conf/ node1/
```
4. Copy node2 and node3 from node1
```
cp -rf node1/ node2
cp -rf node1/ node3
```
5. Change ports for node2 and node3
```
sed -i -e 's/6667/6668/g' node2/conf/iotdb-engine.properties
sed -i -e 's/6667/6669/g' node3/conf/iotdb-engine.properties
sed -i -e 's/31999/32000/g' node2/conf/iotdb-env.sh
sed -i -e 's/31999/32001/g' node3/conf/iotdb-env.sh
```
6. Start each nodes one by one in seperate terminals
```
cd node1/sbin
start-server.sh &
cd ../../node2/sbin
start-server.sh -internal_meta_port 9005 -internal_data_port 40012 -cluster_rpc_port 55561 &
cd ../../node3/sbin
start-server.sh -internal_meta_port 9007 -internal_data_port 40014 -cluster_rpc_port 55562 &
```
7. Download IoTDB Benchmark tool
`git clone https://github.com/thulab/iotdb-benchmark.git`
8. Open with IDE (mine is IntelliJ Ultimate)
9. Add new maven module as iotdb-0.12
10. Copy all files from iotdb-0.11 module and replace version in pom file 0.12.0-SNAPSHOT and class package as "cn.edu.tsinghua.iotdb.benchmark.iotdb012"
11. Replace artifactId as iotdb-0.12 and name Benchmark IoTDB 0.12 in pom.xml
12. Replace dependency version as 0.12.0-SNAPSHOT for dependencies; iotdb-jdbc and iotdb-session
13. Add new version class names in cn.edu.tsinghua.iotdb.benchmark.conf.Constants class;
```
public static final String IOTDB012_JDBC_CLASS = "cn.edu.tsinghua.iotdb.benchmark.iotdb012.IoTDB";
public static final String IOTDB012_SESSION_CLASS = "cn.edu.tsinghua.iotdb.benchmark.iotdb012.IoTDBSession";
```
14. Add version 0.12 support in cn.edu.tsinghua.iotdb.benchmark.tsdb.DBFactory class;
```
case "0.12.0":
if(config.getINSERT_MODE().equals(Constants.INSERT_USE_JDBC)) {
dbClass = Constants.IOTDB012_JDBC_CLASS;
} else {
dbClass = Constants.IOTDB012_SESSION_CLASS;
}
break;
case "0.11.0":
....
```
15. Create dist-dependencies folder in iotdb-0.12 module directory
16. Create jdbc, service-rpc, session, thrift, tsfile folders into dist-dependencies folder
17. copy pom.xml and jar files from iotdb-github folder into jdbc, service-rpc, session, thrift, tsfile folders
18. Manually install SNAPSHOT builds into local maven repository;
```
cd iotdb-github/dist-dependencies
mvn install:install-file -Dpackaging=pom -Dfile=./pom.xml -DpomFile=./pom.xml
mvn install:install-file -Dfile=./jdbc/target/iotdb-jdbc-0.12.0-SNAPSHOT.jar -Dpackaging=jar -DpomFile=./jdbc/pom.xml
mvn install:install-file -Dfile=./session/target/iotdb-session-0.12.0-SNAPSHOT.jar -Dpackaging=jar -DpomFile=./session/pom.xml
mvn install:install-file -Dfile=./service-rpc/target/service-rpc-0.12.0-SNAPSHOT.jar -Dpackaging=jar -DpomFile=./service-rpc/pom.xml
mvn install:install-file -Dfile=./thrift/target/iotdb-thrift-0.12.0-SNAPSHOT.jar -Dpackaging=jar -DpomFile=./thrift/pom.xml
mvn install:install-file -Dfile=./tsfile/target/tsfile-0.12.0-SNAPSHOT.jar -Dpackaging=jar -DpomFile=./tsfile/pom.xml
```
19. Change iotdb-benchmark tool config.properties file as given below, other parameters are left as default;
```
HOST=<lan ip of your server>
PORT=6667
DB_SWITCH=IoTDB
DB_NAME=plant
BENCHMARK_WORK_MODE=testWithDefaultPath
IS_DELETE_DATA=true
#Whether to create the schema before writing
CREATE_SCHEMA=true
#Write data start time
START_TIME=2021-02-01T00:00:00+03:00
VERSION=0.12.0
OPERATION_PROPORTION=1:0:0:0:0:0:0:0:0
INSERT_MODE=session
```
20. maven clean and install iotdb-benchmark tool
21. Execute test
**Expected behavior**
I expect data is replicated into all nodes;
```
cd iotdb-github
cli/sbin/start-cli.sh -p 6667
select s_0 from root.group_1.d_1 LIMIT 3;
<data exists>
cli/sbin/start-cli.sh -p 6668
select s_0 from root.group_1.d_1 LIMIT 3;
<NO data exists>
cli/sbin/start-cli.sh -p 6669
select s_0 from root.group_1.d_1 LIMIT 3;
<NO data exists>
```
I get some errors which given under "Additional Context" title
I found an FAQ document on this link; https://github.com/apache/iotdb/issues/2270
Translated it into English with Google Translate: https://translate.google.com/translate?sl=auto&tl=en&u=https://github.com/apache/iotdb/issues/2270
According to item one i changed port as 55560 in conf/config.properties file under iotdb-benchmark tool and tried to ingest again,
this time i got Connection Refused exception as given below;
```
11:39:08.999 [main] ERROR cn.edu.tsinghua.iotdb.benchmark.iotdb012.IoTDBSession - Failed to add session
org.apache.iotdb.rpc.IoTDBConnectionException: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.iotdb.session.SessionConnection.init(SessionConnection.java:91)
at org.apache.iotdb.session.SessionConnection.<init>(SessionConnection.java:79)
at org.apache.iotdb.session.Session.constructSessionConnection(Session.java:212)
at org.apache.iotdb.session.Session.open(Session.java:181)
at org.apache.iotdb.session.Session.open(Session.java:166)
at cn.edu.tsinghua.iotdb.benchmark.iotdb012.IoTDBSession.<init>(IoTDBSession.java:38)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
at java.base/java.lang.reflect.ReflectAccess.newInstance(ReflectAccess.java:124)
at java.base/jdk.internal.reflect.ReflectionFactory.newInstance(ReflectionFactory.java:346)
at java.base/java.lang.Class.newInstance(Class.java:604)
at cn.edu.tsinghua.iotdb.benchmark.tsdb.DBFactory.getDatabase(DBFactory.java:74)
at cn.edu.tsinghua.iotdb.benchmark.tsdb.DBWrapper.<init>(DBWrapper.java:33)
at cn.edu.tsinghua.iotdb.benchmark.App.testWithDefaultPath(App.java:98)
at cn.edu.tsinghua.iotdb.benchmark.App.main(App.java:68)
Caused by: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
at org.apache.iotdb.rpc.TElasticFramedTransport.open(TElasticFramedTransport.java:87)
at org.apache.iotdb.session.SessionConnection.init(SessionConnection.java:89)
... 16 common frames omitted
```
Whenever i try to insert using Command Line interface as given below;
```
cd iotdb-github
cli/sbin/start-cli.sh -p 55560
insert into root.group_d1.d_1(timestamp,value) values(1612126801000,87.74);
# i get data for;
select * from root.group_d1.d_1;
```
And also i get data by connecting 55561 and 55562 port numbers.
Interestingly when i ingest more data only node1's data folder size is getting larger than node2 and node3 ones.
This means data is actually stored only under node1 data folder.
**Ingest Replicated Data with IotDb benchmark tool**
At the end of the day i am looking for the correct way to replicate ingest operation to other nodes of cluster with iotdb-benchmark tool.
**Desktop (please complete the following information):**
- OS: Centos Linux
- Version: 8 with kernel 4.18.0-240.10.1.el8_3.x86_64
**Additional context**
Exceptions seen in error log file:
```
2021-02-15 11:48:51,749 [pool-8-IoTDB-RPC-Client-thread-2] ERROR org.apache.thrift.server.TThreadPoolServer$WorkerProcess:315 - Thrift error occurred during processing of message.
org.apache.thrift.transport.TTransportException: null
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.iotdb.rpc.TElasticFramedTransport.readFrame(TElasticFramedTransport.java:108)
at org.apache.iotdb.rpc.TElasticFramedTransport.read(TElasticFramedTransport.java:103)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:425)
at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:321)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:225)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:310)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] neuyilan edited a comment on issue #2683: Cluster Data Replication Problem v.0.12.0-SNAPSHOT
Posted by GitBox <gi...@apache.org>.
neuyilan edited a comment on issue #2683:
URL: https://github.com/apache/iotdb/issues/2683#issuecomment-783344085
Thank you very much for your detailed description,
1. The default distributed RPC port is 55560. I see that you have found the answer to this question.
2. For cluster version testing, we generally use the support_ cluster branch[1] for testing, and the master is not used for the cluster version.
> Interestingly when I ingest more data only node1's data folder size is getting larger than node2 and node3 ones.
> This means data is actually stored only under the node1 data folder.
3. About node1's data folder size is getting bigger than node2 and node3. Because the default copy is 2 copies, and the data is partitioned according to storage group name + time, it is not possible to guarantee the same size of the data on the three nodes. You can use this tool[1] to look at data partitioning.
> Lots of "TTransportException: null" exceptions seen on logs.
4. Yes, there are a large number of errors in the currently distributed version, but it does not affect the normal operation and can be ignored for the time being. However, we welcome to find out the essential cause of this problem.
[1] https://github.com/thulab/iotdb-benchmark/tree/support_cluster
[2] http://iotdb.apache.org/UserGuide/Master/System%20Tools/NodeTool.html
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] neuyilan commented on issue #2683: Cluster Data Replication Problem v.0.12.0-SNAPSHOT
Posted by GitBox <gi...@apache.org>.
neuyilan commented on issue #2683:
URL: https://github.com/apache/iotdb/issues/2683#issuecomment-790405249
After today, the RPC ports of the cluster version is the same as those in the stand-alone version, namely 6667.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] HTHou closed issue #2683: Cluster Data Replication Problem v.0.12.0-SNAPSHOT
Posted by GitBox <gi...@apache.org>.
HTHou closed issue #2683:
URL: https://github.com/apache/iotdb/issues/2683
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] neuyilan commented on issue #2683: Cluster Data Replication Problem v.0.12.0-SNAPSHOT
Posted by GitBox <gi...@apache.org>.
neuyilan commented on issue #2683:
URL: https://github.com/apache/iotdb/issues/2683#issuecomment-783344085
Thank you very much for your detailed description,
1. The default distributed RPC port is 55560. I see that you have found the answer to this question.
2. For cluster version testing, we generally use the support_ cluster branch[1] for testing, and the master is not used for the cluster version.
> Interestingly when I ingest more data only node1's data folder size is getting larger than node2 and node3 ones.
> This means data is actually stored only under the node1 data folder.
3. About node1's data folder size is getting bigger than node2 and node3. Because the default copy is 2 copies, and the data is partitioned according to storage group namee+ time, it is not possible to guarantee the same size of the data on the three nodes. You can use this tool[1] to look at data partitioning.
> Lots of "TTransportException: null" exceptions seen on logs.
4. Yes, there are a large number of errors in the currently distributed version, but it does not affect the normal operation and can be ignored for the time being. However, we welcome to find out the essential cause of this problem.
[1] https://github.com/thulab/iotdb-benchmark/tree/support_cluster
[2] http://iotdb.apache.org/UserGuide/Master/System%20Tools/NodeTool.html
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org