You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2021/09/23 03:29:23 UTC
[GitHub] [iotdb] liuminghui233 opened a new pull request #4018: [IOTDB-1607] Optimize Tracing
liuminghui233 opened a new pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018
I do some works in this PR to make tracing more useful for error reporting and discovering performance bottlenecks.
For more details, please refer: https://issues.apache.org/jira/browse/IOTDB-1607
Welcome to discuss and contribute!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43398897)
Coverage decreased (-0.005%) to 67.788% when pulling **a051e0595a1b788cba065c679aad4c1922703d33 on liuminghui233:tracingOpt** into **c1e43e25267a63cf087f07bfc203b4e955198ecd on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] liuminghui233 commented on a change in pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
liuminghui233 commented on a change in pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#discussion_r723802249
##########
File path: server/src/main/java/org/apache/iotdb/db/qp/physical/PhysicalPlan.java
##########
@@ -96,6 +96,7 @@
protected long index;
private boolean debug;
+ private boolean enableTracing = false;
Review comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43415863)
Coverage decreased (-0.07%) to 67.724% when pulling **a9e224eec90e78f0a498cf028d3b948654675c1c on liuminghui233:tracingOpt** into **c1e43e25267a63cf087f07bfc203b4e955198ecd on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43427926)
Coverage decreased (-0.06%) to 67.737% when pulling **3452759a93e0766e201bc960d0e5e2d0805e9054 on liuminghui233:tracingOpt** into **c1e43e25267a63cf087f07bfc203b4e955198ecd on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43431585)
Coverage decreased (-0.04%) to 67.753% when pulling **c9109c9969f06efb677ff459e11968094e33f96f on liuminghui233:tracingOpt** into **c1e43e25267a63cf087f07bfc203b4e955198ecd on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43480954)
Coverage decreased (-0.005%) to 67.723% when pulling **f63195afc9e9b2cd15910fd929cab8a1a8b0895c on liuminghui233:tracingOpt** into **e1a7e1390c809922fcfcfff501b434c15cb130a2 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] liuminghui233 commented on a change in pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
liuminghui233 commented on a change in pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#discussion_r723799385
##########
File path: server/src/main/java/org/apache/iotdb/db/qp/logical/crud/QueryOperator.java
##########
@@ -64,9 +64,12 @@
protected Map<String, Object> props;
protected IndexType indexType;
+ protected boolean enableTracing;
+
public QueryOperator() {
super(SQLConstant.TOK_QUERY);
operatorType = Operator.OperatorType.QUERY;
+ enableTracing = false;
Review comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43462258)
Coverage increased (+0.02%) to 67.749% when pulling **3e57dec68a19ecc232817bc801fbf946a662c12c on liuminghui233:tracingOpt** into **e1a7e1390c809922fcfcfff501b434c15cb130a2 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] Alima777 merged pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
Alima777 merged pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43397196)
Coverage decreased (-0.02%) to 67.783% when pulling **4a9cb1fa7fdf3ce40a7958e341eb7894719ff09f on liuminghui233:tracingOpt** into **ed159dfa4b6cbb8ee9b7e2121b20ef5ec6b1f60a on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43394017)
Coverage increased (+0.02%) to 67.823% when pulling **bd33e06244a972973380c52a6a9fbe86af73824f on liuminghui233:tracingOpt** into **ed159dfa4b6cbb8ee9b7e2121b20ef5ec6b1f60a on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls commented on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls commented on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43014564)
Coverage increased (+0.03%) to 67.449% when pulling **ca133b6079b93fd2074bc33481fe47848137a594 on liuminghui233:tracingOpt** into **6dc45c6c4210f117099a1a31b50288f868630ddc on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43233298)
Coverage increased (+0.4%) to 67.799% when pulling **363f0b82a1894d6ba3eacd6735b9095cb06f42d3 on liuminghui233:tracingOpt** into **6dc45c6c4210f117099a1a31b50288f868630ddc on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] liuminghui233 commented on a change in pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
liuminghui233 commented on a change in pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#discussion_r723802709
##########
File path: server/src/main/java/org/apache/iotdb/db/query/control/tracing/TracingManager.java
##########
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.db.query.control.tracing;
+
+import org.apache.iotdb.db.engine.storagegroup.TsFileResource;
+import org.apache.iotdb.service.rpc.thrift.TSTracingInfo;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+public class TracingManager {
+
+ // (queryId -> TracingInfo)
+ private final Map<Long, TracingInfo> queryIdToTracingInfo = new ConcurrentHashMap<>();
+
+ public TracingManager() {}
+
+ public static TracingManager getInstance() {
+ return TracingManagerHelper.INSTANCE;
+ }
+
+ private TracingInfo getTracingInfo(long queryId) {
+ return queryIdToTracingInfo.computeIfAbsent(queryId, k -> new TracingInfo());
+ }
+
+ public void setSeriesPathNum(long queryId, int pathsNum) {
+ getTracingInfo(queryId).setSeriesPathNum(pathsNum);
+ }
+
+ public void addTsFileSet(
+ long queryId, List<TsFileResource> seqResources, List<TsFileResource> unseqResources) {
+ getTracingInfo(queryId).addTsFileSet(seqResources, unseqResources);
+ }
+
+ public void addChunkInfo(long queryId, int chunkNum, long pointsNum, boolean seq) {
+ getTracingInfo(queryId).addChunkInfo(chunkNum, pointsNum, seq);
+ }
+
+ public void addTotalPageNum(long queryId, int pageNum) {
+ getTracingInfo(queryId).addTotalPageNum(pageNum);
+ }
+
+ public void addOverlappedPageNum(long queryId) {
+ getTracingInfo(queryId).addOverlappedPageNum();
+ }
+
+ public void setStartTime(long queryId, long startTime) {
+ getTracingInfo(queryId).setStartTime(startTime);
+ }
+
+ public void registerActivity(long queryId, String activity, long startTime) {
+ getTracingInfo(queryId).addActivity(activity, startTime);
+ }
+
+ public TSTracingInfo fillRpcReturnTracingInfo(long queryId) {
+ return getTracingInfo(queryId).fillRpcReturnTracingInfo();
+ }
Review comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] liuminghui233 commented on a change in pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
liuminghui233 commented on a change in pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#discussion_r723803904
##########
File path: server/src/main/java/org/apache/iotdb/db/service/TSServiceImpl.java
##########
@@ -748,18 +755,22 @@ private TSExecuteStatementResp internalExecuteQueryStatement(
AUDIT_LOGGER.debug(
"Session {} execute Query: {}", sessionManager.getCurrSessionId(), statement);
- final long startTime = System.currentTimeMillis();
+ final long queryStartTime = System.currentTimeMillis();
Review comment:
Compared with queryStartTime, this.startTime includes the time of parse SQL to physical plan, which is needed in tracing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] Alima777 commented on a change in pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
Alima777 commented on a change in pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#discussion_r722065492
##########
File path: jdbc/src/main/java/org/apache/iotdb/jdbc/AbstractIoTDBJDBCResultSet.java
##########
@@ -90,6 +91,7 @@ public AbstractIoTDBJDBCResultSet(
timeout,
sgColumns,
aliasColumnMap);
+ this.ioTDBRpcTracingInfo = new IoTDBTracingInfo();
Review comment:
Initialization required? We only use it when it's set explicitly.
##########
File path: server/src/main/java/org/apache/iotdb/db/qp/logical/crud/QueryOperator.java
##########
@@ -64,9 +64,12 @@
protected Map<String, Object> props;
protected IndexType indexType;
+ protected boolean enableTracing;
+
public QueryOperator() {
super(SQLConstant.TOK_QUERY);
operatorType = Operator.OperatorType.QUERY;
+ enableTracing = false;
Review comment:
Its default is false. Remove this.
##########
File path: server/src/main/java/org/apache/iotdb/db/service/TSServiceImpl.java
##########
@@ -748,18 +755,22 @@ private TSExecuteStatementResp internalExecuteQueryStatement(
AUDIT_LOGGER.debug(
"Session {} execute Query: {}", sessionManager.getCurrSessionId(), statement);
- final long startTime = System.currentTimeMillis();
+ final long queryStartTime = System.currentTimeMillis();
Review comment:
Why two `startTime` parameter?
##########
File path: server/src/main/java/org/apache/iotdb/db/qp/physical/PhysicalPlan.java
##########
@@ -96,6 +96,7 @@
protected long index;
private boolean debug;
+ private boolean enableTracing = false;
Review comment:
Move this to queryPlan?
##########
File path: server/src/main/java/org/apache/iotdb/db/query/control/tracing/TracingManager.java
##########
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.db.query.control.tracing;
+
+import org.apache.iotdb.db.engine.storagegroup.TsFileResource;
+import org.apache.iotdb.service.rpc.thrift.TSTracingInfo;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+public class TracingManager {
+
+ // (queryId -> TracingInfo)
+ private final Map<Long, TracingInfo> queryIdToTracingInfo = new ConcurrentHashMap<>();
+
+ public TracingManager() {}
+
+ public static TracingManager getInstance() {
+ return TracingManagerHelper.INSTANCE;
+ }
+
+ private TracingInfo getTracingInfo(long queryId) {
+ return queryIdToTracingInfo.computeIfAbsent(queryId, k -> new TracingInfo());
+ }
+
+ public void setSeriesPathNum(long queryId, int pathsNum) {
+ getTracingInfo(queryId).setSeriesPathNum(pathsNum);
+ }
+
+ public void addTsFileSet(
+ long queryId, List<TsFileResource> seqResources, List<TsFileResource> unseqResources) {
+ getTracingInfo(queryId).addTsFileSet(seqResources, unseqResources);
+ }
+
+ public void addChunkInfo(long queryId, int chunkNum, long pointsNum, boolean seq) {
+ getTracingInfo(queryId).addChunkInfo(chunkNum, pointsNum, seq);
+ }
+
+ public void addTotalPageNum(long queryId, int pageNum) {
+ getTracingInfo(queryId).addTotalPageNum(pageNum);
+ }
+
+ public void addOverlappedPageNum(long queryId) {
+ getTracingInfo(queryId).addOverlappedPageNum();
+ }
+
+ public void setStartTime(long queryId, long startTime) {
+ getTracingInfo(queryId).setStartTime(startTime);
+ }
+
+ public void registerActivity(long queryId, String activity, long startTime) {
+ getTracingInfo(queryId).addActivity(activity, startTime);
+ }
+
+ public TSTracingInfo fillRpcReturnTracingInfo(long queryId) {
+ return getTracingInfo(queryId).fillRpcReturnTracingInfo();
+ }
Review comment:
Remove this after using in case memory leak.
```suggestion
public TSTracingInfo fillRpcReturnTracingInfo(long queryId) {
return queryIdToTracingInfo.remove(queryId).fillRpcReturnTracingInfo();
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] liuminghui233 commented on a change in pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
liuminghui233 commented on a change in pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#discussion_r723799153
##########
File path: jdbc/src/main/java/org/apache/iotdb/jdbc/AbstractIoTDBJDBCResultSet.java
##########
@@ -90,6 +91,7 @@ public AbstractIoTDBJDBCResultSet(
timeout,
sgColumns,
aliasColumnMap);
+ this.ioTDBRpcTracingInfo = new IoTDBTracingInfo();
Review comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43330436)
Coverage decreased (-0.004%) to 67.777% when pulling **80a0d60a8c0ab1fdeb26965bb2ed766bbf938f7b on liuminghui233:tracingOpt** into **cbbdc6caf51660e5817a4e9c854831d820315b72 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43271699)
Coverage increased (+0.4%) to 67.782% when pulling **1ecbf6e3ad73a9c434b7b999a354033464b0ac72 on liuminghui233:tracingOpt** into **6dc45c6c4210f117099a1a31b50288f868630ddc on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43044246)
Coverage increased (+0.1%) to 67.543% when pulling **5204f1110d5de0f7cabaeb339cf8510390e8708b on liuminghui233:tracingOpt** into **6dc45c6c4210f117099a1a31b50288f868630ddc on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43019332)
Coverage increased (+0.09%) to 67.513% when pulling **e8a934d71c8eb83da6ed9280f643d701053acc9d on liuminghui233:tracingOpt** into **6dc45c6c4210f117099a1a31b50288f868630ddc on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43330547)
Coverage increased (+0.004%) to 67.784% when pulling **80a0d60a8c0ab1fdeb26965bb2ed766bbf938f7b on liuminghui233:tracingOpt** into **cbbdc6caf51660e5817a4e9c854831d820315b72 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43330952)
Coverage decreased (-0.004%) to 67.777% when pulling **f8110de50f0b7118b2edb9b32a9991afb2bb243d on liuminghui233:tracingOpt** into **cbbdc6caf51660e5817a4e9c854831d820315b72 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] liuminghui233 commented on a change in pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
liuminghui233 commented on a change in pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#discussion_r723799153
##########
File path: jdbc/src/main/java/org/apache/iotdb/jdbc/AbstractIoTDBJDBCResultSet.java
##########
@@ -90,6 +91,7 @@ public AbstractIoTDBJDBCResultSet(
timeout,
sgColumns,
aliasColumnMap);
+ this.ioTDBRpcTracingInfo = new IoTDBTracingInfo();
Review comment:
fixed
##########
File path: server/src/main/java/org/apache/iotdb/db/qp/logical/crud/QueryOperator.java
##########
@@ -64,9 +64,12 @@
protected Map<String, Object> props;
protected IndexType indexType;
+ protected boolean enableTracing;
+
public QueryOperator() {
super(SQLConstant.TOK_QUERY);
operatorType = Operator.OperatorType.QUERY;
+ enableTracing = false;
Review comment:
fixed
##########
File path: server/src/main/java/org/apache/iotdb/db/qp/physical/PhysicalPlan.java
##########
@@ -96,6 +96,7 @@
protected long index;
private boolean debug;
+ private boolean enableTracing = false;
Review comment:
fixed
##########
File path: server/src/main/java/org/apache/iotdb/db/query/control/tracing/TracingManager.java
##########
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.db.query.control.tracing;
+
+import org.apache.iotdb.db.engine.storagegroup.TsFileResource;
+import org.apache.iotdb.service.rpc.thrift.TSTracingInfo;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+public class TracingManager {
+
+ // (queryId -> TracingInfo)
+ private final Map<Long, TracingInfo> queryIdToTracingInfo = new ConcurrentHashMap<>();
+
+ public TracingManager() {}
+
+ public static TracingManager getInstance() {
+ return TracingManagerHelper.INSTANCE;
+ }
+
+ private TracingInfo getTracingInfo(long queryId) {
+ return queryIdToTracingInfo.computeIfAbsent(queryId, k -> new TracingInfo());
+ }
+
+ public void setSeriesPathNum(long queryId, int pathsNum) {
+ getTracingInfo(queryId).setSeriesPathNum(pathsNum);
+ }
+
+ public void addTsFileSet(
+ long queryId, List<TsFileResource> seqResources, List<TsFileResource> unseqResources) {
+ getTracingInfo(queryId).addTsFileSet(seqResources, unseqResources);
+ }
+
+ public void addChunkInfo(long queryId, int chunkNum, long pointsNum, boolean seq) {
+ getTracingInfo(queryId).addChunkInfo(chunkNum, pointsNum, seq);
+ }
+
+ public void addTotalPageNum(long queryId, int pageNum) {
+ getTracingInfo(queryId).addTotalPageNum(pageNum);
+ }
+
+ public void addOverlappedPageNum(long queryId) {
+ getTracingInfo(queryId).addOverlappedPageNum();
+ }
+
+ public void setStartTime(long queryId, long startTime) {
+ getTracingInfo(queryId).setStartTime(startTime);
+ }
+
+ public void registerActivity(long queryId, String activity, long startTime) {
+ getTracingInfo(queryId).addActivity(activity, startTime);
+ }
+
+ public TSTracingInfo fillRpcReturnTracingInfo(long queryId) {
+ return getTracingInfo(queryId).fillRpcReturnTracingInfo();
+ }
Review comment:
fixed
##########
File path: server/src/main/java/org/apache/iotdb/db/service/TSServiceImpl.java
##########
@@ -748,18 +755,22 @@ private TSExecuteStatementResp internalExecuteQueryStatement(
AUDIT_LOGGER.debug(
"Session {} execute Query: {}", sessionManager.getCurrSessionId(), statement);
- final long startTime = System.currentTimeMillis();
+ final long queryStartTime = System.currentTimeMillis();
Review comment:
Compared with queryStartTime, this.startTime includes the time of parse SQL to physical plan, which is needed in tracing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43402174)
Coverage decreased (-0.07%) to 67.718% when pulling **7e0b5ec7f2337984b37faa193577c19646d0c98a on liuminghui233:tracingOpt** into **c1e43e25267a63cf087f07bfc203b4e955198ecd on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43330349)
Coverage decreased (-0.01%) to 67.766% when pulling **80a0d60a8c0ab1fdeb26965bb2ed766bbf938f7b on liuminghui233:tracingOpt** into **cbbdc6caf51660e5817a4e9c854831d820315b72 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43468887)
Coverage increased (+0.02%) to 67.745% when pulling **fa3d8ac5323027d0076e9885e9a00bd955e039ce on liuminghui233:tracingOpt** into **e1a7e1390c809922fcfcfff501b434c15cb130a2 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43399255)
Coverage decreased (-0.02%) to 67.776% when pulling **a051e0595a1b788cba065c679aad4c1922703d33 on liuminghui233:tracingOpt** into **c1e43e25267a63cf087f07bfc203b4e955198ecd on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43041016)
Coverage increased (+0.1%) to 67.561% when pulling **208cca1b03a603c710be38c33d269e19bb330680 on liuminghui233:tracingOpt** into **6dc45c6c4210f117099a1a31b50288f868630ddc on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #4018: [IOTDB-1607] Optimize Tracing
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #4018:
URL: https://github.com/apache/iotdb/pull/4018#issuecomment-925495718
[](https://coveralls.io/builds/43494155)
Coverage increased (+0.03%) to 67.758% when pulling **640078d4b07ad189d44c12d59ba409ab4ba7377c on liuminghui233:tracingOpt** into **e1a7e1390c809922fcfcfff501b434c15cb130a2 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org