You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ignite.apache.org by GitBox <gi...@apache.org> on 2020/10/05 13:41:34 UTC
[GitHub] [ignite] SwirMix opened a new pull request #8316: IGNITE-13489 start and stop clients. Check topology version
SwirMix opened a new pull request #8316:
URL: https://github.com/apache/ignite/pull/8316
Thank you for submitting the pull request to the Apache Ignite.
In order to streamline the review of the contribution
we ask you to ensure the following steps have been taken:
### The Contribution Checklist
- [ ] There is a single JIRA ticket related to the pull request.
- [ ] The web-link to the pull request is attached to the JIRA ticket.
- [ ] The JIRA ticket has the _Patch Available_ state.
- [ ] The pull request body describes changes that have been made.
The description explains _WHAT_ and _WHY_ was made instead of _HOW_.
- [ ] The pull request title is treated as the final commit message.
The following pattern must be used: `IGNITE-XXXX Change summary` where `XXXX` - number of JIRA issue.
- [ ] A reviewer has been mentioned through the JIRA comments
(see [the Maintainers list](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-ReviewProcessandMaintainers))
- [ ] The pull request has been checked by the Teamcity Bot and
the `green visa` attached to the JIRA ticket (see [TC.Bot: Check PR](https://mtcga.gridgain.com/prs.html))
### Notes
- [How to Contribute](https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute)
- [Coding abbreviation rules](https://cwiki.apache.org/confluence/display/IGNITE/Abbreviation+Rules)
- [Coding Guidelines](https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines)
- [Apache Ignite Teamcity Bot](https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot)
If you need any help, please email dev@ignite.apache.org or ask anу advice on http://asf.slack.com _#ignite_ channel.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r504523794
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/reporter/Reporter.java
##########
@@ -0,0 +1,155 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node.reporter;
+
+import java.io.FileWriter;
+import java.io.IOException;
+import java.util.concurrent.ConcurrentLinkedDeque;
+
+/**
+ * report writer thread
+ */
+public class Reporter implements Runnable {
+ /**
+ * Queue with data for report.
+ * */
+ private ConcurrentLinkedDeque<Report> queue;
+
+ /**
+ * Target report file path.
+ * */
+ private String reportFile;
+
+ /** */
+ private String supportInfoTemplate = "Current action name: %s \n" +
+ "Number of threads in the test: %d \n" +
+ "Start time: %d \n";
+
+ /** */
+ private FileWriter fileWriter;
+
+ /**
+ * Title for csv report.
+ * */
+ private String reportHeader = "start_time," +
+ "end_time," +
+ "tx_count," +
+ "min_latency," +
+ "avg_latency," +
+ "max_latency," +
+ "percentile99%," +
+ "variance";
+
+ /**
+ * Template for the final report line.
+ * */
+ private String reportTemplate = "" +
+ "%d," +
+ "%d," +
+ "%d," +
+ "%d," +
+ "%d," +
+ "%d," +
+ "%d," +
+ "%d";
+
+ /** */
+ public Reporter(String reportFile, ConcurrentLinkedDeque concurrentLinkedDeque, Metadata metadata) throws IOException {
+ this.reportFile = reportFile;
+ this.queue = concurrentLinkedDeque;
+ this.fileWriter = new FileWriter(reportFile, true);
+ this.fileWriter.write(supportInfoBuilder(metadata));
+ this.fileWriter.write(this.reportHeader);
+ this.fileWriter.flush();
+ }
+
+ /** */
+ @Override public void run() {
+ while (!Thread.interrupted()) {
+ try {
+ writeAction();
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+ }
+ try {
+ fileWriter.close();
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+ }
+
+ /** */
+ private void writeAction() throws IOException {
+ try {
+
+ } finally {
+ if (!queue.isEmpty()) {
+ String singleReport = buildReportString(queue.pollFirst());
Review comment:
Why do you create a busy loop in finally block? Why to burn CPU waiting for new report?
Just call blocking method queue.take of LinkedBlockedQueue for new updates.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r503974406
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -112,12 +114,14 @@ private Report calculateInterimStatement() {
finally {
//calc dispersion
long x = 0;
- for (Long report : reports) {
- x = x + (report - avgLatency) ^ (2);
- }
+ for (Long report : reports) x = (long) (x + Math.pow(report - avgLatency, 2));
Review comment:
```
for (Long report : reports)
x += Math.pow(report - avgLatency, 2));
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r504186138
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -0,0 +1,137 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+import java.util.ArrayList;
+import java.util.Collections;
+
+import org.apache.ignite.internal.ducktest.tests.start_stop_client.node.reporter.Report;
+import org.apache.log4j.Logger;
+
+/**
+ * Single action thread.
+ */
+public class OperationThread implements Runnable {
+ /** */
+ private Action actionNode;
+
+ /** */
+ private Logger logger;
+
+ /** */
+ private long pacing;
+
+ /** batch*/
+ private static final long count = 1000;
+
+ /** */
+ public Action getActionNode() {
+ return actionNode;
+ }
+
+ /** */
+ public void setActionNode(ActionNode actionNode) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ }
+
+ /** */
+ public OperationThread(Action actionNode, Logger logger, long pacing) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ this.pacing = pacing;
+ }
+
+ /** {@inheritDoc} */
+ @Override public void run() {
+ while (!Thread.interrupted()) {
+ try {
+ actionNode.publishInternalReport(calculateInternalReport());
+ Thread.sleep(pacing);
+ } catch (InterruptedException e) {
+ }
Review comment:
You should not swallow `InterruptedException`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] SwirMix removed a comment on pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
SwirMix removed a comment on pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#issuecomment-706558313
add mini report.
Extract data from agent log
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r503979243
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -0,0 +1,140 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+import java.util.ArrayList;
+import java.util.Collections;
+
+import org.apache.ignite.internal.ducktest.tests.start_stop_client.node.reporter.Report;
+import org.apache.log4j.Logger;
+
+/**
+ * Single action thread.
+ */
+public class OperationThread implements Runnable {
+ /** */
+ private Action actionNode;
+
+ /** */
+ private Logger logger;
+
+ /** */
+ private long pacing;
+
+ /** batch*/
+ private static final long count = 1000;
+
+ /** */
+ public Action getActionNode() {
+ return actionNode;
+ }
+
+ /** */
+ public void setActionNode(ActionNode actionNode) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ }
+
+ /** */
+ public OperationThread(Action actionNode, Logger logger, long pacing) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ this.pacing = pacing;
+ }
+
+ /** {@inheritDoc} */
+ @Override public void run() {
+ while (!Thread.interrupted()) {
+ try {
+ actionNode.publishInternalReport(calculateInternalReport());
+ Thread.sleep(pacing);
+ } catch (InterruptedException e) {
+ }
+ }
+ }
+
+ /** */
+ private Report calculateInternalReport() {
+ long endTime = 0;
+ long txCount = 0;
+ long minLatency = -1;
+ long maxLatency = -1;
+ long avgLatency = 0;
+ long percentile99 = 0;
+
+ ArrayList<Long> reports = new ArrayList();
+
+ long latency = 0;
+ long batch = 0;
+ long startTime = System.nanoTime();
+
+ try {
+ while (batch < (count)) {
+ latency = actionNode.singleAction();
+ txCount++;
+
+ if (minLatency == -1 || minLatency > latency) {
+ minLatency = latency;
+ }
+
+ if (maxLatency < latency) {
+ maxLatency = latency;
+ }
+ reports.add(latency);
+ batch++;
+ }
+ endTime = System.nanoTime();
+
+ long sum = 0;
+ for (Long l : reports) {
+ sum += l;
+ }
+ if (!reports.isEmpty()) {
+ avgLatency = (Long) sum / reports.size();
+ Collections.sort(reports);
+
+ percentile99 = reports.get((int) (reports.size() * 0.99));
+ }
+ }
+ finally {
Review comment:
This is bad idea to use finally like this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r504183746
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -0,0 +1,137 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+import java.util.ArrayList;
+import java.util.Collections;
+
+import org.apache.ignite.internal.ducktest.tests.start_stop_client.node.reporter.Report;
+import org.apache.log4j.Logger;
+
+/**
+ * Single action thread.
+ */
+public class OperationThread implements Runnable {
+ /** */
+ private Action actionNode;
+
+ /** */
+ private Logger logger;
+
+ /** */
+ private long pacing;
+
+ /** batch*/
+ private static final long count = 1000;
+
+ /** */
+ public Action getActionNode() {
+ return actionNode;
+ }
+
+ /** */
+ public void setActionNode(ActionNode actionNode) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ }
+
+ /** */
+ public OperationThread(Action actionNode, Logger logger, long pacing) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ this.pacing = pacing;
+ }
+
+ /** {@inheritDoc} */
+ @Override public void run() {
+ while (!Thread.interrupted()) {
+ try {
+ actionNode.publishInternalReport(calculateInternalReport());
+ Thread.sleep(pacing);
+ } catch (InterruptedException e) {
+ }
+ }
+ }
+
+ /** */
+ private Report calculateInternalReport() {
+ long endTime = 0;
+ long txCount = 0;
+ long minLatency = -1;
+ long maxLatency = -1;
+ double avgLatency = 0;
+ long percentile99 = 0;
+
+ ArrayList<Long> reports = new ArrayList();
+
+ long latency = 0;
+ long batch = 0;
+ long startTime = System.nanoTime();
+
+ while (batch < (count)) {
+ latency = actionNode.singleAction();
+ txCount++;
+
+ if (minLatency == -1 || minLatency > latency) {
+ minLatency = latency;
+ }
+
+ if (maxLatency < latency) {
+ maxLatency = latency;
+ }
+ reports.add(latency);
+ batch++;
+ }
+ endTime = System.nanoTime();
+
+ long sum = 0;
+ for (Long l : reports) {
+ sum += l;
+ }
+
+ double sum_d = sum;
+
+ if (!reports.isEmpty()) {
+ avgLatency = sum_d / reports.size();
+ Collections.sort(reports);
+
+ percentile99 = reports.get((int) (reports.size() * 0.99));
+ }
+
+ //calc dispersion
+ double x = 0;
+ for (Long report : reports) x += Math.pow(report - avgLatency, 2);
Review comment:
Please, use
```
for (long report : reports)
x += Math.pow(report - avgLatency, 2);
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] SwirMix commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
SwirMix commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r503715600
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/Action.java
##########
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+/**
+ * Base interface.
+ */
+public interface Action {
+ /* Target action. **/
+ public long singleAction();
+
+ /* Method for publishing an internal report. **/
+ public void publishInterimReport(Report report);
Review comment:
this is internal report. Ok, renamed method
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] SwirMix commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
SwirMix commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r503715600
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/Action.java
##########
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+/**
+ * Base interface.
+ */
+public interface Action {
+ /* Target action. **/
+ public long singleAction();
+
+ /* Method for publishing an internal report. **/
+ public void publishInterimReport(Report report);
Review comment:
this id internal report. Ok, renamed method
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] SwirMix commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
SwirMix commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r503989104
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -0,0 +1,140 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+import java.util.ArrayList;
+import java.util.Collections;
+
+import org.apache.ignite.internal.ducktest.tests.start_stop_client.node.reporter.Report;
+import org.apache.log4j.Logger;
+
+/**
+ * Single action thread.
+ */
+public class OperationThread implements Runnable {
+ /** */
+ private Action actionNode;
+
+ /** */
+ private Logger logger;
+
+ /** */
+ private long pacing;
+
+ /** batch*/
+ private static final long count = 1000;
+
+ /** */
+ public Action getActionNode() {
+ return actionNode;
+ }
+
+ /** */
+ public void setActionNode(ActionNode actionNode) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ }
+
+ /** */
+ public OperationThread(Action actionNode, Logger logger, long pacing) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ this.pacing = pacing;
+ }
+
+ /** {@inheritDoc} */
+ @Override public void run() {
+ while (!Thread.interrupted()) {
+ try {
+ actionNode.publishInternalReport(calculateInternalReport());
+ Thread.sleep(pacing);
+ } catch (InterruptedException e) {
+ }
+ }
+ }
+
+ /** */
+ private Report calculateInternalReport() {
+ long endTime = 0;
+ long txCount = 0;
+ long minLatency = -1;
+ long maxLatency = -1;
+ long avgLatency = 0;
+ long percentile99 = 0;
+
+ ArrayList<Long> reports = new ArrayList();
+
+ long latency = 0;
+ long batch = 0;
+ long startTime = System.nanoTime();
+
+ try {
+ while (batch < (count)) {
+ latency = actionNode.singleAction();
+ txCount++;
+
+ if (minLatency == -1 || minLatency > latency) {
+ minLatency = latency;
+ }
+
+ if (maxLatency < latency) {
+ maxLatency = latency;
+ }
+ reports.add(latency);
+ batch++;
+ }
+ endTime = System.nanoTime();
+
+ long sum = 0;
+ for (Long l : reports) {
+ sum += l;
+ }
+ if (!reports.isEmpty()) {
+ avgLatency = (Long) sum / reports.size();
+ Collections.sort(reports);
+
+ percentile99 = reports.get((int) (reports.size() * 0.99));
+ }
+ }
+ finally {
Review comment:
could you explain why?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r503688107
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import org.apache.log4j.Logger;
+
+/**
+ * Single action thread.
+ */
+public class OperationThread implements Runnable {
+ /** */
+ private Action actionNode;
+
+ /** */
+ private Logger logger;
+
+ /** */
+ private long pacing;
+
+ /** batch*/
+ private static final long count = 1000;
+
+ /** */
+ public Action getActionNode() {
+ return actionNode;
+ }
+
+ /** */
+ public void setActionNode(ActionNode actionNode) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ }
+
+ /** */
+ public OperationThread(Action actionNode, Logger logger, long pacing) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ this.pacing = pacing;
+ }
+
+ /** {@inheritDoc} */
+ @Override public void run() {
+ while (!Thread.interrupted()) {
+ try {
+ actionNode.publishInterimReport(calculateInterimStatement());
+ Thread.sleep(pacing);
+ } catch (InterruptedException e) {
+ }
+ }
+ }
+
+ /** Building an interim report. */
+ private Report calculateInterimStatement() {
Review comment:
Accoring Oxford dictionary, "Statement is the act of stating or expressing something in words"
I suppose, that you mean smth like "data" or "figures"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r503687282
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/Action.java
##########
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+/**
+ * Base interface.
+ */
+public interface Action {
+ /* Target action. **/
+ public long singleAction();
+
+ /* Method for publishing an internal report. **/
+ public void publishInterimReport(Report report);
Review comment:
Do you mean `iterim` (smth between stages) or `internal`? So it seems that either comment is incorrect or method name.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r503973343
##########
File path: .idea/inspectionProfiles/Project_Default.xml
##########
@@ -1,776 +1,769 @@
-<profile version="1.0">
- <option name="myName" value="ignite_inspections"/>
Review comment:
Oops, revert this :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r503688957
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import org.apache.log4j.Logger;
+
+/**
+ * Single action thread.
+ */
+public class OperationThread implements Runnable {
+ /** */
+ private Action actionNode;
+
+ /** */
+ private Logger logger;
+
+ /** */
+ private long pacing;
+
+ /** batch*/
+ private static final long count = 1000;
+
+ /** */
+ public Action getActionNode() {
+ return actionNode;
+ }
+
+ /** */
+ public void setActionNode(ActionNode actionNode) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ }
+
+ /** */
+ public OperationThread(Action actionNode, Logger logger, long pacing) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ this.pacing = pacing;
+ }
+
+ /** {@inheritDoc} */
+ @Override public void run() {
+ while (!Thread.interrupted()) {
+ try {
+ actionNode.publishInterimReport(calculateInterimStatement());
+ Thread.sleep(pacing);
+ } catch (InterruptedException e) {
+ }
+ }
+ }
+
+ /** Building an interim report. */
+ private Report calculateInterimStatement() {
+ long endTime = 0;
+ long txCount = 0;
+ long minLatency = -1;
+ long maxLatency = -1;
+ long avgLatency = 0;
+ long percentile99 = 0;
+
+ ArrayList<Long> reports = new ArrayList();
+
+ long latency = 0;
+ long batch = 0;
+ long startTime = System.currentTimeMillis();
Review comment:
For measurements `System.currentTimeMillis` is a doubtful choice. For measurements a monotonic `System.nanoTime` is preferable. You can easily convert nanoseconds interval to millis using `Duration.of`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] SwirMix commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
SwirMix commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r503727878
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import org.apache.log4j.Logger;
+
+/**
+ * Single action thread.
+ */
+public class OperationThread implements Runnable {
+ /** */
+ private Action actionNode;
+
+ /** */
+ private Logger logger;
+
+ /** */
+ private long pacing;
+
+ /** batch*/
+ private static final long count = 1000;
+
+ /** */
+ public Action getActionNode() {
+ return actionNode;
+ }
+
+ /** */
+ public void setActionNode(ActionNode actionNode) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ }
+
+ /** */
+ public OperationThread(Action actionNode, Logger logger, long pacing) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ this.pacing = pacing;
+ }
+
+ /** {@inheritDoc} */
+ @Override public void run() {
+ while (!Thread.interrupted()) {
+ try {
+ actionNode.publishInterimReport(calculateInterimStatement());
+ Thread.sleep(pacing);
+ } catch (InterruptedException e) {
+ }
+ }
+ }
+
+ /** Building an interim report. */
+ private Report calculateInterimStatement() {
+ long endTime = 0;
+ long txCount = 0;
+ long minLatency = -1;
+ long maxLatency = -1;
+ long avgLatency = 0;
+ long percentile99 = 0;
+
+ ArrayList<Long> reports = new ArrayList();
+
+ long latency = 0;
+ long batch = 0;
+ long startTime = System.currentTimeMillis();
+
+ try {
+ while (batch < (count)) {
+ latency = actionNode.singleAction();
+ txCount++;
+
+ if (minLatency == -1 || minLatency > latency) {
+ minLatency = latency;
+ }
+
+ if (maxLatency < latency) {
+ maxLatency = latency;
+ }
+ reports.add(latency);
+ batch++;
+ }
+ endTime = System.currentTimeMillis();
+
+ long sum = 0;
+ for (Long l : reports) {
+ sum += l;
+ }
+ if (!reports.isEmpty()) {
+ avgLatency = sum / reports.size();
Review comment:
Yes, rounding is intentional.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r503974406
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -112,12 +114,14 @@ private Report calculateInterimStatement() {
finally {
//calc dispersion
long x = 0;
- for (Long report : reports) {
- x = x + (report - avgLatency) ^ (2);
- }
+ for (Long report : reports) x = (long) (x + Math.pow(report - avgLatency, 2));
Review comment:
```
for (Long report : reports)
x += Math.pow(report - avgLatency, 2));
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r504510532
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/ActionNode.java
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Date;
+import java.util.Optional;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentLinkedDeque;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+import com.fasterxml.jackson.databind.JsonNode;
+import org.apache.ignite.internal.ducktest.tests.start_stop_client.node.reporter.Metadata;
+import org.apache.ignite.internal.ducktest.tests.start_stop_client.node.reporter.Report;
+import org.apache.ignite.internal.ducktest.tests.start_stop_client.node.reporter.Reporter;
+import org.apache.ignite.internal.ducktest.utils.IgniteAwareApplication;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+/**
+ * The base class agent
+ */
+public abstract class ActionNode extends IgniteAwareApplication implements Action {
+ /** The unique name of the agent. */
+ private String nodeId = UUID.randomUUID().toString();
+
+ /**
+ * Queue with data for report.
+ * */
+ private ConcurrentLinkedDeque<Report> queue = new ConcurrentLinkedDeque<Report>();
+
+ /* **/
+ Reporter reporter;
+
+ /* **/
+ Metadata testInfo = new Metadata();
+
+ /** Start test time. */
+ protected Date startTime;
+
+ /** End test time. */
+ protected Date endTime;
+
+ /** Thread count. */
+ private int threads;
+
+ /** The delay between iterations of the action. */
+ protected long pacing;
+
+ /** Current action name. */
+ protected String actionName;
+
+ /** Thread executer. */
+ protected ExecutorService executor;
+
+ /** Threads. */
+ private ArrayList<OperationThread> operationThreads = new ArrayList();
+
+ /** */
+ private Logger log = LogManager.getLogger(ActionNode.class.getName());
+
+ /** {@inheritDoc} */
+ @Override protected void run(JsonNode jsonNode) throws Exception {
+ init(jsonNode);
+
+ markInitialized();
+
+ for (int i = 0; i < threads; i++) {
+ operationThreads.add(new OperationThread(this, log, pacing));
+ }
+ for (OperationThread thread : operationThreads) {
+ executor.execute(thread);
+ }
+
+ while (!terminated()) {
+ TimeUnit.MILLISECONDS.sleep(100);
+ }
+
+ executor.shutdownNow();
+ if (!executor.isShutdown()) {
Review comment:
executor.awaitTermination(100, TimeUnit.MILLISECONDS)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r504184219
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -0,0 +1,137 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+import java.util.ArrayList;
+import java.util.Collections;
+
+import org.apache.ignite.internal.ducktest.tests.start_stop_client.node.reporter.Report;
+import org.apache.log4j.Logger;
+
+/**
+ * Single action thread.
+ */
+public class OperationThread implements Runnable {
+ /** */
+ private Action actionNode;
+
+ /** */
+ private Logger logger;
+
+ /** */
+ private long pacing;
+
+ /** batch*/
+ private static final long count = 1000;
+
+ /** */
+ public Action getActionNode() {
+ return actionNode;
+ }
+
+ /** */
+ public void setActionNode(ActionNode actionNode) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ }
+
+ /** */
+ public OperationThread(Action actionNode, Logger logger, long pacing) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ this.pacing = pacing;
+ }
+
+ /** {@inheritDoc} */
+ @Override public void run() {
+ while (!Thread.interrupted()) {
+ try {
+ actionNode.publishInternalReport(calculateInternalReport());
+ Thread.sleep(pacing);
+ } catch (InterruptedException e) {
+ }
+ }
+ }
+
+ /** */
+ private Report calculateInternalReport() {
+ long endTime = 0;
+ long txCount = 0;
+ long minLatency = -1;
+ long maxLatency = -1;
+ double avgLatency = 0;
+ long percentile99 = 0;
+
+ ArrayList<Long> reports = new ArrayList();
+
+ long latency = 0;
+ long batch = 0;
+ long startTime = System.nanoTime();
+
+ while (batch < (count)) {
+ latency = actionNode.singleAction();
+ txCount++;
+
+ if (minLatency == -1 || minLatency > latency) {
+ minLatency = latency;
+ }
+
+ if (maxLatency < latency) {
+ maxLatency = latency;
+ }
+ reports.add(latency);
+ batch++;
+ }
+ endTime = System.nanoTime();
+
+ long sum = 0;
+ for (Long l : reports) {
+ sum += l;
+ }
+
+ double sum_d = sum;
+
+ if (!reports.isEmpty()) {
+ avgLatency = sum_d / reports.size();
+ Collections.sort(reports);
+
+ percentile99 = reports.get((int) (reports.size() * 0.99));
+ }
+
+ //calc dispersion
+ double x = 0;
+ for (Long report : reports) x += Math.pow(report - avgLatency, 2);
+
+ double dispersion = 0;
Review comment:
this is not dispersion, it's variance (correct translation of Дисперсия)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r504519759
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/ActionNode.java
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+import java.io.File;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Date;
+import java.util.Optional;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentLinkedDeque;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+import com.fasterxml.jackson.databind.JsonNode;
+import org.apache.ignite.internal.ducktest.tests.start_stop_client.node.reporter.Metadata;
+import org.apache.ignite.internal.ducktest.tests.start_stop_client.node.reporter.Report;
+import org.apache.ignite.internal.ducktest.tests.start_stop_client.node.reporter.Reporter;
+import org.apache.ignite.internal.ducktest.utils.IgniteAwareApplication;
+import org.apache.log4j.LogManager;
+import org.apache.log4j.Logger;
+
+/**
+ * The base class agent
+ */
+public abstract class ActionNode extends IgniteAwareApplication implements Action {
+ /** The unique name of the agent. */
+ private String nodeId = UUID.randomUUID().toString();
+
+ /**
+ * Queue with data for report.
+ * */
+ private ConcurrentLinkedDeque<Report> queue = new ConcurrentLinkedDeque<Report>();
+
+ /* **/
+ Reporter reporter;
+
+ /* **/
+ Metadata testInfo = new Metadata();
+
+ /** Start test time. */
+ protected Date startTime;
+
+ /** End test time. */
+ protected Date endTime;
+
+ /** Thread count. */
+ private int threads;
+
+ /** The delay between iterations of the action. */
+ protected long pacing;
+
+ /** Current action name. */
+ protected String actionName;
+
+ /** Thread executer. */
+ protected ExecutorService executor;
+
+ /** Threads. */
+ private ArrayList<OperationThread> operationThreads = new ArrayList();
+
+ /** */
+ private Logger log = LogManager.getLogger(ActionNode.class.getName());
+
+ /** {@inheritDoc} */
+ @Override protected void run(JsonNode jsonNode) throws Exception {
+ init(jsonNode);
+
+ markInitialized();
+
+ for (int i = 0; i < threads; i++) {
+ operationThreads.add(new OperationThread(this, log, pacing));
+ }
+ for (OperationThread thread : operationThreads) {
+ executor.execute(thread);
+ }
+
+ while (!terminated()) {
+ TimeUnit.MILLISECONDS.sleep(100);
+ }
+
+ executor.shutdownNow();
+ if (!executor.isShutdown()) {
+ TimeUnit.MILLISECONDS.sleep(100);
+ }
+
+ endTime = new Date();
+ markFinished();
+ }
+
+ /** */
+ protected void init(JsonNode jsonNode) throws IOException {
+ threads = Optional.ofNullable(jsonNode.get("threads")).map(JsonNode::asInt).orElse(1);
+ pacing = Optional.ofNullable(jsonNode.get("pacing")).map(JsonNode::asLong).orElse(0l);
+ actionName = Optional.ofNullable(jsonNode.get("action")).map(JsonNode::asText).orElse("default-action-name");
+ String reportFolder = jsonNode.get("report_folder").asText();
+
+ startTime = new Date();
+
+ log.info(
+ "test props:" +
+ " action=" + actionName +
+ " pacing=" + pacing +
+ " threads=" + threads
+ );
+
+ testInfo.setActionName(actionName);
+ testInfo.setThreads(threads);
+ testInfo.setStartTime(System.currentTimeMillis());
+
+ File folder = new File(reportFolder);
+ if (!folder.exists()) {
+ folder.mkdir();
+ }
+
+ String reportPath = reportFolder + "/report-" + System.currentTimeMillis() + ".csv";
+ reporter = new Reporter(reportPath, queue, testInfo);
+
+ executor = Executors.newFixedThreadPool(threads + 3);
+ executor.execute(reporter);
+ scriptInit(jsonNode);
+ }
+
+ /** Init method. */
+ protected abstract void scriptInit(JsonNode jsonNode);
+
+ /** {@inheritDoc} */
+ @Override public void publishInternalReport(Report report) {
+ if (!executor.isShutdown()) {
+ executor.execute(new Runnable() {
Review comment:
Why you create new instance of Runnable in queue(execute in executor) to push something to queue?
Looks super weird.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r504185930
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -0,0 +1,137 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+import java.util.ArrayList;
+import java.util.Collections;
+
+import org.apache.ignite.internal.ducktest.tests.start_stop_client.node.reporter.Report;
+import org.apache.log4j.Logger;
+
+/**
+ * Single action thread.
+ */
+public class OperationThread implements Runnable {
+ /** */
+ private Action actionNode;
+
+ /** */
+ private Logger logger;
+
+ /** */
+ private long pacing;
+
+ /** batch*/
+ private static final long count = 1000;
+
+ /** */
+ public Action getActionNode() {
+ return actionNode;
+ }
+
+ /** */
+ public void setActionNode(ActionNode actionNode) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ }
+
+ /** */
+ public OperationThread(Action actionNode, Logger logger, long pacing) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ this.pacing = pacing;
+ }
+
+ /** {@inheritDoc} */
+ @Override public void run() {
+ while (!Thread.interrupted()) {
Review comment:
Thread.isInterrupted()
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r503974406
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -112,12 +114,14 @@ private Report calculateInterimStatement() {
finally {
//calc dispersion
long x = 0;
- for (Long report : reports) {
- x = x + (report - avgLatency) ^ (2);
- }
+ for (Long report : reports) x = (long) (x + Math.pow(report - avgLatency, 2));
Review comment:
```
for (long report : reports)
x += Math.pow(report - avgLatency, 2));
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r503736656
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import org.apache.log4j.Logger;
+
+/**
+ * Single action thread.
+ */
+public class OperationThread implements Runnable {
+ /** */
+ private Action actionNode;
+
+ /** */
+ private Logger logger;
+
+ /** */
+ private long pacing;
+
+ /** batch*/
+ private static final long count = 1000;
+
+ /** */
+ public Action getActionNode() {
+ return actionNode;
+ }
+
+ /** */
+ public void setActionNode(ActionNode actionNode) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ }
+
+ /** */
+ public OperationThread(Action actionNode, Logger logger, long pacing) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ this.pacing = pacing;
+ }
+
+ /** {@inheritDoc} */
+ @Override public void run() {
+ while (!Thread.interrupted()) {
+ try {
+ actionNode.publishInterimReport(calculateInterimStatement());
+ Thread.sleep(pacing);
+ } catch (InterruptedException e) {
+ }
+ }
+ }
+
+ /** Building an interim report. */
+ private Report calculateInterimStatement() {
+ long endTime = 0;
+ long txCount = 0;
+ long minLatency = -1;
+ long maxLatency = -1;
+ long avgLatency = 0;
+ long percentile99 = 0;
+
+ ArrayList<Long> reports = new ArrayList();
+
+ long latency = 0;
+ long batch = 0;
+ long startTime = System.currentTimeMillis();
+
+ try {
+ while (batch < (count)) {
+ latency = actionNode.singleAction();
+ txCount++;
+
+ if (minLatency == -1 || minLatency > latency) {
+ minLatency = latency;
+ }
+
+ if (maxLatency < latency) {
+ maxLatency = latency;
+ }
+ reports.add(latency);
+ batch++;
+ }
+ endTime = System.currentTimeMillis();
+
+ long sum = 0;
+ for (Long l : reports) {
+ sum += l;
+ }
+ if (!reports.isEmpty()) {
+ avgLatency = sum / reports.size();
Review comment:
I suppose, that this is a mistake.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r503692844
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import org.apache.log4j.Logger;
+
+/**
+ * Single action thread.
+ */
+public class OperationThread implements Runnable {
+ /** */
+ private Action actionNode;
+
+ /** */
+ private Logger logger;
+
+ /** */
+ private long pacing;
+
+ /** batch*/
+ private static final long count = 1000;
+
+ /** */
+ public Action getActionNode() {
+ return actionNode;
+ }
+
+ /** */
+ public void setActionNode(ActionNode actionNode) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ }
+
+ /** */
+ public OperationThread(Action actionNode, Logger logger, long pacing) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ this.pacing = pacing;
+ }
+
+ /** {@inheritDoc} */
+ @Override public void run() {
+ while (!Thread.interrupted()) {
+ try {
+ actionNode.publishInterimReport(calculateInterimStatement());
+ Thread.sleep(pacing);
+ } catch (InterruptedException e) {
+ }
+ }
+ }
+
+ /** Building an interim report. */
+ private Report calculateInterimStatement() {
+ long endTime = 0;
+ long txCount = 0;
+ long minLatency = -1;
+ long maxLatency = -1;
+ long avgLatency = 0;
+ long percentile99 = 0;
+
+ ArrayList<Long> reports = new ArrayList();
+
+ long latency = 0;
+ long batch = 0;
+ long startTime = System.currentTimeMillis();
+
+ try {
+ while (batch < (count)) {
+ latency = actionNode.singleAction();
+ txCount++;
+
+ if (minLatency == -1 || minLatency > latency) {
+ minLatency = latency;
+ }
+
+ if (maxLatency < latency) {
+ maxLatency = latency;
+ }
+ reports.add(latency);
+ batch++;
+ }
+ endTime = System.currentTimeMillis();
+
+ long sum = 0;
+ for (Long l : reports) {
Review comment:
Use primitive here `for (long l: reports)`, and according to codestyle extra space after l is unnecessary.
Also, onliner loop should be without curly brackets.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] SwirMix commented on pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
SwirMix commented on pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#issuecomment-706558668
Added a latency monitoring mechanism. Added building a mini report based on the java application log
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] SwirMix commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
SwirMix commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r503730200
##########
File path: modules/ducktests/tests/ignitetest/report/html_report_test.py
##########
@@ -0,0 +1,22 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from ignitetest.report.plotter.report_builder import ReportBuilder
Review comment:
I agree. Scripts reportage temporarily cut out
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r503700968
##########
File path: modules/ducktests/tests/ignitetest/report/plotter/plotter.py
##########
@@ -0,0 +1,111 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+class for plotting operation execution time based on launch logs
+"""
+
+import matplotlib.pyplot as plt
+
+
+class Plotter:
+ REPORTS = []
+
+ # pylint: disable=R0913
+ def __init__(self, data):
+ self.REPORTS = data
+
+ def plot_avg_latency_data(self, name):
+ """
+ Generate graph
+ :param name: image name.
+ """
+ axis_x_res = self.time_axios_prepare()
+ axis_y = []
+
+ for report in self.REPORTS:
+ y = int(int(report.avg_latency))
+ axis_y.append(y)
+
+ plt.figure(figsize=(20, 20))
+ plt.title("average latency (ns)")
+ plt.xlabel("time (s)")
+ plt.ylabel("latency (ns)")
+ plt.plot(axis_x_res, axis_y)
+ plt.savefig(str(name) + ".png")
+
+ def plot_avg_percentile(self, name):
+ """
+ Generate graph
+ :param name: image name.
+ """
+ axis_x_res = self.time_axios_prepare()
+ axis_avg = []
+ axios_percentile = []
+
+ for report in self.REPORTS:
+ axios_percentile.append(int(int(report.percentile99)))
+ axis_avg.append(int(int(report.avg_latency)))
+
+ plt.figure(figsize=(20, 10))
+ plt.title("percentile 99% and avg_latency (ns)")
+ plt.xlabel("time (s)")
+ plt.ylabel("latency (ns)")
+ plt.plot(axis_x_res, axis_avg, label='average latency')
+ plt.plot(axis_x_res, axios_percentile, label='percentile 99%')
+ plt.legend()
+ plt.savefig(str(name) + ".png")
+
+ def plot_percentile_latency_data(self, name):
+ """
+ Generate graph
+ :param name: image name.
+ """
+ axis_x_res = self.time_axios_prepare()
+ axis_y = []
+
+ for report in self.REPORTS:
+ y = int(int(report.percentile99))
+ axis_y.append(y)
+
+ plt.figure(figsize=(20, 20))
+ plt.title("percentile 99% latency (ns)")
+ plt.xlabel("time (s)")
+ plt.ylabel("latency (ns)")
+ plt.plot(axis_x_res, axis_y)
+
+ plt.savefig(str(name) + ".png")
+
+ def time_axios_prepare(self):
Review comment:
If you use matplotlib, may be it's better to use something like numpy and pandas also?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] SwirMix commented on pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
SwirMix commented on pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#issuecomment-706558313
add mini report.
Extract data from agent log
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r503693269
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import org.apache.log4j.Logger;
+
+/**
+ * Single action thread.
+ */
+public class OperationThread implements Runnable {
+ /** */
+ private Action actionNode;
+
+ /** */
+ private Logger logger;
+
+ /** */
+ private long pacing;
+
+ /** batch*/
+ private static final long count = 1000;
+
+ /** */
+ public Action getActionNode() {
+ return actionNode;
+ }
+
+ /** */
+ public void setActionNode(ActionNode actionNode) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ }
+
+ /** */
+ public OperationThread(Action actionNode, Logger logger, long pacing) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ this.pacing = pacing;
+ }
+
+ /** {@inheritDoc} */
+ @Override public void run() {
+ while (!Thread.interrupted()) {
+ try {
+ actionNode.publishInterimReport(calculateInterimStatement());
+ Thread.sleep(pacing);
+ } catch (InterruptedException e) {
+ }
+ }
+ }
+
+ /** Building an interim report. */
+ private Report calculateInterimStatement() {
+ long endTime = 0;
+ long txCount = 0;
+ long minLatency = -1;
+ long maxLatency = -1;
+ long avgLatency = 0;
+ long percentile99 = 0;
+
+ ArrayList<Long> reports = new ArrayList();
+
+ long latency = 0;
+ long batch = 0;
+ long startTime = System.currentTimeMillis();
+
+ try {
+ while (batch < (count)) {
+ latency = actionNode.singleAction();
+ txCount++;
+
+ if (minLatency == -1 || minLatency > latency) {
+ minLatency = latency;
+ }
+
+ if (maxLatency < latency) {
+ maxLatency = latency;
+ }
+ reports.add(latency);
+ batch++;
+ }
+ endTime = System.currentTimeMillis();
+
+ long sum = 0;
+ for (Long l : reports) {
+ sum += l;
+ }
+ if (!reports.isEmpty()) {
+ avgLatency = sum / reports.size();
Review comment:
Are you sure, that avgLatency should be long integer and not double precision floating point number?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r504186138
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -0,0 +1,137 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+import java.util.ArrayList;
+import java.util.Collections;
+
+import org.apache.ignite.internal.ducktest.tests.start_stop_client.node.reporter.Report;
+import org.apache.log4j.Logger;
+
+/**
+ * Single action thread.
+ */
+public class OperationThread implements Runnable {
+ /** */
+ private Action actionNode;
+
+ /** */
+ private Logger logger;
+
+ /** */
+ private long pacing;
+
+ /** batch*/
+ private static final long count = 1000;
+
+ /** */
+ public Action getActionNode() {
+ return actionNode;
+ }
+
+ /** */
+ public void setActionNode(ActionNode actionNode) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ }
+
+ /** */
+ public OperationThread(Action actionNode, Logger logger, long pacing) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ this.pacing = pacing;
+ }
+
+ /** {@inheritDoc} */
+ @Override public void run() {
+ while (!Thread.interrupted()) {
+ try {
+ actionNode.publishInternalReport(calculateInternalReport());
+ Thread.sleep(pacing);
+ } catch (InterruptedException e) {
+ }
Review comment:
You should not swallow `InterruptedException`.
The best way -- restore interrupted flag and rethrow RuntimeException with cause
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] SwirMix closed pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
SwirMix closed pull request #8316:
URL: https://github.com/apache/ignite/pull/8316
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r503697700
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import org.apache.log4j.Logger;
+
+/**
+ * Single action thread.
+ */
+public class OperationThread implements Runnable {
+ /** */
+ private Action actionNode;
+
+ /** */
+ private Logger logger;
+
+ /** */
+ private long pacing;
+
+ /** batch*/
+ private static final long count = 1000;
+
+ /** */
+ public Action getActionNode() {
+ return actionNode;
+ }
+
+ /** */
+ public void setActionNode(ActionNode actionNode) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ }
+
+ /** */
+ public OperationThread(Action actionNode, Logger logger, long pacing) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ this.pacing = pacing;
+ }
+
+ /** {@inheritDoc} */
+ @Override public void run() {
+ while (!Thread.interrupted()) {
+ try {
+ actionNode.publishInterimReport(calculateInterimStatement());
+ Thread.sleep(pacing);
+ } catch (InterruptedException e) {
+ }
+ }
+ }
+
+ /** Building an interim report. */
+ private Report calculateInterimStatement() {
+ long endTime = 0;
+ long txCount = 0;
+ long minLatency = -1;
+ long maxLatency = -1;
+ long avgLatency = 0;
+ long percentile99 = 0;
+
+ ArrayList<Long> reports = new ArrayList();
+
+ long latency = 0;
+ long batch = 0;
+ long startTime = System.currentTimeMillis();
+
+ try {
+ while (batch < (count)) {
+ latency = actionNode.singleAction();
+ txCount++;
+
+ if (minLatency == -1 || minLatency > latency) {
+ minLatency = latency;
+ }
+
+ if (maxLatency < latency) {
+ maxLatency = latency;
+ }
+ reports.add(latency);
+ batch++;
+ }
+ endTime = System.currentTimeMillis();
+
+ long sum = 0;
+ for (Long l : reports) {
+ sum += l;
+ }
+ if (!reports.isEmpty()) {
+ avgLatency = sum / reports.size();
+ Collections.sort(reports);
+
+ percentile99 = reports.get((int) (reports.size() * 0.99));
+ }
+ }
+ finally {
+ //calc dispersion
+ long x = 0;
+ for (Long report : reports) {
+ x = x + (report - avgLatency) ^ (2);
Review comment:
Oh! `^` is a XOR bitwise operator! Use here `Math.pow(report - avgLatency, 2)`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r504525202
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -0,0 +1,151 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.ignite.internal.ducktest.tests.start_stop_client.node.reporter.Report;
+import org.apache.log4j.Logger;
+
+/**
+ * Single action thread.
+ */
+public class OperationThread implements Runnable {
+ /** */
+ private Action actionNode;
+
+ /** */
+ private Logger logger;
+
+ /** */
+ private long pacing;
+
+ /** batch*/
+ private static final long count = 1000;
+
+ /** */
+ public Action getActionNode() {
+ return actionNode;
+ }
+
+ /** */
+ public void setActionNode(ActionNode actionNode) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ }
+
+ /** */
+ public OperationThread(Action actionNode, Logger logger, long pacing) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ this.pacing = pacing;
+ }
+
+ /** {@inheritDoc} */
+ @Override public void run() {
+ Report report;
+ while (!Thread.currentThread().isInterrupted()) {
+ report = calculateInternalReport();
+ if (report != null) {
+ actionNode.publishInternalReport(report);
+ }
+ }
+ }
+
+ /** */
+ private Report calculateInternalReport() {
+ long endTime = 0;
+ long txCount = 0;
+ long minLatency = -1;
+ long maxLatency = -1;
+ double avgLatency = 0;
+ long percentile99 = 0;
+
+ ArrayList<Long> reports = new ArrayList();
+
+ long latency = 0;
+ long batch = 0;
+ long startTime = System.nanoTime();
+
+ while (batch < (count)) {
+ try {
+ latency = actionNode.singleAction();
+ } catch (Exception e) {
+ logger.info("Fail put operation!");
+ return null;
+ }
+
+ txCount++;
+
+ if (minLatency == -1 || minLatency > latency) {
+ minLatency = latency;
+ }
+
+ if (maxLatency < latency) {
+ maxLatency = latency;
+ }
+ reports.add(latency);
+ batch++;
+ }
+ endTime = System.nanoTime();
+
+ long sum = 0;
+ for (Long l : reports) {
+ sum += l;
+ }
+
+ double sum_d = sum;
+
+ if (!reports.isEmpty()) {
+ avgLatency = sum_d / reports.size();
+ Collections.sort(reports);
+
+ percentile99 = reports.get((int) (reports.size() * 0.99));
+ }
+
+ //calc dispersion
+ double x = 0;
+ for (Long report : reports)
+ x += Math.pow(report - avgLatency, 2);
+
+ double variance = 0;
+ if (reports.size() > 1)
+ variance = x / (reports.size() - 1);
+
+ Report report = new Report();
+ report.setAvgLatency(Math.round(avgLatency));
+ report.setStartTime(startTime);
+ report.setEndTime(endTime);
+ report.setMinLatency(minLatency);
+ report.setMaxLatency(maxLatency);
+ report.setTxCount(txCount);
+ report.setThreadName(Thread.currentThread().getName());
+ report.setPercentile99(percentile99);
+ report.setVariance(Math.round(variance));
+
+ try {
+ TimeUnit.MILLISECONDS.sleep(pacing);
+ } catch (Exception e) {
+ e.printStackTrace();
Review comment:
You swallow InterruptedException
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r503696158
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import org.apache.log4j.Logger;
+
+/**
+ * Single action thread.
+ */
+public class OperationThread implements Runnable {
+ /** */
+ private Action actionNode;
+
+ /** */
+ private Logger logger;
+
+ /** */
+ private long pacing;
+
+ /** batch*/
+ private static final long count = 1000;
+
+ /** */
+ public Action getActionNode() {
+ return actionNode;
+ }
+
+ /** */
+ public void setActionNode(ActionNode actionNode) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ }
+
+ /** */
+ public OperationThread(Action actionNode, Logger logger, long pacing) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ this.pacing = pacing;
+ }
+
+ /** {@inheritDoc} */
+ @Override public void run() {
+ while (!Thread.interrupted()) {
+ try {
+ actionNode.publishInterimReport(calculateInterimStatement());
+ Thread.sleep(pacing);
+ } catch (InterruptedException e) {
+ }
+ }
+ }
+
+ /** Building an interim report. */
+ private Report calculateInterimStatement() {
+ long endTime = 0;
+ long txCount = 0;
+ long minLatency = -1;
+ long maxLatency = -1;
+ long avgLatency = 0;
+ long percentile99 = 0;
+
+ ArrayList<Long> reports = new ArrayList();
+
+ long latency = 0;
+ long batch = 0;
+ long startTime = System.currentTimeMillis();
+
+ try {
+ while (batch < (count)) {
+ latency = actionNode.singleAction();
+ txCount++;
+
+ if (minLatency == -1 || minLatency > latency) {
+ minLatency = latency;
+ }
+
+ if (maxLatency < latency) {
+ maxLatency = latency;
+ }
+ reports.add(latency);
+ batch++;
+ }
+ endTime = System.currentTimeMillis();
+
+ long sum = 0;
+ for (Long l : reports) {
+ sum += l;
+ }
+ if (!reports.isEmpty()) {
+ avgLatency = sum / reports.size();
+ Collections.sort(reports);
+
+ percentile99 = reports.get((int) (reports.size() * 0.99));
+ }
+ }
+ finally {
+ //calc dispersion
+ long x = 0;
+ for (Long report : reports) {
+ x = x + (report - avgLatency) ^ (2);
+ }
+ logger.info("dispersion debug: " + x);
+
+ double dispersion = (double) (x / (reports.size() - 1));
Review comment:
Are you sure, that x should be long integer? You will lose information.
And what if reports.size() equals to 1?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r504184219
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -0,0 +1,137 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+import java.util.ArrayList;
+import java.util.Collections;
+
+import org.apache.ignite.internal.ducktest.tests.start_stop_client.node.reporter.Report;
+import org.apache.log4j.Logger;
+
+/**
+ * Single action thread.
+ */
+public class OperationThread implements Runnable {
+ /** */
+ private Action actionNode;
+
+ /** */
+ private Logger logger;
+
+ /** */
+ private long pacing;
+
+ /** batch*/
+ private static final long count = 1000;
+
+ /** */
+ public Action getActionNode() {
+ return actionNode;
+ }
+
+ /** */
+ public void setActionNode(ActionNode actionNode) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ }
+
+ /** */
+ public OperationThread(Action actionNode, Logger logger, long pacing) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ this.pacing = pacing;
+ }
+
+ /** {@inheritDoc} */
+ @Override public void run() {
+ while (!Thread.interrupted()) {
+ try {
+ actionNode.publishInternalReport(calculateInternalReport());
+ Thread.sleep(pacing);
+ } catch (InterruptedException e) {
+ }
+ }
+ }
+
+ /** */
+ private Report calculateInternalReport() {
+ long endTime = 0;
+ long txCount = 0;
+ long minLatency = -1;
+ long maxLatency = -1;
+ double avgLatency = 0;
+ long percentile99 = 0;
+
+ ArrayList<Long> reports = new ArrayList();
+
+ long latency = 0;
+ long batch = 0;
+ long startTime = System.nanoTime();
+
+ while (batch < (count)) {
+ latency = actionNode.singleAction();
+ txCount++;
+
+ if (minLatency == -1 || minLatency > latency) {
+ minLatency = latency;
+ }
+
+ if (maxLatency < latency) {
+ maxLatency = latency;
+ }
+ reports.add(latency);
+ batch++;
+ }
+ endTime = System.nanoTime();
+
+ long sum = 0;
+ for (Long l : reports) {
+ sum += l;
+ }
+
+ double sum_d = sum;
+
+ if (!reports.isEmpty()) {
+ avgLatency = sum_d / reports.size();
+ Collections.sort(reports);
+
+ percentile99 = reports.get((int) (reports.size() * 0.99));
+ }
+
+ //calc dispersion
+ double x = 0;
+ for (Long report : reports) x += Math.pow(report - avgLatency, 2);
+
+ double dispersion = 0;
Review comment:
this is not dispersion, it's variance
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r503702898
##########
File path: modules/ducktests/tests/ignitetest/report/html_report_test.py
##########
@@ -0,0 +1,22 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from ignitetest.report.plotter.report_builder import ReportBuilder
Review comment:
If I understand correctly, you have hardcoded report generation? This is really bad decision.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [ignite] ivandasch commented on a change in pull request #8316: IGNITE-13489 start and stop clients. Check topology version
Posted by GitBox <gi...@apache.org>.
ivandasch commented on a change in pull request #8316:
URL: https://github.com/apache/ignite/pull/8316#discussion_r504183884
##########
File path: modules/ducktests/src/main/java/org/apache/ignite/internal/ducktest/tests/start_stop_client/node/OperationThread.java
##########
@@ -0,0 +1,137 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.ducktest.tests.start_stop_client.node;
+
+import java.util.ArrayList;
+import java.util.Collections;
+
+import org.apache.ignite.internal.ducktest.tests.start_stop_client.node.reporter.Report;
+import org.apache.log4j.Logger;
+
+/**
+ * Single action thread.
+ */
+public class OperationThread implements Runnable {
+ /** */
+ private Action actionNode;
+
+ /** */
+ private Logger logger;
+
+ /** */
+ private long pacing;
+
+ /** batch*/
+ private static final long count = 1000;
+
+ /** */
+ public Action getActionNode() {
+ return actionNode;
+ }
+
+ /** */
+ public void setActionNode(ActionNode actionNode) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ }
+
+ /** */
+ public OperationThread(Action actionNode, Logger logger, long pacing) {
+ this.actionNode = actionNode;
+ this.logger = logger;
+ this.pacing = pacing;
+ }
+
+ /** {@inheritDoc} */
+ @Override public void run() {
+ while (!Thread.interrupted()) {
+ try {
+ actionNode.publishInternalReport(calculateInternalReport());
+ Thread.sleep(pacing);
+ } catch (InterruptedException e) {
+ }
+ }
+ }
+
+ /** */
+ private Report calculateInternalReport() {
+ long endTime = 0;
+ long txCount = 0;
+ long minLatency = -1;
+ long maxLatency = -1;
+ double avgLatency = 0;
+ long percentile99 = 0;
+
+ ArrayList<Long> reports = new ArrayList();
+
+ long latency = 0;
+ long batch = 0;
+ long startTime = System.nanoTime();
+
+ while (batch < (count)) {
+ latency = actionNode.singleAction();
+ txCount++;
+
+ if (minLatency == -1 || minLatency > latency) {
+ minLatency = latency;
+ }
+
+ if (maxLatency < latency) {
+ maxLatency = latency;
+ }
+ reports.add(latency);
+ batch++;
+ }
+ endTime = System.nanoTime();
+
+ long sum = 0;
+ for (Long l : reports) {
+ sum += l;
+ }
+
+ double sum_d = sum;
+
+ if (!reports.isEmpty()) {
+ avgLatency = sum_d / reports.size();
+ Collections.sort(reports);
+
+ percentile99 = reports.get((int) (reports.size() * 0.99));
+ }
+
+ //calc dispersion
+ double x = 0;
+ for (Long report : reports) x += Math.pow(report - avgLatency, 2);
+
+ double dispersion = 0;
+ if (reports.size() > 1) {
Review comment:
Unnecessary curly brackets.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org