You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@geode.apache.org by GitBox <gi...@apache.org> on 2020/07/10 18:21:37 UTC
[GitHub] [geode-benchmarks] bschuchardt opened a new pull request #130: initial implementation of withClusterTopology
bschuchardt opened a new pull request #130:
URL: https://github.com/apache/geode-benchmarks/pull/130
added a new client-less topology and converted put/get tests to optionally use it.
The topology is selected with "-PwithClusterTopology"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode-benchmarks] bschuchardt commented on a change in pull request #130: initial implementation of withClusterTopology
Posted by GitBox <gi...@apache.org>.
bschuchardt commented on a change in pull request #130:
URL: https://github.com/apache/geode-benchmarks/pull/130#discussion_r453713685
##########
File path: geode-benchmarks/src/main/java/org/apache/geode/benchmark/tests/GeodeBenchmark.java
##########
@@ -45,11 +46,20 @@ public static TestConfig createConfig() {
config.durationSeconds(BENCHMARK_DURATION);
config.threads(THREADS);
- final String sniProp = System.getProperty("withSniProxy");
- final boolean doSni = sniProp != null && !sniProp.equals("false");
+ String prop = System.getProperty("withSniProxy");
Review comment:
I thought this code was curious, too. I'll ask Bill about it because this pattern was already in place.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode-benchmarks] Bill commented on a change in pull request #130: Enable geode-benchmarks to run with only servers
Posted by GitBox <gi...@apache.org>.
Bill commented on a change in pull request #130:
URL: https://github.com/apache/geode-benchmarks/pull/130#discussion_r455878798
##########
File path: geode-benchmarks/src/main/java/org/apache/geode/benchmark/tests/GeodeBenchmark.java
##########
@@ -45,11 +46,20 @@ public static TestConfig createConfig() {
config.durationSeconds(BENCHMARK_DURATION);
config.threads(THREADS);
- final String sniProp = System.getProperty("withSniProxy");
- final boolean doSni = sniProp != null && !sniProp.equals("false");
+ String prop = System.getProperty("withSniProxy");
Review comment:
I simply didn't know about `getBoolean()`. `GeodeProperties.isPropertySet(String)` can be replaced with `getBoolean()` too. The latter is true if the prop isn't null and `equalsIgnoreCase("true")`. Changing this is easy but will entail changing `GeodeBenchmarkTest.withSniProxyNotLowercaseFalse()` and `withClusterTopologyNotLowercaseFalse()`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode-benchmarks] pivotal-jbarrett commented on a change in pull request #130: initial implementation of withClusterTopology
Posted by GitBox <gi...@apache.org>.
pivotal-jbarrett commented on a change in pull request #130:
URL: https://github.com/apache/geode-benchmarks/pull/130#discussion_r453098098
##########
File path: geode-benchmarks/build.gradle
##########
@@ -93,6 +93,7 @@ task benchmark(type: Test) {
}
systemProperty 'withSsl', project.hasProperty('withSsl')
systemProperty 'withSniProxy', project.hasProperty('withSniProxy')
+ systemProperty 'withClusterTopology', project.hasProperty('withSniProxy')
Review comment:
I think you want `project.hasProperty('withSniProxy')`.
##########
File path: geode-benchmarks/src/main/java/org/apache/geode/benchmark/tests/GeodeBenchmark.java
##########
@@ -45,11 +46,20 @@ public static TestConfig createConfig() {
config.durationSeconds(BENCHMARK_DURATION);
config.threads(THREADS);
- final String sniProp = System.getProperty("withSniProxy");
- final boolean doSni = sniProp != null && !sniProp.equals("false");
+ String prop = System.getProperty("withSniProxy");
Review comment:
`Boolean.getBoolean(name)` returns false if the property is missing or exists but is not "true".
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode-benchmarks] Bill commented on a change in pull request #130: Enable geode-benchmarks to run with only servers
Posted by GitBox <gi...@apache.org>.
Bill commented on a change in pull request #130:
URL: https://github.com/apache/geode-benchmarks/pull/130#discussion_r455878798
##########
File path: geode-benchmarks/src/main/java/org/apache/geode/benchmark/tests/GeodeBenchmark.java
##########
@@ -45,11 +46,20 @@ public static TestConfig createConfig() {
config.durationSeconds(BENCHMARK_DURATION);
config.threads(THREADS);
- final String sniProp = System.getProperty("withSniProxy");
- final boolean doSni = sniProp != null && !sniProp.equals("false");
+ String prop = System.getProperty("withSniProxy");
Review comment:
I simply didn't know about `getBoolean()`. `GeodeProperties.isPropertySet(String)` can be replaced with `getBoolean()` too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode-benchmarks] Bill edited a comment on pull request #130: initial implementation of withClusterTopology
Posted by GitBox <gi...@apache.org>.
Bill edited a comment on pull request #130:
URL: https://github.com/apache/geode-benchmarks/pull/130#issuecomment-656944271
when I run `./run_tests.sh -t burcham --br https://github.com/bschuchardt/geode-benchmarks.git --bb feature/clusterTopology -- -PwithThreads=10 -PwithWarmup=10 -PwithDuration=30 -i --tests PartitionedPutBenchmark` it fails with:
```
> Task :geode-benchmarks:benchmark FAILED
org.apache.geode.benchmark.tests.PartitionedPutBenchmark > run() FAILED
java.util.concurrent.CompletionException: java.lang.RuntimeException: java.lang.IllegalStateException: The singleton cache was created by CacheFactory not ClientCacheFactory.
```
I wonder if we need some conditional code around creating the client proxy region (if we aren't on a client role then don't create the client proxy region.) Hmm I see in e.g. `PartitionedGetBenchmark` we have precisely that conditional code hmm.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode-benchmarks] Bill commented on a change in pull request #130: initial implementation of withClusterTopology
Posted by GitBox <gi...@apache.org>.
Bill commented on a change in pull request #130:
URL: https://github.com/apache/geode-benchmarks/pull/130#discussion_r453122053
##########
File path: geode-benchmarks/src/main/java/org/apache/geode/benchmark/tests/PartitionedGetBenchmark.java
##########
@@ -56,9 +56,14 @@ public void setKeyRange(final LongRange keyRange) {
public TestConfig configure() {
TestConfig config = GeodeBenchmark.createConfig();
before(config, new CreatePartitionedRegion(), SERVER);
- before(config, new CreateClientProxyRegion(), CLIENT);
- before(config, new PrePopulateRegion(keyRange), CLIENT);
- workload(config, new GetTask(keyRange), CLIENT);
+ if (config.getRoles().containsKey(CLIENT)) {
+ before(config, new CreateClientProxyRegion(), CLIENT);
+ before(config, new PrePopulateRegion(keyRange), CLIENT);
+ workload(config, new GetTask(keyRange), CLIENT);
+ } else {
+ before(config, new PrePopulateRegion(keyRange), CLIENT);
+ workload(config, new GetTask(keyRange), CLIENT);
Review comment:
run these on `SERVER` role, not `CLIENT` one
##########
File path: geode-benchmarks/src/main/java/org/apache/geode/benchmark/topology/ClusterTopology.java
##########
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.benchmark.topology;
+
+import static org.apache.geode.benchmark.Config.before;
+import static org.apache.geode.benchmark.Config.role;
+import static org.apache.geode.benchmark.parameters.Utils.addToTestConfig;
+import static org.apache.geode.benchmark.topology.Ports.LOCATOR_PORT;
+import static org.apache.geode.benchmark.topology.Roles.LOCATOR;
+import static org.apache.geode.benchmark.topology.Roles.SERVER;
+
+import org.apache.geode.benchmark.parameters.GcLoggingParameters;
+import org.apache.geode.benchmark.parameters.GcParameters;
+import org.apache.geode.benchmark.parameters.HeapParameters;
+import org.apache.geode.benchmark.parameters.JvmParameters;
+import org.apache.geode.benchmark.parameters.ProfilerParameters;
+import org.apache.geode.benchmark.tasks.StartLocator;
+import org.apache.geode.benchmark.tasks.StartServer;
+import org.apache.geode.perftest.TestConfig;
+
+public class ClusterTopology {
+ private static final int NUM_LOCATORS = 1;
+ private static final int NUM_SERVERS = 3;
+ private static final String WITH_SSL_ARGUMENT = "-DwithSsl=true";
+ private static final String WITH_SECURITY_MANAGER_ARGUMENT = "-DwithSecurityManager=true";
+
+ public static void configure(TestConfig config) {
+ role(config, LOCATOR, NUM_LOCATORS);
+ role(config, SERVER, NUM_SERVERS);
+
+ JvmParameters.configure(config);
+ HeapParameters.configure(config);
+ GcLoggingParameters.configure(config);
+ GcParameters.configure(config);
+ ProfilerParameters.configure(config);
+
+ addToTestConfig(config, "withSsl", WITH_SSL_ARGUMENT);
Review comment:
TBD if this will do what you want. You're trying to trick `JVMLauncher.buildCommand()` to see the JVM arg `"-DwithSsl=true"`. This might work (I'm testing it now.) But the way `ClientServerTopologyWithSNIProxy` accomplishes the trick is via:
```java
configureGeodeProductJvms(config, WITH_SSL_ARGUMENT);
```
If this way here works better though, I recommend changing the SNI topology to do it this way too since `addToTestConfig(...)` seems like a better way for a topology to communicate its intentions.
##########
File path: geode-benchmarks/build.gradle
##########
@@ -93,6 +93,7 @@ task benchmark(type: Test) {
}
systemProperty 'withSsl', project.hasProperty('withSsl')
systemProperty 'withSniProxy', project.hasProperty('withSniProxy')
+ systemProperty 'withClusterTopology', project.hasProperty('withSniProxy')
Review comment:
I think @pivotal-jbarrett meant to say: `project.hasProperty('withClusterTopology')`
##########
File path: geode-benchmarks/src/test/java/org/apache/geode/benchmark/tests/GeodeBenchmarkTest.java
##########
@@ -36,46 +36,66 @@
*/
class GeodeBenchmarkTest {
+ public static final String WITH_CLUSTER_TOPOLOGY = "withClusterTopology";
+ public static final String WITH_SNI_PROXY = "withSniProxy";
private TestConfig config;
private TestStep startProxyStep;
@BeforeEach
public void beforeEach() {
+ System.clearProperty(WITH_SNI_PROXY);
+ System.clearProperty(WITH_CLUSTER_TOPOLOGY);
Review comment:
The old `@AfterAll` prevented subsequent tests run in the same JVM from seeing these system properties. This approach leaves the system properties in effect after the test finishes.
Is that important? Seems like it might be good to be sure and clear those in an `@AfterAll`.
##########
File path: geode-benchmarks/src/test/java/org/apache/geode/benchmark/tests/GeodeBenchmarkTest.java
##########
@@ -36,46 +36,66 @@
*/
class GeodeBenchmarkTest {
+ public static final String WITH_CLUSTER_TOPOLOGY = "withClusterTopology";
+ public static final String WITH_SNI_PROXY = "withSniProxy";
Review comment:
good DRY
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode-benchmarks] bschuchardt commented on a change in pull request #130: initial implementation of withClusterTopology
Posted by GitBox <gi...@apache.org>.
bschuchardt commented on a change in pull request #130:
URL: https://github.com/apache/geode-benchmarks/pull/130#discussion_r454449211
##########
File path: geode-benchmarks/src/main/java/org/apache/geode/benchmark/topology/ClusterTopology.java
##########
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.benchmark.topology;
+
+import static org.apache.geode.benchmark.Config.before;
+import static org.apache.geode.benchmark.Config.role;
+import static org.apache.geode.benchmark.parameters.Utils.addToTestConfig;
+import static org.apache.geode.benchmark.topology.Ports.LOCATOR_PORT;
+import static org.apache.geode.benchmark.topology.Roles.LOCATOR;
+import static org.apache.geode.benchmark.topology.Roles.SERVER;
+
+import org.apache.geode.benchmark.parameters.GcLoggingParameters;
+import org.apache.geode.benchmark.parameters.GcParameters;
+import org.apache.geode.benchmark.parameters.HeapParameters;
+import org.apache.geode.benchmark.parameters.JvmParameters;
+import org.apache.geode.benchmark.parameters.ProfilerParameters;
+import org.apache.geode.benchmark.tasks.StartLocator;
+import org.apache.geode.benchmark.tasks.StartServer;
+import org.apache.geode.perftest.TestConfig;
+
+public class ClusterTopology {
+ private static final int NUM_LOCATORS = 1;
+ private static final int NUM_SERVERS = 3;
+ private static final String WITH_SSL_ARGUMENT = "-DwithSsl=true";
+ private static final String WITH_SECURITY_MANAGER_ARGUMENT = "-DwithSecurityManager=true";
+
+ public static void configure(TestConfig config) {
+ role(config, LOCATOR, NUM_LOCATORS);
+ role(config, SERVER, NUM_SERVERS);
+
+ JvmParameters.configure(config);
+ HeapParameters.configure(config);
+ GcLoggingParameters.configure(config);
+ GcParameters.configure(config);
+ ProfilerParameters.configure(config);
+
+ addToTestConfig(config, "withSsl", WITH_SSL_ARGUMENT);
Review comment:
This is what the client/server topology does. There's a conditional in addToTestConfig() that does the right thing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode-benchmarks] pivotal-jbarrett closed pull request #130: Enable geode-benchmarks to run with only servers
Posted by GitBox <gi...@apache.org>.
pivotal-jbarrett closed pull request #130:
URL: https://github.com/apache/geode-benchmarks/pull/130
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode-benchmarks] pivotal-jbarrett commented on pull request #130: Enable geode-benchmarks to run with only servers
Posted by GitBox <gi...@apache.org>.
pivotal-jbarrett commented on pull request #130:
URL: https://github.com/apache/geode-benchmarks/pull/130#issuecomment-706430249
I took this and ran with it for some other testing we were doing and opened another PR, #135. We can close this one if #135 gets picked up.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode-benchmarks] bschuchardt commented on a change in pull request #130: initial implementation of withClusterTopology
Posted by GitBox <gi...@apache.org>.
bschuchardt commented on a change in pull request #130:
URL: https://github.com/apache/geode-benchmarks/pull/130#discussion_r453837947
##########
File path: geode-benchmarks/src/test/java/org/apache/geode/benchmark/tests/GeodeBenchmarkTest.java
##########
@@ -36,46 +36,66 @@
*/
class GeodeBenchmarkTest {
+ public static final String WITH_CLUSTER_TOPOLOGY = "withClusterTopology";
+ public static final String WITH_SNI_PROXY = "withSniProxy";
private TestConfig config;
private TestStep startProxyStep;
@BeforeEach
public void beforeEach() {
+ System.clearProperty(WITH_SNI_PROXY);
+ System.clearProperty(WITH_CLUSTER_TOPOLOGY);
Review comment:
I'll put that back - thanks Bill
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode-benchmarks] bschuchardt commented on a change in pull request #130: Enable geode-benchmarks to run with only servers
Posted by GitBox <gi...@apache.org>.
bschuchardt commented on a change in pull request #130:
URL: https://github.com/apache/geode-benchmarks/pull/130#discussion_r456533721
##########
File path: geode-benchmarks/src/main/java/org/apache/geode/benchmark/tests/GeodeBenchmark.java
##########
@@ -45,11 +46,20 @@ public static TestConfig createConfig() {
config.durationSeconds(BENCHMARK_DURATION);
config.threads(THREADS);
- final String sniProp = System.getProperty("withSniProxy");
- final boolean doSni = sniProp != null && !sniProp.equals("false");
+ String prop = System.getProperty("withSniProxy");
Review comment:
Hmm, that's as may be but the code you wrote doesn't do the same thing as Boolean.getBoolean(). The code allows the property to be set with a value that, if not "false", enables the setting. Boolean.getBoolean() returns `false` if the property is not set or if it is set but is not exactly equal to "false".
Changing the code to use Boolean.getBoolean() causes your test GeodeBenchmarkTest.withSniProxyNotLowercaseFalse to fail, so I'd like some assurance that it's okay to delete that test.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode-benchmarks] bschuchardt commented on a change in pull request #130: Enable geode-benchmarks to run with only servers
Posted by GitBox <gi...@apache.org>.
bschuchardt commented on a change in pull request #130:
URL: https://github.com/apache/geode-benchmarks/pull/130#discussion_r456533721
##########
File path: geode-benchmarks/src/main/java/org/apache/geode/benchmark/tests/GeodeBenchmark.java
##########
@@ -45,11 +46,20 @@ public static TestConfig createConfig() {
config.durationSeconds(BENCHMARK_DURATION);
config.threads(THREADS);
- final String sniProp = System.getProperty("withSniProxy");
- final boolean doSni = sniProp != null && !sniProp.equals("false");
+ String prop = System.getProperty("withSniProxy");
Review comment:
Hmm, that's as may be but the code you wrote doesn't do the same thing as Boolean.getBoolean(). The code allows the property to be set with a value that, if not "false", enables the setting. Boolean.getBoolean() returns `false` if the property is not set or if it is set but is not exactly equal to "true".
Changing the code to use Boolean.getBoolean() causes your test GeodeBenchmarkTest.withSniProxyNotLowercaseFalse to fail, so I'd like some assurance that it's okay to delete that test.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode-benchmarks] pivotal-jbarrett commented on pull request #130: Enable geode-benchmarks to run with only servers
Posted by GitBox <gi...@apache.org>.
pivotal-jbarrett commented on pull request #130:
URL: https://github.com/apache/geode-benchmarks/pull/130#issuecomment-726154443
Close in favor of #135
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode-benchmarks] Bill commented on pull request #130: initial implementation of withClusterTopology
Posted by GitBox <gi...@apache.org>.
Bill commented on pull request #130:
URL: https://github.com/apache/geode-benchmarks/pull/130#issuecomment-656944271
when I run `./run_tests.sh -t burcham --br https://github.com/bschuchardt/geode-benchmarks.git --bb feature/clusterTopology -- -PwithThreads=10 -PwithWarmup=10 -PwithDuration=30 -i --tests PartitionedPutBenchmark` it fails with:
```
> Task :geode-benchmarks:benchmark FAILED
org.apache.geode.benchmark.tests.PartitionedPutBenchmark > run() FAILED
java.util.concurrent.CompletionException: java.lang.RuntimeException: java.lang.IllegalStateException: The singleton cache was created by CacheFactory not ClientCacheFactory.
```
I wonder if we need some conditional code around creating the client proxy region (if we aren't on a client role then don't create the client proxy region.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [geode-benchmarks] bschuchardt commented on a change in pull request #130: initial implementation of withClusterTopology
Posted by GitBox <gi...@apache.org>.
bschuchardt commented on a change in pull request #130:
URL: https://github.com/apache/geode-benchmarks/pull/130#discussion_r453712325
##########
File path: geode-benchmarks/build.gradle
##########
@@ -93,6 +93,7 @@ task benchmark(type: Test) {
}
systemProperty 'withSsl', project.hasProperty('withSsl')
systemProperty 'withSniProxy', project.hasProperty('withSniProxy')
+ systemProperty 'withClusterTopology', project.hasProperty('withSniProxy')
Review comment:
yes, I'll fix that
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org