You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/06/16 10:11:29 UTC
[GitHub] [pulsar] nicoloboschi opened a new pull request, #16090: [improve][pulsar-perf] Transactions: client TLS options
nicoloboschi opened a new pull request, #16090:
URL: https://github.com/apache/pulsar/pull/16090
### Motivation
`pulsar-perf transaction` does not support tls parameter for the pulsar client like `--trust-cert-file` and `-tls-allow-insecure` as the other perf command do (produce, consume, read)
### Modifications
* Refactored the common arguments for the perf commands to a single class to avoid code duplication
* Added support for `--trust-cert-file` and `-tls-allow-insecure` for the `pulsar-perf transaction` command
- [x] `doc`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] michaeljmarshall commented on a diff in pull request #16090: [improve][pulsar-perf] Transactions: improve client options
Posted by GitBox <gi...@apache.org>.
michaeljmarshall commented on code in PR #16090:
URL: https://github.com/apache/pulsar/pull/16090#discussion_r906248569
##########
pulsar-testclient/src/main/java/org/apache/pulsar/testclient/PerformanceBaseArguments.java:
##########
@@ -0,0 +1,135 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.testclient;
+
+import static org.apache.commons.lang3.StringUtils.isBlank;
+import com.beust.jcommander.Parameter;
+import java.io.FileInputStream;
+import java.util.Properties;
+import lombok.SneakyThrows;
+
+
+public abstract class PerformanceBaseArguments {
+
+ @Parameter(names = { "-h", "--help" }, description = "Help message", help = true)
+ boolean help;
+
+ @Parameter(names = { "-cf", "--conf-file" }, description = "Configuration file")
+ public String confFile;
+
+ @Parameter(names = { "-u", "--service-url" }, description = "Pulsar Service URL")
+ public String serviceURL;
+
+ @Parameter(names = { "--auth-plugin" }, description = "Authentication plugin class name")
+ public String authPluginClassName;
+
+ @Parameter(
+ names = { "--auth-params" },
+ description = "Authentication parameters, whose format is determined by the implementation "
+ + "of method `configure` in authentication plugin class, for example \"key1:val1,key2:val2\" "
+ + "or \"{\"key1\":\"val1\",\"key2\":\"val2\"}.")
+ public String authParams;
+
+ @Parameter(names = {
+ "--trust-cert-file" }, description = "Path for the trusted TLS certificate file")
+ public String tlsTrustCertsFilePath = "";
+
+ @Parameter(names = {
+ "--tls-allow-insecure" }, description = "Allow insecure TLS connection")
+ public Boolean tlsAllowInsecureConnection = null;
+
+ @Parameter(names = {
+ "--tls-hostname-verification" }, description = "Enable TLS hostname verification")
Review Comment:
Good point, I agree with using `--tls-enable-hostname-verification`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nicoloboschi commented on pull request #16090: [improve][pulsar-perf] Transactions: client TLS options
Posted by GitBox <gi...@apache.org>.
nicoloboschi commented on PR #16090:
URL: https://github.com/apache/pulsar/pull/16090#issuecomment-1157783881
/pulsarbot rerun-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] liangyepianzhou commented on pull request #16090: [improve][pulsar-perf] Transactions: improve client options
Posted by GitBox <gi...@apache.org>.
liangyepianzhou commented on PR #16090:
URL: https://github.com/apache/pulsar/pull/16090#issuecomment-1158777507
Could you please add a test to test the new parameters for transaction-perf?
Just like this [test](https://github.com/apache/pulsar/blob/master/pulsar-testclient/src/test/java/org/apache/pulsar/testclient/Oauth2PerformanceTransactionTest.java).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nicoloboschi commented on pull request #16090: [improve][pulsar-perf] Transactions: improve client options
Posted by GitBox <gi...@apache.org>.
nicoloboschi commented on PR #16090:
URL: https://github.com/apache/pulsar/pull/16090#issuecomment-1158785745
@liangyepianzhou the new parameters are mostly for TLS connections and perf tests do not use TLS. we don't need to test TLS parameters are working for Pulsar Client, I believe just the arguments + client builder unit tests are enough
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nicoloboschi commented on a diff in pull request #16090: [improve][pulsar-perf] Transactions: improve client options
Posted by GitBox <gi...@apache.org>.
nicoloboschi commented on code in PR #16090:
URL: https://github.com/apache/pulsar/pull/16090#discussion_r904769601
##########
pulsar-testclient/src/main/java/org/apache/pulsar/testclient/PerformanceBaseArguments.java:
##########
@@ -0,0 +1,135 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.testclient;
+
+import static org.apache.commons.lang3.StringUtils.isBlank;
+import com.beust.jcommander.Parameter;
+import java.io.FileInputStream;
+import java.util.Properties;
+import lombok.SneakyThrows;
+
+
+public abstract class PerformanceBaseArguments {
+
+ @Parameter(names = { "-h", "--help" }, description = "Help message", help = true)
+ boolean help;
+
+ @Parameter(names = { "-cf", "--conf-file" }, description = "Configuration file")
+ public String confFile;
+
+ @Parameter(names = { "-u", "--service-url" }, description = "Pulsar Service URL")
+ public String serviceURL;
+
+ @Parameter(names = { "--auth-plugin" }, description = "Authentication plugin class name")
+ public String authPluginClassName;
+
+ @Parameter(
+ names = { "--auth-params" },
+ description = "Authentication parameters, whose format is determined by the implementation "
+ + "of method `configure` in authentication plugin class, for example \"key1:val1,key2:val2\" "
+ + "or \"{\"key1\":\"val1\",\"key2\":\"val2\"}.")
+ public String authParams;
+
+ @Parameter(names = {
+ "--trust-cert-file" }, description = "Path for the trusted TLS certificate file")
+ public String tlsTrustCertsFilePath = "";
+
+ @Parameter(names = {
+ "--tls-allow-insecure" }, description = "Allow insecure TLS connection")
+ public Boolean tlsAllowInsecureConnection = null;
+
+ @Parameter(names = {
+ "--tls-hostname-verification" }, description = "Enable TLS hostname verification")
Review Comment:
Good point @michaeljmarshall
Since other TLS params in the perf cli have `--tls` as prefix, I'd prefer to go with `--tls-enable-hostname-verification`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] liangyepianzhou commented on pull request #16090: [improve][pulsar-perf] Transactions: client TLS options
Posted by GitBox <gi...@apache.org>.
liangyepianzhou commented on PR #16090:
URL: https://github.com/apache/pulsar/pull/16090#issuecomment-1158688073
Good work!
Could you please add a test for it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] michaeljmarshall commented on a diff in pull request #16090: [improve][pulsar-perf] Transactions: improve client options
Posted by GitBox <gi...@apache.org>.
michaeljmarshall commented on code in PR #16090:
URL: https://github.com/apache/pulsar/pull/16090#discussion_r902794380
##########
pulsar-testclient/src/main/java/org/apache/pulsar/testclient/PerformanceBaseArguments.java:
##########
@@ -0,0 +1,135 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.testclient;
+
+import static org.apache.commons.lang3.StringUtils.isBlank;
+import com.beust.jcommander.Parameter;
+import java.io.FileInputStream;
+import java.util.Properties;
+import lombok.SneakyThrows;
+
+
+public abstract class PerformanceBaseArguments {
+
+ @Parameter(names = { "-h", "--help" }, description = "Help message", help = true)
+ boolean help;
+
+ @Parameter(names = { "-cf", "--conf-file" }, description = "Configuration file")
+ public String confFile;
+
+ @Parameter(names = { "-u", "--service-url" }, description = "Pulsar Service URL")
+ public String serviceURL;
+
+ @Parameter(names = { "--auth-plugin" }, description = "Authentication plugin class name")
+ public String authPluginClassName;
+
+ @Parameter(
+ names = { "--auth-params" },
+ description = "Authentication parameters, whose format is determined by the implementation "
+ + "of method `configure` in authentication plugin class, for example \"key1:val1,key2:val2\" "
+ + "or \"{\"key1\":\"val1\",\"key2\":\"val2\"}.")
+ public String authParams;
+
+ @Parameter(names = {
+ "--trust-cert-file" }, description = "Path for the trusted TLS certificate file")
+ public String tlsTrustCertsFilePath = "";
+
+ @Parameter(names = {
+ "--tls-allow-insecure" }, description = "Allow insecure TLS connection")
+ public Boolean tlsAllowInsecureConnection = null;
+
+ @Parameter(names = {
+ "--tls-hostname-verification" }, description = "Enable TLS hostname verification")
Review Comment:
For other CLI tools in the project, we already use either `--tls-enable-hostname-verification` or `--hostname-verification-enabled`. Since this hasn't been released yet, I'd prefer not to add another name for the same configuration. I like `--hostname-verification-enabled`, do you have a preference @nicoloboschi?
https://github.com/apache/pulsar/blob/09be79742473b7686f44d3f47e7c10fb9b441792/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/PulsarAdminTool.java#L73
https://github.com/apache/pulsar/blob/09be79742473b7686f44d3f47e7c10fb9b441792/pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdSinks.java#L170
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on a diff in pull request #16090: [improve][pulsar-perf] Transactions: client TLS options
Posted by GitBox <gi...@apache.org>.
eolivelli commented on code in PR #16090:
URL: https://github.com/apache/pulsar/pull/16090#discussion_r899229330
##########
pulsar-testclient/src/main/java/org/apache/pulsar/testclient/PerformanceTransaction.java:
##########
@@ -267,12 +224,17 @@ public static void main(String[] args)
}
if (arguments.partitions != null) {
PulsarAdminBuilder clientBuilder = PulsarAdmin.builder()
- .serviceHttpUrl(arguments.adminURL);
+ .serviceHttpUrl(arguments.adminURL)
+ .tlsTrustCertsFilePath(arguments.tlsTrustCertsFilePath);
if (isNotBlank(arguments.authPluginClassName)) {
clientBuilder.authentication(arguments.authPluginClassName, arguments.authParams);
}
+ if (arguments.tlsAllowInsecureConnection != null) {
+ clientBuilder.allowTlsInsecureConnection(arguments.tlsAllowInsecureConnection);
Review Comment:
what about enableTlsHostnameVerification ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] nicoloboschi commented on pull request #16090: [improve][pulsar-perf] Transactions: improve client options
Posted by GitBox <gi...@apache.org>.
nicoloboschi commented on PR #16090:
URL: https://github.com/apache/pulsar/pull/16090#issuecomment-1158739261
@liangyepianzhou PTAL again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli merged pull request #16090: [improve][pulsar-perf] Transactions: improve client options
Posted by GitBox <gi...@apache.org>.
eolivelli merged PR #16090:
URL: https://github.com/apache/pulsar/pull/16090
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org