You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/09/25 04:30:22 UTC
[GitHub] [pulsar] yuruguo opened a new pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
yuruguo opened a new pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187
### Motivation
The option `--payload-file` is just defined but does not work in PerformanceClient, this PR supports to use payload from a file instead of empty buffer.
### Modifications
- Add option `--payload-delimiter`, `--format-payload`, and `--format-class`
- produceSocket sends payloadData
### Documentation
Need to update docs
- doc-required
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-928430733
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo removed a comment on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo removed a comment on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-930642723
@merlimat @lhotari @aahmed-se @sijie @codelipenghui @jiazhai PTAL, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-938219849
@merlimat @lhotari @aahmed-se @sijie @codelipenghui @jiazhai @BewareMyPower @hangc0276 @gaoran10 PTAL, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo removed a comment on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo removed a comment on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-927279612
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-927681502
@merlimat @lhotari @codelipenghui @jiazhai
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-928430733
@merlimat @k2la @codelipenghui @jiazhai PTAL, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-934131261
@merlimat @lhotari @aahmed-se @sijie @codelipenghui @jiazhai @BewareMyPower @hangc0276 @gaoran10 PTAL, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo removed a comment on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo removed a comment on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-930810678
@merlimat @lhotari @aahmed-se @sijie @codelipenghui @jiazhai - PTAL, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-927681772
@merlimat @lhotari @codelipenghui @jiazhai PTAL, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] k2la commented on a change in pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
k2la commented on a change in pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#discussion_r717153523
##########
File path: pulsar-testclient/src/main/java/org/apache/pulsar/proxy/socket/client/PerformanceClient.java
##########
@@ -193,23 +212,49 @@ public Arguments loadArguments(String[] args) {
}
- public void runPerformanceTest(long messages, long limit, int numOfTopic, int sizeOfMessage, String baseUrl,
- String topicName, String authPluginClassName, String authParams) throws InterruptedException, FileNotFoundException {
+ public void runPerformanceTest(Arguments arguments) throws InterruptedException, IOException {
+ // Read payload data from file if needed
+ final byte[] payloadBytes = new byte[arguments.msgSize];
+ Random random = new Random(0);
+ List<byte[]> payloadByteList = Lists.newArrayList();
+ if (arguments.payloadFilename != null) {
+ Path payloadFilePath = Paths.get(arguments.payloadFilename);
+ if (Files.notExists(payloadFilePath) || Files.size(payloadFilePath) == 0) {
+ throw new IllegalArgumentException("Payload file doesn't exist or it is empty.");
+ }
+ // here escaping the default payload delimiter to correct value
+ String delimiter = arguments.payloadDelimiter.equals("\\n") ? "\n" : arguments.payloadDelimiter;
+ String[] payloadList = new String(Files.readAllBytes(payloadFilePath), StandardCharsets.UTF_8).split(delimiter);
+ log.info("Reading payloads from {} and {} records read", payloadFilePath.toAbsolutePath(), payloadList.length);
+ for (String payload : payloadList) {
+ payloadByteList.add(payload.getBytes(StandardCharsets.UTF_8));
+ }
+
+ if (arguments.formatPayload) {
+ messageFormatter = getMessageFormatter(arguments.formatterClass);
+ }
+ } else {
+ for (int i = 0; i < payloadBytes.length; ++i) {
+ payloadBytes[i] = (byte) (random.nextInt(26) + 65);
Review comment:
What is this `65`?
##########
File path: pulsar-testclient/src/main/java/org/apache/pulsar/proxy/socket/client/PerformanceClient.java
##########
@@ -193,23 +212,49 @@ public Arguments loadArguments(String[] args) {
}
- public void runPerformanceTest(long messages, long limit, int numOfTopic, int sizeOfMessage, String baseUrl,
- String topicName, String authPluginClassName, String authParams) throws InterruptedException, FileNotFoundException {
+ public void runPerformanceTest(Arguments arguments) throws InterruptedException, IOException {
+ // Read payload data from file if needed
+ final byte[] payloadBytes = new byte[arguments.msgSize];
+ Random random = new Random(0);
+ List<byte[]> payloadByteList = Lists.newArrayList();
+ if (arguments.payloadFilename != null) {
+ Path payloadFilePath = Paths.get(arguments.payloadFilename);
+ if (Files.notExists(payloadFilePath) || Files.size(payloadFilePath) == 0) {
+ throw new IllegalArgumentException("Payload file doesn't exist or it is empty.");
+ }
+ // here escaping the default payload delimiter to correct value
+ String delimiter = arguments.payloadDelimiter.equals("\\n") ? "\n" : arguments.payloadDelimiter;
+ String[] payloadList = new String(Files.readAllBytes(payloadFilePath), StandardCharsets.UTF_8).split(delimiter);
+ log.info("Reading payloads from {} and {} records read", payloadFilePath.toAbsolutePath(), payloadList.length);
+ for (String payload : payloadList) {
+ payloadByteList.add(payload.getBytes(StandardCharsets.UTF_8));
+ }
+
+ if (arguments.formatPayload) {
+ messageFormatter = getMessageFormatter(arguments.formatterClass);
+ }
+ } else {
+ for (int i = 0; i < payloadBytes.length; ++i) {
+ payloadBytes[i] = (byte) (random.nextInt(26) + 65);
Review comment:
I see.
I think it would be easier for developers to understand that if 65 is replaced with a constant or if comments are added.
##########
File path: pulsar-testclient/src/main/java/org/apache/pulsar/proxy/socket/client/PerformanceClient.java
##########
@@ -193,23 +212,49 @@ public Arguments loadArguments(String[] args) {
}
- public void runPerformanceTest(long messages, long limit, int numOfTopic, int sizeOfMessage, String baseUrl,
- String topicName, String authPluginClassName, String authParams) throws InterruptedException, FileNotFoundException {
+ public void runPerformanceTest(Arguments arguments) throws InterruptedException, IOException {
+ // Read payload data from file if needed
+ final byte[] payloadBytes = new byte[arguments.msgSize];
+ Random random = new Random(0);
+ List<byte[]> payloadByteList = Lists.newArrayList();
+ if (arguments.payloadFilename != null) {
+ Path payloadFilePath = Paths.get(arguments.payloadFilename);
+ if (Files.notExists(payloadFilePath) || Files.size(payloadFilePath) == 0) {
+ throw new IllegalArgumentException("Payload file doesn't exist or it is empty.");
+ }
+ // here escaping the default payload delimiter to correct value
+ String delimiter = arguments.payloadDelimiter.equals("\\n") ? "\n" : arguments.payloadDelimiter;
+ String[] payloadList = new String(Files.readAllBytes(payloadFilePath), StandardCharsets.UTF_8).split(delimiter);
+ log.info("Reading payloads from {} and {} records read", payloadFilePath.toAbsolutePath(), payloadList.length);
+ for (String payload : payloadList) {
+ payloadByteList.add(payload.getBytes(StandardCharsets.UTF_8));
+ }
+
+ if (arguments.formatPayload) {
+ messageFormatter = getMessageFormatter(arguments.formatterClass);
+ }
+ } else {
+ for (int i = 0; i < payloadBytes.length; ++i) {
+ payloadBytes[i] = (byte) (random.nextInt(26) + 65);
Review comment:
I see.
I think it would be easier for developers to understand that if `65` is replaced with a constant or if comments are added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo removed a comment on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo removed a comment on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-927681772
@merlimat @lhotari @codelipenghui @jiazhai PTAL, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo removed a comment on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo removed a comment on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-927681772
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-929724930
@merlimat @aahmed-se @sijie @codelipenghui @jiazhai PTAL, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-930642723
@merlimat @aahmed-se @sijie @codelipenghui @jiazhai PTAL, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo removed a comment on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo removed a comment on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-928430733
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-929724930
@merlimat @aahmed-se @sijie @codelipenghui @jiazhai PTAL, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo removed a comment on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo removed a comment on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-933896283
@merlimat @lhotari @aahmed-se @sijie @codelipenghui @jiazhai PTAL, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo removed a comment on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo removed a comment on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-928914376
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-941799068
> @yuruguo done
thx :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Anonymitaet commented on a change in pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on a change in pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#discussion_r716155333
##########
File path: site2/docs/reference-cli-tools.md
##########
@@ -525,6 +525,9 @@ Options
|`-m`, `--num-messages`|Number of messages to publish in total. If this value is less than or equal to 0, it keeps publishing messages.|0|
|`-t`, `--num-topic`|The number of topics|1|
|`-f`, `--payload-file`|Use payload from a file instead of empty buffer||
+|`-e`, `--payload-delimiter`|The delimiter used to split lines when using payload from a file|\n|
+|`-fp`, `--format-payload`|Format %i as a message index in the stream from producer and/or %t as the timestamp nanoseconds|false|
+|`-fc`, `--format-class`|Custom Formatter class name|org.apache.pulsar.testclient.DefaultMessageFormatter|
Review comment:
```suggestion
|`-fc`, `--format-class`|Custom formatter class name|`org.apache.pulsar.testclient.DefaultMessageFormatter`|
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo edited a comment on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo edited a comment on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-934131261
@merlimat @lhotari @aahmed-se @sijie @codelipenghui @jiazhai @BewareMyPower @hangc0276 @gaoran10 PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Anonymitaet commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-941793697
@yuruguo done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-928914376
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] k2la commented on a change in pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
k2la commented on a change in pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#discussion_r717245893
##########
File path: pulsar-testclient/src/main/java/org/apache/pulsar/proxy/socket/client/PerformanceClient.java
##########
@@ -193,23 +212,49 @@ public Arguments loadArguments(String[] args) {
}
- public void runPerformanceTest(long messages, long limit, int numOfTopic, int sizeOfMessage, String baseUrl,
- String topicName, String authPluginClassName, String authParams) throws InterruptedException, FileNotFoundException {
+ public void runPerformanceTest(Arguments arguments) throws InterruptedException, IOException {
+ // Read payload data from file if needed
+ final byte[] payloadBytes = new byte[arguments.msgSize];
+ Random random = new Random(0);
+ List<byte[]> payloadByteList = Lists.newArrayList();
+ if (arguments.payloadFilename != null) {
+ Path payloadFilePath = Paths.get(arguments.payloadFilename);
+ if (Files.notExists(payloadFilePath) || Files.size(payloadFilePath) == 0) {
+ throw new IllegalArgumentException("Payload file doesn't exist or it is empty.");
+ }
+ // here escaping the default payload delimiter to correct value
+ String delimiter = arguments.payloadDelimiter.equals("\\n") ? "\n" : arguments.payloadDelimiter;
+ String[] payloadList = new String(Files.readAllBytes(payloadFilePath), StandardCharsets.UTF_8).split(delimiter);
+ log.info("Reading payloads from {} and {} records read", payloadFilePath.toAbsolutePath(), payloadList.length);
+ for (String payload : payloadList) {
+ payloadByteList.add(payload.getBytes(StandardCharsets.UTF_8));
+ }
+
+ if (arguments.formatPayload) {
+ messageFormatter = getMessageFormatter(arguments.formatterClass);
+ }
+ } else {
+ for (int i = 0; i < payloadBytes.length; ++i) {
+ payloadBytes[i] = (byte) (random.nextInt(26) + 65);
Review comment:
I see.
I think it would be easier for developers to understand that if `65` is replaced with a constant or if comments are added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo edited a comment on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo edited a comment on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-927279528
> Thanks for your contribution. Does this affect only master or other versioned docs?
> If latter, could you please help update [all affected versions](https://github.com/apache/pulsar/tree/master/site2/website/versioned_docs)? Thanks
This only affects `master` branch because they are new options.
please add `doc` label for this PR, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-927205251
PTAL @merlimat @codelipenghui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Anonymitaet commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
Anonymitaet commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-941793697
@yuruguo done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-941799068
> @yuruguo done
thx :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo edited a comment on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo edited a comment on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-933896283
@merlimat @lhotari @aahmed-se @sijie @codelipenghui @jiazhai PTAL, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-927279612
/pulsarbot run-failure-checks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo removed a comment on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo removed a comment on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-927205251
PTAL @merlimat @codelipenghui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo edited a comment on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo edited a comment on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-930642723
@merlimat @lhotari @aahmed-se @sijie @codelipenghui @jiazhai PTAL, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] k2la commented on a change in pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
k2la commented on a change in pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#discussion_r717245893
##########
File path: pulsar-testclient/src/main/java/org/apache/pulsar/proxy/socket/client/PerformanceClient.java
##########
@@ -193,23 +212,49 @@ public Arguments loadArguments(String[] args) {
}
- public void runPerformanceTest(long messages, long limit, int numOfTopic, int sizeOfMessage, String baseUrl,
- String topicName, String authPluginClassName, String authParams) throws InterruptedException, FileNotFoundException {
+ public void runPerformanceTest(Arguments arguments) throws InterruptedException, IOException {
+ // Read payload data from file if needed
+ final byte[] payloadBytes = new byte[arguments.msgSize];
+ Random random = new Random(0);
+ List<byte[]> payloadByteList = Lists.newArrayList();
+ if (arguments.payloadFilename != null) {
+ Path payloadFilePath = Paths.get(arguments.payloadFilename);
+ if (Files.notExists(payloadFilePath) || Files.size(payloadFilePath) == 0) {
+ throw new IllegalArgumentException("Payload file doesn't exist or it is empty.");
+ }
+ // here escaping the default payload delimiter to correct value
+ String delimiter = arguments.payloadDelimiter.equals("\\n") ? "\n" : arguments.payloadDelimiter;
+ String[] payloadList = new String(Files.readAllBytes(payloadFilePath), StandardCharsets.UTF_8).split(delimiter);
+ log.info("Reading payloads from {} and {} records read", payloadFilePath.toAbsolutePath(), payloadList.length);
+ for (String payload : payloadList) {
+ payloadByteList.add(payload.getBytes(StandardCharsets.UTF_8));
+ }
+
+ if (arguments.formatPayload) {
+ messageFormatter = getMessageFormatter(arguments.formatterClass);
+ }
+ } else {
+ for (int i = 0; i < payloadBytes.length; ++i) {
+ payloadBytes[i] = (byte) (random.nextInt(26) + 65);
Review comment:
I see.
I think it would be easier for developers to understand that if 65 is replaced with a constant or if comments are added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-927279528
> Thanks for your contribution. Does this affect only master or other versioned docs?
> If latter, could you please help update [all affected versions](https://github.com/apache/pulsar/tree/master/site2/website/versioned_docs)? Thanks
This only affects `master` branch because they are new options.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Anonymitaet merged pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
Anonymitaet merged pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo removed a comment on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo removed a comment on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-927681502
@merlimat @lhotari @codelipenghui @jiazhai
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo removed a comment on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo removed a comment on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-928430733
@merlimat @k2la @codelipenghui @jiazhai PTAL, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on a change in pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on a change in pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#discussion_r717238556
##########
File path: pulsar-testclient/src/main/java/org/apache/pulsar/proxy/socket/client/PerformanceClient.java
##########
@@ -193,23 +212,49 @@ public Arguments loadArguments(String[] args) {
}
- public void runPerformanceTest(long messages, long limit, int numOfTopic, int sizeOfMessage, String baseUrl,
- String topicName, String authPluginClassName, String authParams) throws InterruptedException, FileNotFoundException {
+ public void runPerformanceTest(Arguments arguments) throws InterruptedException, IOException {
+ // Read payload data from file if needed
+ final byte[] payloadBytes = new byte[arguments.msgSize];
+ Random random = new Random(0);
+ List<byte[]> payloadByteList = Lists.newArrayList();
+ if (arguments.payloadFilename != null) {
+ Path payloadFilePath = Paths.get(arguments.payloadFilename);
+ if (Files.notExists(payloadFilePath) || Files.size(payloadFilePath) == 0) {
+ throw new IllegalArgumentException("Payload file doesn't exist or it is empty.");
+ }
+ // here escaping the default payload delimiter to correct value
+ String delimiter = arguments.payloadDelimiter.equals("\\n") ? "\n" : arguments.payloadDelimiter;
+ String[] payloadList = new String(Files.readAllBytes(payloadFilePath), StandardCharsets.UTF_8).split(delimiter);
+ log.info("Reading payloads from {} and {} records read", payloadFilePath.toAbsolutePath(), payloadList.length);
+ for (String payload : payloadList) {
+ payloadByteList.add(payload.getBytes(StandardCharsets.UTF_8));
+ }
+
+ if (arguments.formatPayload) {
+ messageFormatter = getMessageFormatter(arguments.formatterClass);
+ }
+ } else {
+ for (int i = 0; i < payloadBytes.length; ++i) {
+ payloadBytes[i] = (byte) (random.nextInt(26) + 65);
Review comment:
The value of the `payloadBytes` is from `A` to `Z` and the ASCII of `A`-`Z` is `65` - `90`, so we need to add 65.
##########
File path: pulsar-testclient/src/main/java/org/apache/pulsar/proxy/socket/client/PerformanceClient.java
##########
@@ -193,23 +212,49 @@ public Arguments loadArguments(String[] args) {
}
- public void runPerformanceTest(long messages, long limit, int numOfTopic, int sizeOfMessage, String baseUrl,
- String topicName, String authPluginClassName, String authParams) throws InterruptedException, FileNotFoundException {
+ public void runPerformanceTest(Arguments arguments) throws InterruptedException, IOException {
+ // Read payload data from file if needed
+ final byte[] payloadBytes = new byte[arguments.msgSize];
+ Random random = new Random(0);
+ List<byte[]> payloadByteList = Lists.newArrayList();
+ if (arguments.payloadFilename != null) {
+ Path payloadFilePath = Paths.get(arguments.payloadFilename);
+ if (Files.notExists(payloadFilePath) || Files.size(payloadFilePath) == 0) {
+ throw new IllegalArgumentException("Payload file doesn't exist or it is empty.");
+ }
+ // here escaping the default payload delimiter to correct value
+ String delimiter = arguments.payloadDelimiter.equals("\\n") ? "\n" : arguments.payloadDelimiter;
+ String[] payloadList = new String(Files.readAllBytes(payloadFilePath), StandardCharsets.UTF_8).split(delimiter);
+ log.info("Reading payloads from {} and {} records read", payloadFilePath.toAbsolutePath(), payloadList.length);
+ for (String payload : payloadList) {
+ payloadByteList.add(payload.getBytes(StandardCharsets.UTF_8));
+ }
+
+ if (arguments.formatPayload) {
+ messageFormatter = getMessageFormatter(arguments.formatterClass);
+ }
+ } else {
+ for (int i = 0; i < payloadBytes.length; ++i) {
+ payloadBytes[i] = (byte) (random.nextInt(26) + 65);
Review comment:
I have added a comment to explain about `65`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo removed a comment on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo removed a comment on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-938219849
@merlimat @lhotari @aahmed-se @sijie @codelipenghui @jiazhai @BewareMyPower @hangc0276 @gaoran10 PTAL, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-940572596
@hangc0276 PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo removed a comment on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo removed a comment on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-940572596
@hangc0276 PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on a change in pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on a change in pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#discussion_r717268368
##########
File path: pulsar-testclient/src/main/java/org/apache/pulsar/proxy/socket/client/PerformanceClient.java
##########
@@ -193,23 +212,49 @@ public Arguments loadArguments(String[] args) {
}
- public void runPerformanceTest(long messages, long limit, int numOfTopic, int sizeOfMessage, String baseUrl,
- String topicName, String authPluginClassName, String authParams) throws InterruptedException, FileNotFoundException {
+ public void runPerformanceTest(Arguments arguments) throws InterruptedException, IOException {
+ // Read payload data from file if needed
+ final byte[] payloadBytes = new byte[arguments.msgSize];
+ Random random = new Random(0);
+ List<byte[]> payloadByteList = Lists.newArrayList();
+ if (arguments.payloadFilename != null) {
+ Path payloadFilePath = Paths.get(arguments.payloadFilename);
+ if (Files.notExists(payloadFilePath) || Files.size(payloadFilePath) == 0) {
+ throw new IllegalArgumentException("Payload file doesn't exist or it is empty.");
+ }
+ // here escaping the default payload delimiter to correct value
+ String delimiter = arguments.payloadDelimiter.equals("\\n") ? "\n" : arguments.payloadDelimiter;
+ String[] payloadList = new String(Files.readAllBytes(payloadFilePath), StandardCharsets.UTF_8).split(delimiter);
+ log.info("Reading payloads from {} and {} records read", payloadFilePath.toAbsolutePath(), payloadList.length);
+ for (String payload : payloadList) {
+ payloadByteList.add(payload.getBytes(StandardCharsets.UTF_8));
+ }
+
+ if (arguments.formatPayload) {
+ messageFormatter = getMessageFormatter(arguments.formatterClass);
+ }
+ } else {
+ for (int i = 0; i < payloadBytes.length; ++i) {
+ payloadBytes[i] = (byte) (random.nextInt(26) + 65);
Review comment:
I have added a comment to explain about `65`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-933896283
@merlimat @lhotari @aahmed-se @sijie @codelipenghui @jiazhai - PTAL, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo removed a comment on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo removed a comment on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-934131261
@merlimat @lhotari @aahmed-se @sijie @codelipenghui @jiazhai @BewareMyPower @hangc0276 @gaoran10 PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-940859894
@Anonymitaet Thank you for merging this PR, please also assign it to me and mark the corresponding label (`component/tools`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#issuecomment-930810678
@merlimat @lhotari @aahmed-se @sijie @codelipenghui @jiazhai - PTAL, thx!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] k2la commented on a change in pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
k2la commented on a change in pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#discussion_r717153523
##########
File path: pulsar-testclient/src/main/java/org/apache/pulsar/proxy/socket/client/PerformanceClient.java
##########
@@ -193,23 +212,49 @@ public Arguments loadArguments(String[] args) {
}
- public void runPerformanceTest(long messages, long limit, int numOfTopic, int sizeOfMessage, String baseUrl,
- String topicName, String authPluginClassName, String authParams) throws InterruptedException, FileNotFoundException {
+ public void runPerformanceTest(Arguments arguments) throws InterruptedException, IOException {
+ // Read payload data from file if needed
+ final byte[] payloadBytes = new byte[arguments.msgSize];
+ Random random = new Random(0);
+ List<byte[]> payloadByteList = Lists.newArrayList();
+ if (arguments.payloadFilename != null) {
+ Path payloadFilePath = Paths.get(arguments.payloadFilename);
+ if (Files.notExists(payloadFilePath) || Files.size(payloadFilePath) == 0) {
+ throw new IllegalArgumentException("Payload file doesn't exist or it is empty.");
+ }
+ // here escaping the default payload delimiter to correct value
+ String delimiter = arguments.payloadDelimiter.equals("\\n") ? "\n" : arguments.payloadDelimiter;
+ String[] payloadList = new String(Files.readAllBytes(payloadFilePath), StandardCharsets.UTF_8).split(delimiter);
+ log.info("Reading payloads from {} and {} records read", payloadFilePath.toAbsolutePath(), payloadList.length);
+ for (String payload : payloadList) {
+ payloadByteList.add(payload.getBytes(StandardCharsets.UTF_8));
+ }
+
+ if (arguments.formatPayload) {
+ messageFormatter = getMessageFormatter(arguments.formatterClass);
+ }
+ } else {
+ for (int i = 0; i < payloadBytes.length; ++i) {
+ payloadBytes[i] = (byte) (random.nextInt(26) + 65);
Review comment:
What is this `65`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] yuruguo commented on a change in pull request #12187: [testclient] Make --payload-file take effect in PerformanceClient
Posted by GitBox <gi...@apache.org>.
yuruguo commented on a change in pull request #12187:
URL: https://github.com/apache/pulsar/pull/12187#discussion_r717238556
##########
File path: pulsar-testclient/src/main/java/org/apache/pulsar/proxy/socket/client/PerformanceClient.java
##########
@@ -193,23 +212,49 @@ public Arguments loadArguments(String[] args) {
}
- public void runPerformanceTest(long messages, long limit, int numOfTopic, int sizeOfMessage, String baseUrl,
- String topicName, String authPluginClassName, String authParams) throws InterruptedException, FileNotFoundException {
+ public void runPerformanceTest(Arguments arguments) throws InterruptedException, IOException {
+ // Read payload data from file if needed
+ final byte[] payloadBytes = new byte[arguments.msgSize];
+ Random random = new Random(0);
+ List<byte[]> payloadByteList = Lists.newArrayList();
+ if (arguments.payloadFilename != null) {
+ Path payloadFilePath = Paths.get(arguments.payloadFilename);
+ if (Files.notExists(payloadFilePath) || Files.size(payloadFilePath) == 0) {
+ throw new IllegalArgumentException("Payload file doesn't exist or it is empty.");
+ }
+ // here escaping the default payload delimiter to correct value
+ String delimiter = arguments.payloadDelimiter.equals("\\n") ? "\n" : arguments.payloadDelimiter;
+ String[] payloadList = new String(Files.readAllBytes(payloadFilePath), StandardCharsets.UTF_8).split(delimiter);
+ log.info("Reading payloads from {} and {} records read", payloadFilePath.toAbsolutePath(), payloadList.length);
+ for (String payload : payloadList) {
+ payloadByteList.add(payload.getBytes(StandardCharsets.UTF_8));
+ }
+
+ if (arguments.formatPayload) {
+ messageFormatter = getMessageFormatter(arguments.formatterClass);
+ }
+ } else {
+ for (int i = 0; i < payloadBytes.length; ++i) {
+ payloadBytes[i] = (byte) (random.nextInt(26) + 65);
Review comment:
The value of the `payloadBytes` is from `A` to `Z` and the ASCII of `A`-`Z` is `65` - `90`, so we need to add 65.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org