You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/10/22 11:28:37 UTC
[GitHub] [pulsar] murong00 opened a new issue #8340: Provide options to specify existing topics when allowAutoTopicCreation is disable and using multiple topics
murong00 opened a new issue #8340:
URL: https://github.com/apache/pulsar/issues/8340
**Is your enhancement request related to a problem? Please describe.**
Some users may encounter the following issues when using multiple topics with option `--num-topics` but `allowAutoTopicCreation` is disable:
```
Exception in thread "main" java.util.concurrent.ExecutionException: org.apache.pulsar.client.api.PulsarClientException$TopicDoesNotExistException: Topic does not exist
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at org.apache.pulsar.testclient.PerformanceConsumer.main(PerformanceConsumer.java:343)
Caused by: org.apache.pulsar.client.api.PulsarClientException$TopicDoesNotExistException: Topic does not exist
at org.apache.pulsar.client.impl.ClientCnx.getPulsarClientException(ClientCnx.java:1003)
at org.apache.pulsar.client.impl.ClientCnx.handleError(ClientCnx.java:610)
at org.apache.pulsar.common.protocol.PulsarDecoder.channelRead(PulsarDecoder.java:170)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
```
or using multiple subscriptions with option `--num-consumers` but `allowAutoSubscriptionCreation` is disable:
```
[pulsar-client-io-1-8] WARN org.apache.pulsar.client.impl.ConnectionHandler - [persistent://public/default/test-partition-3] [sub-1] Could not get connection to broker: Subscription does not exist -- Will try again in 0.1 s
[pulsar-client-io-1-7] WARN org.apache.pulsar.client.impl.ConnectionHandler - [persistent://public/default/test-partition-4] [sub-1] Could not get connection to broker: Subscription does not exist -- Will try again in 0.1 s
[pulsar-client-io-1-3] WARN org.apache.pulsar.client.impl.ConnectionHandler - [persistent://public/default/test-partition-2] [sub-0] Could not get connection to broker: Subscription does not exist -- Will try again in 0.1 s
[pulsar-client-io-1-4] WARN org.apache.pulsar.client.impl.ConnectionHandler - [persistent://public/default/test-partition-1] [sub-0] Could not get connection to broker: Subscription does not exist -- Will try again in 0.1 s
```
**Describe the solution you'd like**
Add some options to specify existing topics or subscriptions to meet the above scenario.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] murong00 removed a comment on issue #8340: [pulsar-testclient]Provide options to specify existing topics when allowAutoTopicCreation is disable and using multiple topics
Posted by GitBox <gi...@apache.org>.
murong00 removed a comment on issue #8340:
URL: https://github.com/apache/pulsar/issues/8340#issuecomment-724532309
@codelipenghui Thanks for pointing this out, however, the user may not have permissions to create new topics and subscriptions. In addtion, if we fix --topic (which is `List<String>` but only one is allowed) to support using the specified topics and check its size by `--num-topics`, it is inconvenient when auto-creation is allowed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] murong00 commented on issue #8340: [pulsar-testclient]Provide options to specify existing topics when allowAutoTopicCreation is disable and using multiple topics
Posted by GitBox <gi...@apache.org>.
murong00 commented on issue #8340:
URL: https://github.com/apache/pulsar/issues/8340#issuecomment-722939860
@codelipenghui I agree with you, however, when `--num-topics` or `--num-consumers` is larger than one current `--topic` and `--subscription`(whose size are not allowed to be greater than one) are regarded as a prefix and topics and subscriptions prefixed with this are auto-created. If we fix it by allowing multi topis/subs, I think it is a break change, what's more, how do users decide whether to automatically create topics(default behavior) with the prefix or to use the specified ones?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] murong00 commented on issue #8340: [pulsar-testclient]Provide options to specify existing topics when allowAutoTopicCreation is disable and using multiple topics
Posted by GitBox <gi...@apache.org>.
murong00 commented on issue #8340:
URL: https://github.com/apache/pulsar/issues/8340#issuecomment-724532309
@codelipenghui Thanks for pointing this out, however, the user may not have permissions to create new topics and subscriptions. In addtion, if we fix --topic (which is `List<String>` but only one is allowed) to support using the specified topics and check its size by `--num-topics`, it is inconvenient when auto-creation is allowed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #8340: [pulsar-testclient]Provide options to specify existing topics when allowAutoTopicCreation is disable and using multiple topics
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #8340:
URL: https://github.com/apache/pulsar/issues/8340#issuecomment-722980362
@murong00 I think we can check if the specified topic name and subscription name is already created, if created we should use the existing topic and namespace directly, if not we should create new topics and subscriptions as expected. I think it's more clear for users(Pulsar client also following this pattern).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui edited a comment on issue #8340: [pulsar-testclient]Provide options to specify existing topics when allowAutoTopicCreation is disable and using multiple topics
Posted by GitBox <gi...@apache.org>.
codelipenghui edited a comment on issue #8340:
URL: https://github.com/apache/pulsar/issues/8340#issuecomment-722267662
@murong00 If users disabled the topic auto-creation or subscription auto-creation. Users should create a Topic or Subscription for perf test, any reason for introducing new options in #8342? If current --topic and --subscription can't use the existing topic and subscription, I think we should fix it rather than adding more options.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] zymap closed issue #8340: [pulsar-testclient]Provide options to specify existing topics when allowAutoTopicCreation is disable and using multiple topics
Posted by GitBox <gi...@apache.org>.
zymap closed issue #8340:
URL: https://github.com/apache/pulsar/issues/8340
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #8340: [pulsar-testclient]Provide options to specify existing topics when allowAutoTopicCreation is disable and using multiple topics
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on issue #8340:
URL: https://github.com/apache/pulsar/issues/8340#issuecomment-722267662
@murong00 If users disabled the topic auto-creation or subscription auto-creation. Users should create a Topic or Subscription for perf test, any reason for introducing new options in #8342?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org