You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "dmonopoly (via GitHub)" <gi...@apache.org> on 2024/12/17 21:59:23 UTC
[I] Get not allowed error, Unable to create topic list watcher: Pattern longer than maximum: 50 [pulsar]
dmonopoly opened a new issue, #23750:
URL: https://github.com/apache/pulsar/issues/23750
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Motivation
I have topics that are very long: about 100 characters. The regex matcher cannot handle this apparently.
```
var patternStr = topic + ".*"
val topicPattern: Pattern = Pattern.compile(patternStr)
pulsarConsumer = consumerBuilder.topicsPattern(topicPattern).subscribe()
```
=> [pulsar-client-io-1-3] ERROR org.apache.pulsar.client.impl.ClientCnx - Get not allowed error, Unable to create topic list watcher: Pattern longer than maximum: 50
If I try to shorten the topic pattern and only use the last 50 or so characters, this is also not allowed:
```
var patternStr = topic + ".*"
if (topic.length > 46) {
patternStr = ".*" + topic.slice(topic.length - 46..topic.length - 1) + ".*"
logger.info("Using shortened topic regex matcher: $patternStr")
}
val topicPattern: Pattern = Pattern.compile(patternStr)
pulsarConsumer = consumerBuilder.topicsPattern(topicPattern).subscribe()
```
=>
Exception in thread "main" org.apache.pulsar.client.api.PulsarClientException: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Invalid short topic name '.*place513/consumption-metamagic542153-coverage-v0.*', it should be in the format of <tenant>/<namespace>/<topic> or <topic>
at org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:1111)
at org.apache.pulsar.client.impl.ConsumerBuilderImpl.subscribe(ConsumerBuilderImpl.java:103)
### Solution
_No response_
### Alternatives
_No response_
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Get not allowed error, Unable to create topic list watcher: Pattern longer than maximum: 50 [pulsar]
Posted by "tsturzl (via GitHub)" <gi...@apache.org>.
tsturzl commented on issue #23750:
URL: https://github.com/apache/pulsar/issues/23750#issuecomment-2578940992
I'm running into this, seems very strange to limit the pattern by string length
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Get not allowed error, Unable to create topic list watcher: Pattern longer than maximum: 50 [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat commented on issue #23750:
URL: https://github.com/apache/pulsar/issues/23750#issuecomment-6388724031
Thanks @dmonopoly. The maximum is configurable: raise `subscriptionPatternMaxLength` in `broker.conf` (default 50), as @tsturzl pointed out. The broker limits pattern length because it evaluates the regex against every topic in the namespace for each topic-list watcher. The second error happens because `topicsPattern` needs the `persistent://tenant/namespace/` prefix outside the regex, so the pattern can't start with `.*`. Closing as completed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Get not allowed error, Unable to create topic list watcher: Pattern longer than maximum: 50 [pulsar]
Posted by "tsturzl (via GitHub)" <gi...@apache.org>.
tsturzl commented on issue #23750:
URL: https://github.com/apache/pulsar/issues/23750#issuecomment-2580884734
There is a broker configuration field `subscriptionPatternMaxLength` you can set to whatever limit you want. I'm still not entirely sure why this is limited. I've run into issues trying to match large quantities of topics, but I can't imagine that regex length is a major factor in how fast topic lookup happens.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Get not allowed error, Unable to create topic list watcher: Pattern longer than maximum: 50 [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat closed issue #23750: Get not allowed error, Unable to create topic list watcher: Pattern longer than maximum: 50
URL: https://github.com/apache/pulsar/issues/23750
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org