You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "hecter-java-architect (via GitHub)" <gi...@apache.org> on 2023/08/21 08:16:29 UTC
[GitHub] [pulsar] hecter-java-architect opened a new issue, #21038: [Bug] Topic cannot be matched using regular expressions
hecter-java-architect opened a new issue, #21038:
URL: https://github.com/apache/pulsar/issues/21038
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Version
pulsarVersion:3.0.0
pulsarClientVersion:3.0.0
### Minimal reproduce step
When the server receives a request of type GET_TOPICS_OF_NAMESPACE, it will return the Topic and perform regular matching
However, the returned Topic contains the "-partition-0" partition suffix, but when matching again, the partition suffix is not removed
`getBrokerService().pulsar().getNamespaceService().getListOfTopics(namespaceName, mode)
.thenAccept(topics -> {
boolean filterTopics = false;
// filter system topic
List<String> filteredTopics = TopicList.filterSystemTopic(topics);
if (enableSubscriptionPatternEvaluation && topicsPattern.isPresent()) {
if (topicsPattern.get().length() <= maxSubscriptionPatternLength) {
filterTopics = true;
filteredTopics = TopicList.filterTopics(filteredTopics, topicsPattern.get());
} else {
log.info("[{}] Subscription pattern provided was longer than maximum {}.",
remoteAddress, maxSubscriptionPatternLength);
}
}
}`
Let's take a look`filterSystemTopic`
`List<String> filteredTopics = TopicList.filterSystemTopic(topics);`
`
public static List<String> filterSystemTopic(List<String> original) {
return original.stream()
.filter(topic -> !SystemTopicNames.isSystemTopic(TopicName.get(topic)))
.collect(Collectors.toList());
}
public static boolean isSystemTopic(TopicName topicName) {
TopicName nonPartitionedTopicName = TopicName.get(topicName.getPartitionedTopicName());
return isEventSystemTopic(nonPartitionedTopicName) || isTransactionInternalName(nonPartitionedTopicName);
}`
### What did you expect to see?
Hope that the matching topic can be removed from the partition suffix
### What did you see instead?
The matching topic does not remove the partition suffix
### Anything else?
_No response_
### Are you willing to submit a PR?
- [X] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hecter-java-architect commented on issue #21038: [Bug] Partition topic cannot be matched using regex
Posted by "hecter-java-architect (via GitHub)" <gi...@apache.org>.
hecter-java-architect commented on issue #21038:
URL: https://github.com/apache/pulsar/issues/21038#issuecomment-1690193152
> 你好，@hecter-java-architect
>
> 您可以查看此处的讨论以获取解决方法。[#20716（在帖子中回复）](https://github.com/apache/pulsar/discussions/20716#discussioncomment-6679380)
>
> 我会开始讨论，看看什么行为是正确的，我们就可以让他们达成共识然后。
Thanks，I think that after this configuration `enableBrokerSideSubscriptionPatternEvaluation` is false, all topics will be returned to the client for filtering, which is not a good practice, And the `filterTopics` method also hopes not to include partition part，So I want to know why not remove the partition information on the server side for filtering。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hecter-java-architect commented on issue #21038: [Bug] Partition topic cannot be matched using regex
Posted by "hecter-java-architect (via GitHub)" <gi...@apache.org>.
hecter-java-architect commented on issue #21038:
URL: https://github.com/apache/pulsar/issues/21038#issuecomment-1691182991
> @hecter-java-architect
>
> Yep, You are right. We should consider changing it, but it was introduced by a new proposal. let me issue a discussion for it.
sure，when we can start discussing this proposal，i want to join
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] mattisonchao commented on issue #21038: [Bug] Partition topic cannot be matched using regex
Posted by "mattisonchao (via GitHub)" <gi...@apache.org>.
mattisonchao commented on issue #21038:
URL: https://github.com/apache/pulsar/issues/21038#issuecomment-1689401568
Hi, @hecter-java-architect
You can check the discussion here to get the workaround fix. https://github.com/apache/pulsar/discussions/20716#discussioncomment-6679380
I'll start talking about it, see what behaviour is correct, and then we can reach a consensus.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hecter-java-architect commented on issue #21038: [Bug] Topic cannot be matched using regular expressions
Posted by "hecter-java-architect (via GitHub)" <gi...@apache.org>.
hecter-java-architect commented on issue #21038:
URL: https://github.com/apache/pulsar/issues/21038#issuecomment-1685883959
我发现在 filterTopics 方法中，有注释显示不应该包含分区的部分
```java
// get topics that match 'topicsPattern' from original topics list
// return result should contain only topic names, without partition part
public static List<String> filterTopics(List<String> original, String regex) {
Pattern topicsPattern = Pattern.compile(regex);
return filterTopics(original, topicsPattern);
}
public static List<String> filterTopics(List<String> original, Pattern topicsPattern) {
final Pattern shortenedTopicsPattern = topicsPattern.toString().contains(SCHEME_SEPARATOR)
? Pattern.compile(SCHEME_SEPARATOR_PATTERN.split(topicsPattern.toString())[1]) : topicsPattern;
return original.stream()
.map(TopicName::get)
.map(TopicName::toString)
.filter(topic -> shortenedTopicsPattern.matcher(SCHEME_SEPARATOR_PATTERN.split(topic)[1]).matches())
.collect(Collectors.toList());
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Partition topic cannot be matched using regex [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat commented on issue #21038:
URL: https://github.com/apache/pulsar/issues/21038#issuecomment-6381732859
Thanks for the report and the analysis, @hecter-java-architect. This was fixed by https://github.com/apache/pulsar/pull/21885, which makes `TopicList.filterTopics` match the pattern against the partitioned topic name without the `-partition-N` suffix (see [TopicList.java](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/java/org/apache/pulsar/common/topics/TopicList.java#L76-L84)). https://github.com/apache/pulsar/pull/22025 fixed the same problem for topic-list watcher notifications. Both fixes are in 4.0.x and later. I'm closing this. Please reopen if a pattern like `tenant/ns/mixed-load-1-\d{1,3}` still doesn't match partitioned topics on a currently supported release (4.0.x, 4.2.x or 5.0.x).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hecter-java-architect commented on issue #21038: [Bug] Topic cannot be matched using regular expressions
Posted by "hecter-java-architect (via GitHub)" <gi...@apache.org>.
hecter-java-architect commented on issue #21038:
URL: https://github.com/apache/pulsar/issues/21038#issuecomment-1685878287
The data of the /managed-ledgers path under zookeeper is very clear, so it will not be displayed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on issue #21038: [Bug] Partition topic cannot be matched using regex
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #21038:
URL: https://github.com/apache/pulsar/issues/21038#issuecomment-1732453539
The issue had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] mattisonchao commented on issue #21038: [Bug] Partition topic cannot be matched using regex
Posted by "mattisonchao (via GitHub)" <gi...@apache.org>.
mattisonchao commented on issue #21038:
URL: https://github.com/apache/pulsar/issues/21038#issuecomment-1690785418
@hecter-java-architect
Yep, You are right. We should consider changing it, but it was introduced by a new proposal. let me issue a discussion for it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hecter-java-architect commented on issue #21038: [Bug] Partition topic cannot be matched using regex
Posted by "hecter-java-architect (via GitHub)" <gi...@apache.org>.
hecter-java-architect commented on issue #21038:
URL: https://github.com/apache/pulsar/issues/21038#issuecomment-1691178469
sure，when we can start discussing this proposal，i want to join us
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hecter-java-architect commented on issue #21038: [Bug] Topic cannot be matched using regular expressions
Posted by "hecter-java-architect (via GitHub)" <gi...@apache.org>.
hecter-java-architect commented on issue #21038:
URL: https://github.com/apache/pulsar/issues/21038#issuecomment-1685876180
test
```java
public static void main(String[] args) {
List<String> topics = Lists.newArrayList(
"persistent://local-public/default/mixed-load-1-0-partition-0",
"persistent://local-public/default/mixed-load-1-1-partition-0",
"persistent://local-public/default/mixed-load-no-partten-1-0-partition-0");
List<String> filteredTopics = TopicList.filterSystemTopic(topics);
filteredTopics = TopicList.filterTopics(filteredTopics, "local-public/default/mixed-load-1-\\d{1,3}");
System.out.println(filteredTopics);
}
```
result
```
[]
Process finished with exit code 0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] hecter-java-architect commented on issue #21038: [Bug] Topic cannot be matched using regular expressions
Posted by "hecter-java-architect (via GitHub)" <gi...@apache.org>.
hecter-java-architect commented on issue #21038:
URL: https://github.com/apache/pulsar/issues/21038#issuecomment-1685907094
```java
return original.stream()
.map(TopicName::get)
// should use TopicName::getPartitionedTopicName
.map(TopicName::toString)
.filter(topic -> shortenedTopicsPattern.matcher(SCHEME_SEPARATOR_PATTERN.split(topic)[1]).matches())
.collect(Collectors.toList());
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Partition topic cannot be matched using regex [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat closed issue #21038: [Bug] Partition topic cannot be matched using regex
URL: https://github.com/apache/pulsar/issues/21038
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org