Re: [Discuss] Optimize the performance of creating Topic

[Ivan Kelly <iv...@apache.org>]

> Creating a topic will first check whether the topic already exists.
> The verification will read all topics under the namespace, and then
> traverse these topics to see if the topic already exists.
> When there are a large number of topics under the namespace(about 300,000
> topics),
> less than 10 topics can be created in one second.
Why do we need to read all topics at all? We really just need to check
whether TOPIC or TOPIC-partition-0 exist.

Even if they do not exist, is there anything to stop one client
creating TOPIC and another creating TOPIC-partition-0?

-Ivan

Re: [Discuss] Optimize the performance of creating Topic

[Lin Lin <li...@apache.org>]

On 2021/08/03 11:12:34, Ivan Kelly <iv...@apache.org> wrote: 
> > Creating a topic will first check whether the topic already exists.
> > The verification will read all topics under the namespace, and then
> > traverse these topics to see if the topic already exists.
> > When there are a large number of topics under the namespace(about 300,000
> > topics),
> > less than 10 topics can be created in one second.
> Why do we need to read all topics at all? We really just need to check
> whether TOPIC or TOPIC-partition-0 exist.
> 
> Even if they do not exist, is there anything to stop one client
> creating TOPIC and another creating TOPIC-partition-0?
> 
> -Ivan
> 

Such as the test case "testCreatePartitionedTopicHavingNonPartitionTopicWithPartitionSuffix". Some non partition topic has the partition suffix. In this case, we can not use the cache to check anymore. And we have to traverse