[Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Xiangying Meng <xi...@apache.org>]

Dear Apache Pulsar Community,

I hope this email finds you well.I am writing to suggest a potential
improvement to the Pulsar-admin tool,
 which I believe could simplify the process of cleaning up tenants and
namespaces in Apache Pulsar.

Currently, cleaning up all the namespaces and topics within a tenant or
cleaning up all the topics within a namespace requires several manual
steps,
such as listing the namespaces, listing the topics, and then deleting each
topic individually.
This process can be time-consuming and error-prone for users.

To address this issue, I propose the addition of a "clear" parameter to the
Pulsar-admin tool,
which would automate the cleanup process for tenants and namespaces. Here's
a conceptual implementation:

1. To clean up all namespaces and topics within a tenant:
``` bash
pulsar-admin tenants clear <tenant-name>
```
2. To clean up all topics within a namespace:
```bash
pulsar-admin namespaces clear <tenant-name>/<namespace-name>
```

By implementing these new parameters, users would be able to perform
cleanup operations more efficiently and with fewer manual steps.
I believe this improvement would greatly enhance the user experience when
working with Apache Pulsar.

I'd like to discuss the feasibility of this suggestion and gather feedback
from the community.
If everyone agrees, I can work on implementing this feature and submit a
pull request for review.

Looking forward to hearing your thoughts on this.

Best regards,
Xiangying

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Michael Marshall <mm...@apache.org>]

> `delete-all-namespaces` and `delete-all-topics`

I support these terms as well. Regarding the concern about CLI
usability, bash completion could help a lot.

Additional questions:

1. What permission level is required for each action?
2. How will system topics be treated?

Thanks,
Michael


On Mon, Apr 17, 2023 at 2:59 AM Xiangying Meng <xi...@apache.org> wrote:
>
> Hi Penghui,
>
> I appreciate your feedback and completely agree with your concern about the
> learning curve for Pulsar users. Introducing additional keywords could
> potentially increase the complexity for users who need to understand the
> new terms. Therefore, I accept your suggestion to use delete-all-namespaces
> and delete-all-topics for the proposed improvement.
>
> Thank you for sharing your insights, and I look forward to working on this
> enhancement with the community's support.
>
> Best regards,
> Xiangying
>
> On Mon, Apr 17, 2023 at 3:24 PM PengHui Li <pe...@apache.org> wrote:
>
> > The new operation will delete all the data and the metadata under
> > a tenant or namespace. I would like to suggest to use
> >
> > `delete-all-namespaces` and `delete-all-topics`
> >
> > The `delete` actually acts as a fact of deleting metadata and data.
> > And `truncate` is for deleting the data. IMO, we'd better not
> > introduce another new keyword, either `clear` or `wipe`, because
> > it will bring more knowledge to Pulsar users who must understand.
> >
> >
> > Thanks,
> > Penghui
> >
> > On Mon, Apr 17, 2023 at 10:56 AM Xiangying Meng <xi...@apache.org>
> > wrote:
> >
> > > Hi Enrico,
> > >
> > > Thank you for your feedback. While I understand that
> > > "delete-all-namespaces" is more explicit,
> > > I also think it's a bit lengthy for a command-line parameter.
> > > I personally believe the "wipe" option, combined with a safety
> > confirmation
> > > step,
> > >  would be more user-friendly and efficient.
> > >
> > > By adding a safety confirmation step, we can minimize the risk of
> > > accidental mass deletion.
> > > Users would be required to confirm their intention to perform the
> > deletion
> > > by
> > > typing 'YES' or a similar confirmation word before the operation
> > proceeds.
> > >
> > > What do you think about this approach?
> > > If there's a consensus, I can work on implementing this feature with the
> > > "wipe" option and the safety confirmation step.
> > >
> > > Best regards,
> > > Xiangying
> > >
> > > On Sun, Apr 16, 2023 at 11:25 PM Enrico Olivelli <eo...@gmail.com>
> > > wrote:
> > >
> > > > Il Dom 16 Apr 2023, 15:45 Asaf Mesika <as...@gmail.com> ha
> > > scritto:
> > > >
> > > > > How about "truncate" instead of "clear"?
> > > > >
> > > >
> > > >
> > > > Truncate is better, or maybe 'wipe' (because truncate means another
> > > > operation for topics currently)
> > > >
> > > > Another alternative, more explicit:
> > > > pulsar-admin tenants delete-all-namespaces TENANT
> > > >
> > > > Enrico
> > > >
> > > > >
> > > > > Just wondering - since it is such a dangerous command, how can we
> > help
> > > > the
> > > > > user not make an accidental mass deletion?
> > > > >
> > > > > On Sat, Apr 15, 2023 at 1:12 PM Girish Sharma <
> > scrapmachines@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > > However, the current goal is to keep the tenant and namespace
> > > intact
> > > > > > while
> > > > > > > cleaning up their contents.
> > > > > > Ah, I see now. Yes, in that case a clear command is better. Will
> > this
> > > > > > command also take into account the value of the broker config
> > > > > > `forceDeleteNamespaceAllowed` in case someone is clearing the owner
> > > > > tenant?
> > > > > >
> > > > > > Regards
> > > > > >
> > > > > > On Sat, Apr 15, 2023 at 3:39 PM Enrico Olivelli <
> > eolivelli@gmail.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > The proposal sounds really useful, especially for automated
> > > testing.
> > > > > > > +1
> > > > > > >
> > > > > > > Enrico
> > > > > > >
> > > > > > > Il giorno sab 15 apr 2023 alle ore 12:07 Xiangying Meng
> > > > > > > <xi...@apache.org> ha scritto:
> > > > > > > >
> > > > > > > > Dear Girish,
> > > > > > > >
> > > > > > > > Thank you for your response and suggestion to extend the use of
> > > the
> > > > > > > > `boolean force` flag for namespaces and tenants.
> > > > > > > > I understand that the `force` flag is already implemented for
> > > > > deleting
> > > > > > > > topics, namespaces, and tenants,
> > > > > > > > and it provides a consistent way to perform these actions.
> > > > > > > >
> > > > > > > > However, the current goal is to keep the tenant and namespace
> > > > intact
> > > > > > > while
> > > > > > > > cleaning up their contents.
> > > > > > > > In other words, I want to have a way to remove all topics
> > within
> > > a
> > > > > > > > namespace or all namespaces and topics
> > > > > > > > within a tenant without actually deleting the namespace or
> > tenant
> > > > > > itself.
> > > > > > > >
> > > > > > > > To achieve this goal, I proposed adding a `clear` command for
> > > > > > > `namespaces`
> > > > > > > > and `tenants`.
> > > > > > > >
> > > > > > > > This approach would allow users to keep the tenant and
> > namespace
> > > > > > > structures
> > > > > > > > in place
> > > > > > > > while cleaning up their contents.
> > > > > > > > I hope this clarifies my intention, and I would like to hear
> > your
> > > > > > > thoughts
> > > > > > > > on this proposal.
> > > > > > > >
> > > > > > > > Best regards,
> > > > > > > > Xiangying
> > > > > > > >
> > > > > > > > On Sat, Apr 15, 2023 at 5:49 PM Girish Sharma <
> > > > > scrapmachines@gmail.com
> > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hello Xiangying,
> > > > > > > > > This indeed is a cumbersome task to delete a filled namespace
> > > or
> > > > > > > tenant. We
> > > > > > > > > face this challenge in our organization where we use the
> > > > > > multi-tenancy
> > > > > > > > > feature of pulsar heavily.
> > > > > > > > >
> > > > > > > > > I would like to suggest a different command to do this
> > though..
> > > > > > > Similar to
> > > > > > > > > how you cannot delete a topic without deleting its
> > > > > > > > > subscribers/producers/consumers, unless we use the `boolean
> > > > force`
> > > > > > > flag.
> > > > > > > > > Why not extend this to namespace and tenant as well and let
> > the
> > > > > force
> > > > > > > param
> > > > > > > > > do the cleanup (which your suggested `clear` command would
> > do).
> > > > > > > > >
> > > > > > > > > As of today, using force to delete a namespace just returns
> > 405
> > > > > > saying
> > > > > > > > > broker doesn't allow force delete of namespace containing
> > > topics.
> > > > > > > > >
> > > > > > > > > Any thoughts?
> > > > > > > > >
> > > > > > > > > On Sat, Apr 15, 2023 at 3:07 PM Xiangying Meng <
> > > > > xiangying@apache.org
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Dear Apache Pulsar Community,
> > > > > > > > > >
> > > > > > > > > > I hope this email finds you well.I am writing to suggest a
> > > > > > potential
> > > > > > > > > > improvement to the Pulsar-admin tool,
> > > > > > > > > >  which I believe could simplify the process of cleaning up
> > > > > tenants
> > > > > > > and
> > > > > > > > > > namespaces in Apache Pulsar.
> > > > > > > > > >
> > > > > > > > > > Currently, cleaning up all the namespaces and topics
> > within a
> > > > > > tenant
> > > > > > > or
> > > > > > > > > > cleaning up all the topics within a namespace requires
> > > several
> > > > > > manual
> > > > > > > > > > steps,
> > > > > > > > > > such as listing the namespaces, listing the topics, and
> > then
> > > > > > deleting
> > > > > > > > > each
> > > > > > > > > > topic individually.
> > > > > > > > > > This process can be time-consuming and error-prone for
> > users.
> > > > > > > > > >
> > > > > > > > > > To address this issue, I propose the addition of a "clear"
> > > > > > parameter
> > > > > > > to
> > > > > > > > > the
> > > > > > > > > > Pulsar-admin tool,
> > > > > > > > > > which would automate the cleanup process for tenants and
> > > > > > namespaces.
> > > > > > > > > Here's
> > > > > > > > > > a conceptual implementation:
> > > > > > > > > >
> > > > > > > > > > 1. To clean up all namespaces and topics within a tenant:
> > > > > > > > > > ``` bash
> > > > > > > > > > pulsar-admin tenants clear <tenant-name>
> > > > > > > > > > ```
> > > > > > > > > > 2. To clean up all topics within a namespace:
> > > > > > > > > > ```bash
> > > > > > > > > > pulsar-admin namespaces clear
> > <tenant-name>/<namespace-name>
> > > > > > > > > > ```
> > > > > > > > > >
> > > > > > > > > > By implementing these new parameters, users would be able
> > to
> > > > > > perform
> > > > > > > > > > cleanup operations more efficiently and with fewer manual
> > > > steps.
> > > > > > > > > > I believe this improvement would greatly enhance the user
> > > > > > experience
> > > > > > > when
> > > > > > > > > > working with Apache Pulsar.
> > > > > > > > > >
> > > > > > > > > > I'd like to discuss the feasibility of this suggestion and
> > > > gather
> > > > > > > > > feedback
> > > > > > > > > > from the community.
> > > > > > > > > > If everyone agrees, I can work on implementing this feature
> > > and
> > > > > > > submit a
> > > > > > > > > > pull request for review.
> > > > > > > > > >
> > > > > > > > > > Looking forward to hearing your thoughts on this.
> > > > > > > > > >
> > > > > > > > > > Best regards,
> > > > > > > > > > Xiangying
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Girish Sharma
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Girish Sharma
> > > > > >
> > > > >
> > > >
> > >
> >

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Xiangying Meng <xi...@apache.org>]

Hi Penghui,

I appreciate your feedback and completely agree with your concern about the
learning curve for Pulsar users. Introducing additional keywords could
potentially increase the complexity for users who need to understand the
new terms. Therefore, I accept your suggestion to use delete-all-namespaces
and delete-all-topics for the proposed improvement.

Thank you for sharing your insights, and I look forward to working on this
enhancement with the community's support.

Best regards,
Xiangying

On Mon, Apr 17, 2023 at 3:24 PM PengHui Li <pe...@apache.org> wrote:

> The new operation will delete all the data and the metadata under
> a tenant or namespace. I would like to suggest to use
>
> `delete-all-namespaces` and `delete-all-topics`
>
> The `delete` actually acts as a fact of deleting metadata and data.
> And `truncate` is for deleting the data. IMO, we'd better not
> introduce another new keyword, either `clear` or `wipe`, because
> it will bring more knowledge to Pulsar users who must understand.
>
>
> Thanks,
> Penghui
>
> On Mon, Apr 17, 2023 at 10:56 AM Xiangying Meng <xi...@apache.org>
> wrote:
>
> > Hi Enrico,
> >
> > Thank you for your feedback. While I understand that
> > "delete-all-namespaces" is more explicit,
> > I also think it's a bit lengthy for a command-line parameter.
> > I personally believe the "wipe" option, combined with a safety
> confirmation
> > step,
> >  would be more user-friendly and efficient.
> >
> > By adding a safety confirmation step, we can minimize the risk of
> > accidental mass deletion.
> > Users would be required to confirm their intention to perform the
> deletion
> > by
> > typing 'YES' or a similar confirmation word before the operation
> proceeds.
> >
> > What do you think about this approach?
> > If there's a consensus, I can work on implementing this feature with the
> > "wipe" option and the safety confirmation step.
> >
> > Best regards,
> > Xiangying
> >
> > On Sun, Apr 16, 2023 at 11:25 PM Enrico Olivelli <eo...@gmail.com>
> > wrote:
> >
> > > Il Dom 16 Apr 2023, 15:45 Asaf Mesika <as...@gmail.com> ha
> > scritto:
> > >
> > > > How about "truncate" instead of "clear"?
> > > >
> > >
> > >
> > > Truncate is better, or maybe 'wipe' (because truncate means another
> > > operation for topics currently)
> > >
> > > Another alternative, more explicit:
> > > pulsar-admin tenants delete-all-namespaces TENANT
> > >
> > > Enrico
> > >
> > > >
> > > > Just wondering - since it is such a dangerous command, how can we
> help
> > > the
> > > > user not make an accidental mass deletion?
> > > >
> > > > On Sat, Apr 15, 2023 at 1:12 PM Girish Sharma <
> scrapmachines@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > > However, the current goal is to keep the tenant and namespace
> > intact
> > > > > while
> > > > > > cleaning up their contents.
> > > > > Ah, I see now. Yes, in that case a clear command is better. Will
> this
> > > > > command also take into account the value of the broker config
> > > > > `forceDeleteNamespaceAllowed` in case someone is clearing the owner
> > > > tenant?
> > > > >
> > > > > Regards
> > > > >
> > > > > On Sat, Apr 15, 2023 at 3:39 PM Enrico Olivelli <
> eolivelli@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > The proposal sounds really useful, especially for automated
> > testing.
> > > > > > +1
> > > > > >
> > > > > > Enrico
> > > > > >
> > > > > > Il giorno sab 15 apr 2023 alle ore 12:07 Xiangying Meng
> > > > > > <xi...@apache.org> ha scritto:
> > > > > > >
> > > > > > > Dear Girish,
> > > > > > >
> > > > > > > Thank you for your response and suggestion to extend the use of
> > the
> > > > > > > `boolean force` flag for namespaces and tenants.
> > > > > > > I understand that the `force` flag is already implemented for
> > > > deleting
> > > > > > > topics, namespaces, and tenants,
> > > > > > > and it provides a consistent way to perform these actions.
> > > > > > >
> > > > > > > However, the current goal is to keep the tenant and namespace
> > > intact
> > > > > > while
> > > > > > > cleaning up their contents.
> > > > > > > In other words, I want to have a way to remove all topics
> within
> > a
> > > > > > > namespace or all namespaces and topics
> > > > > > > within a tenant without actually deleting the namespace or
> tenant
> > > > > itself.
> > > > > > >
> > > > > > > To achieve this goal, I proposed adding a `clear` command for
> > > > > > `namespaces`
> > > > > > > and `tenants`.
> > > > > > >
> > > > > > > This approach would allow users to keep the tenant and
> namespace
> > > > > > structures
> > > > > > > in place
> > > > > > > while cleaning up their contents.
> > > > > > > I hope this clarifies my intention, and I would like to hear
> your
> > > > > > thoughts
> > > > > > > on this proposal.
> > > > > > >
> > > > > > > Best regards,
> > > > > > > Xiangying
> > > > > > >
> > > > > > > On Sat, Apr 15, 2023 at 5:49 PM Girish Sharma <
> > > > scrapmachines@gmail.com
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hello Xiangying,
> > > > > > > > This indeed is a cumbersome task to delete a filled namespace
> > or
> > > > > > tenant. We
> > > > > > > > face this challenge in our organization where we use the
> > > > > multi-tenancy
> > > > > > > > feature of pulsar heavily.
> > > > > > > >
> > > > > > > > I would like to suggest a different command to do this
> though..
> > > > > > Similar to
> > > > > > > > how you cannot delete a topic without deleting its
> > > > > > > > subscribers/producers/consumers, unless we use the `boolean
> > > force`
> > > > > > flag.
> > > > > > > > Why not extend this to namespace and tenant as well and let
> the
> > > > force
> > > > > > param
> > > > > > > > do the cleanup (which your suggested `clear` command would
> do).
> > > > > > > >
> > > > > > > > As of today, using force to delete a namespace just returns
> 405
> > > > > saying
> > > > > > > > broker doesn't allow force delete of namespace containing
> > topics.
> > > > > > > >
> > > > > > > > Any thoughts?
> > > > > > > >
> > > > > > > > On Sat, Apr 15, 2023 at 3:07 PM Xiangying Meng <
> > > > xiangying@apache.org
> > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Dear Apache Pulsar Community,
> > > > > > > > >
> > > > > > > > > I hope this email finds you well.I am writing to suggest a
> > > > > potential
> > > > > > > > > improvement to the Pulsar-admin tool,
> > > > > > > > >  which I believe could simplify the process of cleaning up
> > > > tenants
> > > > > > and
> > > > > > > > > namespaces in Apache Pulsar.
> > > > > > > > >
> > > > > > > > > Currently, cleaning up all the namespaces and topics
> within a
> > > > > tenant
> > > > > > or
> > > > > > > > > cleaning up all the topics within a namespace requires
> > several
> > > > > manual
> > > > > > > > > steps,
> > > > > > > > > such as listing the namespaces, listing the topics, and
> then
> > > > > deleting
> > > > > > > > each
> > > > > > > > > topic individually.
> > > > > > > > > This process can be time-consuming and error-prone for
> users.
> > > > > > > > >
> > > > > > > > > To address this issue, I propose the addition of a "clear"
> > > > > parameter
> > > > > > to
> > > > > > > > the
> > > > > > > > > Pulsar-admin tool,
> > > > > > > > > which would automate the cleanup process for tenants and
> > > > > namespaces.
> > > > > > > > Here's
> > > > > > > > > a conceptual implementation:
> > > > > > > > >
> > > > > > > > > 1. To clean up all namespaces and topics within a tenant:
> > > > > > > > > ``` bash
> > > > > > > > > pulsar-admin tenants clear <tenant-name>
> > > > > > > > > ```
> > > > > > > > > 2. To clean up all topics within a namespace:
> > > > > > > > > ```bash
> > > > > > > > > pulsar-admin namespaces clear
> <tenant-name>/<namespace-name>
> > > > > > > > > ```
> > > > > > > > >
> > > > > > > > > By implementing these new parameters, users would be able
> to
> > > > > perform
> > > > > > > > > cleanup operations more efficiently and with fewer manual
> > > steps.
> > > > > > > > > I believe this improvement would greatly enhance the user
> > > > > experience
> > > > > > when
> > > > > > > > > working with Apache Pulsar.
> > > > > > > > >
> > > > > > > > > I'd like to discuss the feasibility of this suggestion and
> > > gather
> > > > > > > > feedback
> > > > > > > > > from the community.
> > > > > > > > > If everyone agrees, I can work on implementing this feature
> > and
> > > > > > submit a
> > > > > > > > > pull request for review.
> > > > > > > > >
> > > > > > > > > Looking forward to hearing your thoughts on this.
> > > > > > > > >
> > > > > > > > > Best regards,
> > > > > > > > > Xiangying
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Girish Sharma
> > > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Girish Sharma
> > > > >
> > > >
> > >
> >
>

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[PengHui Li <pe...@apache.org>]

The new operation will delete all the data and the metadata under
a tenant or namespace. I would like to suggest to use

`delete-all-namespaces` and `delete-all-topics`

The `delete` actually acts as a fact of deleting metadata and data.
And `truncate` is for deleting the data. IMO, we'd better not
introduce another new keyword, either `clear` or `wipe`, because
it will bring more knowledge to Pulsar users who must understand.


Thanks,
Penghui

On Mon, Apr 17, 2023 at 10:56 AM Xiangying Meng <xi...@apache.org>
wrote:

> Hi Enrico,
>
> Thank you for your feedback. While I understand that
> "delete-all-namespaces" is more explicit,
> I also think it's a bit lengthy for a command-line parameter.
> I personally believe the "wipe" option, combined with a safety confirmation
> step,
>  would be more user-friendly and efficient.
>
> By adding a safety confirmation step, we can minimize the risk of
> accidental mass deletion.
> Users would be required to confirm their intention to perform the deletion
> by
> typing 'YES' or a similar confirmation word before the operation proceeds.
>
> What do you think about this approach?
> If there's a consensus, I can work on implementing this feature with the
> "wipe" option and the safety confirmation step.
>
> Best regards,
> Xiangying
>
> On Sun, Apr 16, 2023 at 11:25 PM Enrico Olivelli <eo...@gmail.com>
> wrote:
>
> > Il Dom 16 Apr 2023, 15:45 Asaf Mesika <as...@gmail.com> ha
> scritto:
> >
> > > How about "truncate" instead of "clear"?
> > >
> >
> >
> > Truncate is better, or maybe 'wipe' (because truncate means another
> > operation for topics currently)
> >
> > Another alternative, more explicit:
> > pulsar-admin tenants delete-all-namespaces TENANT
> >
> > Enrico
> >
> > >
> > > Just wondering - since it is such a dangerous command, how can we help
> > the
> > > user not make an accidental mass deletion?
> > >
> > > On Sat, Apr 15, 2023 at 1:12 PM Girish Sharma <scrapmachines@gmail.com
> >
> > > wrote:
> > >
> > > > > However, the current goal is to keep the tenant and namespace
> intact
> > > > while
> > > > > cleaning up their contents.
> > > > Ah, I see now. Yes, in that case a clear command is better. Will this
> > > > command also take into account the value of the broker config
> > > > `forceDeleteNamespaceAllowed` in case someone is clearing the owner
> > > tenant?
> > > >
> > > > Regards
> > > >
> > > > On Sat, Apr 15, 2023 at 3:39 PM Enrico Olivelli <eolivelli@gmail.com
> >
> > > > wrote:
> > > >
> > > > > The proposal sounds really useful, especially for automated
> testing.
> > > > > +1
> > > > >
> > > > > Enrico
> > > > >
> > > > > Il giorno sab 15 apr 2023 alle ore 12:07 Xiangying Meng
> > > > > <xi...@apache.org> ha scritto:
> > > > > >
> > > > > > Dear Girish,
> > > > > >
> > > > > > Thank you for your response and suggestion to extend the use of
> the
> > > > > > `boolean force` flag for namespaces and tenants.
> > > > > > I understand that the `force` flag is already implemented for
> > > deleting
> > > > > > topics, namespaces, and tenants,
> > > > > > and it provides a consistent way to perform these actions.
> > > > > >
> > > > > > However, the current goal is to keep the tenant and namespace
> > intact
> > > > > while
> > > > > > cleaning up their contents.
> > > > > > In other words, I want to have a way to remove all topics within
> a
> > > > > > namespace or all namespaces and topics
> > > > > > within a tenant without actually deleting the namespace or tenant
> > > > itself.
> > > > > >
> > > > > > To achieve this goal, I proposed adding a `clear` command for
> > > > > `namespaces`
> > > > > > and `tenants`.
> > > > > >
> > > > > > This approach would allow users to keep the tenant and namespace
> > > > > structures
> > > > > > in place
> > > > > > while cleaning up their contents.
> > > > > > I hope this clarifies my intention, and I would like to hear your
> > > > > thoughts
> > > > > > on this proposal.
> > > > > >
> > > > > > Best regards,
> > > > > > Xiangying
> > > > > >
> > > > > > On Sat, Apr 15, 2023 at 5:49 PM Girish Sharma <
> > > scrapmachines@gmail.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hello Xiangying,
> > > > > > > This indeed is a cumbersome task to delete a filled namespace
> or
> > > > > tenant. We
> > > > > > > face this challenge in our organization where we use the
> > > > multi-tenancy
> > > > > > > feature of pulsar heavily.
> > > > > > >
> > > > > > > I would like to suggest a different command to do this though..
> > > > > Similar to
> > > > > > > how you cannot delete a topic without deleting its
> > > > > > > subscribers/producers/consumers, unless we use the `boolean
> > force`
> > > > > flag.
> > > > > > > Why not extend this to namespace and tenant as well and let the
> > > force
> > > > > param
> > > > > > > do the cleanup (which your suggested `clear` command would do).
> > > > > > >
> > > > > > > As of today, using force to delete a namespace just returns 405
> > > > saying
> > > > > > > broker doesn't allow force delete of namespace containing
> topics.
> > > > > > >
> > > > > > > Any thoughts?
> > > > > > >
> > > > > > > On Sat, Apr 15, 2023 at 3:07 PM Xiangying Meng <
> > > xiangying@apache.org
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Dear Apache Pulsar Community,
> > > > > > > >
> > > > > > > > I hope this email finds you well.I am writing to suggest a
> > > > potential
> > > > > > > > improvement to the Pulsar-admin tool,
> > > > > > > >  which I believe could simplify the process of cleaning up
> > > tenants
> > > > > and
> > > > > > > > namespaces in Apache Pulsar.
> > > > > > > >
> > > > > > > > Currently, cleaning up all the namespaces and topics within a
> > > > tenant
> > > > > or
> > > > > > > > cleaning up all the topics within a namespace requires
> several
> > > > manual
> > > > > > > > steps,
> > > > > > > > such as listing the namespaces, listing the topics, and then
> > > > deleting
> > > > > > > each
> > > > > > > > topic individually.
> > > > > > > > This process can be time-consuming and error-prone for users.
> > > > > > > >
> > > > > > > > To address this issue, I propose the addition of a "clear"
> > > > parameter
> > > > > to
> > > > > > > the
> > > > > > > > Pulsar-admin tool,
> > > > > > > > which would automate the cleanup process for tenants and
> > > > namespaces.
> > > > > > > Here's
> > > > > > > > a conceptual implementation:
> > > > > > > >
> > > > > > > > 1. To clean up all namespaces and topics within a tenant:
> > > > > > > > ``` bash
> > > > > > > > pulsar-admin tenants clear <tenant-name>
> > > > > > > > ```
> > > > > > > > 2. To clean up all topics within a namespace:
> > > > > > > > ```bash
> > > > > > > > pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> > > > > > > > ```
> > > > > > > >
> > > > > > > > By implementing these new parameters, users would be able to
> > > > perform
> > > > > > > > cleanup operations more efficiently and with fewer manual
> > steps.
> > > > > > > > I believe this improvement would greatly enhance the user
> > > > experience
> > > > > when
> > > > > > > > working with Apache Pulsar.
> > > > > > > >
> > > > > > > > I'd like to discuss the feasibility of this suggestion and
> > gather
> > > > > > > feedback
> > > > > > > > from the community.
> > > > > > > > If everyone agrees, I can work on implementing this feature
> and
> > > > > submit a
> > > > > > > > pull request for review.
> > > > > > > >
> > > > > > > > Looking forward to hearing your thoughts on this.
> > > > > > > >
> > > > > > > > Best regards,
> > > > > > > > Xiangying
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Girish Sharma
> > > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Girish Sharma
> > > >
> > >
> >
>

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Xiangying Meng <xi...@apache.org>]

Hi Enrico,

Thank you for your feedback. While I understand that
"delete-all-namespaces" is more explicit,
I also think it's a bit lengthy for a command-line parameter.
I personally believe the "wipe" option, combined with a safety confirmation
step,
 would be more user-friendly and efficient.

By adding a safety confirmation step, we can minimize the risk of
accidental mass deletion.
Users would be required to confirm their intention to perform the deletion
by
typing 'YES' or a similar confirmation word before the operation proceeds.

What do you think about this approach?
If there's a consensus, I can work on implementing this feature with the
"wipe" option and the safety confirmation step.

Best regards,
Xiangying

On Sun, Apr 16, 2023 at 11:25 PM Enrico Olivelli <eo...@gmail.com>
wrote:

> Il Dom 16 Apr 2023, 15:45 Asaf Mesika <as...@gmail.com> ha scritto:
>
> > How about "truncate" instead of "clear"?
> >
>
>
> Truncate is better, or maybe 'wipe' (because truncate means another
> operation for topics currently)
>
> Another alternative, more explicit:
> pulsar-admin tenants delete-all-namespaces TENANT
>
> Enrico
>
> >
> > Just wondering - since it is such a dangerous command, how can we help
> the
> > user not make an accidental mass deletion?
> >
> > On Sat, Apr 15, 2023 at 1:12 PM Girish Sharma <sc...@gmail.com>
> > wrote:
> >
> > > > However, the current goal is to keep the tenant and namespace intact
> > > while
> > > > cleaning up their contents.
> > > Ah, I see now. Yes, in that case a clear command is better. Will this
> > > command also take into account the value of the broker config
> > > `forceDeleteNamespaceAllowed` in case someone is clearing the owner
> > tenant?
> > >
> > > Regards
> > >
> > > On Sat, Apr 15, 2023 at 3:39 PM Enrico Olivelli <eo...@gmail.com>
> > > wrote:
> > >
> > > > The proposal sounds really useful, especially for automated testing.
> > > > +1
> > > >
> > > > Enrico
> > > >
> > > > Il giorno sab 15 apr 2023 alle ore 12:07 Xiangying Meng
> > > > <xi...@apache.org> ha scritto:
> > > > >
> > > > > Dear Girish,
> > > > >
> > > > > Thank you for your response and suggestion to extend the use of the
> > > > > `boolean force` flag for namespaces and tenants.
> > > > > I understand that the `force` flag is already implemented for
> > deleting
> > > > > topics, namespaces, and tenants,
> > > > > and it provides a consistent way to perform these actions.
> > > > >
> > > > > However, the current goal is to keep the tenant and namespace
> intact
> > > > while
> > > > > cleaning up their contents.
> > > > > In other words, I want to have a way to remove all topics within a
> > > > > namespace or all namespaces and topics
> > > > > within a tenant without actually deleting the namespace or tenant
> > > itself.
> > > > >
> > > > > To achieve this goal, I proposed adding a `clear` command for
> > > > `namespaces`
> > > > > and `tenants`.
> > > > >
> > > > > This approach would allow users to keep the tenant and namespace
> > > > structures
> > > > > in place
> > > > > while cleaning up their contents.
> > > > > I hope this clarifies my intention, and I would like to hear your
> > > > thoughts
> > > > > on this proposal.
> > > > >
> > > > > Best regards,
> > > > > Xiangying
> > > > >
> > > > > On Sat, Apr 15, 2023 at 5:49 PM Girish Sharma <
> > scrapmachines@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hello Xiangying,
> > > > > > This indeed is a cumbersome task to delete a filled namespace or
> > > > tenant. We
> > > > > > face this challenge in our organization where we use the
> > > multi-tenancy
> > > > > > feature of pulsar heavily.
> > > > > >
> > > > > > I would like to suggest a different command to do this though..
> > > > Similar to
> > > > > > how you cannot delete a topic without deleting its
> > > > > > subscribers/producers/consumers, unless we use the `boolean
> force`
> > > > flag.
> > > > > > Why not extend this to namespace and tenant as well and let the
> > force
> > > > param
> > > > > > do the cleanup (which your suggested `clear` command would do).
> > > > > >
> > > > > > As of today, using force to delete a namespace just returns 405
> > > saying
> > > > > > broker doesn't allow force delete of namespace containing topics.
> > > > > >
> > > > > > Any thoughts?
> > > > > >
> > > > > > On Sat, Apr 15, 2023 at 3:07 PM Xiangying Meng <
> > xiangying@apache.org
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Dear Apache Pulsar Community,
> > > > > > >
> > > > > > > I hope this email finds you well.I am writing to suggest a
> > > potential
> > > > > > > improvement to the Pulsar-admin tool,
> > > > > > >  which I believe could simplify the process of cleaning up
> > tenants
> > > > and
> > > > > > > namespaces in Apache Pulsar.
> > > > > > >
> > > > > > > Currently, cleaning up all the namespaces and topics within a
> > > tenant
> > > > or
> > > > > > > cleaning up all the topics within a namespace requires several
> > > manual
> > > > > > > steps,
> > > > > > > such as listing the namespaces, listing the topics, and then
> > > deleting
> > > > > > each
> > > > > > > topic individually.
> > > > > > > This process can be time-consuming and error-prone for users.
> > > > > > >
> > > > > > > To address this issue, I propose the addition of a "clear"
> > > parameter
> > > > to
> > > > > > the
> > > > > > > Pulsar-admin tool,
> > > > > > > which would automate the cleanup process for tenants and
> > > namespaces.
> > > > > > Here's
> > > > > > > a conceptual implementation:
> > > > > > >
> > > > > > > 1. To clean up all namespaces and topics within a tenant:
> > > > > > > ``` bash
> > > > > > > pulsar-admin tenants clear <tenant-name>
> > > > > > > ```
> > > > > > > 2. To clean up all topics within a namespace:
> > > > > > > ```bash
> > > > > > > pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> > > > > > > ```
> > > > > > >
> > > > > > > By implementing these new parameters, users would be able to
> > > perform
> > > > > > > cleanup operations more efficiently and with fewer manual
> steps.
> > > > > > > I believe this improvement would greatly enhance the user
> > > experience
> > > > when
> > > > > > > working with Apache Pulsar.
> > > > > > >
> > > > > > > I'd like to discuss the feasibility of this suggestion and
> gather
> > > > > > feedback
> > > > > > > from the community.
> > > > > > > If everyone agrees, I can work on implementing this feature and
> > > > submit a
> > > > > > > pull request for review.
> > > > > > >
> > > > > > > Looking forward to hearing your thoughts on this.
> > > > > > >
> > > > > > > Best regards,
> > > > > > > Xiangying
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Girish Sharma
> > > > > >
> > > >
> > >
> > >
> > > --
> > > Girish Sharma
> > >
> >
>

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Enrico Olivelli <eo...@gmail.com>]

Il Dom 16 Apr 2023, 15:45 Asaf Mesika <as...@gmail.com> ha scritto:

> How about "truncate" instead of "clear"?
>


Truncate is better, or maybe 'wipe' (because truncate means another
operation for topics currently)

Another alternative, more explicit:
pulsar-admin tenants delete-all-namespaces TENANT

Enrico

>
> Just wondering - since it is such a dangerous command, how can we help the
> user not make an accidental mass deletion?
>
> On Sat, Apr 15, 2023 at 1:12 PM Girish Sharma <sc...@gmail.com>
> wrote:
>
> > > However, the current goal is to keep the tenant and namespace intact
> > while
> > > cleaning up their contents.
> > Ah, I see now. Yes, in that case a clear command is better. Will this
> > command also take into account the value of the broker config
> > `forceDeleteNamespaceAllowed` in case someone is clearing the owner
> tenant?
> >
> > Regards
> >
> > On Sat, Apr 15, 2023 at 3:39 PM Enrico Olivelli <eo...@gmail.com>
> > wrote:
> >
> > > The proposal sounds really useful, especially for automated testing.
> > > +1
> > >
> > > Enrico
> > >
> > > Il giorno sab 15 apr 2023 alle ore 12:07 Xiangying Meng
> > > <xi...@apache.org> ha scritto:
> > > >
> > > > Dear Girish,
> > > >
> > > > Thank you for your response and suggestion to extend the use of the
> > > > `boolean force` flag for namespaces and tenants.
> > > > I understand that the `force` flag is already implemented for
> deleting
> > > > topics, namespaces, and tenants,
> > > > and it provides a consistent way to perform these actions.
> > > >
> > > > However, the current goal is to keep the tenant and namespace intact
> > > while
> > > > cleaning up their contents.
> > > > In other words, I want to have a way to remove all topics within a
> > > > namespace or all namespaces and topics
> > > > within a tenant without actually deleting the namespace or tenant
> > itself.
> > > >
> > > > To achieve this goal, I proposed adding a `clear` command for
> > > `namespaces`
> > > > and `tenants`.
> > > >
> > > > This approach would allow users to keep the tenant and namespace
> > > structures
> > > > in place
> > > > while cleaning up their contents.
> > > > I hope this clarifies my intention, and I would like to hear your
> > > thoughts
> > > > on this proposal.
> > > >
> > > > Best regards,
> > > > Xiangying
> > > >
> > > > On Sat, Apr 15, 2023 at 5:49 PM Girish Sharma <
> scrapmachines@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Hello Xiangying,
> > > > > This indeed is a cumbersome task to delete a filled namespace or
> > > tenant. We
> > > > > face this challenge in our organization where we use the
> > multi-tenancy
> > > > > feature of pulsar heavily.
> > > > >
> > > > > I would like to suggest a different command to do this though..
> > > Similar to
> > > > > how you cannot delete a topic without deleting its
> > > > > subscribers/producers/consumers, unless we use the `boolean force`
> > > flag.
> > > > > Why not extend this to namespace and tenant as well and let the
> force
> > > param
> > > > > do the cleanup (which your suggested `clear` command would do).
> > > > >
> > > > > As of today, using force to delete a namespace just returns 405
> > saying
> > > > > broker doesn't allow force delete of namespace containing topics.
> > > > >
> > > > > Any thoughts?
> > > > >
> > > > > On Sat, Apr 15, 2023 at 3:07 PM Xiangying Meng <
> xiangying@apache.org
> > >
> > > > > wrote:
> > > > >
> > > > > > Dear Apache Pulsar Community,
> > > > > >
> > > > > > I hope this email finds you well.I am writing to suggest a
> > potential
> > > > > > improvement to the Pulsar-admin tool,
> > > > > >  which I believe could simplify the process of cleaning up
> tenants
> > > and
> > > > > > namespaces in Apache Pulsar.
> > > > > >
> > > > > > Currently, cleaning up all the namespaces and topics within a
> > tenant
> > > or
> > > > > > cleaning up all the topics within a namespace requires several
> > manual
> > > > > > steps,
> > > > > > such as listing the namespaces, listing the topics, and then
> > deleting
> > > > > each
> > > > > > topic individually.
> > > > > > This process can be time-consuming and error-prone for users.
> > > > > >
> > > > > > To address this issue, I propose the addition of a "clear"
> > parameter
> > > to
> > > > > the
> > > > > > Pulsar-admin tool,
> > > > > > which would automate the cleanup process for tenants and
> > namespaces.
> > > > > Here's
> > > > > > a conceptual implementation:
> > > > > >
> > > > > > 1. To clean up all namespaces and topics within a tenant:
> > > > > > ``` bash
> > > > > > pulsar-admin tenants clear <tenant-name>
> > > > > > ```
> > > > > > 2. To clean up all topics within a namespace:
> > > > > > ```bash
> > > > > > pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> > > > > > ```
> > > > > >
> > > > > > By implementing these new parameters, users would be able to
> > perform
> > > > > > cleanup operations more efficiently and with fewer manual steps.
> > > > > > I believe this improvement would greatly enhance the user
> > experience
> > > when
> > > > > > working with Apache Pulsar.
> > > > > >
> > > > > > I'd like to discuss the feasibility of this suggestion and gather
> > > > > feedback
> > > > > > from the community.
> > > > > > If everyone agrees, I can work on implementing this feature and
> > > submit a
> > > > > > pull request for review.
> > > > > >
> > > > > > Looking forward to hearing your thoughts on this.
> > > > > >
> > > > > > Best regards,
> > > > > > Xiangying
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Girish Sharma
> > > > >
> > >
> >
> >
> > --
> > Girish Sharma
> >
>

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Xiangying Meng <xi...@apache.org>]

Hi Zike,

I agree that using regex to delete topics is a good approach, and I also
support the idea of letting users confirm the topics they're about to
delete. However, there's a concern that this might introduce a breaking
change, as users may have already used `pulsar-admin topics delete` in
their scripts.

To address this concern, we can introduce a new option flag (e.g.,
`--regex` or `--pattern`) to the existing `pulsar-admin topics delete`
command, which will be specifically used for regex-based deletion. This
way, the default behavior of the command remains the same and won't affect
existing users' scripts, while the new functionality is introduced for
those who want to use regex for deletion.

An example of how the updated command could look like:

```
pulsar-admin topics delete --regex "public/default/test-topic-\d+"
```

Please let me know what you think about this solution.

Best regards,
Xiangying

On Wed, Apr 19, 2023 at 4:36 PM Zike Yang <zi...@apache.org> wrote:

> Is it possible to use regex to delete namespaces/topics here?
> For instance, we can use the following command to delete all topics in
> the default namespace:
> ```
> pulsar-admin topics delete public/default/*
> ```
> We can also use it to delete some specific topics:
> ```
> pulsar-admin topics delete public/default/*topic
> ```
> This way, we don't need to introduce a new command
> `delete-all-topics`. It also makes it easier for users to delete
> specific topic lists.
>
> > Part-1: Give a summary print of the namespaces, and topics to delete,
> >  and ask the user to confirm if they want to delete the resources.
> > Part-2: If users select “yes”, the deletion will be really executed.
> > Part-3: Print a summary of the results
>
> +1 for this. This will minimize the risk of accidentally deleting
> important resources.
>
> BR,
> Zike Yang
>
> On Wed, Apr 19, 2023 at 3:34 PM Yubiao Feng
> <yu...@streamnative.io.invalid> wrote:
> >
> > > Just wondering - since it is such a dangerous
> > > command, how can we help the user not make
> > > an accidental mass deletion?
> >
> > Just a suggestion: We can make this command executed on two-part
> >
> > Part-1: Give a summary print of the namespaces, and topics to delete,
> >  and ask the user to confirm if they want to delete the resources.
> >
> > Part-2: If users select “yes”, the deletion will be really executed.
> >
> > Part-3: Print a summary of the results
> >
> >
> > Thanks
> > Yubiao Feng
> >
> > On Sun, Apr 16, 2023 at 9:45 PM Asaf Mesika <as...@gmail.com>
> wrote:
> >
> > > How about "truncate" instead of "clear"?
> > >
> > > Just wondering - since it is such a dangerous command, how can we help
> the
> > > user not make an accidental mass deletion?
> > >
> > > On Sat, Apr 15, 2023 at 1:12 PM Girish Sharma <scrapmachines@gmail.com
> >
> > > wrote:
> > >
> > > > > However, the current goal is to keep the tenant and namespace
> intact
> > > > while
> > > > > cleaning up their contents.
> > > > Ah, I see now. Yes, in that case a clear command is better. Will this
> > > > command also take into account the value of the broker config
> > > > `forceDeleteNamespaceAllowed` in case someone is clearing the owner
> > > tenant?
> > > >
> > > > Regards
> > > >
> > > > On Sat, Apr 15, 2023 at 3:39 PM Enrico Olivelli <eolivelli@gmail.com
> >
> > > > wrote:
> > > >
> > > > > The proposal sounds really useful, especially for automated
> testing.
> > > > > +1
> > > > >
> > > > > Enrico
> > > > >
> > > > > Il giorno sab 15 apr 2023 alle ore 12:07 Xiangying Meng
> > > > > <xi...@apache.org> ha scritto:
> > > > > >
> > > > > > Dear Girish,
> > > > > >
> > > > > > Thank you for your response and suggestion to extend the use of
> the
> > > > > > `boolean force` flag for namespaces and tenants.
> > > > > > I understand that the `force` flag is already implemented for
> > > deleting
> > > > > > topics, namespaces, and tenants,
> > > > > > and it provides a consistent way to perform these actions.
> > > > > >
> > > > > > However, the current goal is to keep the tenant and namespace
> intact
> > > > > while
> > > > > > cleaning up their contents.
> > > > > > In other words, I want to have a way to remove all topics within
> a
> > > > > > namespace or all namespaces and topics
> > > > > > within a tenant without actually deleting the namespace or tenant
> > > > itself.
> > > > > >
> > > > > > To achieve this goal, I proposed adding a `clear` command for
> > > > > `namespaces`
> > > > > > and `tenants`.
> > > > > >
> > > > > > This approach would allow users to keep the tenant and namespace
> > > > > structures
> > > > > > in place
> > > > > > while cleaning up their contents.
> > > > > > I hope this clarifies my intention, and I would like to hear your
> > > > > thoughts
> > > > > > on this proposal.
> > > > > >
> > > > > > Best regards,
> > > > > > Xiangying
> > > > > >
> > > > > > On Sat, Apr 15, 2023 at 5:49 PM Girish Sharma <
> > > scrapmachines@gmail.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hello Xiangying,
> > > > > > > This indeed is a cumbersome task to delete a filled namespace
> or
> > > > > tenant. We
> > > > > > > face this challenge in our organization where we use the
> > > > multi-tenancy
> > > > > > > feature of pulsar heavily.
> > > > > > >
> > > > > > > I would like to suggest a different command to do this though..
> > > > > Similar to
> > > > > > > how you cannot delete a topic without deleting its
> > > > > > > subscribers/producers/consumers, unless we use the `boolean
> force`
> > > > > flag.
> > > > > > > Why not extend this to namespace and tenant as well and let the
> > > force
> > > > > param
> > > > > > > do the cleanup (which your suggested `clear` command would do).
> > > > > > >
> > > > > > > As of today, using force to delete a namespace just returns 405
> > > > saying
> > > > > > > broker doesn't allow force delete of namespace containing
> topics.
> > > > > > >
> > > > > > > Any thoughts?
> > > > > > >
> > > > > > > On Sat, Apr 15, 2023 at 3:07 PM Xiangying Meng <
> > > xiangying@apache.org
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Dear Apache Pulsar Community,
> > > > > > > >
> > > > > > > > I hope this email finds you well.I am writing to suggest a
> > > > potential
> > > > > > > > improvement to the Pulsar-admin tool,
> > > > > > > >  which I believe could simplify the process of cleaning up
> > > tenants
> > > > > and
> > > > > > > > namespaces in Apache Pulsar.
> > > > > > > >
> > > > > > > > Currently, cleaning up all the namespaces and topics within a
> > > > tenant
> > > > > or
> > > > > > > > cleaning up all the topics within a namespace requires
> several
> > > > manual
> > > > > > > > steps,
> > > > > > > > such as listing the namespaces, listing the topics, and then
> > > > deleting
> > > > > > > each
> > > > > > > > topic individually.
> > > > > > > > This process can be time-consuming and error-prone for users.
> > > > > > > >
> > > > > > > > To address this issue, I propose the addition of a "clear"
> > > > parameter
> > > > > to
> > > > > > > the
> > > > > > > > Pulsar-admin tool,
> > > > > > > > which would automate the cleanup process for tenants and
> > > > namespaces.
> > > > > > > Here's
> > > > > > > > a conceptual implementation:
> > > > > > > >
> > > > > > > > 1. To clean up all namespaces and topics within a tenant:
> > > > > > > > ``` bash
> > > > > > > > pulsar-admin tenants clear <tenant-name>
> > > > > > > > ```
> > > > > > > > 2. To clean up all topics within a namespace:
> > > > > > > > ```bash
> > > > > > > > pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> > > > > > > > ```
> > > > > > > >
> > > > > > > > By implementing these new parameters, users would be able to
> > > > perform
> > > > > > > > cleanup operations more efficiently and with fewer manual
> steps.
> > > > > > > > I believe this improvement would greatly enhance the user
> > > > experience
> > > > > when
> > > > > > > > working with Apache Pulsar.
> > > > > > > >
> > > > > > > > I'd like to discuss the feasibility of this suggestion and
> gather
> > > > > > > feedback
> > > > > > > > from the community.
> > > > > > > > If everyone agrees, I can work on implementing this feature
> and
> > > > > submit a
> > > > > > > > pull request for review.
> > > > > > > >
> > > > > > > > Looking forward to hearing your thoughts on this.
> > > > > > > >
> > > > > > > > Best regards,
> > > > > > > > Xiangying
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Girish Sharma
> > > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Girish Sharma
> > > >
> > >
>

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Zike Yang <zi...@apache.org>]

Is it possible to use regex to delete namespaces/topics here?
For instance, we can use the following command to delete all topics in
the default namespace:
```
pulsar-admin topics delete public/default/*
```
We can also use it to delete some specific topics:
```
pulsar-admin topics delete public/default/*topic
```
This way, we don't need to introduce a new command
`delete-all-topics`. It also makes it easier for users to delete
specific topic lists.

> Part-1: Give a summary print of the namespaces, and topics to delete,
>  and ask the user to confirm if they want to delete the resources.
> Part-2: If users select “yes”, the deletion will be really executed.
> Part-3: Print a summary of the results

+1 for this. This will minimize the risk of accidentally deleting
important resources.

BR,
Zike Yang

On Wed, Apr 19, 2023 at 3:34 PM Yubiao Feng
<yu...@streamnative.io.invalid> wrote:
>
> > Just wondering - since it is such a dangerous
> > command, how can we help the user not make
> > an accidental mass deletion?
>
> Just a suggestion: We can make this command executed on two-part
>
> Part-1: Give a summary print of the namespaces, and topics to delete,
>  and ask the user to confirm if they want to delete the resources.
>
> Part-2: If users select “yes”, the deletion will be really executed.
>
> Part-3: Print a summary of the results
>
>
> Thanks
> Yubiao Feng
>
> On Sun, Apr 16, 2023 at 9:45 PM Asaf Mesika <as...@gmail.com> wrote:
>
> > How about "truncate" instead of "clear"?
> >
> > Just wondering - since it is such a dangerous command, how can we help the
> > user not make an accidental mass deletion?
> >
> > On Sat, Apr 15, 2023 at 1:12 PM Girish Sharma <sc...@gmail.com>
> > wrote:
> >
> > > > However, the current goal is to keep the tenant and namespace intact
> > > while
> > > > cleaning up their contents.
> > > Ah, I see now. Yes, in that case a clear command is better. Will this
> > > command also take into account the value of the broker config
> > > `forceDeleteNamespaceAllowed` in case someone is clearing the owner
> > tenant?
> > >
> > > Regards
> > >
> > > On Sat, Apr 15, 2023 at 3:39 PM Enrico Olivelli <eo...@gmail.com>
> > > wrote:
> > >
> > > > The proposal sounds really useful, especially for automated testing.
> > > > +1
> > > >
> > > > Enrico
> > > >
> > > > Il giorno sab 15 apr 2023 alle ore 12:07 Xiangying Meng
> > > > <xi...@apache.org> ha scritto:
> > > > >
> > > > > Dear Girish,
> > > > >
> > > > > Thank you for your response and suggestion to extend the use of the
> > > > > `boolean force` flag for namespaces and tenants.
> > > > > I understand that the `force` flag is already implemented for
> > deleting
> > > > > topics, namespaces, and tenants,
> > > > > and it provides a consistent way to perform these actions.
> > > > >
> > > > > However, the current goal is to keep the tenant and namespace intact
> > > > while
> > > > > cleaning up their contents.
> > > > > In other words, I want to have a way to remove all topics within a
> > > > > namespace or all namespaces and topics
> > > > > within a tenant without actually deleting the namespace or tenant
> > > itself.
> > > > >
> > > > > To achieve this goal, I proposed adding a `clear` command for
> > > > `namespaces`
> > > > > and `tenants`.
> > > > >
> > > > > This approach would allow users to keep the tenant and namespace
> > > > structures
> > > > > in place
> > > > > while cleaning up their contents.
> > > > > I hope this clarifies my intention, and I would like to hear your
> > > > thoughts
> > > > > on this proposal.
> > > > >
> > > > > Best regards,
> > > > > Xiangying
> > > > >
> > > > > On Sat, Apr 15, 2023 at 5:49 PM Girish Sharma <
> > scrapmachines@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hello Xiangying,
> > > > > > This indeed is a cumbersome task to delete a filled namespace or
> > > > tenant. We
> > > > > > face this challenge in our organization where we use the
> > > multi-tenancy
> > > > > > feature of pulsar heavily.
> > > > > >
> > > > > > I would like to suggest a different command to do this though..
> > > > Similar to
> > > > > > how you cannot delete a topic without deleting its
> > > > > > subscribers/producers/consumers, unless we use the `boolean force`
> > > > flag.
> > > > > > Why not extend this to namespace and tenant as well and let the
> > force
> > > > param
> > > > > > do the cleanup (which your suggested `clear` command would do).
> > > > > >
> > > > > > As of today, using force to delete a namespace just returns 405
> > > saying
> > > > > > broker doesn't allow force delete of namespace containing topics.
> > > > > >
> > > > > > Any thoughts?
> > > > > >
> > > > > > On Sat, Apr 15, 2023 at 3:07 PM Xiangying Meng <
> > xiangying@apache.org
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Dear Apache Pulsar Community,
> > > > > > >
> > > > > > > I hope this email finds you well.I am writing to suggest a
> > > potential
> > > > > > > improvement to the Pulsar-admin tool,
> > > > > > >  which I believe could simplify the process of cleaning up
> > tenants
> > > > and
> > > > > > > namespaces in Apache Pulsar.
> > > > > > >
> > > > > > > Currently, cleaning up all the namespaces and topics within a
> > > tenant
> > > > or
> > > > > > > cleaning up all the topics within a namespace requires several
> > > manual
> > > > > > > steps,
> > > > > > > such as listing the namespaces, listing the topics, and then
> > > deleting
> > > > > > each
> > > > > > > topic individually.
> > > > > > > This process can be time-consuming and error-prone for users.
> > > > > > >
> > > > > > > To address this issue, I propose the addition of a "clear"
> > > parameter
> > > > to
> > > > > > the
> > > > > > > Pulsar-admin tool,
> > > > > > > which would automate the cleanup process for tenants and
> > > namespaces.
> > > > > > Here's
> > > > > > > a conceptual implementation:
> > > > > > >
> > > > > > > 1. To clean up all namespaces and topics within a tenant:
> > > > > > > ``` bash
> > > > > > > pulsar-admin tenants clear <tenant-name>
> > > > > > > ```
> > > > > > > 2. To clean up all topics within a namespace:
> > > > > > > ```bash
> > > > > > > pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> > > > > > > ```
> > > > > > >
> > > > > > > By implementing these new parameters, users would be able to
> > > perform
> > > > > > > cleanup operations more efficiently and with fewer manual steps.
> > > > > > > I believe this improvement would greatly enhance the user
> > > experience
> > > > when
> > > > > > > working with Apache Pulsar.
> > > > > > >
> > > > > > > I'd like to discuss the feasibility of this suggestion and gather
> > > > > > feedback
> > > > > > > from the community.
> > > > > > > If everyone agrees, I can work on implementing this feature and
> > > > submit a
> > > > > > > pull request for review.
> > > > > > >
> > > > > > > Looking forward to hearing your thoughts on this.
> > > > > > >
> > > > > > > Best regards,
> > > > > > > Xiangying
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Girish Sharma
> > > > > >
> > > >
> > >
> > >
> > > --
> > > Girish Sharma
> > >
> >

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Yubiao Feng <yu...@streamnative.io.INVALID>]

> Just wondering - since it is such a dangerous
> command, how can we help the user not make
> an accidental mass deletion?

Just a suggestion: We can make this command executed on two-part

Part-1: Give a summary print of the namespaces, and topics to delete,
 and ask the user to confirm if they want to delete the resources.

Part-2: If users select “yes”, the deletion will be really executed.

Part-3: Print a summary of the results


Thanks
Yubiao Feng

On Sun, Apr 16, 2023 at 9:45 PM Asaf Mesika <as...@gmail.com> wrote:

> How about "truncate" instead of "clear"?
>
> Just wondering - since it is such a dangerous command, how can we help the
> user not make an accidental mass deletion?
>
> On Sat, Apr 15, 2023 at 1:12 PM Girish Sharma <sc...@gmail.com>
> wrote:
>
> > > However, the current goal is to keep the tenant and namespace intact
> > while
> > > cleaning up their contents.
> > Ah, I see now. Yes, in that case a clear command is better. Will this
> > command also take into account the value of the broker config
> > `forceDeleteNamespaceAllowed` in case someone is clearing the owner
> tenant?
> >
> > Regards
> >
> > On Sat, Apr 15, 2023 at 3:39 PM Enrico Olivelli <eo...@gmail.com>
> > wrote:
> >
> > > The proposal sounds really useful, especially for automated testing.
> > > +1
> > >
> > > Enrico
> > >
> > > Il giorno sab 15 apr 2023 alle ore 12:07 Xiangying Meng
> > > <xi...@apache.org> ha scritto:
> > > >
> > > > Dear Girish,
> > > >
> > > > Thank you for your response and suggestion to extend the use of the
> > > > `boolean force` flag for namespaces and tenants.
> > > > I understand that the `force` flag is already implemented for
> deleting
> > > > topics, namespaces, and tenants,
> > > > and it provides a consistent way to perform these actions.
> > > >
> > > > However, the current goal is to keep the tenant and namespace intact
> > > while
> > > > cleaning up their contents.
> > > > In other words, I want to have a way to remove all topics within a
> > > > namespace or all namespaces and topics
> > > > within a tenant without actually deleting the namespace or tenant
> > itself.
> > > >
> > > > To achieve this goal, I proposed adding a `clear` command for
> > > `namespaces`
> > > > and `tenants`.
> > > >
> > > > This approach would allow users to keep the tenant and namespace
> > > structures
> > > > in place
> > > > while cleaning up their contents.
> > > > I hope this clarifies my intention, and I would like to hear your
> > > thoughts
> > > > on this proposal.
> > > >
> > > > Best regards,
> > > > Xiangying
> > > >
> > > > On Sat, Apr 15, 2023 at 5:49 PM Girish Sharma <
> scrapmachines@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Hello Xiangying,
> > > > > This indeed is a cumbersome task to delete a filled namespace or
> > > tenant. We
> > > > > face this challenge in our organization where we use the
> > multi-tenancy
> > > > > feature of pulsar heavily.
> > > > >
> > > > > I would like to suggest a different command to do this though..
> > > Similar to
> > > > > how you cannot delete a topic without deleting its
> > > > > subscribers/producers/consumers, unless we use the `boolean force`
> > > flag.
> > > > > Why not extend this to namespace and tenant as well and let the
> force
> > > param
> > > > > do the cleanup (which your suggested `clear` command would do).
> > > > >
> > > > > As of today, using force to delete a namespace just returns 405
> > saying
> > > > > broker doesn't allow force delete of namespace containing topics.
> > > > >
> > > > > Any thoughts?
> > > > >
> > > > > On Sat, Apr 15, 2023 at 3:07 PM Xiangying Meng <
> xiangying@apache.org
> > >
> > > > > wrote:
> > > > >
> > > > > > Dear Apache Pulsar Community,
> > > > > >
> > > > > > I hope this email finds you well.I am writing to suggest a
> > potential
> > > > > > improvement to the Pulsar-admin tool,
> > > > > >  which I believe could simplify the process of cleaning up
> tenants
> > > and
> > > > > > namespaces in Apache Pulsar.
> > > > > >
> > > > > > Currently, cleaning up all the namespaces and topics within a
> > tenant
> > > or
> > > > > > cleaning up all the topics within a namespace requires several
> > manual
> > > > > > steps,
> > > > > > such as listing the namespaces, listing the topics, and then
> > deleting
> > > > > each
> > > > > > topic individually.
> > > > > > This process can be time-consuming and error-prone for users.
> > > > > >
> > > > > > To address this issue, I propose the addition of a "clear"
> > parameter
> > > to
> > > > > the
> > > > > > Pulsar-admin tool,
> > > > > > which would automate the cleanup process for tenants and
> > namespaces.
> > > > > Here's
> > > > > > a conceptual implementation:
> > > > > >
> > > > > > 1. To clean up all namespaces and topics within a tenant:
> > > > > > ``` bash
> > > > > > pulsar-admin tenants clear <tenant-name>
> > > > > > ```
> > > > > > 2. To clean up all topics within a namespace:
> > > > > > ```bash
> > > > > > pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> > > > > > ```
> > > > > >
> > > > > > By implementing these new parameters, users would be able to
> > perform
> > > > > > cleanup operations more efficiently and with fewer manual steps.
> > > > > > I believe this improvement would greatly enhance the user
> > experience
> > > when
> > > > > > working with Apache Pulsar.
> > > > > >
> > > > > > I'd like to discuss the feasibility of this suggestion and gather
> > > > > feedback
> > > > > > from the community.
> > > > > > If everyone agrees, I can work on implementing this feature and
> > > submit a
> > > > > > pull request for review.
> > > > > >
> > > > > > Looking forward to hearing your thoughts on this.
> > > > > >
> > > > > > Best regards,
> > > > > > Xiangying
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Girish Sharma
> > > > >
> > >
> >
> >
> > --
> > Girish Sharma
> >
>

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Asaf Mesika <as...@gmail.com>]

How about "truncate" instead of "clear"?

Just wondering - since it is such a dangerous command, how can we help the
user not make an accidental mass deletion?

On Sat, Apr 15, 2023 at 1:12 PM Girish Sharma <sc...@gmail.com>
wrote:

> > However, the current goal is to keep the tenant and namespace intact
> while
> > cleaning up their contents.
> Ah, I see now. Yes, in that case a clear command is better. Will this
> command also take into account the value of the broker config
> `forceDeleteNamespaceAllowed` in case someone is clearing the owner tenant?
>
> Regards
>
> On Sat, Apr 15, 2023 at 3:39 PM Enrico Olivelli <eo...@gmail.com>
> wrote:
>
> > The proposal sounds really useful, especially for automated testing.
> > +1
> >
> > Enrico
> >
> > Il giorno sab 15 apr 2023 alle ore 12:07 Xiangying Meng
> > <xi...@apache.org> ha scritto:
> > >
> > > Dear Girish,
> > >
> > > Thank you for your response and suggestion to extend the use of the
> > > `boolean force` flag for namespaces and tenants.
> > > I understand that the `force` flag is already implemented for deleting
> > > topics, namespaces, and tenants,
> > > and it provides a consistent way to perform these actions.
> > >
> > > However, the current goal is to keep the tenant and namespace intact
> > while
> > > cleaning up their contents.
> > > In other words, I want to have a way to remove all topics within a
> > > namespace or all namespaces and topics
> > > within a tenant without actually deleting the namespace or tenant
> itself.
> > >
> > > To achieve this goal, I proposed adding a `clear` command for
> > `namespaces`
> > > and `tenants`.
> > >
> > > This approach would allow users to keep the tenant and namespace
> > structures
> > > in place
> > > while cleaning up their contents.
> > > I hope this clarifies my intention, and I would like to hear your
> > thoughts
> > > on this proposal.
> > >
> > > Best regards,
> > > Xiangying
> > >
> > > On Sat, Apr 15, 2023 at 5:49 PM Girish Sharma <scrapmachines@gmail.com
> >
> > > wrote:
> > >
> > > > Hello Xiangying,
> > > > This indeed is a cumbersome task to delete a filled namespace or
> > tenant. We
> > > > face this challenge in our organization where we use the
> multi-tenancy
> > > > feature of pulsar heavily.
> > > >
> > > > I would like to suggest a different command to do this though..
> > Similar to
> > > > how you cannot delete a topic without deleting its
> > > > subscribers/producers/consumers, unless we use the `boolean force`
> > flag.
> > > > Why not extend this to namespace and tenant as well and let the force
> > param
> > > > do the cleanup (which your suggested `clear` command would do).
> > > >
> > > > As of today, using force to delete a namespace just returns 405
> saying
> > > > broker doesn't allow force delete of namespace containing topics.
> > > >
> > > > Any thoughts?
> > > >
> > > > On Sat, Apr 15, 2023 at 3:07 PM Xiangying Meng <xiangying@apache.org
> >
> > > > wrote:
> > > >
> > > > > Dear Apache Pulsar Community,
> > > > >
> > > > > I hope this email finds you well.I am writing to suggest a
> potential
> > > > > improvement to the Pulsar-admin tool,
> > > > >  which I believe could simplify the process of cleaning up tenants
> > and
> > > > > namespaces in Apache Pulsar.
> > > > >
> > > > > Currently, cleaning up all the namespaces and topics within a
> tenant
> > or
> > > > > cleaning up all the topics within a namespace requires several
> manual
> > > > > steps,
> > > > > such as listing the namespaces, listing the topics, and then
> deleting
> > > > each
> > > > > topic individually.
> > > > > This process can be time-consuming and error-prone for users.
> > > > >
> > > > > To address this issue, I propose the addition of a "clear"
> parameter
> > to
> > > > the
> > > > > Pulsar-admin tool,
> > > > > which would automate the cleanup process for tenants and
> namespaces.
> > > > Here's
> > > > > a conceptual implementation:
> > > > >
> > > > > 1. To clean up all namespaces and topics within a tenant:
> > > > > ``` bash
> > > > > pulsar-admin tenants clear <tenant-name>
> > > > > ```
> > > > > 2. To clean up all topics within a namespace:
> > > > > ```bash
> > > > > pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> > > > > ```
> > > > >
> > > > > By implementing these new parameters, users would be able to
> perform
> > > > > cleanup operations more efficiently and with fewer manual steps.
> > > > > I believe this improvement would greatly enhance the user
> experience
> > when
> > > > > working with Apache Pulsar.
> > > > >
> > > > > I'd like to discuss the feasibility of this suggestion and gather
> > > > feedback
> > > > > from the community.
> > > > > If everyone agrees, I can work on implementing this feature and
> > submit a
> > > > > pull request for review.
> > > > >
> > > > > Looking forward to hearing your thoughts on this.
> > > > >
> > > > > Best regards,
> > > > > Xiangying
> > > > >
> > > >
> > > >
> > > > --
> > > > Girish Sharma
> > > >
> >
>
>
> --
> Girish Sharma
>

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Girish Sharma <sc...@gmail.com>]

> However, the current goal is to keep the tenant and namespace intact while
> cleaning up their contents.
Ah, I see now. Yes, in that case a clear command is better. Will this
command also take into account the value of the broker config
`forceDeleteNamespaceAllowed` in case someone is clearing the owner tenant?

Regards

On Sat, Apr 15, 2023 at 3:39 PM Enrico Olivelli <eo...@gmail.com> wrote:

> The proposal sounds really useful, especially for automated testing.
> +1
>
> Enrico
>
> Il giorno sab 15 apr 2023 alle ore 12:07 Xiangying Meng
> <xi...@apache.org> ha scritto:
> >
> > Dear Girish,
> >
> > Thank you for your response and suggestion to extend the use of the
> > `boolean force` flag for namespaces and tenants.
> > I understand that the `force` flag is already implemented for deleting
> > topics, namespaces, and tenants,
> > and it provides a consistent way to perform these actions.
> >
> > However, the current goal is to keep the tenant and namespace intact
> while
> > cleaning up their contents.
> > In other words, I want to have a way to remove all topics within a
> > namespace or all namespaces and topics
> > within a tenant without actually deleting the namespace or tenant itself.
> >
> > To achieve this goal, I proposed adding a `clear` command for
> `namespaces`
> > and `tenants`.
> >
> > This approach would allow users to keep the tenant and namespace
> structures
> > in place
> > while cleaning up their contents.
> > I hope this clarifies my intention, and I would like to hear your
> thoughts
> > on this proposal.
> >
> > Best regards,
> > Xiangying
> >
> > On Sat, Apr 15, 2023 at 5:49 PM Girish Sharma <sc...@gmail.com>
> > wrote:
> >
> > > Hello Xiangying,
> > > This indeed is a cumbersome task to delete a filled namespace or
> tenant. We
> > > face this challenge in our organization where we use the multi-tenancy
> > > feature of pulsar heavily.
> > >
> > > I would like to suggest a different command to do this though..
> Similar to
> > > how you cannot delete a topic without deleting its
> > > subscribers/producers/consumers, unless we use the `boolean force`
> flag.
> > > Why not extend this to namespace and tenant as well and let the force
> param
> > > do the cleanup (which your suggested `clear` command would do).
> > >
> > > As of today, using force to delete a namespace just returns 405 saying
> > > broker doesn't allow force delete of namespace containing topics.
> > >
> > > Any thoughts?
> > >
> > > On Sat, Apr 15, 2023 at 3:07 PM Xiangying Meng <xi...@apache.org>
> > > wrote:
> > >
> > > > Dear Apache Pulsar Community,
> > > >
> > > > I hope this email finds you well.I am writing to suggest a potential
> > > > improvement to the Pulsar-admin tool,
> > > >  which I believe could simplify the process of cleaning up tenants
> and
> > > > namespaces in Apache Pulsar.
> > > >
> > > > Currently, cleaning up all the namespaces and topics within a tenant
> or
> > > > cleaning up all the topics within a namespace requires several manual
> > > > steps,
> > > > such as listing the namespaces, listing the topics, and then deleting
> > > each
> > > > topic individually.
> > > > This process can be time-consuming and error-prone for users.
> > > >
> > > > To address this issue, I propose the addition of a "clear" parameter
> to
> > > the
> > > > Pulsar-admin tool,
> > > > which would automate the cleanup process for tenants and namespaces.
> > > Here's
> > > > a conceptual implementation:
> > > >
> > > > 1. To clean up all namespaces and topics within a tenant:
> > > > ``` bash
> > > > pulsar-admin tenants clear <tenant-name>
> > > > ```
> > > > 2. To clean up all topics within a namespace:
> > > > ```bash
> > > > pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> > > > ```
> > > >
> > > > By implementing these new parameters, users would be able to perform
> > > > cleanup operations more efficiently and with fewer manual steps.
> > > > I believe this improvement would greatly enhance the user experience
> when
> > > > working with Apache Pulsar.
> > > >
> > > > I'd like to discuss the feasibility of this suggestion and gather
> > > feedback
> > > > from the community.
> > > > If everyone agrees, I can work on implementing this feature and
> submit a
> > > > pull request for review.
> > > >
> > > > Looking forward to hearing your thoughts on this.
> > > >
> > > > Best regards,
> > > > Xiangying
> > > >
> > >
> > >
> > > --
> > > Girish Sharma
> > >
>


-- 
Girish Sharma

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Enrico Olivelli <eo...@gmail.com>]

The proposal sounds really useful, especially for automated testing.
+1

Enrico

Il giorno sab 15 apr 2023 alle ore 12:07 Xiangying Meng
<xi...@apache.org> ha scritto:
>
> Dear Girish,
>
> Thank you for your response and suggestion to extend the use of the
> `boolean force` flag for namespaces and tenants.
> I understand that the `force` flag is already implemented for deleting
> topics, namespaces, and tenants,
> and it provides a consistent way to perform these actions.
>
> However, the current goal is to keep the tenant and namespace intact while
> cleaning up their contents.
> In other words, I want to have a way to remove all topics within a
> namespace or all namespaces and topics
> within a tenant without actually deleting the namespace or tenant itself.
>
> To achieve this goal, I proposed adding a `clear` command for `namespaces`
> and `tenants`.
>
> This approach would allow users to keep the tenant and namespace structures
> in place
> while cleaning up their contents.
> I hope this clarifies my intention, and I would like to hear your thoughts
> on this proposal.
>
> Best regards,
> Xiangying
>
> On Sat, Apr 15, 2023 at 5:49 PM Girish Sharma <sc...@gmail.com>
> wrote:
>
> > Hello Xiangying,
> > This indeed is a cumbersome task to delete a filled namespace or tenant. We
> > face this challenge in our organization where we use the multi-tenancy
> > feature of pulsar heavily.
> >
> > I would like to suggest a different command to do this though.. Similar to
> > how you cannot delete a topic without deleting its
> > subscribers/producers/consumers, unless we use the `boolean force` flag.
> > Why not extend this to namespace and tenant as well and let the force param
> > do the cleanup (which your suggested `clear` command would do).
> >
> > As of today, using force to delete a namespace just returns 405 saying
> > broker doesn't allow force delete of namespace containing topics.
> >
> > Any thoughts?
> >
> > On Sat, Apr 15, 2023 at 3:07 PM Xiangying Meng <xi...@apache.org>
> > wrote:
> >
> > > Dear Apache Pulsar Community,
> > >
> > > I hope this email finds you well.I am writing to suggest a potential
> > > improvement to the Pulsar-admin tool,
> > >  which I believe could simplify the process of cleaning up tenants and
> > > namespaces in Apache Pulsar.
> > >
> > > Currently, cleaning up all the namespaces and topics within a tenant or
> > > cleaning up all the topics within a namespace requires several manual
> > > steps,
> > > such as listing the namespaces, listing the topics, and then deleting
> > each
> > > topic individually.
> > > This process can be time-consuming and error-prone for users.
> > >
> > > To address this issue, I propose the addition of a "clear" parameter to
> > the
> > > Pulsar-admin tool,
> > > which would automate the cleanup process for tenants and namespaces.
> > Here's
> > > a conceptual implementation:
> > >
> > > 1. To clean up all namespaces and topics within a tenant:
> > > ``` bash
> > > pulsar-admin tenants clear <tenant-name>
> > > ```
> > > 2. To clean up all topics within a namespace:
> > > ```bash
> > > pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> > > ```
> > >
> > > By implementing these new parameters, users would be able to perform
> > > cleanup operations more efficiently and with fewer manual steps.
> > > I believe this improvement would greatly enhance the user experience when
> > > working with Apache Pulsar.
> > >
> > > I'd like to discuss the feasibility of this suggestion and gather
> > feedback
> > > from the community.
> > > If everyone agrees, I can work on implementing this feature and submit a
> > > pull request for review.
> > >
> > > Looking forward to hearing your thoughts on this.
> > >
> > > Best regards,
> > > Xiangying
> > >
> >
> >
> > --
> > Girish Sharma
> >

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Xiangying Meng <xi...@apache.org>]

Dear Girish,

Thank you for your response and suggestion to extend the use of the
`boolean force` flag for namespaces and tenants.
I understand that the `force` flag is already implemented for deleting
topics, namespaces, and tenants,
and it provides a consistent way to perform these actions.

However, the current goal is to keep the tenant and namespace intact while
cleaning up their contents.
In other words, I want to have a way to remove all topics within a
namespace or all namespaces and topics
within a tenant without actually deleting the namespace or tenant itself.

To achieve this goal, I proposed adding a `clear` command for `namespaces`
and `tenants`.

This approach would allow users to keep the tenant and namespace structures
in place
while cleaning up their contents.
I hope this clarifies my intention, and I would like to hear your thoughts
on this proposal.

Best regards,
Xiangying

On Sat, Apr 15, 2023 at 5:49 PM Girish Sharma <sc...@gmail.com>
wrote:

> Hello Xiangying,
> This indeed is a cumbersome task to delete a filled namespace or tenant. We
> face this challenge in our organization where we use the multi-tenancy
> feature of pulsar heavily.
>
> I would like to suggest a different command to do this though.. Similar to
> how you cannot delete a topic without deleting its
> subscribers/producers/consumers, unless we use the `boolean force` flag.
> Why not extend this to namespace and tenant as well and let the force param
> do the cleanup (which your suggested `clear` command would do).
>
> As of today, using force to delete a namespace just returns 405 saying
> broker doesn't allow force delete of namespace containing topics.
>
> Any thoughts?
>
> On Sat, Apr 15, 2023 at 3:07 PM Xiangying Meng <xi...@apache.org>
> wrote:
>
> > Dear Apache Pulsar Community,
> >
> > I hope this email finds you well.I am writing to suggest a potential
> > improvement to the Pulsar-admin tool,
> >  which I believe could simplify the process of cleaning up tenants and
> > namespaces in Apache Pulsar.
> >
> > Currently, cleaning up all the namespaces and topics within a tenant or
> > cleaning up all the topics within a namespace requires several manual
> > steps,
> > such as listing the namespaces, listing the topics, and then deleting
> each
> > topic individually.
> > This process can be time-consuming and error-prone for users.
> >
> > To address this issue, I propose the addition of a "clear" parameter to
> the
> > Pulsar-admin tool,
> > which would automate the cleanup process for tenants and namespaces.
> Here's
> > a conceptual implementation:
> >
> > 1. To clean up all namespaces and topics within a tenant:
> > ``` bash
> > pulsar-admin tenants clear <tenant-name>
> > ```
> > 2. To clean up all topics within a namespace:
> > ```bash
> > pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> > ```
> >
> > By implementing these new parameters, users would be able to perform
> > cleanup operations more efficiently and with fewer manual steps.
> > I believe this improvement would greatly enhance the user experience when
> > working with Apache Pulsar.
> >
> > I'd like to discuss the feasibility of this suggestion and gather
> feedback
> > from the community.
> > If everyone agrees, I can work on implementing this feature and submit a
> > pull request for review.
> >
> > Looking forward to hearing your thoughts on this.
> >
> > Best regards,
> > Xiangying
> >
>
>
> --
> Girish Sharma
>

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Girish Sharma <sc...@gmail.com>]

Hello Xiangying,
This indeed is a cumbersome task to delete a filled namespace or tenant. We
face this challenge in our organization where we use the multi-tenancy
feature of pulsar heavily.

I would like to suggest a different command to do this though.. Similar to
how you cannot delete a topic without deleting its
subscribers/producers/consumers, unless we use the `boolean force` flag.
Why not extend this to namespace and tenant as well and let the force param
do the cleanup (which your suggested `clear` command would do).

As of today, using force to delete a namespace just returns 405 saying
broker doesn't allow force delete of namespace containing topics.

Any thoughts?

On Sat, Apr 15, 2023 at 3:07 PM Xiangying Meng <xi...@apache.org> wrote:

> Dear Apache Pulsar Community,
>
> I hope this email finds you well.I am writing to suggest a potential
> improvement to the Pulsar-admin tool,
>  which I believe could simplify the process of cleaning up tenants and
> namespaces in Apache Pulsar.
>
> Currently, cleaning up all the namespaces and topics within a tenant or
> cleaning up all the topics within a namespace requires several manual
> steps,
> such as listing the namespaces, listing the topics, and then deleting each
> topic individually.
> This process can be time-consuming and error-prone for users.
>
> To address this issue, I propose the addition of a "clear" parameter to the
> Pulsar-admin tool,
> which would automate the cleanup process for tenants and namespaces. Here's
> a conceptual implementation:
>
> 1. To clean up all namespaces and topics within a tenant:
> ``` bash
> pulsar-admin tenants clear <tenant-name>
> ```
> 2. To clean up all topics within a namespace:
> ```bash
> pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> ```
>
> By implementing these new parameters, users would be able to perform
> cleanup operations more efficiently and with fewer manual steps.
> I believe this improvement would greatly enhance the user experience when
> working with Apache Pulsar.
>
> I'd like to discuss the feasibility of this suggestion and gather feedback
> from the community.
> If everyone agrees, I can work on implementing this feature and submit a
> pull request for review.
>
> Looking forward to hearing your thoughts on this.
>
> Best regards,
> Xiangying
>


-- 
Girish Sharma

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Girish Sharma <sc...@gmail.com>]

Hello Xiangying,
Thank you for the summary .

I would like to understand if this feature would delete the regex based
topics or namespaces from all available clusters in the global zookeeper,
or only from the local cluster where this command is run.

Regards

On Tue, Apr 25, 2023, 8:52 AM Xiangying Meng <xi...@apache.org> wrote:

> Hi Girish,
>
> Thank you for raising concerns about the proposed feature. I would
> like to address the points you mentioned in your email.
>
> 1. I understand that one of your concerns is whether the proposed
> regex-based deletion feature would provide significant advantages over
> using a simple one-liner script to call the delete topic command for
> each topic. As Yubiao pointed out, using scripts to delete topics one
> by one can lead to increased network overhead and slow performance,
> particularly when dealing with a large number of topics. Implementing
> regex support for delete operations would provide a more efficient and
> convenient way to manage resources in Pulsar.
>
> 2. In addition to the benefits for testing purposes, we have
> communicated with business users of Pulsar and found that the proposed
> regex-based deletion feature can be helpful in production environments
> as well. For instance, it can be used to efficiently clean up
> subscriptions associated with deprecated services, ensuring better
> resource management and reducing clutter in the system.
>
> 3. As I suggested earlier, we can introduce a new option flag (e.g.,
> `--regex` or `--pattern`) to the existing `pulsar-admin topics delete`
> command to prevent breaking changes for users who have already used
> the command in their scripts. This would ensure backward compatibility
> while providing the new functionality for those who want to use regex
> for deletion.
>
> I hope this clears up any confusion and addresses your concerns.
> Please let me know if you have any further questions or suggestions.
>
> Best regards,
> Xiangying Meng
>
> On Mon, Apr 24, 2023 at 6:23 PM Girish Sharma <sc...@gmail.com>
> wrote:
> >
> > Hello Yubiao,
> > As per my understanding, this feature suggestion is intended to delete
> the
> > topics from all replicated clusters under the namespace. Thus, the
> example
> > you are providing may not be a good fit for this?
> >
> > Xiangying, please clarify if my understanding is incorrect.
> >
> > On Mon, Apr 24, 2023 at 3:24 PM Yubiao Feng
> > <yu...@streamnative.io.invalid> wrote:
> >
> > > Hi Girish Sharma
> > >
> > > > What additional advantage would one get by using that approach
> > > > rather than simply using a one liner script to just call delete
> > > > topic for each of those topics if the list of topics is known.
> > >
> > > If users enabled `Geo-Replication` on a namespace in mistake(expected
> > > only to enable one topic),
> > > it is possible to create many topics on the remote cluster in one
> second.
> > >
> > > Not long ago, 10,000 topics were created per second because of this
> > > mistake. It took us a long time to
> > > remove these topics. We delete these topics in this way:
> > > ```
> > > cat topics_name_file | awk  '{system("bin/pulsar-admin topics delete
> "$0)}'
> > > )
> > > ```
> > > It deletes topics one by one.
> > >
> > > We conclude later that stress test tools such as `Jmeter` or `ab`
> should be
> > > used to delete so many topics.
> > >
> > > If Pulsar could provide these APIs, it would be better.
> > >
> > > Thanks
> > > Yubiao Feng
> > >
> > >
> > >
> > >
> > > On Wed, Apr 19, 2023 at 3:29 PM Girish Sharma <scrapmachines@gmail.com
> >
> > > wrote:
> > >
> > > > Hello Yubiao,
> > > >
> > > > What additional advantage would one get by using that approach rather
> > > than
> > > > simply using a one liner script to just call delete topic for each of
> > > those
> > > > topics if the list of topics is known.
> > > >
> > > > Regards
> > > >
> > > > On Wed, Apr 19, 2023 at 12:54 PM Yubiao Feng
> > > > <yu...@streamnative.io.invalid> wrote:
> > > >
> > > > > In addition to these two, It is recommended to add a method to
> batch
> > > > delete
> > > > > topics, such as this:
> > > > >
> > > > > ```
> > > > > pulsar-admin topics delete-all-topics <topic_1>, <topic_2>
> > > > >
> > > > > or
> > > > >
> > > > > pulsar-admin topics delete-all-topic <a file contains a topic_name
> > > lists>
> > > > > ```
> > > > >
> > > > > Thanks
> > > > > Yubiao Feng
> > > > >
> > > > > On Sat, Apr 15, 2023 at 5:37 PM Xiangying Meng <
> xiangying@apache.org>
> > > > > wrote:
> > > > >
> > > > > > Dear Apache Pulsar Community,
> > > > > >
> > > > > > I hope this email finds you well.I am writing to suggest a
> potential
> > > > > > improvement to the Pulsar-admin tool,
> > > > > >  which I believe could simplify the process of cleaning up
> tenants
> > > and
> > > > > > namespaces in Apache Pulsar.
> > > > > >
> > > > > > Currently, cleaning up all the namespaces and topics within a
> tenant
> > > or
> > > > > > cleaning up all the topics within a namespace requires several
> manual
> > > > > > steps,
> > > > > > such as listing the namespaces, listing the topics, and then
> deleting
> > > > > each
> > > > > > topic individually.
> > > > > > This process can be time-consuming and error-prone for users.
> > > > > >
> > > > > > To address this issue, I propose the addition of a "clear"
> parameter
> > > to
> > > > > the
> > > > > > Pulsar-admin tool,
> > > > > > which would automate the cleanup process for tenants and
> namespaces.
> > > > > Here's
> > > > > > a conceptual implementation:
> > > > > >
> > > > > > 1. To clean up all namespaces and topics within a tenant:
> > > > > > ``` bash
> > > > > > pulsar-admin tenants clear <tenant-name>
> > > > > > ```
> > > > > > 2. To clean up all topics within a namespace:
> > > > > > ```bash
> > > > > > pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> > > > > > ```
> > > > > >
> > > > > > By implementing these new parameters, users would be able to
> perform
> > > > > > cleanup operations more efficiently and with fewer manual steps.
> > > > > > I believe this improvement would greatly enhance the user
> experience
> > > > when
> > > > > > working with Apache Pulsar.
> > > > > >
> > > > > > I'd like to discuss the feasibility of this suggestion and gather
> > > > > feedback
> > > > > > from the community.
> > > > > > If everyone agrees, I can work on implementing this feature and
> > > submit
> > > > a
> > > > > > pull request for review.
> > > > > >
> > > > > > Looking forward to hearing your thoughts on this.
> > > > > >
> > > > > > Best regards,
> > > > > > Xiangying
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Girish Sharma
> > > >
> > >
> >
> >
> > --
> > Girish Sharma
>

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Asaf Mesika <as...@gmail.com>]

The vote triggered the existence of the PIP, so I just read it now quickly.

When someone deletes topics using regex, you list the topics to filter
based on the user permissions? You first obtain the permission of the user,
see what tenants, namespaces it has access to and only then, list the
topics there, and then filter it using reg-ex?


On Tue, Apr 25, 2023 at 3:22 PM Xiangying Meng <xi...@apache.org> wrote:

> Hi Girish and Zike,
>
> @Girish,
> Regarding your question about the scope of the regex-based deletion
> feature, the initial proposal is to delete the specified topics or
> namespaces only in the local cluster where the command is run. If you
> need to delete them in remote clusters as well, you can simply run the
> command again in the respective clusters.
>
> @Zike,
> You raised a valid point about the regex-based deletion feature not
> covering the case Yubiao mentioned. To address this, we can introduce
> another flag (e.g., `--from-file`) that allows users to read a list of
> namespaces and topics from a file for deletion. This would cater to
> situations where an arbitrary list of topics/namespaces needs to be
> deleted, providing a more comprehensive solution.
>
> Let me know if you have any further questions or suggestions.
>
> Best regards,
> Xiangying Meng
>
> On Tue, Apr 25, 2023 at 5:23 PM Zike Yang <zi...@apache.org> wrote:
> >
> > Hi, Xiangying
> >
> > > 1. I understand that one of your concerns is whether the proposed
> > > regex-based deletion feature would provide significant advantages over
> > > using a simple one-liner script to call the delete topic command for
> > > each topic. As Yubiao pointed out, using scripts to delete topics one
> > > by one can lead to increased network overhead and slow performance,
> > > particularly when dealing with a large number of topics. Implementing
> > > regex support for delete operations would provide a more efficient and
> > > convenient way to manage resources in Pulsar.
> >
> > From my understanding, introducing regex-based deletion feature
> > doesn't solve the case yubiao mentioned above. They are two different
> > cases.
> > Regex-based deletion can only delete topics of the same format, not an
> > arbitrary list of topics.
> >
> >
> > BR,
> > Zike Yang
> > On Tue, Apr 25, 2023 at 11:22 AM Xiangying Meng <xi...@apache.org>
> wrote:
> > >
> > > Hi Girish,
> > >
> > > Thank you for raising concerns about the proposed feature. I would
> > > like to address the points you mentioned in your email.
> > >
> > > 1. I understand that one of your concerns is whether the proposed
> > > regex-based deletion feature would provide significant advantages over
> > > using a simple one-liner script to call the delete topic command for
> > > each topic. As Yubiao pointed out, using scripts to delete topics one
> > > by one can lead to increased network overhead and slow performance,
> > > particularly when dealing with a large number of topics. Implementing
> > > regex support for delete operations would provide a more efficient and
> > > convenient way to manage resources in Pulsar.
> > >
> > > 2. In addition to the benefits for testing purposes, we have
> > > communicated with business users of Pulsar and found that the proposed
> > > regex-based deletion feature can be helpful in production environments
> > > as well. For instance, it can be used to efficiently clean up
> > > subscriptions associated with deprecated services, ensuring better
> > > resource management and reducing clutter in the system.
> > >
> > > 3. As I suggested earlier, we can introduce a new option flag (e.g.,
> > > `--regex` or `--pattern`) to the existing `pulsar-admin topics delete`
> > > command to prevent breaking changes for users who have already used
> > > the command in their scripts. This would ensure backward compatibility
> > > while providing the new functionality for those who want to use regex
> > > for deletion.
> > >
> > > I hope this clears up any confusion and addresses your concerns.
> > > Please let me know if you have any further questions or suggestions.
> > >
> > > Best regards,
> > > Xiangying Meng
> > >
> > > On Mon, Apr 24, 2023 at 6:23 PM Girish Sharma <sc...@gmail.com>
> wrote:
> > > >
> > > > Hello Yubiao,
> > > > As per my understanding, this feature suggestion is intended to
> delete the
> > > > topics from all replicated clusters under the namespace. Thus, the
> example
> > > > you are providing may not be a good fit for this?
> > > >
> > > > Xiangying, please clarify if my understanding is incorrect.
> > > >
> > > > On Mon, Apr 24, 2023 at 3:24 PM Yubiao Feng
> > > > <yu...@streamnative.io.invalid> wrote:
> > > >
> > > > > Hi Girish Sharma
> > > > >
> > > > > > What additional advantage would one get by using that approach
> > > > > > rather than simply using a one liner script to just call delete
> > > > > > topic for each of those topics if the list of topics is known.
> > > > >
> > > > > If users enabled `Geo-Replication` on a namespace in
> mistake(expected
> > > > > only to enable one topic),
> > > > > it is possible to create many topics on the remote cluster in one
> second.
> > > > >
> > > > > Not long ago, 10,000 topics were created per second because of this
> > > > > mistake. It took us a long time to
> > > > > remove these topics. We delete these topics in this way:
> > > > > ```
> > > > > cat topics_name_file | awk  '{system("bin/pulsar-admin topics
> delete "$0)}'
> > > > > )
> > > > > ```
> > > > > It deletes topics one by one.
> > > > >
> > > > > We conclude later that stress test tools such as `Jmeter` or `ab`
> should be
> > > > > used to delete so many topics.
> > > > >
> > > > > If Pulsar could provide these APIs, it would be better.
> > > > >
> > > > > Thanks
> > > > > Yubiao Feng
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Apr 19, 2023 at 3:29 PM Girish Sharma <
> scrapmachines@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hello Yubiao,
> > > > > >
> > > > > > What additional advantage would one get by using that approach
> rather
> > > > > than
> > > > > > simply using a one liner script to just call delete topic for
> each of
> > > > > those
> > > > > > topics if the list of topics is known.
> > > > > >
> > > > > > Regards
> > > > > >
> > > > > > On Wed, Apr 19, 2023 at 12:54 PM Yubiao Feng
> > > > > > <yu...@streamnative.io.invalid> wrote:
> > > > > >
> > > > > > > In addition to these two, It is recommended to add a method to
> batch
> > > > > > delete
> > > > > > > topics, such as this:
> > > > > > >
> > > > > > > ```
> > > > > > > pulsar-admin topics delete-all-topics <topic_1>, <topic_2>
> > > > > > >
> > > > > > > or
> > > > > > >
> > > > > > > pulsar-admin topics delete-all-topic <a file contains a
> topic_name
> > > > > lists>
> > > > > > > ```
> > > > > > >
> > > > > > > Thanks
> > > > > > > Yubiao Feng
> > > > > > >
> > > > > > > On Sat, Apr 15, 2023 at 5:37 PM Xiangying Meng <
> xiangying@apache.org>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Dear Apache Pulsar Community,
> > > > > > > >
> > > > > > > > I hope this email finds you well.I am writing to suggest a
> potential
> > > > > > > > improvement to the Pulsar-admin tool,
> > > > > > > >  which I believe could simplify the process of cleaning up
> tenants
> > > > > and
> > > > > > > > namespaces in Apache Pulsar.
> > > > > > > >
> > > > > > > > Currently, cleaning up all the namespaces and topics within
> a tenant
> > > > > or
> > > > > > > > cleaning up all the topics within a namespace requires
> several manual
> > > > > > > > steps,
> > > > > > > > such as listing the namespaces, listing the topics, and then
> deleting
> > > > > > > each
> > > > > > > > topic individually.
> > > > > > > > This process can be time-consuming and error-prone for users.
> > > > > > > >
> > > > > > > > To address this issue, I propose the addition of a "clear"
> parameter
> > > > > to
> > > > > > > the
> > > > > > > > Pulsar-admin tool,
> > > > > > > > which would automate the cleanup process for tenants and
> namespaces.
> > > > > > > Here's
> > > > > > > > a conceptual implementation:
> > > > > > > >
> > > > > > > > 1. To clean up all namespaces and topics within a tenant:
> > > > > > > > ``` bash
> > > > > > > > pulsar-admin tenants clear <tenant-name>
> > > > > > > > ```
> > > > > > > > 2. To clean up all topics within a namespace:
> > > > > > > > ```bash
> > > > > > > > pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> > > > > > > > ```
> > > > > > > >
> > > > > > > > By implementing these new parameters, users would be able to
> perform
> > > > > > > > cleanup operations more efficiently and with fewer manual
> steps.
> > > > > > > > I believe this improvement would greatly enhance the user
> experience
> > > > > > when
> > > > > > > > working with Apache Pulsar.
> > > > > > > >
> > > > > > > > I'd like to discuss the feasibility of this suggestion and
> gather
> > > > > > > feedback
> > > > > > > > from the community.
> > > > > > > > If everyone agrees, I can work on implementing this feature
> and
> > > > > submit
> > > > > > a
> > > > > > > > pull request for review.
> > > > > > > >
> > > > > > > > Looking forward to hearing your thoughts on this.
> > > > > > > >
> > > > > > > > Best regards,
> > > > > > > > Xiangying
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Girish Sharma
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Girish Sharma
>

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Xiangying Meng <xi...@apache.org>]

Hi Girish and Zike,

@Girish,
Regarding your question about the scope of the regex-based deletion
feature, the initial proposal is to delete the specified topics or
namespaces only in the local cluster where the command is run. If you
need to delete them in remote clusters as well, you can simply run the
command again in the respective clusters.

@Zike,
You raised a valid point about the regex-based deletion feature not
covering the case Yubiao mentioned. To address this, we can introduce
another flag (e.g., `--from-file`) that allows users to read a list of
namespaces and topics from a file for deletion. This would cater to
situations where an arbitrary list of topics/namespaces needs to be
deleted, providing a more comprehensive solution.

Let me know if you have any further questions or suggestions.

Best regards,
Xiangying Meng

On Tue, Apr 25, 2023 at 5:23 PM Zike Yang <zi...@apache.org> wrote:
>
> Hi, Xiangying
>
> > 1. I understand that one of your concerns is whether the proposed
> > regex-based deletion feature would provide significant advantages over
> > using a simple one-liner script to call the delete topic command for
> > each topic. As Yubiao pointed out, using scripts to delete topics one
> > by one can lead to increased network overhead and slow performance,
> > particularly when dealing with a large number of topics. Implementing
> > regex support for delete operations would provide a more efficient and
> > convenient way to manage resources in Pulsar.
>
> From my understanding, introducing regex-based deletion feature
> doesn't solve the case yubiao mentioned above. They are two different
> cases.
> Regex-based deletion can only delete topics of the same format, not an
> arbitrary list of topics.
>
>
> BR,
> Zike Yang
> On Tue, Apr 25, 2023 at 11:22 AM Xiangying Meng <xi...@apache.org> wrote:
> >
> > Hi Girish,
> >
> > Thank you for raising concerns about the proposed feature. I would
> > like to address the points you mentioned in your email.
> >
> > 1. I understand that one of your concerns is whether the proposed
> > regex-based deletion feature would provide significant advantages over
> > using a simple one-liner script to call the delete topic command for
> > each topic. As Yubiao pointed out, using scripts to delete topics one
> > by one can lead to increased network overhead and slow performance,
> > particularly when dealing with a large number of topics. Implementing
> > regex support for delete operations would provide a more efficient and
> > convenient way to manage resources in Pulsar.
> >
> > 2. In addition to the benefits for testing purposes, we have
> > communicated with business users of Pulsar and found that the proposed
> > regex-based deletion feature can be helpful in production environments
> > as well. For instance, it can be used to efficiently clean up
> > subscriptions associated with deprecated services, ensuring better
> > resource management and reducing clutter in the system.
> >
> > 3. As I suggested earlier, we can introduce a new option flag (e.g.,
> > `--regex` or `--pattern`) to the existing `pulsar-admin topics delete`
> > command to prevent breaking changes for users who have already used
> > the command in their scripts. This would ensure backward compatibility
> > while providing the new functionality for those who want to use regex
> > for deletion.
> >
> > I hope this clears up any confusion and addresses your concerns.
> > Please let me know if you have any further questions or suggestions.
> >
> > Best regards,
> > Xiangying Meng
> >
> > On Mon, Apr 24, 2023 at 6:23 PM Girish Sharma <sc...@gmail.com> wrote:
> > >
> > > Hello Yubiao,
> > > As per my understanding, this feature suggestion is intended to delete the
> > > topics from all replicated clusters under the namespace. Thus, the example
> > > you are providing may not be a good fit for this?
> > >
> > > Xiangying, please clarify if my understanding is incorrect.
> > >
> > > On Mon, Apr 24, 2023 at 3:24 PM Yubiao Feng
> > > <yu...@streamnative.io.invalid> wrote:
> > >
> > > > Hi Girish Sharma
> > > >
> > > > > What additional advantage would one get by using that approach
> > > > > rather than simply using a one liner script to just call delete
> > > > > topic for each of those topics if the list of topics is known.
> > > >
> > > > If users enabled `Geo-Replication` on a namespace in mistake(expected
> > > > only to enable one topic),
> > > > it is possible to create many topics on the remote cluster in one second.
> > > >
> > > > Not long ago, 10,000 topics were created per second because of this
> > > > mistake. It took us a long time to
> > > > remove these topics. We delete these topics in this way:
> > > > ```
> > > > cat topics_name_file | awk  '{system("bin/pulsar-admin topics delete "$0)}'
> > > > )
> > > > ```
> > > > It deletes topics one by one.
> > > >
> > > > We conclude later that stress test tools such as `Jmeter` or `ab` should be
> > > > used to delete so many topics.
> > > >
> > > > If Pulsar could provide these APIs, it would be better.
> > > >
> > > > Thanks
> > > > Yubiao Feng
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, Apr 19, 2023 at 3:29 PM Girish Sharma <sc...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hello Yubiao,
> > > > >
> > > > > What additional advantage would one get by using that approach rather
> > > > than
> > > > > simply using a one liner script to just call delete topic for each of
> > > > those
> > > > > topics if the list of topics is known.
> > > > >
> > > > > Regards
> > > > >
> > > > > On Wed, Apr 19, 2023 at 12:54 PM Yubiao Feng
> > > > > <yu...@streamnative.io.invalid> wrote:
> > > > >
> > > > > > In addition to these two, It is recommended to add a method to batch
> > > > > delete
> > > > > > topics, such as this:
> > > > > >
> > > > > > ```
> > > > > > pulsar-admin topics delete-all-topics <topic_1>, <topic_2>
> > > > > >
> > > > > > or
> > > > > >
> > > > > > pulsar-admin topics delete-all-topic <a file contains a topic_name
> > > > lists>
> > > > > > ```
> > > > > >
> > > > > > Thanks
> > > > > > Yubiao Feng
> > > > > >
> > > > > > On Sat, Apr 15, 2023 at 5:37 PM Xiangying Meng <xi...@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > Dear Apache Pulsar Community,
> > > > > > >
> > > > > > > I hope this email finds you well.I am writing to suggest a potential
> > > > > > > improvement to the Pulsar-admin tool,
> > > > > > >  which I believe could simplify the process of cleaning up tenants
> > > > and
> > > > > > > namespaces in Apache Pulsar.
> > > > > > >
> > > > > > > Currently, cleaning up all the namespaces and topics within a tenant
> > > > or
> > > > > > > cleaning up all the topics within a namespace requires several manual
> > > > > > > steps,
> > > > > > > such as listing the namespaces, listing the topics, and then deleting
> > > > > > each
> > > > > > > topic individually.
> > > > > > > This process can be time-consuming and error-prone for users.
> > > > > > >
> > > > > > > To address this issue, I propose the addition of a "clear" parameter
> > > > to
> > > > > > the
> > > > > > > Pulsar-admin tool,
> > > > > > > which would automate the cleanup process for tenants and namespaces.
> > > > > > Here's
> > > > > > > a conceptual implementation:
> > > > > > >
> > > > > > > 1. To clean up all namespaces and topics within a tenant:
> > > > > > > ``` bash
> > > > > > > pulsar-admin tenants clear <tenant-name>
> > > > > > > ```
> > > > > > > 2. To clean up all topics within a namespace:
> > > > > > > ```bash
> > > > > > > pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> > > > > > > ```
> > > > > > >
> > > > > > > By implementing these new parameters, users would be able to perform
> > > > > > > cleanup operations more efficiently and with fewer manual steps.
> > > > > > > I believe this improvement would greatly enhance the user experience
> > > > > when
> > > > > > > working with Apache Pulsar.
> > > > > > >
> > > > > > > I'd like to discuss the feasibility of this suggestion and gather
> > > > > > feedback
> > > > > > > from the community.
> > > > > > > If everyone agrees, I can work on implementing this feature and
> > > > submit
> > > > > a
> > > > > > > pull request for review.
> > > > > > >
> > > > > > > Looking forward to hearing your thoughts on this.
> > > > > > >
> > > > > > > Best regards,
> > > > > > > Xiangying
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Girish Sharma
> > > > >
> > > >
> > >
> > >
> > > --
> > > Girish Sharma

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Zike Yang <zi...@apache.org>]

Hi, Xiangying

> 1. I understand that one of your concerns is whether the proposed
> regex-based deletion feature would provide significant advantages over
> using a simple one-liner script to call the delete topic command for
> each topic. As Yubiao pointed out, using scripts to delete topics one
> by one can lead to increased network overhead and slow performance,
> particularly when dealing with a large number of topics. Implementing
> regex support for delete operations would provide a more efficient and
> convenient way to manage resources in Pulsar.

From my understanding, introducing regex-based deletion feature
doesn't solve the case yubiao mentioned above. They are two different
cases.
Regex-based deletion can only delete topics of the same format, not an
arbitrary list of topics.


BR,
Zike Yang
On Tue, Apr 25, 2023 at 11:22 AM Xiangying Meng <xi...@apache.org> wrote:
>
> Hi Girish,
>
> Thank you for raising concerns about the proposed feature. I would
> like to address the points you mentioned in your email.
>
> 1. I understand that one of your concerns is whether the proposed
> regex-based deletion feature would provide significant advantages over
> using a simple one-liner script to call the delete topic command for
> each topic. As Yubiao pointed out, using scripts to delete topics one
> by one can lead to increased network overhead and slow performance,
> particularly when dealing with a large number of topics. Implementing
> regex support for delete operations would provide a more efficient and
> convenient way to manage resources in Pulsar.
>
> 2. In addition to the benefits for testing purposes, we have
> communicated with business users of Pulsar and found that the proposed
> regex-based deletion feature can be helpful in production environments
> as well. For instance, it can be used to efficiently clean up
> subscriptions associated with deprecated services, ensuring better
> resource management and reducing clutter in the system.
>
> 3. As I suggested earlier, we can introduce a new option flag (e.g.,
> `--regex` or `--pattern`) to the existing `pulsar-admin topics delete`
> command to prevent breaking changes for users who have already used
> the command in their scripts. This would ensure backward compatibility
> while providing the new functionality for those who want to use regex
> for deletion.
>
> I hope this clears up any confusion and addresses your concerns.
> Please let me know if you have any further questions or suggestions.
>
> Best regards,
> Xiangying Meng
>
> On Mon, Apr 24, 2023 at 6:23 PM Girish Sharma <sc...@gmail.com> wrote:
> >
> > Hello Yubiao,
> > As per my understanding, this feature suggestion is intended to delete the
> > topics from all replicated clusters under the namespace. Thus, the example
> > you are providing may not be a good fit for this?
> >
> > Xiangying, please clarify if my understanding is incorrect.
> >
> > On Mon, Apr 24, 2023 at 3:24 PM Yubiao Feng
> > <yu...@streamnative.io.invalid> wrote:
> >
> > > Hi Girish Sharma
> > >
> > > > What additional advantage would one get by using that approach
> > > > rather than simply using a one liner script to just call delete
> > > > topic for each of those topics if the list of topics is known.
> > >
> > > If users enabled `Geo-Replication` on a namespace in mistake(expected
> > > only to enable one topic),
> > > it is possible to create many topics on the remote cluster in one second.
> > >
> > > Not long ago, 10,000 topics were created per second because of this
> > > mistake. It took us a long time to
> > > remove these topics. We delete these topics in this way:
> > > ```
> > > cat topics_name_file | awk  '{system("bin/pulsar-admin topics delete "$0)}'
> > > )
> > > ```
> > > It deletes topics one by one.
> > >
> > > We conclude later that stress test tools such as `Jmeter` or `ab` should be
> > > used to delete so many topics.
> > >
> > > If Pulsar could provide these APIs, it would be better.
> > >
> > > Thanks
> > > Yubiao Feng
> > >
> > >
> > >
> > >
> > > On Wed, Apr 19, 2023 at 3:29 PM Girish Sharma <sc...@gmail.com>
> > > wrote:
> > >
> > > > Hello Yubiao,
> > > >
> > > > What additional advantage would one get by using that approach rather
> > > than
> > > > simply using a one liner script to just call delete topic for each of
> > > those
> > > > topics if the list of topics is known.
> > > >
> > > > Regards
> > > >
> > > > On Wed, Apr 19, 2023 at 12:54 PM Yubiao Feng
> > > > <yu...@streamnative.io.invalid> wrote:
> > > >
> > > > > In addition to these two, It is recommended to add a method to batch
> > > > delete
> > > > > topics, such as this:
> > > > >
> > > > > ```
> > > > > pulsar-admin topics delete-all-topics <topic_1>, <topic_2>
> > > > >
> > > > > or
> > > > >
> > > > > pulsar-admin topics delete-all-topic <a file contains a topic_name
> > > lists>
> > > > > ```
> > > > >
> > > > > Thanks
> > > > > Yubiao Feng
> > > > >
> > > > > On Sat, Apr 15, 2023 at 5:37 PM Xiangying Meng <xi...@apache.org>
> > > > > wrote:
> > > > >
> > > > > > Dear Apache Pulsar Community,
> > > > > >
> > > > > > I hope this email finds you well.I am writing to suggest a potential
> > > > > > improvement to the Pulsar-admin tool,
> > > > > >  which I believe could simplify the process of cleaning up tenants
> > > and
> > > > > > namespaces in Apache Pulsar.
> > > > > >
> > > > > > Currently, cleaning up all the namespaces and topics within a tenant
> > > or
> > > > > > cleaning up all the topics within a namespace requires several manual
> > > > > > steps,
> > > > > > such as listing the namespaces, listing the topics, and then deleting
> > > > > each
> > > > > > topic individually.
> > > > > > This process can be time-consuming and error-prone for users.
> > > > > >
> > > > > > To address this issue, I propose the addition of a "clear" parameter
> > > to
> > > > > the
> > > > > > Pulsar-admin tool,
> > > > > > which would automate the cleanup process for tenants and namespaces.
> > > > > Here's
> > > > > > a conceptual implementation:
> > > > > >
> > > > > > 1. To clean up all namespaces and topics within a tenant:
> > > > > > ``` bash
> > > > > > pulsar-admin tenants clear <tenant-name>
> > > > > > ```
> > > > > > 2. To clean up all topics within a namespace:
> > > > > > ```bash
> > > > > > pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> > > > > > ```
> > > > > >
> > > > > > By implementing these new parameters, users would be able to perform
> > > > > > cleanup operations more efficiently and with fewer manual steps.
> > > > > > I believe this improvement would greatly enhance the user experience
> > > > when
> > > > > > working with Apache Pulsar.
> > > > > >
> > > > > > I'd like to discuss the feasibility of this suggestion and gather
> > > > > feedback
> > > > > > from the community.
> > > > > > If everyone agrees, I can work on implementing this feature and
> > > submit
> > > > a
> > > > > > pull request for review.
> > > > > >
> > > > > > Looking forward to hearing your thoughts on this.
> > > > > >
> > > > > > Best regards,
> > > > > > Xiangying
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Girish Sharma
> > > >
> > >
> >
> >
> > --
> > Girish Sharma

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Xiangying Meng <xi...@apache.org>]

Hi Girish,

Thank you for raising concerns about the proposed feature. I would
like to address the points you mentioned in your email.

1. I understand that one of your concerns is whether the proposed
regex-based deletion feature would provide significant advantages over
using a simple one-liner script to call the delete topic command for
each topic. As Yubiao pointed out, using scripts to delete topics one
by one can lead to increased network overhead and slow performance,
particularly when dealing with a large number of topics. Implementing
regex support for delete operations would provide a more efficient and
convenient way to manage resources in Pulsar.

2. In addition to the benefits for testing purposes, we have
communicated with business users of Pulsar and found that the proposed
regex-based deletion feature can be helpful in production environments
as well. For instance, it can be used to efficiently clean up
subscriptions associated with deprecated services, ensuring better
resource management and reducing clutter in the system.

3. As I suggested earlier, we can introduce a new option flag (e.g.,
`--regex` or `--pattern`) to the existing `pulsar-admin topics delete`
command to prevent breaking changes for users who have already used
the command in their scripts. This would ensure backward compatibility
while providing the new functionality for those who want to use regex
for deletion.

I hope this clears up any confusion and addresses your concerns.
Please let me know if you have any further questions or suggestions.

Best regards,
Xiangying Meng

On Mon, Apr 24, 2023 at 6:23 PM Girish Sharma <sc...@gmail.com> wrote:
>
> Hello Yubiao,
> As per my understanding, this feature suggestion is intended to delete the
> topics from all replicated clusters under the namespace. Thus, the example
> you are providing may not be a good fit for this?
>
> Xiangying, please clarify if my understanding is incorrect.
>
> On Mon, Apr 24, 2023 at 3:24 PM Yubiao Feng
> <yu...@streamnative.io.invalid> wrote:
>
> > Hi Girish Sharma
> >
> > > What additional advantage would one get by using that approach
> > > rather than simply using a one liner script to just call delete
> > > topic for each of those topics if the list of topics is known.
> >
> > If users enabled `Geo-Replication` on a namespace in mistake(expected
> > only to enable one topic),
> > it is possible to create many topics on the remote cluster in one second.
> >
> > Not long ago, 10,000 topics were created per second because of this
> > mistake. It took us a long time to
> > remove these topics. We delete these topics in this way:
> > ```
> > cat topics_name_file | awk  '{system("bin/pulsar-admin topics delete "$0)}'
> > )
> > ```
> > It deletes topics one by one.
> >
> > We conclude later that stress test tools such as `Jmeter` or `ab` should be
> > used to delete so many topics.
> >
> > If Pulsar could provide these APIs, it would be better.
> >
> > Thanks
> > Yubiao Feng
> >
> >
> >
> >
> > On Wed, Apr 19, 2023 at 3:29 PM Girish Sharma <sc...@gmail.com>
> > wrote:
> >
> > > Hello Yubiao,
> > >
> > > What additional advantage would one get by using that approach rather
> > than
> > > simply using a one liner script to just call delete topic for each of
> > those
> > > topics if the list of topics is known.
> > >
> > > Regards
> > >
> > > On Wed, Apr 19, 2023 at 12:54 PM Yubiao Feng
> > > <yu...@streamnative.io.invalid> wrote:
> > >
> > > > In addition to these two, It is recommended to add a method to batch
> > > delete
> > > > topics, such as this:
> > > >
> > > > ```
> > > > pulsar-admin topics delete-all-topics <topic_1>, <topic_2>
> > > >
> > > > or
> > > >
> > > > pulsar-admin topics delete-all-topic <a file contains a topic_name
> > lists>
> > > > ```
> > > >
> > > > Thanks
> > > > Yubiao Feng
> > > >
> > > > On Sat, Apr 15, 2023 at 5:37 PM Xiangying Meng <xi...@apache.org>
> > > > wrote:
> > > >
> > > > > Dear Apache Pulsar Community,
> > > > >
> > > > > I hope this email finds you well.I am writing to suggest a potential
> > > > > improvement to the Pulsar-admin tool,
> > > > >  which I believe could simplify the process of cleaning up tenants
> > and
> > > > > namespaces in Apache Pulsar.
> > > > >
> > > > > Currently, cleaning up all the namespaces and topics within a tenant
> > or
> > > > > cleaning up all the topics within a namespace requires several manual
> > > > > steps,
> > > > > such as listing the namespaces, listing the topics, and then deleting
> > > > each
> > > > > topic individually.
> > > > > This process can be time-consuming and error-prone for users.
> > > > >
> > > > > To address this issue, I propose the addition of a "clear" parameter
> > to
> > > > the
> > > > > Pulsar-admin tool,
> > > > > which would automate the cleanup process for tenants and namespaces.
> > > > Here's
> > > > > a conceptual implementation:
> > > > >
> > > > > 1. To clean up all namespaces and topics within a tenant:
> > > > > ``` bash
> > > > > pulsar-admin tenants clear <tenant-name>
> > > > > ```
> > > > > 2. To clean up all topics within a namespace:
> > > > > ```bash
> > > > > pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> > > > > ```
> > > > >
> > > > > By implementing these new parameters, users would be able to perform
> > > > > cleanup operations more efficiently and with fewer manual steps.
> > > > > I believe this improvement would greatly enhance the user experience
> > > when
> > > > > working with Apache Pulsar.
> > > > >
> > > > > I'd like to discuss the feasibility of this suggestion and gather
> > > > feedback
> > > > > from the community.
> > > > > If everyone agrees, I can work on implementing this feature and
> > submit
> > > a
> > > > > pull request for review.
> > > > >
> > > > > Looking forward to hearing your thoughts on this.
> > > > >
> > > > > Best regards,
> > > > > Xiangying
> > > > >
> > > >
> > >
> > >
> > > --
> > > Girish Sharma
> > >
> >
>
>
> --
> Girish Sharma

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Girish Sharma <sc...@gmail.com>]

Hello Yubiao,
As per my understanding, this feature suggestion is intended to delete the
topics from all replicated clusters under the namespace. Thus, the example
you are providing may not be a good fit for this?

Xiangying, please clarify if my understanding is incorrect.

On Mon, Apr 24, 2023 at 3:24 PM Yubiao Feng
<yu...@streamnative.io.invalid> wrote:

> Hi Girish Sharma
>
> > What additional advantage would one get by using that approach
> > rather than simply using a one liner script to just call delete
> > topic for each of those topics if the list of topics is known.
>
> If users enabled `Geo-Replication` on a namespace in mistake(expected
> only to enable one topic),
> it is possible to create many topics on the remote cluster in one second.
>
> Not long ago, 10,000 topics were created per second because of this
> mistake. It took us a long time to
> remove these topics. We delete these topics in this way:
> ```
> cat topics_name_file | awk  '{system("bin/pulsar-admin topics delete "$0)}'
> )
> ```
> It deletes topics one by one.
>
> We conclude later that stress test tools such as `Jmeter` or `ab` should be
> used to delete so many topics.
>
> If Pulsar could provide these APIs, it would be better.
>
> Thanks
> Yubiao Feng
>
>
>
>
> On Wed, Apr 19, 2023 at 3:29 PM Girish Sharma <sc...@gmail.com>
> wrote:
>
> > Hello Yubiao,
> >
> > What additional advantage would one get by using that approach rather
> than
> > simply using a one liner script to just call delete topic for each of
> those
> > topics if the list of topics is known.
> >
> > Regards
> >
> > On Wed, Apr 19, 2023 at 12:54 PM Yubiao Feng
> > <yu...@streamnative.io.invalid> wrote:
> >
> > > In addition to these two, It is recommended to add a method to batch
> > delete
> > > topics, such as this:
> > >
> > > ```
> > > pulsar-admin topics delete-all-topics <topic_1>, <topic_2>
> > >
> > > or
> > >
> > > pulsar-admin topics delete-all-topic <a file contains a topic_name
> lists>
> > > ```
> > >
> > > Thanks
> > > Yubiao Feng
> > >
> > > On Sat, Apr 15, 2023 at 5:37 PM Xiangying Meng <xi...@apache.org>
> > > wrote:
> > >
> > > > Dear Apache Pulsar Community,
> > > >
> > > > I hope this email finds you well.I am writing to suggest a potential
> > > > improvement to the Pulsar-admin tool,
> > > >  which I believe could simplify the process of cleaning up tenants
> and
> > > > namespaces in Apache Pulsar.
> > > >
> > > > Currently, cleaning up all the namespaces and topics within a tenant
> or
> > > > cleaning up all the topics within a namespace requires several manual
> > > > steps,
> > > > such as listing the namespaces, listing the topics, and then deleting
> > > each
> > > > topic individually.
> > > > This process can be time-consuming and error-prone for users.
> > > >
> > > > To address this issue, I propose the addition of a "clear" parameter
> to
> > > the
> > > > Pulsar-admin tool,
> > > > which would automate the cleanup process for tenants and namespaces.
> > > Here's
> > > > a conceptual implementation:
> > > >
> > > > 1. To clean up all namespaces and topics within a tenant:
> > > > ``` bash
> > > > pulsar-admin tenants clear <tenant-name>
> > > > ```
> > > > 2. To clean up all topics within a namespace:
> > > > ```bash
> > > > pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> > > > ```
> > > >
> > > > By implementing these new parameters, users would be able to perform
> > > > cleanup operations more efficiently and with fewer manual steps.
> > > > I believe this improvement would greatly enhance the user experience
> > when
> > > > working with Apache Pulsar.
> > > >
> > > > I'd like to discuss the feasibility of this suggestion and gather
> > > feedback
> > > > from the community.
> > > > If everyone agrees, I can work on implementing this feature and
> submit
> > a
> > > > pull request for review.
> > > >
> > > > Looking forward to hearing your thoughts on this.
> > > >
> > > > Best regards,
> > > > Xiangying
> > > >
> > >
> >
> >
> > --
> > Girish Sharma
> >
>


-- 
Girish Sharma

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Zike Yang <zi...@apache.org>]

Hi, Yubiao,

> It took us a long time to
> remove these topics. We delete these topics in this way:
> ```
> cat topics_name_file | awk  '{system("bin/pulsar-admin topics delete "$0)}'
> )
> ```
> It deletes topics one by one.

The main overhead here is the frequent creation and release of the JVM.
Maybe we can use `pulsar-shell` here. It will save a lot of time.

Zike Yang




Zike Yang

On Mon, Apr 24, 2023 at 5:54 PM Yubiao Feng
<yu...@streamnative.io.invalid> wrote:
>
> Hi Girish Sharma
>
> > What additional advantage would one get by using that approach
> > rather than simply using a one liner script to just call delete
> > topic for each of those topics if the list of topics is known.
>
> If users enabled `Geo-Replication` on a namespace in mistake(expected
> only to enable one topic),
> it is possible to create many topics on the remote cluster in one second.
>
> Not long ago, 10,000 topics were created per second because of this
> mistake. It took us a long time to
> remove these topics. We delete these topics in this way:
> ```
> cat topics_name_file | awk  '{system("bin/pulsar-admin topics delete "$0)}'
> )
> ```
> It deletes topics one by one.
>
> We conclude later that stress test tools such as `Jmeter` or `ab` should be
> used to delete so many topics.
>
> If Pulsar could provide these APIs, it would be better.
>
> Thanks
> Yubiao Feng
>
>
>
>
> On Wed, Apr 19, 2023 at 3:29 PM Girish Sharma <sc...@gmail.com>
> wrote:
>
> > Hello Yubiao,
> >
> > What additional advantage would one get by using that approach rather than
> > simply using a one liner script to just call delete topic for each of those
> > topics if the list of topics is known.
> >
> > Regards
> >
> > On Wed, Apr 19, 2023 at 12:54 PM Yubiao Feng
> > <yu...@streamnative.io.invalid> wrote:
> >
> > > In addition to these two, It is recommended to add a method to batch
> > delete
> > > topics, such as this:
> > >
> > > ```
> > > pulsar-admin topics delete-all-topics <topic_1>, <topic_2>
> > >
> > > or
> > >
> > > pulsar-admin topics delete-all-topic <a file contains a topic_name lists>
> > > ```
> > >
> > > Thanks
> > > Yubiao Feng
> > >
> > > On Sat, Apr 15, 2023 at 5:37 PM Xiangying Meng <xi...@apache.org>
> > > wrote:
> > >
> > > > Dear Apache Pulsar Community,
> > > >
> > > > I hope this email finds you well.I am writing to suggest a potential
> > > > improvement to the Pulsar-admin tool,
> > > >  which I believe could simplify the process of cleaning up tenants and
> > > > namespaces in Apache Pulsar.
> > > >
> > > > Currently, cleaning up all the namespaces and topics within a tenant or
> > > > cleaning up all the topics within a namespace requires several manual
> > > > steps,
> > > > such as listing the namespaces, listing the topics, and then deleting
> > > each
> > > > topic individually.
> > > > This process can be time-consuming and error-prone for users.
> > > >
> > > > To address this issue, I propose the addition of a "clear" parameter to
> > > the
> > > > Pulsar-admin tool,
> > > > which would automate the cleanup process for tenants and namespaces.
> > > Here's
> > > > a conceptual implementation:
> > > >
> > > > 1. To clean up all namespaces and topics within a tenant:
> > > > ``` bash
> > > > pulsar-admin tenants clear <tenant-name>
> > > > ```
> > > > 2. To clean up all topics within a namespace:
> > > > ```bash
> > > > pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> > > > ```
> > > >
> > > > By implementing these new parameters, users would be able to perform
> > > > cleanup operations more efficiently and with fewer manual steps.
> > > > I believe this improvement would greatly enhance the user experience
> > when
> > > > working with Apache Pulsar.
> > > >
> > > > I'd like to discuss the feasibility of this suggestion and gather
> > > feedback
> > > > from the community.
> > > > If everyone agrees, I can work on implementing this feature and submit
> > a
> > > > pull request for review.
> > > >
> > > > Looking forward to hearing your thoughts on this.
> > > >
> > > > Best regards,
> > > > Xiangying
> > > >
> > >
> >
> >
> > --
> > Girish Sharma
> >

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Yubiao Feng <yu...@streamnative.io.INVALID>]

Hi Girish Sharma

> What additional advantage would one get by using that approach
> rather than simply using a one liner script to just call delete
> topic for each of those topics if the list of topics is known.

If users enabled `Geo-Replication` on a namespace in mistake(expected
only to enable one topic),
it is possible to create many topics on the remote cluster in one second.

Not long ago, 10,000 topics were created per second because of this
mistake. It took us a long time to
remove these topics. We delete these topics in this way:
```
cat topics_name_file | awk  '{system("bin/pulsar-admin topics delete "$0)}'
)
```
It deletes topics one by one.

We conclude later that stress test tools such as `Jmeter` or `ab` should be
used to delete so many topics.

If Pulsar could provide these APIs, it would be better.

Thanks
Yubiao Feng




On Wed, Apr 19, 2023 at 3:29 PM Girish Sharma <sc...@gmail.com>
wrote:

> Hello Yubiao,
>
> What additional advantage would one get by using that approach rather than
> simply using a one liner script to just call delete topic for each of those
> topics if the list of topics is known.
>
> Regards
>
> On Wed, Apr 19, 2023 at 12:54 PM Yubiao Feng
> <yu...@streamnative.io.invalid> wrote:
>
> > In addition to these two, It is recommended to add a method to batch
> delete
> > topics, such as this:
> >
> > ```
> > pulsar-admin topics delete-all-topics <topic_1>, <topic_2>
> >
> > or
> >
> > pulsar-admin topics delete-all-topic <a file contains a topic_name lists>
> > ```
> >
> > Thanks
> > Yubiao Feng
> >
> > On Sat, Apr 15, 2023 at 5:37 PM Xiangying Meng <xi...@apache.org>
> > wrote:
> >
> > > Dear Apache Pulsar Community,
> > >
> > > I hope this email finds you well.I am writing to suggest a potential
> > > improvement to the Pulsar-admin tool,
> > >  which I believe could simplify the process of cleaning up tenants and
> > > namespaces in Apache Pulsar.
> > >
> > > Currently, cleaning up all the namespaces and topics within a tenant or
> > > cleaning up all the topics within a namespace requires several manual
> > > steps,
> > > such as listing the namespaces, listing the topics, and then deleting
> > each
> > > topic individually.
> > > This process can be time-consuming and error-prone for users.
> > >
> > > To address this issue, I propose the addition of a "clear" parameter to
> > the
> > > Pulsar-admin tool,
> > > which would automate the cleanup process for tenants and namespaces.
> > Here's
> > > a conceptual implementation:
> > >
> > > 1. To clean up all namespaces and topics within a tenant:
> > > ``` bash
> > > pulsar-admin tenants clear <tenant-name>
> > > ```
> > > 2. To clean up all topics within a namespace:
> > > ```bash
> > > pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> > > ```
> > >
> > > By implementing these new parameters, users would be able to perform
> > > cleanup operations more efficiently and with fewer manual steps.
> > > I believe this improvement would greatly enhance the user experience
> when
> > > working with Apache Pulsar.
> > >
> > > I'd like to discuss the feasibility of this suggestion and gather
> > feedback
> > > from the community.
> > > If everyone agrees, I can work on implementing this feature and submit
> a
> > > pull request for review.
> > >
> > > Looking forward to hearing your thoughts on this.
> > >
> > > Best regards,
> > > Xiangying
> > >
> >
>
>
> --
> Girish Sharma
>

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Girish Sharma <sc...@gmail.com>]

Hello Yubiao,

What additional advantage would one get by using that approach rather than
simply using a one liner script to just call delete topic for each of those
topics if the list of topics is known.

Regards

On Wed, Apr 19, 2023 at 12:54 PM Yubiao Feng
<yu...@streamnative.io.invalid> wrote:

> In addition to these two, It is recommended to add a method to batch delete
> topics, such as this:
>
> ```
> pulsar-admin topics delete-all-topics <topic_1>, <topic_2>
>
> or
>
> pulsar-admin topics delete-all-topic <a file contains a topic_name lists>
> ```
>
> Thanks
> Yubiao Feng
>
> On Sat, Apr 15, 2023 at 5:37 PM Xiangying Meng <xi...@apache.org>
> wrote:
>
> > Dear Apache Pulsar Community,
> >
> > I hope this email finds you well.I am writing to suggest a potential
> > improvement to the Pulsar-admin tool,
> >  which I believe could simplify the process of cleaning up tenants and
> > namespaces in Apache Pulsar.
> >
> > Currently, cleaning up all the namespaces and topics within a tenant or
> > cleaning up all the topics within a namespace requires several manual
> > steps,
> > such as listing the namespaces, listing the topics, and then deleting
> each
> > topic individually.
> > This process can be time-consuming and error-prone for users.
> >
> > To address this issue, I propose the addition of a "clear" parameter to
> the
> > Pulsar-admin tool,
> > which would automate the cleanup process for tenants and namespaces.
> Here's
> > a conceptual implementation:
> >
> > 1. To clean up all namespaces and topics within a tenant:
> > ``` bash
> > pulsar-admin tenants clear <tenant-name>
> > ```
> > 2. To clean up all topics within a namespace:
> > ```bash
> > pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> > ```
> >
> > By implementing these new parameters, users would be able to perform
> > cleanup operations more efficiently and with fewer manual steps.
> > I believe this improvement would greatly enhance the user experience when
> > working with Apache Pulsar.
> >
> > I'd like to discuss the feasibility of this suggestion and gather
> feedback
> > from the community.
> > If everyone agrees, I can work on implementing this feature and submit a
> > pull request for review.
> >
> > Looking forward to hearing your thoughts on this.
> >
> > Best regards,
> > Xiangying
> >
>


-- 
Girish Sharma

Re: [Discuss] Suggestion for a "clear" parameter in Pulsar-admin to simplify tenant and namespace cleanup

[Yubiao Feng <yu...@streamnative.io.INVALID>]

In addition to these two, It is recommended to add a method to batch delete
topics, such as this:

```
pulsar-admin topics delete-all-topics <topic_1>, <topic_2>

or

pulsar-admin topics delete-all-topic <a file contains a topic_name lists>
```

Thanks
Yubiao Feng

On Sat, Apr 15, 2023 at 5:37 PM Xiangying Meng <xi...@apache.org> wrote:

> Dear Apache Pulsar Community,
>
> I hope this email finds you well.I am writing to suggest a potential
> improvement to the Pulsar-admin tool,
>  which I believe could simplify the process of cleaning up tenants and
> namespaces in Apache Pulsar.
>
> Currently, cleaning up all the namespaces and topics within a tenant or
> cleaning up all the topics within a namespace requires several manual
> steps,
> such as listing the namespaces, listing the topics, and then deleting each
> topic individually.
> This process can be time-consuming and error-prone for users.
>
> To address this issue, I propose the addition of a "clear" parameter to the
> Pulsar-admin tool,
> which would automate the cleanup process for tenants and namespaces. Here's
> a conceptual implementation:
>
> 1. To clean up all namespaces and topics within a tenant:
> ``` bash
> pulsar-admin tenants clear <tenant-name>
> ```
> 2. To clean up all topics within a namespace:
> ```bash
> pulsar-admin namespaces clear <tenant-name>/<namespace-name>
> ```
>
> By implementing these new parameters, users would be able to perform
> cleanup operations more efficiently and with fewer manual steps.
> I believe this improvement would greatly enhance the user experience when
> working with Apache Pulsar.
>
> I'd like to discuss the feasibility of this suggestion and gather feedback
> from the community.
> If everyone agrees, I can work on implementing this feature and submit a
> pull request for review.
>
> Looking forward to hearing your thoughts on this.
>
> Best regards,
> Xiangying
>