Re: Introducer Pulsar admin api to pulsar-client-go

[ZhangJian He <sh...@gmail.com>]

ping @PengHuiLi. Please let me know your thoughts on this matter.

Thanks
ZhangJian He


On Tue, 28 Feb 2023 at 20:24, Enrico Olivelli <eo...@gmail.com> wrote:

> Il giorno mar 28 feb 2023 alle ore 10:40 ZhangJian He
> <sh...@gmail.com> ha scritto:
> >
> > ping @PengHuiLi @Enrico Olivelli
> >
> > Thanks
> > ZhangJian He
> >
> >
> > On Mon, 27 Feb 2023 at 14:02, ZhangJian He <sh...@gmail.com> wrote:
> >>
> >> > Another idea is whether it is possible to generate clients for various
> >> languages based on the current pulsar rest swagger files
> >> Yes. But I don't think this is a suitable choice for the go client. Due
> to the existing swagger file not fully complying with the specification, it
> cannot be used directly to generate client code. Personally, I don't think
> fixing it is particularly easy and there is currently no one working on it.
> >> For the go client, I would still recommend creating a new repository
> and having everyone contribute to it from there. WDYT?
>
> I agree with this approach
>
> Enrico
>
> >>
> >> Also, I would appreciate it if others could let me know your thoughts
> on this matter. @PengHuiLi @Enrico Olivelli
> >>
> >> Thanks
> >> ZhangJian He
> >>
> >>
> >> On Sun, 26 Feb 2023 at 21:04, Guangning E <eg...@gmail.com> wrote:
> >>>
> >>> Another idea is whether it is possible to generate clients for various
> >>> languages based on the current pulsar rest swagger files?
> >>>
> >>> Thanks,
> >>> Guangning
> >>>
> >>> On Sat, Feb 25, 2023 at 1:44 PM Zixuan Liu <no...@gmail.com> wrote:
> >>>
> >>> > > I would like to suggest creating a new repository called
> >>> > "pulsar-admin-go"
> >>> >
> >>> > +1, I agreed with you.
> >>> >
> >>> > Thanks,
> >>> > Zixuan
> >>> >
> >>> > ZhangJian He <sh...@gmail.com> 于2023年2月25日周六 11:05写道：
> >>> >
> >>> > > Following our previous discussions via email, I would like to
> suggest
> >>> > > creating a new repository called "pulsar-admin-go". This
> repository will
> >>> > > serve as a platform for all of us to contribute our code.
> Pulsarctl[0] go
> >>> > > first.
> >>> > >
> >>> > > Please let me know your thoughts on this matter.
> >>> > >
> >>> > > [0] - https://github.com/streamnative/pulsarctl
> >>> > >
> >>> > > Thanks
> >>> > > ZhangJian He
> >>> > >
> >>> > >
> >>> > > On Fri, 17 Feb 2023 at 23:24, ZhangJian He <sh...@gmail.com>
> wrote:
> >>> > >
> >>> > > > Thank for StreamNative for willing to donate this project. This
> means
> >>> > we
> >>> > > > don't have to develop and maintain a set of HTTP code from
> scratch. My
> >>> > > idea
> >>> > > > aligns with Yunze's, and separating it into a standalone
> >>> > pulsar-admin-go
> >>> > > > project would be better. The **pulsarctl** repo contains
> bookkeeper
> >>> > http
> >>> > > > call too. Maybe we can have a project bookkeeper-admin-go ?（it's
> a
> >>> > liitle
> >>> > > > going off-topic ）
> >>> > > >
> >>> > > > Thanks
> >>> > > > ZhangJian He
> >>> > > >
> >>> > > >
> >>> > > > On Fri, 17 Feb 2023 at 20:29, PengHui Li <
> codelipenghui@gmail.com>
> >>> > > wrote:
> >>> > > >
> >>> > > >> Hi Yunze,
> >>> > > >>
> >>> > > >> Yes, we can split it.
> >>> > > >> Both one repo with two modules or two repos works for me.
> >>> > > >>
> >>> > > >> The pulsarctl already have the admin API and CLI.
> >>> > > >> So I think we don’t need to develop another one.
> >>> > > >>
> >>> > > >> Best,
> >>> > > >> Penghui
> >>> > > >>
> >>> > > >> > On Feb 17, 2023, at 17:44, Yunze Xu
> <yz...@streamnative.io.INVALID>
> >>> > > >> wrote:
> >>> > > >> >
> >>> > > >> > Hi PengHui,
> >>> > > >> >
> >>> > > >> > Now I changed my mind a bit. Even if the pulsarctl was
> contributed
> >>> > to
> >>> > > >> > the Apache Foundation, I think we should also avoid adding it
> as the
> >>> > > >> > dependency. What we need is an API layer but not the CLI,
> while
> >>> > > >> > pulsarctl couples the API and CLI.
> >>> > > >> >
> >>> > > >> > At the moment, my expectation is:
> >>> > > >> > 1. Use a separate repo (e.g. pulsar-admin-go) to implement
> the admin
> >>> > > >> > APIs in Golang.
> >>> > > >> > 2. Depend this new repo in pulsarctl.
> >>> > > >> >
> >>> > > >> > Then we will have three Go projects:
> >>> > > >> > - pulsar-client-go: The Pulsar Go client APIs
> >>> > > >> > - pulsar-admin-go: The Pulsar Go admin APIs
> >>> > > >> > - pulsarctl: The admin CLI tool written in Go
> >>> > > >> >
> >>> > > >> > Thanks,
> >>> > > >> > Yunze
> >>> > > >> >
> >>> > > >> > On Fri, Feb 17, 2023 at 4:22 PM PengHui Li <
> penghui@apache.org>
> >>> > > wrote:
> >>> > > >> >>
> >>> > > >> >> I checked with Sijie today.
> >>> > > >> >> StreamNative can contribute the pulsarctl project to Apache
> >>> > > Foundation.
> >>> > > >> >>
> >>> > > >> >> Regards,
> >>> > > >> >> Penghui
> >>> > > >> >>
> >>> > > >> >> On Fri, Feb 17, 2023 at 4:02 PM Enrico Olivelli <
> >>> > eolivelli@gmail.com
> >>> > > >
> >>> > > >> wrote:
> >>> > > >> >>
> >>> > > >> >>> I agree to add an admin API to the go client, this would be
> very
> >>> > > >> helpful.
> >>> > > >> >>>
> >>> > > >> >>> Il giorno ven 17 feb 2023 alle ore 08:44 Zixuan Liu
> >>> > > >> >>> <no...@gmail.com> ha scritto:
> >>> > > >> >>>>
> >>> > > >> >>>> Hi Zhangjian,
> >>> > > >> >>>>
> >>> > > >> >>>> This is a good idea to write the admin client by golang,
> but I
> >>> > > don't
> >>> > > >> >>>> suggest add the admin features to pulsar-go-client, it's
> better
> >>> > to
> >>> > > >> use a
> >>> > > >> >>>> new repository to do that to  separate dependencies.
> >>> > > >> >>>>
> >>> > > >> >>>> BTW, StreamNative has a pulsarctl [0] tool, which includes
> the
> >>> > > admin
> >>> > > >> api.
> >>> > > >> >>>>
> >>> > > >> >>>>>> It's better to reuse existing code rather than
> reinventing the
> >>> > > >> wheel.
> >>> > > >> >>>>
> >>> > > >> >>>> I aggred this point. If possible, we can integrate the
> pulsarctl
> >>> > to
> >>> > > >> this
> >>> > > >> >>>> new project.
> >>> > > >> >>>
> >>> > > >> >>> We are talking about adding a client that calls a
> >>> > > >> >>> well defined and maintained REST API.
> >>> > > >> >>> It is better to have our implementation and not rely on
> third
> >>> > > parties
> >>> > > >> >>> when it is possible.
> >>> > > >> >>> If there is a security issue in pulsarctl, how would we
> handle
> >>> > that
> >>> > > ?
> >>> > > >> >>> Also the Pulsar community maintains the Pulsar API and this
> is the
> >>> > > >> >>> place where it is easier to keep the client up-to-date with
> the
> >>> > new
> >>> > > >> >>> APIs that we will develop,
> >>> > > >> >>> we can't wait for a third party project to implement our
> own APIs
> >>> > > and
> >>> > > >> >>> wait for an upgrade (even if it is OSS, we cannot cut
> releases or
> >>> > > have
> >>> > > >> >>> control over the release cycle)
> >>> > > >> >>>
> >>> > > >> >>>
> >>> > > >> >>> Enrico
> >>> > > >> >>>
> >>> > > >> >>>
> >>> > > >> >>>
> >>> > > >> >>>>
> >>> > > >> >>>> [0] - https://github.com/streamnative/pulsarctl
> >>> > > >> >>>>
> >>> > > >> >>>> Thanks,
> >>> > > >> >>>> Zixuan
> >>> > > >> >>>>
> >>> > > >> >>>>
> >>> > > >> >>>> ZhangJian He <sh...@gmail.com> 于2023年2月17日周五 13:47写道：
> >>> > > >> >>>>
> >>> > > >> >>>>> Separating dependencies is better. For example, I think
> >>> > > >> >>> Pulsar-admin-go can
> >>> > > >> >>>>> only have golang standard tls and http dependencies.
> >>> > > >> >>>>> But it seems impossible to have two go modules when
> publishing
> >>> > > >> packages
> >>> > > >> >>>>> using github.
> >>> > > >> >>>>>
> >>> > > >> >>>>>> Has anyone tried generating an admin client from our
> generated
> >>> > > open
> >>> > > >> >>>>> api spec?
> >>> > > >> >>>>>
> >>> > > >> >>>>> I have attempted it, but it requires us to modify our
> Swagger
> >>> > > file.
> >>> > > >> Our
> >>> > > >> >>>>> existing Swagger file can't generate HTTP clients
> directly.
> >>> > > Perhaps
> >>> > > >> we
> >>> > > >> >>> can
> >>> > > >> >>>>> rewrite a unified and standardized Swagger file, and then
> >>> > generate
> >>> > > >> all
> >>> > > >> >>>>> code, including brokers, from there gradually.
> >>> > > >> >>>>>
> >>> > > >> >>>>> Thanks
> >>> > > >> >>>>> ZhangJian He
> >>> > > >> >>>>>
> >>> > > >> >>>>>
> >>> > > >> >>>>> On Fri, 17 Feb 2023 at 12:37, Yunze Xu
> >>> > > <yzxu@streamnative.io.invalid
> >>> > > >> >
> >>> > > >> >>>>> wrote:
> >>> > > >> >>>>>
> >>> > > >> >>>>>>> I notice that the Java Client and the Java Admin Client
> are
> >>> > > >> >>> separate
> >>> > > >> >>>>>> dependencies. Is this boundary important to maintain for
> other
> >>> > > >> >>>>>> language admin clients?
> >>> > > >> >>>>>>
> >>> > > >> >>>>>> IMO, separating them is better to maintain. I had an
> idea to
> >>> > > >> >>> implement
> >>> > > >> >>>>>> a pure C implementation of the Pulsar admin API. Only
> libcurl
> >>> > and
> >>> > > >> >>>>>> openssl dependencies are required. Compared with the C++
> >>> > library,
> >>> > > >> the
> >>> > > >> >>>>>> pure C library has smaller size, better ABI
> compatibility, and
> >>> > > much
> >>> > > >> >>>>>> quicker compilation speed than the C++ library, which
> has some
> >>> > > more
> >>> > > >> >>>>>> heavy dependencies like Boost and Protobuf.
> >>> > > >> >>>>>>
> >>> > > >> >>>>>> Thanks,
> >>> > > >> >>>>>> Yunze
> >>> > > >> >>>>>>
> >>> > > >> >>>>>> On Fri, Feb 17, 2023 at 11:58 AM Michael Marshall <
> >>> > > >> >>> mmarshall@apache.org>
> >>> > > >> >>>>>> wrote:
> >>> > > >> >>>>>>>
> >>> > > >> >>>>>>> I think it would be valuable to have admin clients in
> many
> >>> > > >> >>> languages.
> >>> > > >> >>>>>>> Has anyone tried generating an admin client from our
> generated
> >>> > > >> open
> >>> > > >> >>>>>>> api spec?
> >>> > > >> >>>>>>>
> >>> > > >> >>>>>>> I notice that the Java Client and the Java Admin Client
> are
> >>> > > >> >>> separate
> >>> > > >> >>>>>>> dependencies. Is this boundary important to maintain
> for other
> >>> > > >> >>>>>>> language admin clients?
> >>> > > >> >>>>>>>
> >>> > > >> >>>>>>> Thanks,
> >>> > > >> >>>>>>> Michael
> >>> > > >> >>>>>>>
> >>> > > >> >>>>>>> On Thu, Feb 16, 2023 at 7:47 PM ZhangJian He <
> >>> > > shoothzj@gmail.com>
> >>> > > >> >>>>> wrote:
> >>> > > >> >>>>>>>>
> >>> > > >> >>>>>>>> I would like to express that the current Pulsar client
> for Go
> >>> > > >> >>>>>>>> (pulsar-client-go) is missing the pulsar Admin API. As
> such,
> >>> > I
> >>> > > >> >>> would
> >>> > > >> >>>>>> like
> >>> > > >> >>>>>>>> to propose that we work towards adding this feature to
> >>> > > >> >>>>>> pulsar-client-go.
> >>> > > >> >>>>>>>>
> >>> > > >> >>>>>>>> I believe that this new feature would be a valuable
> addition
> >>> > to
> >>> > > >> >>>>>>>> pulsar-client-go, and I am excited to work to make it
> happen.
> >>> > > >> >>>>>>>>
> >>> > > >> >>>>>>>> I have submitted a PR:
> >>> > > >> >>>>>> https://github.com/apache/pulsar-client-go/pull/959
> >>> > > >> >>>>>>>> The full api is not currently available, but we are
> adding.
> >>> > > >> >>>>>>>>
> >>> > > >> >>>>>>>> Below is a simple example about how to use
> >>> > > >> >>>>>>>>
> >>> > > >> >>>>>>>> ## usage
> >>> > > >> >>>>>>>>
> >>> > > >> >>>>>>>> ```go
> >>> > > >> >>>>>>>> package main
> >>> > > >> >>>>>>>>
> >>> > > >> >>>>>>>> import (
> >>> > > >> >>>>>>>> "fmt"
> >>> > > >> >>>>>>>> "github.com/apache/pulsar-client-go/padmin"
> >>> > > >> >>>>>>>> )
> >>> > > >> >>>>>>>>
> >>> > > >> >>>>>>>> func main() {
> >>> > > >> >>>>>>>> admin, err := padmin.NewDefaultPulsarAdmin()
> >>> > > >> >>>>>>>> if err != nil {
> >>> > > >> >>>>>>>> panic(err)
> >>> > > >> >>>>>>>> }
> >>> > > >> >>>>>>>> // get namespace topic list
> >>> > > >> >>>>>>>> topics, err :=
> >>> > > >> >>> admin.PersistentTopics.ListNamespaceTopics("tenant",
> >>> > > >> >>>>>>>> "namespace")
> >>> > > >> >>>>>>>> if err != nil {
> >>> > > >> >>>>>>>> panic(err)
> >>> > > >> >>>>>>>> }
> >>> > > >> >>>>>>>> fmt.Println(topics)
> >>> > > >> >>>>>>>> }
> >>> > > >> >>>>>>>> ```
> >>> > > >> >>>>>>>>
> >>> > > >> >>>>>>>> Thanks
> >>> > > >> >>>>>>>> ZhangJian He
> >>> > > >> >>>>>>
> >>> > > >> >>>>>
> >>> > > >> >>>
> >>> > > >>
> >>> > > >>
> >>> > >
> >>> >
>