Providing a TestingPulsarServer ?

[Enrico Olivelli <eo...@gmail.com>]

Hello,
Basically every time I write an application or library for Pulsar I
fall into the need of starting a simple embedded Pulsar server in
order to run the tests.

Currently I have two approaches:
- testcontainers
- craft my own "PulsarService" classes

Testcontainers is okay, especially for real integration tests, and if
you want to test against different versions of Pulsar "server".

But for other kind of "unit tests" (even if the need to start the
Server, let me call them 'unit tests') it is better to have something
to start the Pulsar cluster.

In Apache ZooKeeper project we have ZooKeeperServerEmbedded [1], or
you can use TestingServer from Apache Curator.
Pravega project already provides a LocalPravegaEmulator service.

I totally miss this feature in Pulsar.

What about providing such API for our users ?

try (PulsarService service = PulsarService.start(configuration)) {
      String url = service.getBrokerServiceUrl();
      ..
}

I will be happy to write up a PIP and also to provide an
implementation (I already have 3-4 similar classes in my projects)

Please note that 'mocking' the Pulsar Service is not enough,
especially for an user that tries to use Pulsar and it is not a Pulsar
expert, because you cannot mock properly all of the features, and also
it is difficult to keep the mock in pair with Pulsar evolutions.

Enrico

[1] https://github.com/apache/zookeeper/blob/12b4e6821997534e1ff58e2e29b9df0beab817d3/zookeeper-server/src/main/java/org/apache/zookeeper/server/embedded/ZooKeeperServerEmbedded.java#L41

Re: Providing a TestingPulsarServer ?

[Enrico Olivelli <eo...@gmail.com>]

Il giorno ven 30 apr 2021 alle ore 23:50 Sijie Guo
<gu...@gmail.com> ha scritto:
>
> You can use PulsarStandaloneBuilder to build a standalone Pulsar server to
> test your applications.
>
> Does that meet your needs?

Yes it is what I was looking for.

I believe that PulsarStandaloneBuilder will be a great API for
the users.

But It looks like I have  a few problems with PulsarStandaloneBuilder,
I will open a ticket.
My problems are about starting the server without a configuration
file, it looks like it needs a real file in order to boot the bookie

Also you cannot access the broker url and the http url, we should
provide convenience methods.


it is just a matter of cleaning it up and probably adding a few tests

Thanks
Enrico

>
> - Sijie
>
> On Fri, Apr 30, 2021 at 5:39 AM Enrico Olivelli <eo...@gmail.com> wrote:
>
> > Hello,
> > Basically every time I write an application or library for Pulsar I
> > fall into the need of starting a simple embedded Pulsar server in
> > order to run the tests.
> >
> > Currently I have two approaches:
> > - testcontainers
> > - craft my own "PulsarService" classes
> >
> > Testcontainers is okay, especially for real integration tests, and if
> > you want to test against different versions of Pulsar "server".
> >
> > But for other kind of "unit tests" (even if the need to start the
> > Server, let me call them 'unit tests') it is better to have something
> > to start the Pulsar cluster.
> >
> > In Apache ZooKeeper project we have ZooKeeperServerEmbedded [1], or
> > you can use TestingServer from Apache Curator.
> > Pravega project already provides a LocalPravegaEmulator service.
> >
> > I totally miss this feature in Pulsar.
> >
> > What about providing such API for our users ?
> >
> > try (PulsarService service = PulsarService.start(configuration)) {
> >       String url = service.getBrokerServiceUrl();
> >       ..
> > }
> >
> > I will be happy to write up a PIP and also to provide an
> > implementation (I already have 3-4 similar classes in my projects)
> >
> > Please note that 'mocking' the Pulsar Service is not enough,
> > especially for an user that tries to use Pulsar and it is not a Pulsar
> > expert, because you cannot mock properly all of the features, and also
> > it is difficult to keep the mock in pair with Pulsar evolutions.
> >
> > Enrico
> >
> > [1]
> > https://github.com/apache/zookeeper/blob/12b4e6821997534e1ff58e2e29b9df0beab817d3/zookeeper-server/src/main/java/org/apache/zookeeper/server/embedded/ZooKeeperServerEmbedded.java#L41
> >

Re: Providing a TestingPulsarServer ?

[Sijie Guo <gu...@gmail.com>]

You can use PulsarStandaloneBuilder to build a standalone Pulsar server to
test your applications.

Does that meet your needs?

- Sijie

On Fri, Apr 30, 2021 at 5:39 AM Enrico Olivelli <eo...@gmail.com> wrote:

> Hello,
> Basically every time I write an application or library for Pulsar I
> fall into the need of starting a simple embedded Pulsar server in
> order to run the tests.
>
> Currently I have two approaches:
> - testcontainers
> - craft my own "PulsarService" classes
>
> Testcontainers is okay, especially for real integration tests, and if
> you want to test against different versions of Pulsar "server".
>
> But for other kind of "unit tests" (even if the need to start the
> Server, let me call them 'unit tests') it is better to have something
> to start the Pulsar cluster.
>
> In Apache ZooKeeper project we have ZooKeeperServerEmbedded [1], or
> you can use TestingServer from Apache Curator.
> Pravega project already provides a LocalPravegaEmulator service.
>
> I totally miss this feature in Pulsar.
>
> What about providing such API for our users ?
>
> try (PulsarService service = PulsarService.start(configuration)) {
>       String url = service.getBrokerServiceUrl();
>       ..
> }
>
> I will be happy to write up a PIP and also to provide an
> implementation (I already have 3-4 similar classes in my projects)
>
> Please note that 'mocking' the Pulsar Service is not enough,
> especially for an user that tries to use Pulsar and it is not a Pulsar
> expert, because you cannot mock properly all of the features, and also
> it is difficult to keep the mock in pair with Pulsar evolutions.
>
> Enrico
>
> [1]
> https://github.com/apache/zookeeper/blob/12b4e6821997534e1ff58e2e29b9df0beab817d3/zookeeper-server/src/main/java/org/apache/zookeeper/server/embedded/ZooKeeperServerEmbedded.java#L41
>