Mock BookKeeper for tests or downstream applications

[Enrico Olivelli <eo...@gmail.com>]

Hi,
I would like to ease the development of BookKeeper based applications.

Currently if you want to perform unit tests of your application you
have to start a ZooKeeper server and at least one Bookie.
This is more like an integration test ! and it is really heavy weight.

In order to get rid of ZooKeeper dependency in tests we could set up a
mock Metadata Driver that holds all Metadata in memory.
In order to not use network we can use the Netty Local Transport
option (already present in BK since 4.5)
We could have a mock LedgerStorageManager that holds data in memory
Then we only have to implement a mock Journal and all is done.


Does anyone have experience ?
Does anyone would like these features ?

I would move this conversation to dev@bookkeeper if there is any interest.


Cheers
Enrico

Re: Mock BookKeeper for tests or downstream applications

[Enrico Olivelli <eo...@gmail.com>]

Ivan,
Thank you for adding this experience

I knew about MockBookKeeper
I think that the major limit it that you have to hard code intoyour
application the selection this particular implementation.

In my approach you will be using the real BK client, so no change in
the application (as soon as you can configure both ClientConfiguration
and ServerConfiguration) but only selecting simpler implementations

Enrico

Il giorno ven 21 feb 2020 alle ore 15:46 Ivan Kelly <iv...@apache.org>
ha scritto:
>
> > Does anyone have experience ?
> > Does anyone would like these features ?
> There's already something in the tree.
> https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/src/test/java/org/apache/bookkeeper/client/MockBookKeeper.java
>
> The problem with this is that sometimes the mock in incomplete or you
> want to trigger certain behaviour. If you rely on the mock in the
> bookkeeper release, you need to wait for a new release to get what you
> want. In most cases you're better off copying the mock into your own
> code and modifying it accordingly. Pulsar does this. In fact, this
> mock was originally part of pulsar, was copied into BK then copied
> back into pulsar when we realized some behaviour was broken (I think
> it was close behaviour at the time).
>
> -Ivan

Re: Mock BookKeeper for tests or downstream applications

[Ivan Kelly <iv...@apache.org>]

> Does anyone have experience ?
> Does anyone would like these features ?
There's already something in the tree.
https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/src/test/java/org/apache/bookkeeper/client/MockBookKeeper.java

The problem with this is that sometimes the mock in incomplete or you
want to trigger certain behaviour. If you rely on the mock in the
bookkeeper release, you need to wait for a new release to get what you
want. In most cases you're better off copying the mock into your own
code and modifying it accordingly. Pulsar does this. In fact, this
mock was originally part of pulsar, was copied into BK then copied
back into pulsar when we realized some behaviour was broken (I think
it was close behaviour at the time).

-Ivan