You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by Nicolò Boschi <bo...@gmail.com> on 2022/04/01 13:17:39 UTC
RocksDB upgrade - wait for BookKeeper 4.15.0 ?
Hi all,
this is a follow-up discussion about the RocksDB upgrade that happened at
yesterday's Apache Pulsar community meeting (
https://lists.apache.org/thread/tq7dsws72zf9r7qzr4l567z9w346ksbm).
We recently reverted the RocksDB upgrade due to runtime compatibility
errors (more details here https://github.com/apache/pulsar/pull/14962).
The main reason to upgrade RocksDB is the Apple M1 compatibility.
Shortly, we cannot upgrade RocksDB to a version greater than 6.17.2 because
BookKeeper needs to be recompiled with the newer version of RocksDB.
There's an ongoing discussion on BookKeeper mailing list about including
the RocksDB upgrade in 4.14.5. Also 4.15.0 is going to be released soon
(with the RocksDB upgrade).
https://lists.apache.org/thread/dsdltmoy5ggqx4oj7hrt13hqok80d4jf
At the moment, Pulsar 2.8, 2.9, 2.10 and master are using 4.14.4.
From the Pulsar side we are interested in upgrading to 4.14.5 because
there's an important bugfix (https://github.com/apache/bookkeeper/pull/3110)
that we should port to all the active branches.
So basically if 4.14.5 will contain the RocksDB upgrade, we'll need to bump
to the same version also in the active branches to match the runtime
compatibility. If not, we're safe to upgrade to 4.14.5 without RocksDB.
My personal opinion is that the best scenario would be:
- Update BK to 4.14.5 on active branches without RocksDB
- Update BK to 4.15.0 on master (and so Pulsar 2.11) with RocksDB and Apple
M1 compatibility.
I think we could push to the BookKeeper ML to not include the RocksDB
upgrade in 4.14.5
Nicolò Boschi
Re: RocksDB upgrade - wait for BookKeeper 4.15.0 ?
Posted by Nicolò Boschi <bo...@gmail.com>.
At the end we'll have Pulsar 2.11 with BookKeeper 4.15.x (Andrey is working
on the BK release and the upgrade in Pulsar) and therefore with M1 support.
Pulsar 2.8, 2.9, 2.10 lines will have BookKeeper 4.14 (without the M1
support).
Nicolò Boschi
Il giorno mar 12 apr 2022 alle ore 15:10 Devin Bost <de...@gmail.com>
ha scritto:
> So, based on this, it looks like we'd need to upgrade Pulsar to use BK
> 4.15.0 to get M1 compatibility?
> What kind of changes will that introduce into Pulsar?
>
> Devin G. Bost
>
>
> On Fri, Apr 1, 2022 at 8:17 AM Nicolò Boschi <bo...@gmail.com> wrote:
>
> > Hi all,
> >
> > this is a follow-up discussion about the RocksDB upgrade that happened at
> > yesterday's Apache Pulsar community meeting (
> > https://lists.apache.org/thread/tq7dsws72zf9r7qzr4l567z9w346ksbm).
> >
> > We recently reverted the RocksDB upgrade due to runtime compatibility
> > errors (more details here https://github.com/apache/pulsar/pull/14962).
> > The main reason to upgrade RocksDB is the Apple M1 compatibility.
> >
> > Shortly, we cannot upgrade RocksDB to a version greater than 6.17.2
> because
> > BookKeeper needs to be recompiled with the newer version of RocksDB.
> >
> > There's an ongoing discussion on BookKeeper mailing list about including
> > the RocksDB upgrade in 4.14.5. Also 4.15.0 is going to be released soon
> > (with the RocksDB upgrade).
> > https://lists.apache.org/thread/dsdltmoy5ggqx4oj7hrt13hqok80d4jf
> >
> > At the moment, Pulsar 2.8, 2.9, 2.10 and master are using 4.14.4.
> >
> > From the Pulsar side we are interested in upgrading to 4.14.5 because
> > there's an important bugfix (
> > https://github.com/apache/bookkeeper/pull/3110)
> > that we should port to all the active branches.
> > So basically if 4.14.5 will contain the RocksDB upgrade, we'll need to
> bump
> > to the same version also in the active branches to match the runtime
> > compatibility. If not, we're safe to upgrade to 4.14.5 without RocksDB.
> >
> > My personal opinion is that the best scenario would be:
> > - Update BK to 4.14.5 on active branches without RocksDB
> > - Update BK to 4.15.0 on master (and so Pulsar 2.11) with RocksDB and
> Apple
> > M1 compatibility.
> >
> > I think we could push to the BookKeeper ML to not include the RocksDB
> > upgrade in 4.14.5
> >
> > Nicolò Boschi
> >
>
Re: RocksDB upgrade - wait for BookKeeper 4.15.0 ?
Posted by Devin Bost <de...@gmail.com>.
So, based on this, it looks like we'd need to upgrade Pulsar to use BK
4.15.0 to get M1 compatibility?
What kind of changes will that introduce into Pulsar?
Devin G. Bost
On Fri, Apr 1, 2022 at 8:17 AM Nicolò Boschi <bo...@gmail.com> wrote:
> Hi all,
>
> this is a follow-up discussion about the RocksDB upgrade that happened at
> yesterday's Apache Pulsar community meeting (
> https://lists.apache.org/thread/tq7dsws72zf9r7qzr4l567z9w346ksbm).
>
> We recently reverted the RocksDB upgrade due to runtime compatibility
> errors (more details here https://github.com/apache/pulsar/pull/14962).
> The main reason to upgrade RocksDB is the Apple M1 compatibility.
>
> Shortly, we cannot upgrade RocksDB to a version greater than 6.17.2 because
> BookKeeper needs to be recompiled with the newer version of RocksDB.
>
> There's an ongoing discussion on BookKeeper mailing list about including
> the RocksDB upgrade in 4.14.5. Also 4.15.0 is going to be released soon
> (with the RocksDB upgrade).
> https://lists.apache.org/thread/dsdltmoy5ggqx4oj7hrt13hqok80d4jf
>
> At the moment, Pulsar 2.8, 2.9, 2.10 and master are using 4.14.4.
>
> From the Pulsar side we are interested in upgrading to 4.14.5 because
> there's an important bugfix (
> https://github.com/apache/bookkeeper/pull/3110)
> that we should port to all the active branches.
> So basically if 4.14.5 will contain the RocksDB upgrade, we'll need to bump
> to the same version also in the active branches to match the runtime
> compatibility. If not, we're safe to upgrade to 4.14.5 without RocksDB.
>
> My personal opinion is that the best scenario would be:
> - Update BK to 4.14.5 on active branches without RocksDB
> - Update BK to 4.15.0 on master (and so Pulsar 2.11) with RocksDB and Apple
> M1 compatibility.
>
> I think we could push to the BookKeeper ML to not include the RocksDB
> upgrade in 4.14.5
>
> Nicolò Boschi
>