You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by Sijie Guo <gu...@gmail.com> on 2018/02/01 20:14:20 UTC
[DISCUSS] Deprecate FlatLedgerManager
Hi all,
`FlatLedgerManager` is not useful at production and most of the production
environment I knew don't use `FlatLedgerManager`. But the default value in
the configuration is still setting to `FlatLedgerManager`, which doesn't
sound right for the new users. so I think it is the time to raise the
discussion of deprecating `FlatLedgerManager` and make
`HierarchicalLedgerManager` as the default.
I created an issue for tracking this. Please comment if you see any problem
of making `HierarchicalLedgerManager` as the default.
https://github.com/apache/bookkeeper/issues/1101
- Sijie
Re: [DISCUSS] Deprecate FlatLedgerManager
Posted by Sijie Guo <gu...@gmail.com>.
Sent out a PR for this: https://github.com/apache/bookkeeper/pull/1115
Please take a look.
- Sijie
On Thu, Feb 1, 2018 at 2:50 PM, Sijie Guo <gu...@gmail.com> wrote:
>
> On Thu, Feb 1, 2018 at 2:22 PM Enrico Olivelli <eo...@gmail.com>
> wrote:
>
>> Il gio 1 feb 2018, 22:43 Ivan Kelly <iv...@apache.org> ha scritto:
>>
>> > > well I don't think it is an easy effort to upgrade from flat ledger
>> > manager
>> > > to hierarchical ledger manager. There was a huge long discussion about
>> > this
>> > > migration 2-3 years ago.
>> >
>>
>> I remember one of my first questions as an user was about migrating away
>> from FlatLM. We did not finish and so for some cases my customers decided
>> to nuke the cluster and start a new one.
>>
>>
>> > Do you recall where the discussion was? JIRA or ML? Superficially it
>> > seems like a simple operation if the cluster is taken down while it is
>> > happening.
>> >
>>
>> Sijie,
>> I have tens of running BK clusters of my customers which are using FlatLM
>> because it is the default. Most of them are very little clusters with 3
>> bookies.
>
>
>
>
>
>>
>> So I am ok with this move but I need a tool to migrate.
>
>
> We will not remove this class. But the default value will be flipped to
> hierarchical. We will provide the tool if we decide to drop it in future.
> But probably not at this moment. If anyone in the community has the time to
> work on this, it would be great as well!
>
>
>>
>> Having the full cluster down can be a option for my cases so I think my
>> customers will be happy even with a simple tool.
>>
>> Maybe we can have a tool which creates a new structure using a different
>> base path so that is case of failure the cluster is not compromised.
>
>
> Yes if downtime is acceptable, the solution will be easy, otherwise it is
> going to be very complicated to support live migration.
>
> Sijie
>
>
>>
>> Enrico
>>
>>
>> > -Ivan
>> >
>>
>>
>> --
>>
>>
>> -- Enrico Olivelli
>>
>
Re: [DISCUSS] Deprecate FlatLedgerManager
Posted by Sijie Guo <gu...@gmail.com>.
On Thu, Feb 1, 2018 at 2:22 PM Enrico Olivelli <eo...@gmail.com> wrote:
> Il gio 1 feb 2018, 22:43 Ivan Kelly <iv...@apache.org> ha scritto:
>
> > > well I don't think it is an easy effort to upgrade from flat ledger
> > manager
> > > to hierarchical ledger manager. There was a huge long discussion about
> > this
> > > migration 2-3 years ago.
> >
>
> I remember one of my first questions as an user was about migrating away
> from FlatLM. We did not finish and so for some cases my customers decided
> to nuke the cluster and start a new one.
>
>
> > Do you recall where the discussion was? JIRA or ML? Superficially it
> > seems like a simple operation if the cluster is taken down while it is
> > happening.
> >
>
> Sijie,
> I have tens of running BK clusters of my customers which are using FlatLM
> because it is the default. Most of them are very little clusters with 3
> bookies.
>
> So I am ok with this move but I need a tool to migrate.
We will not remove this class. But the default value will be flipped to
hierarchical. We will provide the tool if we decide to drop it in future.
But probably not at this moment. If anyone in the community has the time to
work on this, it would be great as well!
>
> Having the full cluster down can be a option for my cases so I think my
> customers will be happy even with a simple tool.
>
> Maybe we can have a tool which creates a new structure using a different
> base path so that is case of failure the cluster is not compromised.
Yes if downtime is acceptable, the solution will be easy, otherwise it is
going to be very complicated to support live migration.
Sijie
>
> Enrico
>
>
> > -Ivan
> >
>
>
> --
>
>
> -- Enrico Olivelli
>
Re: [DISCUSS] Deprecate FlatLedgerManager
Posted by Enrico Olivelli <eo...@gmail.com>.
Il gio 1 feb 2018, 22:43 Ivan Kelly <iv...@apache.org> ha scritto:
> > well I don't think it is an easy effort to upgrade from flat ledger
> manager
> > to hierarchical ledger manager. There was a huge long discussion about
> this
> > migration 2-3 years ago.
>
I remember one of my first questions as an user was about migrating away
from FlatLM. We did not finish and so for some cases my customers decided
to nuke the cluster and start a new one.
> Do you recall where the discussion was? JIRA or ML? Superficially it
> seems like a simple operation if the cluster is taken down while it is
> happening.
>
Sijie,
I have tens of running BK clusters of my customers which are using FlatLM
because it is the default. Most of them are very little clusters with 3
bookies.
So I am ok with this move but I need a tool to migrate.
Having the full cluster down can be a option for my cases so I think my
customers will be happy even with a simple tool.
Maybe we can have a tool which creates a new structure using a different
base path so that is case of failure the cluster is not compromised.
Enrico
> -Ivan
>
--
-- Enrico Olivelli
Re: [DISCUSS] Deprecate FlatLedgerManager
Posted by Sijie Guo <gu...@gmail.com>.
The major concern was about id generation. Yes if you take the cluster
down, you can do it easily. But it is not really useful then. So my take is
not to support it at all. It is not worth spending time and energy on it.
Sijie
On Thu, Feb 1, 2018 at 1:43 PM Ivan Kelly <iv...@apache.org> wrote:
> > well I don't think it is an easy effort to upgrade from flat ledger
> manager
> > to hierarchical ledger manager. There was a huge long discussion about
> this
> > migration 2-3 years ago.
>
> Do you recall where the discussion was? JIRA or ML? Superficially it
> seems like a simple operation if the cluster is taken down while it is
> happening.
>
> -Ivan
>
Re: [DISCUSS] Deprecate FlatLedgerManager
Posted by Ivan Kelly <iv...@apache.org>.
> well I don't think it is an easy effort to upgrade from flat ledger manager
> to hierarchical ledger manager. There was a huge long discussion about this
> migration 2-3 years ago.
Do you recall where the discussion was? JIRA or ML? Superficially it
seems like a simple operation if the cluster is taken down while it is
happening.
-Ivan
Re: [DISCUSS] Deprecate FlatLedgerManager
Posted by Sijie Guo <gu...@gmail.com>.
On Thu, Feb 1, 2018 at 12:20 PM, Ivan Kelly <iv...@apache.org> wrote:
> +1
>
> Flat is an vestigial. Prereq to removal is a tool for upgrading ledger
> metadata.
>
well I don't think it is an easy effort to upgrade from flat ledger manager
to hierarchical ledger manager. There was a huge long discussion about this
migration 2-3 years ago.
Given there is no one really using flat ledger manager on production, I
don't think it is worth attempting this effort again.
But I will limit the scope of this topic only to "deprecation" and changing
the default value. If we want to keep BC for this, we can keep this class
forever in the repo. So no "removal" at any time soon of this class.
- Sijie
>
> -Ivan
>
> On Thu, Feb 1, 2018 at 8:14 PM, Sijie Guo <gu...@gmail.com> wrote:
> > Hi all,
> >
> > `FlatLedgerManager` is not useful at production and most of the
> production
> > environment I knew don't use `FlatLedgerManager`. But the default value
> in
> > the configuration is still setting to `FlatLedgerManager`, which doesn't
> > sound right for the new users. so I think it is the time to raise the
> > discussion of deprecating `FlatLedgerManager` and make
> > `HierarchicalLedgerManager` as the default.
> >
> > I created an issue for tracking this. Please comment if you see any
> problem
> > of making `HierarchicalLedgerManager` as the default.
> >
> > https://github.com/apache/bookkeeper/issues/1101
> >
> > - Sijie
>
Re: [DISCUSS] Deprecate FlatLedgerManager
Posted by Venkateswara Rao Jujjuri <ju...@gmail.com>.
+1
On Thu, Feb 1, 2018 at 12:20 PM, Ivan Kelly <iv...@apache.org> wrote:
> +1
>
> Flat is an vestigial. Prereq to removal is a tool for upgrading ledger
> metadata.
>
> -Ivan
>
> On Thu, Feb 1, 2018 at 8:14 PM, Sijie Guo <gu...@gmail.com> wrote:
> > Hi all,
> >
> > `FlatLedgerManager` is not useful at production and most of the
> production
> > environment I knew don't use `FlatLedgerManager`. But the default value
> in
> > the configuration is still setting to `FlatLedgerManager`, which doesn't
> > sound right for the new users. so I think it is the time to raise the
> > discussion of deprecating `FlatLedgerManager` and make
> > `HierarchicalLedgerManager` as the default.
> >
> > I created an issue for tracking this. Please comment if you see any
> problem
> > of making `HierarchicalLedgerManager` as the default.
> >
> > https://github.com/apache/bookkeeper/issues/1101
> >
> > - Sijie
>
--
Jvrao
---
First they ignore you, then they laugh at you, then they fight you, then
you win. - Mahatma Gandhi
Re: [DISCUSS] Deprecate FlatLedgerManager
Posted by Ivan Kelly <iv...@apache.org>.
+1
Flat is an vestigial. Prereq to removal is a tool for upgrading ledger metadata.
-Ivan
On Thu, Feb 1, 2018 at 8:14 PM, Sijie Guo <gu...@gmail.com> wrote:
> Hi all,
>
> `FlatLedgerManager` is not useful at production and most of the production
> environment I knew don't use `FlatLedgerManager`. But the default value in
> the configuration is still setting to `FlatLedgerManager`, which doesn't
> sound right for the new users. so I think it is the time to raise the
> discussion of deprecating `FlatLedgerManager` and make
> `HierarchicalLedgerManager` as the default.
>
> I created an issue for tracking this. Please comment if you see any problem
> of making `HierarchicalLedgerManager` as the default.
>
> https://github.com/apache/bookkeeper/issues/1101
>
> - Sijie