You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by Sijie Guo <gu...@gmail.com> on 2018/08/17 21:36:43 UTC
[DISCUSS] BP-35: 128 bits support
Hi all,
As promised in last community meeting, I put up all the thoughts and
discussion with JV I had into a BP for supporting 128-bit ledger id. I
tried my best to cover all the aspects that I can think of. There can be
places that I miss. so please take a look and let me know what you think.
BP PR: https://github.com/apache/bookkeeper/pull/1611
Google doc:
https://docs.google.com/document/d/1cu54dNSV2ZrdWCi40LcyX8NxXGRCW0609T_ewmK9BWM
- Sijie
Re: [DISCUSS] BP-35: 128 bits support
Posted by Venkateswara Rao Jujjuri <ju...@gmail.com>.
I think we should avoid the long representation and interpretation in the
new code, basically -1 should be looked as MSB of the lsb-ledgerIDset to 1.
Also, we should be able to provide BC without special cases. Branching
should be based on if msb-ledgerId(scopeID) is 0 or not.
Maybe we should treat the entire 128 bit (two longs) as a bitmap and add
corresponding utility library?
> We are treating the global space of ledgerids as a 2 level hierarchy
My thought is, we are treating it this way because of the lack of native
support. Not for the backward compatibility reasons.
> the full 128bit space is handled, but there is a class of combinations
which won't work (scopeid = 0 and ledgerid < 0).
Not sure, I understand this. We need to thrive towards handling full 128
bits. What is stopping us from that?
On Sat, Aug 18, 2018 at 10:37 AM, Enrico Olivelli <eo...@gmail.com>
wrote:
> Il sab 18 ago 2018, 18:43 Sijie Guo <gu...@gmail.com> ha scritto:
>
> > On Sat, Aug 18, 2018 at 9:28 AM Enrico Olivelli <eo...@gmail.com>
> > wrote:
> >
> > > Il sab 18 ago 2018, 17:26 Sijie Guo <gu...@gmail.com> ha scritto:
> > >
> > > > On Sat, Aug 18, 2018 at 2:12 AM Enrico Olivelli <eolivelli@gmail.com
> >
> > > > wrote:
> > > >
> > > > > Nice work Sijie,
> > > > > Some comments after the first read, just thinking out loud...
> > > > > Overall looks a good idea and implementable.
> > > > >
> > > > > We should cite somewhere the case of scope=0 and ledgerId = -1
> which
> > > is a
> > > > > very special case for usthe.
> > > > >
> > > >
> > > > For BC reason, negative ledger id are not supported when scope == 0.
> We
> > > > effectively only support 63 bit.
> > > >
> > >
> > > Fine to me
> > >
> > > We should state clearly in the BP and then in javadocs and docs that
> not
> > > the full 128bit space is handled,
> >
> > but there is a class of combinations
> > > which won't work (scopeid = 0 and ledgerid < 0).
> > >
> >
> > Sure. But I think that would be clearly until we implement it. So I would
> > defer the clarification until then. Because there can be other more
> corner
> > cases.
> >
> > >
> > >
> > >
> > >
> > >
> > > > We can use versions on clients to distinguish whether they are old
> > > clients
> > > > or new clients, then handle this case differently.
> > > >
> > > > So this special case should not be special for any new clients.
> > > >
> > > > We should also state that scopeid and ledgerid < 0 is not supported
> or
> > it
> > > > > is not clear to me how we will handle that case.
> > > > > The case of scope < 0 is not clear to me as well, maybe I have to
> > read
> > > > more
> > > > > carefully the doc.
> > > >
> > > >
> > > > I think I answer this with my comment above.
> > > >
> > > >
> > > > >
> > > > > We are treating the global space of ledgerids as a 2 level
> hierarchy
> > > and
> > > > > this is good for compatibility.
> > > > > But thinking about a new application, which starts with uuids I
> think
> > > it
> > > > > can be tricky, maybe we should add utilities/APIs to work with UUID
> > > > > directly.
> > > >
> > > >
> > > > The ledgerQualifiedName is the hex presentation of UUID, proposed to
> > add
> > > to
> > > > CLI tool.
> > > >
> > > > We can add it to API as well. However I would defer adding it until
> we
> > > need
> > > > it or after we are on 128 bits. BC is the top concern.
> > > >
> > > >
> > > > > For this new kind of application the space of ids would be flat and
> > not
> > > > > hierarchical (UUID are not hierarchical as far as I know, but I may
> > be
> > > > > wrong).
> > > >
> > > >
> > > > In a lot of places, 128 bit is not well supported. So people would
> > divide
> > > > into two parts any way, msb and lsb.
> > > >
> > > >
> > > > >
> > > > >
> > > > > We should also support listing the whole flat space
> > > >
> > > >
> > > > Sure it can be added. I don’t see a reason we can’t. However IMO
> these
> > > are
> > > > new features after we are on 128 bit. For the concerns in this BP, we
> > are
> > > > more focused on how we can transition from 64 bit to 128 bit.
> > > >
> > >
> > > Okay
> > >
> > > Last comment:
> > > We are going to have a lot of special cases in layout of data/zk ledger
> > > manager, to handle scopeid = 0.
> >
> >
> > > I am thinking to a brand new installation, without upgrade/BC
> concerns, I
> > > wonder if we have some flag not to have that tricks, I am talking about
> > the
> > > different dbs on rocks db, the layout of indexes...and so on.
> >
> >
> > I don’t think we should provide this flag since it would make things
> very
> > complicated. In that case, you can just provide a brand new metadata
> > driver, a new ledger storage. The you don’t need to worry about this
> > upgrade story. However that will be brand new implementation, which would
> > be out of the scope of this BP.
> >
>
> Okay
>
> To me it is clear.
>
> +1
>
> Enrico
>
>
> > Sijie
> >
> >
> >
> > >
> > > This is not a real problem but I wonder if we ca handle this case of
> new
> > > applications cleanly
> > >
> > > Enrico
> > >
> > >
> > >
> > > > or as a last
> > > > > alternative we should add a way to list the space of scopeids,
> > because
> > > > > otherwise the application must store the list of scopeids, and for
> it
> > > it
> > > > > will be a set of half UUIDs.
> > > > >
> > > > > Thanks for driving this
> > > > >
> > > > > Enrico
> > > > >
> > > > >
> > > > >
> > > > > Il ven 17 ago 2018, 23:52 Venkateswara Rao Jujjuri <
> > jujjuri@gmail.com>
> > > > ha
> > > > > scritto:
> > > > >
> > > > > > Thanks a lot, Sijie for the awesome doc.
> > > > > > Let's pool some more thoughts into this and size the work.
> > > > > >
> > > > > > Thanks,
> > > > > > JV
> > > > > >
> > > > > > On Fri, Aug 17, 2018 at 2:36 PM, Sijie Guo <gu...@gmail.com>
> > > wrote:
> > > > > >
> > > > > > > Hi all,
> > > > > > >
> > > > > > > As promised in last community meeting, I put up all the
> thoughts
> > > and
> > > > > > > discussion with JV I had into a BP for supporting 128-bit
> ledger
> > > id.
> > > > I
> > > > > > > tried my best to cover all the aspects that I can think of.
> There
> > > can
> > > > > be
> > > > > > > places that I miss. so please take a look and let me know what
> > you
> > > > > think.
> > > > > > >
> > > > > > > BP PR: https://github.com/apache/bookkeeper/pull/1611
> > > > > > > Google doc:
> > > > > > >
> > https://docs.google.com/document/d/1cu54dNSV2ZrdWCi40LcyX8NxXGRCW
> > > > > > > 0609T_ewmK9BWM
> > > > > > >
> > > > > > > - Sijie
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Jvrao
> > > > > > ---
> > > > > > First they ignore you, then they laugh at you, then they fight
> you,
> > > > then
> > > > > > you win. - Mahatma Gandhi
> > > > > >
> > > > > --
> > > > >
> > > > >
> > > > > -- Enrico Olivelli
> > > > >
> > > >
> > > --
> > >
> > >
> > > -- Enrico Olivelli
> > >
> >
> --
>
>
> -- Enrico Olivelli
>
--
Jvrao
---
First they ignore you, then they laugh at you, then they fight you, then
you win. - Mahatma Gandhi
Re: [DISCUSS] BP-35: 128 bits support
Posted by Enrico Olivelli <eo...@gmail.com>.
Il sab 18 ago 2018, 18:43 Sijie Guo <gu...@gmail.com> ha scritto:
> On Sat, Aug 18, 2018 at 9:28 AM Enrico Olivelli <eo...@gmail.com>
> wrote:
>
> > Il sab 18 ago 2018, 17:26 Sijie Guo <gu...@gmail.com> ha scritto:
> >
> > > On Sat, Aug 18, 2018 at 2:12 AM Enrico Olivelli <eo...@gmail.com>
> > > wrote:
> > >
> > > > Nice work Sijie,
> > > > Some comments after the first read, just thinking out loud...
> > > > Overall looks a good idea and implementable.
> > > >
> > > > We should cite somewhere the case of scope=0 and ledgerId = -1 which
> > is a
> > > > very special case for usthe.
> > > >
> > >
> > > For BC reason, negative ledger id are not supported when scope == 0. We
> > > effectively only support 63 bit.
> > >
> >
> > Fine to me
> >
> > We should state clearly in the BP and then in javadocs and docs that not
> > the full 128bit space is handled,
>
> but there is a class of combinations
> > which won't work (scopeid = 0 and ledgerid < 0).
> >
>
> Sure. But I think that would be clearly until we implement it. So I would
> defer the clarification until then. Because there can be other more corner
> cases.
>
> >
> >
> >
> >
> >
> > > We can use versions on clients to distinguish whether they are old
> > clients
> > > or new clients, then handle this case differently.
> > >
> > > So this special case should not be special for any new clients.
> > >
> > > We should also state that scopeid and ledgerid < 0 is not supported or
> it
> > > > is not clear to me how we will handle that case.
> > > > The case of scope < 0 is not clear to me as well, maybe I have to
> read
> > > more
> > > > carefully the doc.
> > >
> > >
> > > I think I answer this with my comment above.
> > >
> > >
> > > >
> > > > We are treating the global space of ledgerids as a 2 level hierarchy
> > and
> > > > this is good for compatibility.
> > > > But thinking about a new application, which starts with uuids I think
> > it
> > > > can be tricky, maybe we should add utilities/APIs to work with UUID
> > > > directly.
> > >
> > >
> > > The ledgerQualifiedName is the hex presentation of UUID, proposed to
> add
> > to
> > > CLI tool.
> > >
> > > We can add it to API as well. However I would defer adding it until we
> > need
> > > it or after we are on 128 bits. BC is the top concern.
> > >
> > >
> > > > For this new kind of application the space of ids would be flat and
> not
> > > > hierarchical (UUID are not hierarchical as far as I know, but I may
> be
> > > > wrong).
> > >
> > >
> > > In a lot of places, 128 bit is not well supported. So people would
> divide
> > > into two parts any way, msb and lsb.
> > >
> > >
> > > >
> > > >
> > > > We should also support listing the whole flat space
> > >
> > >
> > > Sure it can be added. I don’t see a reason we can’t. However IMO these
> > are
> > > new features after we are on 128 bit. For the concerns in this BP, we
> are
> > > more focused on how we can transition from 64 bit to 128 bit.
> > >
> >
> > Okay
> >
> > Last comment:
> > We are going to have a lot of special cases in layout of data/zk ledger
> > manager, to handle scopeid = 0.
>
>
> > I am thinking to a brand new installation, without upgrade/BC concerns, I
> > wonder if we have some flag not to have that tricks, I am talking about
> the
> > different dbs on rocks db, the layout of indexes...and so on.
>
>
> I don’t think we should provide this flag since it would make things very
> complicated. In that case, you can just provide a brand new metadata
> driver, a new ledger storage. The you don’t need to worry about this
> upgrade story. However that will be brand new implementation, which would
> be out of the scope of this BP.
>
Okay
To me it is clear.
+1
Enrico
> Sijie
>
>
>
> >
> > This is not a real problem but I wonder if we ca handle this case of new
> > applications cleanly
> >
> > Enrico
> >
> >
> >
> > > or as a last
> > > > alternative we should add a way to list the space of scopeids,
> because
> > > > otherwise the application must store the list of scopeids, and for it
> > it
> > > > will be a set of half UUIDs.
> > > >
> > > > Thanks for driving this
> > > >
> > > > Enrico
> > > >
> > > >
> > > >
> > > > Il ven 17 ago 2018, 23:52 Venkateswara Rao Jujjuri <
> jujjuri@gmail.com>
> > > ha
> > > > scritto:
> > > >
> > > > > Thanks a lot, Sijie for the awesome doc.
> > > > > Let's pool some more thoughts into this and size the work.
> > > > >
> > > > > Thanks,
> > > > > JV
> > > > >
> > > > > On Fri, Aug 17, 2018 at 2:36 PM, Sijie Guo <gu...@gmail.com>
> > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > As promised in last community meeting, I put up all the thoughts
> > and
> > > > > > discussion with JV I had into a BP for supporting 128-bit ledger
> > id.
> > > I
> > > > > > tried my best to cover all the aspects that I can think of. There
> > can
> > > > be
> > > > > > places that I miss. so please take a look and let me know what
> you
> > > > think.
> > > > > >
> > > > > > BP PR: https://github.com/apache/bookkeeper/pull/1611
> > > > > > Google doc:
> > > > > >
> https://docs.google.com/document/d/1cu54dNSV2ZrdWCi40LcyX8NxXGRCW
> > > > > > 0609T_ewmK9BWM
> > > > > >
> > > > > > - Sijie
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Jvrao
> > > > > ---
> > > > > First they ignore you, then they laugh at you, then they fight you,
> > > then
> > > > > you win. - Mahatma Gandhi
> > > > >
> > > > --
> > > >
> > > >
> > > > -- Enrico Olivelli
> > > >
> > >
> > --
> >
> >
> > -- Enrico Olivelli
> >
>
--
-- Enrico Olivelli
Re: [DISCUSS] BP-35: 128 bits support
Posted by Sijie Guo <gu...@gmail.com>.
On Sat, Aug 18, 2018 at 9:28 AM Enrico Olivelli <eo...@gmail.com> wrote:
> Il sab 18 ago 2018, 17:26 Sijie Guo <gu...@gmail.com> ha scritto:
>
> > On Sat, Aug 18, 2018 at 2:12 AM Enrico Olivelli <eo...@gmail.com>
> > wrote:
> >
> > > Nice work Sijie,
> > > Some comments after the first read, just thinking out loud...
> > > Overall looks a good idea and implementable.
> > >
> > > We should cite somewhere the case of scope=0 and ledgerId = -1 which
> is a
> > > very special case for usthe.
> > >
> >
> > For BC reason, negative ledger id are not supported when scope == 0. We
> > effectively only support 63 bit.
> >
>
> Fine to me
>
> We should state clearly in the BP and then in javadocs and docs that not
> the full 128bit space is handled,
but there is a class of combinations
> which won't work (scopeid = 0 and ledgerid < 0).
>
Sure. But I think that would be clearly until we implement it. So I would
defer the clarification until then. Because there can be other more corner
cases.
>
>
>
>
>
> > We can use versions on clients to distinguish whether they are old
> clients
> > or new clients, then handle this case differently.
> >
> > So this special case should not be special for any new clients.
> >
> > We should also state that scopeid and ledgerid < 0 is not supported or it
> > > is not clear to me how we will handle that case.
> > > The case of scope < 0 is not clear to me as well, maybe I have to read
> > more
> > > carefully the doc.
> >
> >
> > I think I answer this with my comment above.
> >
> >
> > >
> > > We are treating the global space of ledgerids as a 2 level hierarchy
> and
> > > this is good for compatibility.
> > > But thinking about a new application, which starts with uuids I think
> it
> > > can be tricky, maybe we should add utilities/APIs to work with UUID
> > > directly.
> >
> >
> > The ledgerQualifiedName is the hex presentation of UUID, proposed to add
> to
> > CLI tool.
> >
> > We can add it to API as well. However I would defer adding it until we
> need
> > it or after we are on 128 bits. BC is the top concern.
> >
> >
> > > For this new kind of application the space of ids would be flat and not
> > > hierarchical (UUID are not hierarchical as far as I know, but I may be
> > > wrong).
> >
> >
> > In a lot of places, 128 bit is not well supported. So people would divide
> > into two parts any way, msb and lsb.
> >
> >
> > >
> > >
> > > We should also support listing the whole flat space
> >
> >
> > Sure it can be added. I don’t see a reason we can’t. However IMO these
> are
> > new features after we are on 128 bit. For the concerns in this BP, we are
> > more focused on how we can transition from 64 bit to 128 bit.
> >
>
> Okay
>
> Last comment:
> We are going to have a lot of special cases in layout of data/zk ledger
> manager, to handle scopeid = 0.
> I am thinking to a brand new installation, without upgrade/BC concerns, I
> wonder if we have some flag not to have that tricks, I am talking about the
> different dbs on rocks db, the layout of indexes...and so on.
I don’t think we should provide this flag since it would make things very
complicated. In that case, you can just provide a brand new metadata
driver, a new ledger storage. The you don’t need to worry about this
upgrade story. However that will be brand new implementation, which would
be out of the scope of this BP.
Sijie
>
> This is not a real problem but I wonder if we ca handle this case of new
> applications cleanly
>
> Enrico
>
>
>
> > or as a last
> > > alternative we should add a way to list the space of scopeids, because
> > > otherwise the application must store the list of scopeids, and for it
> it
> > > will be a set of half UUIDs.
> > >
> > > Thanks for driving this
> > >
> > > Enrico
> > >
> > >
> > >
> > > Il ven 17 ago 2018, 23:52 Venkateswara Rao Jujjuri <ju...@gmail.com>
> > ha
> > > scritto:
> > >
> > > > Thanks a lot, Sijie for the awesome doc.
> > > > Let's pool some more thoughts into this and size the work.
> > > >
> > > > Thanks,
> > > > JV
> > > >
> > > > On Fri, Aug 17, 2018 at 2:36 PM, Sijie Guo <gu...@gmail.com>
> wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > As promised in last community meeting, I put up all the thoughts
> and
> > > > > discussion with JV I had into a BP for supporting 128-bit ledger
> id.
> > I
> > > > > tried my best to cover all the aspects that I can think of. There
> can
> > > be
> > > > > places that I miss. so please take a look and let me know what you
> > > think.
> > > > >
> > > > > BP PR: https://github.com/apache/bookkeeper/pull/1611
> > > > > Google doc:
> > > > > https://docs.google.com/document/d/1cu54dNSV2ZrdWCi40LcyX8NxXGRCW
> > > > > 0609T_ewmK9BWM
> > > > >
> > > > > - Sijie
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Jvrao
> > > > ---
> > > > First they ignore you, then they laugh at you, then they fight you,
> > then
> > > > you win. - Mahatma Gandhi
> > > >
> > > --
> > >
> > >
> > > -- Enrico Olivelli
> > >
> >
> --
>
>
> -- Enrico Olivelli
>
Re: [DISCUSS] BP-35: 128 bits support
Posted by Enrico Olivelli <eo...@gmail.com>.
Il sab 18 ago 2018, 17:26 Sijie Guo <gu...@gmail.com> ha scritto:
> On Sat, Aug 18, 2018 at 2:12 AM Enrico Olivelli <eo...@gmail.com>
> wrote:
>
> > Nice work Sijie,
> > Some comments after the first read, just thinking out loud...
> > Overall looks a good idea and implementable.
> >
> > We should cite somewhere the case of scope=0 and ledgerId = -1 which is a
> > very special case for usthe.
> >
>
> For BC reason, negative ledger id are not supported when scope == 0. We
> effectively only support 63 bit.
>
Fine to me
We should state clearly in the BP and then in javadocs and docs that not
the full 128bit space is handled, but there is a class of combinations
which won't work (scopeid = 0 and ledgerid < 0).
> We can use versions on clients to distinguish whether they are old clients
> or new clients, then handle this case differently.
>
> So this special case should not be special for any new clients.
>
> We should also state that scopeid and ledgerid < 0 is not supported or it
> > is not clear to me how we will handle that case.
> > The case of scope < 0 is not clear to me as well, maybe I have to read
> more
> > carefully the doc.
>
>
> I think I answer this with my comment above.
>
>
> >
> > We are treating the global space of ledgerids as a 2 level hierarchy and
> > this is good for compatibility.
> > But thinking about a new application, which starts with uuids I think it
> > can be tricky, maybe we should add utilities/APIs to work with UUID
> > directly.
>
>
> The ledgerQualifiedName is the hex presentation of UUID, proposed to add to
> CLI tool.
>
> We can add it to API as well. However I would defer adding it until we need
> it or after we are on 128 bits. BC is the top concern.
>
>
> > For this new kind of application the space of ids would be flat and not
> > hierarchical (UUID are not hierarchical as far as I know, but I may be
> > wrong).
>
>
> In a lot of places, 128 bit is not well supported. So people would divide
> into two parts any way, msb and lsb.
>
>
> >
> >
> > We should also support listing the whole flat space
>
>
> Sure it can be added. I don’t see a reason we can’t. However IMO these are
> new features after we are on 128 bit. For the concerns in this BP, we are
> more focused on how we can transition from 64 bit to 128 bit.
>
Okay
Last comment:
We are going to have a lot of special cases in layout of data/zk ledger
manager, to handle scopeid = 0.
I am thinking to a brand new installation, without upgrade/BC concerns, I
wonder if we have some flag not to have that tricks, I am talking about the
different dbs on rocks db, the layout of indexes...and so on.
This is not a real problem but I wonder if we ca handle this case of new
applications cleanly
Enrico
> or as a last
> > alternative we should add a way to list the space of scopeids, because
> > otherwise the application must store the list of scopeids, and for it it
> > will be a set of half UUIDs.
> >
> > Thanks for driving this
> >
> > Enrico
> >
> >
> >
> > Il ven 17 ago 2018, 23:52 Venkateswara Rao Jujjuri <ju...@gmail.com>
> ha
> > scritto:
> >
> > > Thanks a lot, Sijie for the awesome doc.
> > > Let's pool some more thoughts into this and size the work.
> > >
> > > Thanks,
> > > JV
> > >
> > > On Fri, Aug 17, 2018 at 2:36 PM, Sijie Guo <gu...@gmail.com> wrote:
> > >
> > > > Hi all,
> > > >
> > > > As promised in last community meeting, I put up all the thoughts and
> > > > discussion with JV I had into a BP for supporting 128-bit ledger id.
> I
> > > > tried my best to cover all the aspects that I can think of. There can
> > be
> > > > places that I miss. so please take a look and let me know what you
> > think.
> > > >
> > > > BP PR: https://github.com/apache/bookkeeper/pull/1611
> > > > Google doc:
> > > > https://docs.google.com/document/d/1cu54dNSV2ZrdWCi40LcyX8NxXGRCW
> > > > 0609T_ewmK9BWM
> > > >
> > > > - Sijie
> > > >
> > >
> > >
> > >
> > > --
> > > Jvrao
> > > ---
> > > First they ignore you, then they laugh at you, then they fight you,
> then
> > > you win. - Mahatma Gandhi
> > >
> > --
> >
> >
> > -- Enrico Olivelli
> >
>
--
-- Enrico Olivelli
Re: [DISCUSS] BP-35: 128 bits support
Posted by Sijie Guo <gu...@gmail.com>.
On Sat, Aug 18, 2018 at 2:12 AM Enrico Olivelli <eo...@gmail.com> wrote:
> Nice work Sijie,
> Some comments after the first read, just thinking out loud...
> Overall looks a good idea and implementable.
>
> We should cite somewhere the case of scope=0 and ledgerId = -1 which is a
> very special case for usthe.
>
For BC reason, negative ledger id are not supported when scope == 0. We
effectively only support 63 bit.
We can use versions on clients to distinguish whether they are old clients
or new clients, then handle this case differently.
So this special case should not be special for any new clients.
We should also state that scopeid and ledgerid < 0 is not supported or it
> is not clear to me how we will handle that case.
> The case of scope < 0 is not clear to me as well, maybe I have to read more
> carefully the doc.
I think I answer this with my comment above.
>
> We are treating the global space of ledgerids as a 2 level hierarchy and
> this is good for compatibility.
> But thinking about a new application, which starts with uuids I think it
> can be tricky, maybe we should add utilities/APIs to work with UUID
> directly.
The ledgerQualifiedName is the hex presentation of UUID, proposed to add to
CLI tool.
We can add it to API as well. However I would defer adding it until we need
it or after we are on 128 bits. BC is the top concern.
> For this new kind of application the space of ids would be flat and not
> hierarchical (UUID are not hierarchical as far as I know, but I may be
> wrong).
In a lot of places, 128 bit is not well supported. So people would divide
into two parts any way, msb and lsb.
>
>
> We should also support listing the whole flat space
Sure it can be added. I don’t see a reason we can’t. However IMO these are
new features after we are on 128 bit. For the concerns in this BP, we are
more focused on how we can transition from 64 bit to 128 bit.
or as a last
> alternative we should add a way to list the space of scopeids, because
> otherwise the application must store the list of scopeids, and for it it
> will be a set of half UUIDs.
>
> Thanks for driving this
>
> Enrico
>
>
>
> Il ven 17 ago 2018, 23:52 Venkateswara Rao Jujjuri <ju...@gmail.com> ha
> scritto:
>
> > Thanks a lot, Sijie for the awesome doc.
> > Let's pool some more thoughts into this and size the work.
> >
> > Thanks,
> > JV
> >
> > On Fri, Aug 17, 2018 at 2:36 PM, Sijie Guo <gu...@gmail.com> wrote:
> >
> > > Hi all,
> > >
> > > As promised in last community meeting, I put up all the thoughts and
> > > discussion with JV I had into a BP for supporting 128-bit ledger id. I
> > > tried my best to cover all the aspects that I can think of. There can
> be
> > > places that I miss. so please take a look and let me know what you
> think.
> > >
> > > BP PR: https://github.com/apache/bookkeeper/pull/1611
> > > Google doc:
> > > https://docs.google.com/document/d/1cu54dNSV2ZrdWCi40LcyX8NxXGRCW
> > > 0609T_ewmK9BWM
> > >
> > > - Sijie
> > >
> >
> >
> >
> > --
> > Jvrao
> > ---
> > First they ignore you, then they laugh at you, then they fight you, then
> > you win. - Mahatma Gandhi
> >
> --
>
>
> -- Enrico Olivelli
>
Re: [DISCUSS] BP-35: 128 bits support
Posted by Enrico Olivelli <eo...@gmail.com>.
Nice work Sijie,
Some comments after the first read, just thinking out loud...
Overall looks a good idea and implementable.
We should cite somewhere the case of scope=0 and ledgerId = -1 which is a
very special case for usthe.
We should also state that scopeid and ledgerid < 0 is not supported or it
is not clear to me how we will handle that case.
The case of scope < 0 is not clear to me as well, maybe I have to read more
carefully the doc.
We are treating the global space of ledgerids as a 2 level hierarchy and
this is good for compatibility.
But thinking about a new application, which starts with uuids I think it
can be tricky, maybe we should add utilities/APIs to work with UUID
directly.
For this new kind of application the space of ids would be flat and not
hierarchical (UUID are not hierarchical as far as I know, but I may be
wrong).
We should also support listing the whole flat space or as a last
alternative we should add a way to list the space of scopeids, because
otherwise the application must store the list of scopeids, and for it it
will be a set of half UUIDs.
Thanks for driving this
Enrico
Il ven 17 ago 2018, 23:52 Venkateswara Rao Jujjuri <ju...@gmail.com> ha
scritto:
> Thanks a lot, Sijie for the awesome doc.
> Let's pool some more thoughts into this and size the work.
>
> Thanks,
> JV
>
> On Fri, Aug 17, 2018 at 2:36 PM, Sijie Guo <gu...@gmail.com> wrote:
>
> > Hi all,
> >
> > As promised in last community meeting, I put up all the thoughts and
> > discussion with JV I had into a BP for supporting 128-bit ledger id. I
> > tried my best to cover all the aspects that I can think of. There can be
> > places that I miss. so please take a look and let me know what you think.
> >
> > BP PR: https://github.com/apache/bookkeeper/pull/1611
> > Google doc:
> > https://docs.google.com/document/d/1cu54dNSV2ZrdWCi40LcyX8NxXGRCW
> > 0609T_ewmK9BWM
> >
> > - Sijie
> >
>
>
>
> --
> Jvrao
> ---
> First they ignore you, then they laugh at you, then they fight you, then
> you win. - Mahatma Gandhi
>
--
-- Enrico Olivelli
Re: [DISCUSS] BP-35: 128 bits support
Posted by Venkateswara Rao Jujjuri <ju...@gmail.com>.
Thanks a lot, Sijie for the awesome doc.
Let's pool some more thoughts into this and size the work.
Thanks,
JV
On Fri, Aug 17, 2018 at 2:36 PM, Sijie Guo <gu...@gmail.com> wrote:
> Hi all,
>
> As promised in last community meeting, I put up all the thoughts and
> discussion with JV I had into a BP for supporting 128-bit ledger id. I
> tried my best to cover all the aspects that I can think of. There can be
> places that I miss. so please take a look and let me know what you think.
>
> BP PR: https://github.com/apache/bookkeeper/pull/1611
> Google doc:
> https://docs.google.com/document/d/1cu54dNSV2ZrdWCi40LcyX8NxXGRCW
> 0609T_ewmK9BWM
>
> - Sijie
>
--
Jvrao
---
First they ignore you, then they laugh at you, then they fight you, then
you win. - Mahatma Gandhi