You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by Venkateswara Rao Jujjuri <ju...@gmail.com> on 2019/05/19 01:48:07 UTC
Cookie for FaultDomain Info?
In the current code, bookie to faultDomain mapping is supplied through
different methods. Salesforce uses a script to read yaml file which
contains racks/machines mapping. But I am wondering why can't we put this
info in the Cookie? Assuming that these machines can never move across
fault zones.
Currently cookies contain version, Host, JourlanDirs, ledgerDirs, and
instanceId.
If we add faultzoneId to it, it will be always available for everyone to
look into.
Is there any reason why it would be a bad idea?
Thanks,
--
Jvrao
---
First they ignore you, then they laugh at you, then they fight you, then
you win. - Mahatma Gandhi
Re: Cookie for FaultDomain Info?
Posted by Enrico Olivelli <eo...@gmail.com>.
Thinking more about this topic....
What about retrieving this information with getBookieInfo?
We can add faultDomainId and we can add other info useful both for
placement.
For instance we could add info about the ability to accept secure
communications, the current bookkeeper version.....
An important thing is that this information is available to the placement
policy in a future proof way
Enrico
Il lun 27 mag 2019, 09:19 Enrico Olivelli <eo...@gmail.com> ha scritto:
>
>
> Il lun 27 mag 2019, 03:13 Venkateswara Rao Jujjuri <ju...@gmail.com> ha
> scritto:
>
>> > I will also send soon a request for enhancements and a PR to have the
>> list of bookies
>>
>> List of bookies in the cluster? how the available and RO are listed
>
>
> If a bookie is down it is not present in any of the two lists.
>
> or list
>> of bookies in the metadata?
>>
>
> Actually we are listing /ledgers/cookies
> There is no API for this scan, you have to query ZK directly
>
> Enrico
>
>
>> On Sat, May 25, 2019 at 9:02 AM Enrico Olivelli <eo...@gmail.com>
>> wrote:
>>
>> > JV,
>> > sorry for so late reply.
>> >
>> >
>> > Il lun 20 mag 2019, 07:54 Venkateswara Rao Jujjuri <ju...@gmail.com>
>> ha
>> > scritto:
>> >
>> > > > we should take care of designing a better API for the placement
>> policy
>> > > +1 Our placement policy is super complex and confusing.
>> > >
>> > > > We could also take into account the ability of adding labels/tags to
>> > > bookies.
>> > >
>> > > Can you add more context and color to your statement?
>> > > In the k8s world we do have a way to add tags. Can you please
>> elaborate
>> > > what you are thinking?
>> > >
>> >
>> > Most of my products have strong requirements about multi tenancy and
>> smart
>> > resource allocation.
>> > I have always very small clusters (3-5 bookies) so rack/region
>> awareness is
>> > not my primary focus (I have very few cases of multi region clusters).
>> > Currently if you want to drive the allocation of resources you have to
>> > write your own placement policy, put some tenant binding info on 'custom
>> > metadata' and use some out-of-band (in respect to BK) information about
>> > bookies and which bookie can be used for the tenant who is requesting
>> the
>> > ledger placement.
>> >
>> > Having some sort of tags directly set on bookies zk directory will drop
>> the
>> > need for such out of bound information distribution.
>> >
>> > Some year ago we discussed about this topic but the discussion did not
>> go
>> > anywhere.
>> > IIRC the reason was that generic labels may be too generic and make the
>> > placement rules too complex.
>> >
>> > I see value on this fault domain id.
>> >
>> > I think we should have some information on zookeeper about which fault
>> > domains are defined.
>> > We can't discover new fault domains just by listing bookies metadata.
>> >
>> > I will also send soon a request for enhancements and a PR to have the
>> list
>> > of bookies (currently we only have APIs to discovery bookies that are up
>> > and running).
>> > One of my colleagues is working on management tools and we found the
>> lack
>> > of this information.
>> > The script approach for dns to switch mapping also makes it impossible
>> to
>> > understand the topology of the cluster without testing every bookie
>> > address.
>> >
>> > Enrico
>> >
>> >
>> >
>> > >
>> > > On Sun, May 19, 2019 at 10:23 PM Enrico Olivelli <eolivelli@gmail.com
>> >
>> > > wrote:
>> > >
>> > > > Il lun 20 mag 2019, 05:03 Sijie Guo <gu...@gmail.com> ha
>> scritto:
>> > > >
>> > > > > +1 from me. `Cookie` was designed for keeping the informations
>> that
>> > is
>> > > > > associated with a bookie (e.g. disk layouts, bookie id and etc).
>> > > > >
>> > > > > I think it is making sense to have `FaultZoneId` stored as part of
>> > the
>> > > > > cookie.
>> > > > >
>> > > >
>> > > > I agree.
>> > > >
>> > > > But we should take care of designing a better API for the placement
>> > > policy.
>> > > > We are changing signatures quite often, adding parameters, changing
>> > > return
>> > > > type....
>> > > >
>> > > > We could also take into account the ability of adding labels/tags to
>> > > > bookies.
>> > > >
>> > > > Enrico
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > > - Sijie
>> > > > >
>> > > > > On Sun, May 19, 2019 at 9:48 AM Venkateswara Rao Jujjuri <
>> > > > > jujjuri@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > > > In the current code, bookie to faultDomain mapping is supplied
>> > > through
>> > > > > > different methods. Salesforce uses a script to read yaml file
>> which
>> > > > > > contains racks/machines mapping. But I am wondering why can't we
>> > put
>> > > > this
>> > > > > > info in the Cookie? Assuming that these machines can never move
>> > > across
>> > > > > > fault zones.
>> > > > > >
>> > > > > > Currently cookies contain version, Host, JourlanDirs,
>> ledgerDirs,
>> > > and
>> > > > > > instanceId.
>> > > > > > If we add faultzoneId to it, it will be always available for
>> > everyone
>> > > > to
>> > > > > > look into.
>> > > > > > Is there any reason why it would be a bad idea?
>> > > > > >
>> > > > > > Thanks,
>> > > > > > --
>> > > > > > Jvrao
>> > > > > > ---
>> > > > > > First they ignore you, then they laugh at you, then they fight
>> you,
>> > > > then
>> > > > > > you win. - Mahatma Gandhi
>> > > > > >
>> > > > >
>> > > >
>> > >
>> > >
>> > > --
>> > > Jvrao
>> > > ---
>> > > First they ignore you, then they laugh at you, then they fight you,
>> then
>> > > you win. - Mahatma Gandhi
>> > >
>> >
>>
>>
>> --
>> Jvrao
>> ---
>> First they ignore you, then they laugh at you, then they fight you, then
>> you win. - Mahatma Gandhi
>>
>
Re: Cookie for FaultDomain Info?
Posted by Enrico Olivelli <eo...@gmail.com>.
Il lun 27 mag 2019, 03:13 Venkateswara Rao Jujjuri <ju...@gmail.com> ha
scritto:
> > I will also send soon a request for enhancements and a PR to have the
> list of bookies
>
> List of bookies in the cluster? how the available and RO are listed
If a bookie is down it is not present in any of the two lists.
or list
> of bookies in the metadata?
>
Actually we are listing /ledgers/cookies
There is no API for this scan, you have to query ZK directly
Enrico
> On Sat, May 25, 2019 at 9:02 AM Enrico Olivelli <eo...@gmail.com>
> wrote:
>
> > JV,
> > sorry for so late reply.
> >
> >
> > Il lun 20 mag 2019, 07:54 Venkateswara Rao Jujjuri <ju...@gmail.com>
> ha
> > scritto:
> >
> > > > we should take care of designing a better API for the placement
> policy
> > > +1 Our placement policy is super complex and confusing.
> > >
> > > > We could also take into account the ability of adding labels/tags to
> > > bookies.
> > >
> > > Can you add more context and color to your statement?
> > > In the k8s world we do have a way to add tags. Can you please elaborate
> > > what you are thinking?
> > >
> >
> > Most of my products have strong requirements about multi tenancy and
> smart
> > resource allocation.
> > I have always very small clusters (3-5 bookies) so rack/region awareness
> is
> > not my primary focus (I have very few cases of multi region clusters).
> > Currently if you want to drive the allocation of resources you have to
> > write your own placement policy, put some tenant binding info on 'custom
> > metadata' and use some out-of-band (in respect to BK) information about
> > bookies and which bookie can be used for the tenant who is requesting the
> > ledger placement.
> >
> > Having some sort of tags directly set on bookies zk directory will drop
> the
> > need for such out of bound information distribution.
> >
> > Some year ago we discussed about this topic but the discussion did not go
> > anywhere.
> > IIRC the reason was that generic labels may be too generic and make the
> > placement rules too complex.
> >
> > I see value on this fault domain id.
> >
> > I think we should have some information on zookeeper about which fault
> > domains are defined.
> > We can't discover new fault domains just by listing bookies metadata.
> >
> > I will also send soon a request for enhancements and a PR to have the
> list
> > of bookies (currently we only have APIs to discovery bookies that are up
> > and running).
> > One of my colleagues is working on management tools and we found the
> lack
> > of this information.
> > The script approach for dns to switch mapping also makes it impossible to
> > understand the topology of the cluster without testing every bookie
> > address.
> >
> > Enrico
> >
> >
> >
> > >
> > > On Sun, May 19, 2019 at 10:23 PM Enrico Olivelli <eo...@gmail.com>
> > > wrote:
> > >
> > > > Il lun 20 mag 2019, 05:03 Sijie Guo <gu...@gmail.com> ha scritto:
> > > >
> > > > > +1 from me. `Cookie` was designed for keeping the informations that
> > is
> > > > > associated with a bookie (e.g. disk layouts, bookie id and etc).
> > > > >
> > > > > I think it is making sense to have `FaultZoneId` stored as part of
> > the
> > > > > cookie.
> > > > >
> > > >
> > > > I agree.
> > > >
> > > > But we should take care of designing a better API for the placement
> > > policy.
> > > > We are changing signatures quite often, adding parameters, changing
> > > return
> > > > type....
> > > >
> > > > We could also take into account the ability of adding labels/tags to
> > > > bookies.
> > > >
> > > > Enrico
> > > >
> > > >
> > > >
> > > >
> > > > > - Sijie
> > > > >
> > > > > On Sun, May 19, 2019 at 9:48 AM Venkateswara Rao Jujjuri <
> > > > > jujjuri@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > In the current code, bookie to faultDomain mapping is supplied
> > > through
> > > > > > different methods. Salesforce uses a script to read yaml file
> which
> > > > > > contains racks/machines mapping. But I am wondering why can't we
> > put
> > > > this
> > > > > > info in the Cookie? Assuming that these machines can never move
> > > across
> > > > > > fault zones.
> > > > > >
> > > > > > Currently cookies contain version, Host, JourlanDirs,
> ledgerDirs,
> > > and
> > > > > > instanceId.
> > > > > > If we add faultzoneId to it, it will be always available for
> > everyone
> > > > to
> > > > > > look into.
> > > > > > Is there any reason why it would be a bad idea?
> > > > > >
> > > > > > Thanks,
> > > > > > --
> > > > > > Jvrao
> > > > > > ---
> > > > > > First they ignore you, then they laugh at you, then they fight
> you,
> > > > then
> > > > > > you win. - Mahatma Gandhi
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > > Jvrao
> > > ---
> > > First they ignore you, then they laugh at you, then they fight you,
> then
> > > you win. - Mahatma Gandhi
> > >
> >
>
>
> --
> Jvrao
> ---
> First they ignore you, then they laugh at you, then they fight you, then
> you win. - Mahatma Gandhi
>
Re: Cookie for FaultDomain Info?
Posted by Venkateswara Rao Jujjuri <ju...@gmail.com>.
> I will also send soon a request for enhancements and a PR to have the
list of bookies
List of bookies in the cluster? how the available and RO are listed or list
of bookies in the metadata?
On Sat, May 25, 2019 at 9:02 AM Enrico Olivelli <eo...@gmail.com> wrote:
> JV,
> sorry for so late reply.
>
>
> Il lun 20 mag 2019, 07:54 Venkateswara Rao Jujjuri <ju...@gmail.com> ha
> scritto:
>
> > > we should take care of designing a better API for the placement policy
> > +1 Our placement policy is super complex and confusing.
> >
> > > We could also take into account the ability of adding labels/tags to
> > bookies.
> >
> > Can you add more context and color to your statement?
> > In the k8s world we do have a way to add tags. Can you please elaborate
> > what you are thinking?
> >
>
> Most of my products have strong requirements about multi tenancy and smart
> resource allocation.
> I have always very small clusters (3-5 bookies) so rack/region awareness is
> not my primary focus (I have very few cases of multi region clusters).
> Currently if you want to drive the allocation of resources you have to
> write your own placement policy, put some tenant binding info on 'custom
> metadata' and use some out-of-band (in respect to BK) information about
> bookies and which bookie can be used for the tenant who is requesting the
> ledger placement.
>
> Having some sort of tags directly set on bookies zk directory will drop the
> need for such out of bound information distribution.
>
> Some year ago we discussed about this topic but the discussion did not go
> anywhere.
> IIRC the reason was that generic labels may be too generic and make the
> placement rules too complex.
>
> I see value on this fault domain id.
>
> I think we should have some information on zookeeper about which fault
> domains are defined.
> We can't discover new fault domains just by listing bookies metadata.
>
> I will also send soon a request for enhancements and a PR to have the list
> of bookies (currently we only have APIs to discovery bookies that are up
> and running).
> One of my colleagues is working on management tools and we found the lack
> of this information.
> The script approach for dns to switch mapping also makes it impossible to
> understand the topology of the cluster without testing every bookie
> address.
>
> Enrico
>
>
>
> >
> > On Sun, May 19, 2019 at 10:23 PM Enrico Olivelli <eo...@gmail.com>
> > wrote:
> >
> > > Il lun 20 mag 2019, 05:03 Sijie Guo <gu...@gmail.com> ha scritto:
> > >
> > > > +1 from me. `Cookie` was designed for keeping the informations that
> is
> > > > associated with a bookie (e.g. disk layouts, bookie id and etc).
> > > >
> > > > I think it is making sense to have `FaultZoneId` stored as part of
> the
> > > > cookie.
> > > >
> > >
> > > I agree.
> > >
> > > But we should take care of designing a better API for the placement
> > policy.
> > > We are changing signatures quite often, adding parameters, changing
> > return
> > > type....
> > >
> > > We could also take into account the ability of adding labels/tags to
> > > bookies.
> > >
> > > Enrico
> > >
> > >
> > >
> > >
> > > > - Sijie
> > > >
> > > > On Sun, May 19, 2019 at 9:48 AM Venkateswara Rao Jujjuri <
> > > > jujjuri@gmail.com>
> > > > wrote:
> > > >
> > > > > In the current code, bookie to faultDomain mapping is supplied
> > through
> > > > > different methods. Salesforce uses a script to read yaml file which
> > > > > contains racks/machines mapping. But I am wondering why can't we
> put
> > > this
> > > > > info in the Cookie? Assuming that these machines can never move
> > across
> > > > > fault zones.
> > > > >
> > > > > Currently cookies contain version, Host, JourlanDirs, ledgerDirs,
> > and
> > > > > instanceId.
> > > > > If we add faultzoneId to it, it will be always available for
> everyone
> > > to
> > > > > look into.
> > > > > Is there any reason why it would be a bad idea?
> > > > >
> > > > > Thanks,
> > > > > --
> > > > > Jvrao
> > > > > ---
> > > > > First they ignore you, then they laugh at you, then they fight you,
> > > then
> > > > > you win. - Mahatma Gandhi
> > > > >
> > > >
> > >
> >
> >
> > --
> > Jvrao
> > ---
> > First they ignore you, then they laugh at you, then they fight you, then
> > you win. - Mahatma Gandhi
> >
>
--
Jvrao
---
First they ignore you, then they laugh at you, then they fight you, then
you win. - Mahatma Gandhi
Re: Cookie for FaultDomain Info?
Posted by Enrico Olivelli <eo...@gmail.com>.
JV,
sorry for so late reply.
Il lun 20 mag 2019, 07:54 Venkateswara Rao Jujjuri <ju...@gmail.com> ha
scritto:
> > we should take care of designing a better API for the placement policy
> +1 Our placement policy is super complex and confusing.
>
> > We could also take into account the ability of adding labels/tags to
> bookies.
>
> Can you add more context and color to your statement?
> In the k8s world we do have a way to add tags. Can you please elaborate
> what you are thinking?
>
Most of my products have strong requirements about multi tenancy and smart
resource allocation.
I have always very small clusters (3-5 bookies) so rack/region awareness is
not my primary focus (I have very few cases of multi region clusters).
Currently if you want to drive the allocation of resources you have to
write your own placement policy, put some tenant binding info on 'custom
metadata' and use some out-of-band (in respect to BK) information about
bookies and which bookie can be used for the tenant who is requesting the
ledger placement.
Having some sort of tags directly set on bookies zk directory will drop the
need for such out of bound information distribution.
Some year ago we discussed about this topic but the discussion did not go
anywhere.
IIRC the reason was that generic labels may be too generic and make the
placement rules too complex.
I see value on this fault domain id.
I think we should have some information on zookeeper about which fault
domains are defined.
We can't discover new fault domains just by listing bookies metadata.
I will also send soon a request for enhancements and a PR to have the list
of bookies (currently we only have APIs to discovery bookies that are up
and running).
One of my colleagues is working on management tools and we found the lack
of this information.
The script approach for dns to switch mapping also makes it impossible to
understand the topology of the cluster without testing every bookie address.
Enrico
>
> On Sun, May 19, 2019 at 10:23 PM Enrico Olivelli <eo...@gmail.com>
> wrote:
>
> > Il lun 20 mag 2019, 05:03 Sijie Guo <gu...@gmail.com> ha scritto:
> >
> > > +1 from me. `Cookie` was designed for keeping the informations that is
> > > associated with a bookie (e.g. disk layouts, bookie id and etc).
> > >
> > > I think it is making sense to have `FaultZoneId` stored as part of the
> > > cookie.
> > >
> >
> > I agree.
> >
> > But we should take care of designing a better API for the placement
> policy.
> > We are changing signatures quite often, adding parameters, changing
> return
> > type....
> >
> > We could also take into account the ability of adding labels/tags to
> > bookies.
> >
> > Enrico
> >
> >
> >
> >
> > > - Sijie
> > >
> > > On Sun, May 19, 2019 at 9:48 AM Venkateswara Rao Jujjuri <
> > > jujjuri@gmail.com>
> > > wrote:
> > >
> > > > In the current code, bookie to faultDomain mapping is supplied
> through
> > > > different methods. Salesforce uses a script to read yaml file which
> > > > contains racks/machines mapping. But I am wondering why can't we put
> > this
> > > > info in the Cookie? Assuming that these machines can never move
> across
> > > > fault zones.
> > > >
> > > > Currently cookies contain version, Host, JourlanDirs, ledgerDirs,
> and
> > > > instanceId.
> > > > If we add faultzoneId to it, it will be always available for everyone
> > to
> > > > look into.
> > > > Is there any reason why it would be a bad idea?
> > > >
> > > > Thanks,
> > > > --
> > > > Jvrao
> > > > ---
> > > > First they ignore you, then they laugh at you, then they fight you,
> > then
> > > > you win. - Mahatma Gandhi
> > > >
> > >
> >
>
>
> --
> Jvrao
> ---
> First they ignore you, then they laugh at you, then they fight you, then
> you win. - Mahatma Gandhi
>
Re: Cookie for FaultDomain Info?
Posted by Venkateswara Rao Jujjuri <ju...@gmail.com>.
> we should take care of designing a better API for the placement policy
+1 Our placement policy is super complex and confusing.
> We could also take into account the ability of adding labels/tags to
bookies.
Can you add more context and color to your statement?
In the k8s world we do have a way to add tags. Can you please elaborate
what you are thinking?
On Sun, May 19, 2019 at 10:23 PM Enrico Olivelli <eo...@gmail.com>
wrote:
> Il lun 20 mag 2019, 05:03 Sijie Guo <gu...@gmail.com> ha scritto:
>
> > +1 from me. `Cookie` was designed for keeping the informations that is
> > associated with a bookie (e.g. disk layouts, bookie id and etc).
> >
> > I think it is making sense to have `FaultZoneId` stored as part of the
> > cookie.
> >
>
> I agree.
>
> But we should take care of designing a better API for the placement policy.
> We are changing signatures quite often, adding parameters, changing return
> type....
>
> We could also take into account the ability of adding labels/tags to
> bookies.
>
> Enrico
>
>
>
>
> > - Sijie
> >
> > On Sun, May 19, 2019 at 9:48 AM Venkateswara Rao Jujjuri <
> > jujjuri@gmail.com>
> > wrote:
> >
> > > In the current code, bookie to faultDomain mapping is supplied through
> > > different methods. Salesforce uses a script to read yaml file which
> > > contains racks/machines mapping. But I am wondering why can't we put
> this
> > > info in the Cookie? Assuming that these machines can never move across
> > > fault zones.
> > >
> > > Currently cookies contain version, Host, JourlanDirs, ledgerDirs, and
> > > instanceId.
> > > If we add faultzoneId to it, it will be always available for everyone
> to
> > > look into.
> > > Is there any reason why it would be a bad idea?
> > >
> > > Thanks,
> > > --
> > > Jvrao
> > > ---
> > > First they ignore you, then they laugh at you, then they fight you,
> then
> > > you win. - Mahatma Gandhi
> > >
> >
>
--
Jvrao
---
First they ignore you, then they laugh at you, then they fight you, then
you win. - Mahatma Gandhi
Re: Cookie for FaultDomain Info?
Posted by Enrico Olivelli <eo...@gmail.com>.
Il lun 20 mag 2019, 05:03 Sijie Guo <gu...@gmail.com> ha scritto:
> +1 from me. `Cookie` was designed for keeping the informations that is
> associated with a bookie (e.g. disk layouts, bookie id and etc).
>
> I think it is making sense to have `FaultZoneId` stored as part of the
> cookie.
>
I agree.
But we should take care of designing a better API for the placement policy.
We are changing signatures quite often, adding parameters, changing return
type....
We could also take into account the ability of adding labels/tags to
bookies.
Enrico
> - Sijie
>
> On Sun, May 19, 2019 at 9:48 AM Venkateswara Rao Jujjuri <
> jujjuri@gmail.com>
> wrote:
>
> > In the current code, bookie to faultDomain mapping is supplied through
> > different methods. Salesforce uses a script to read yaml file which
> > contains racks/machines mapping. But I am wondering why can't we put this
> > info in the Cookie? Assuming that these machines can never move across
> > fault zones.
> >
> > Currently cookies contain version, Host, JourlanDirs, ledgerDirs, and
> > instanceId.
> > If we add faultzoneId to it, it will be always available for everyone to
> > look into.
> > Is there any reason why it would be a bad idea?
> >
> > Thanks,
> > --
> > Jvrao
> > ---
> > First they ignore you, then they laugh at you, then they fight you, then
> > you win. - Mahatma Gandhi
> >
>
Re: Cookie for FaultDomain Info?
Posted by Sijie Guo <gu...@gmail.com>.
+1 from me. `Cookie` was designed for keeping the informations that is
associated with a bookie (e.g. disk layouts, bookie id and etc).
I think it is making sense to have `FaultZoneId` stored as part of the
cookie.
- Sijie
On Sun, May 19, 2019 at 9:48 AM Venkateswara Rao Jujjuri <ju...@gmail.com>
wrote:
> In the current code, bookie to faultDomain mapping is supplied through
> different methods. Salesforce uses a script to read yaml file which
> contains racks/machines mapping. But I am wondering why can't we put this
> info in the Cookie? Assuming that these machines can never move across
> fault zones.
>
> Currently cookies contain version, Host, JourlanDirs, ledgerDirs, and
> instanceId.
> If we add faultzoneId to it, it will be always available for everyone to
> look into.
> Is there any reason why it would be a bad idea?
>
> Thanks,
> --
> Jvrao
> ---
> First they ignore you, then they laugh at you, then they fight you, then
> you win. - Mahatma Gandhi
>