You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by Flavio Junqueira <fp...@yahoo-inc.com> on 2012/11/12 18:10:34 UTC
BK Blog post
FYI: http://developer.yahoo.com/blogs/ydn/posts/2012/11/bookkeeper-durability-at-scale/
-Flavio
Re: BK Blog post
Posted by Uma Maheswara Rao G <ha...@gmail.com>.
On Thu, Nov 15, 2012 at 12:19 AM, Flavio Junqueira <fp...@yahoo-inc.com> wrote:
> Hi Uma, Thanks for the feedback.
>
> On Nov 14, 2012, at 7:30 PM, Uma Maheswara Rao G wrote:
>
>> Hi Flavio,
>>
>> Thanks a lot for the post. It is greatly explained how BK used with Hedwig.
>>
>> How about having some more info about BKJM? I think Ivan might have
>> written docs about it already.
>>
>
> I think Ivan should have written a blog post about BKJM a while back... but no pressure really. :-)
>
>> One question, currently BookKeeper does not have security feature
>> ready right? How you mange this in production cluster? is that systems
>> running in trusted zone and because running under push gateway?
>
> Great question. We have considered a number of options for BK:
>
> 1- Using something like iptables to block connection requests from non-listed hosts. Given that BK is actually an internal service (internal to a product), this solution works ok;
Oh, In our case also, we followed similar approach right now. That we
have made it configurable and provided for some customers (list of IPs
to allow). Still need Kerberos support for others
> 2- Authenticating through ZooKeeper, which means that a connection request only goes through the client requesting it is able to create a znode. We then rely upon ZooKeeper authentication to do the rest of the job;
This is also done from BKJM perspective in our internal version. Vinay
will post a patch soon in HDFS-3399 along with Rakesh patch in BK
about zk authentication.
> 3- We have developed an experimental authentication code that is able to use the authentication framework we use internally.
>
> Different properties opted for different solutions. Perhaps folks on the list from Yahoo! could comment more on the choices for their respective products.
>
It would be great if they also comment.
> -Flavio
>
>>
>> On Wed, Nov 14, 2012 at 9:08 PM, Flavio Junqueira <fp...@yahoo-inc.com> wrote:
>>> I haven't seen reactions from the community about the post, and I was wondering if people felt it was useful, what kind of detail was missing, etc. This kind of feedback would be great for a follow-up post perhaps.
>>>
>>> Let me know, please.
>>>
>>> -Flavio
>>>
>>> On Nov 12, 2012, at 6:10 PM, Flavio Junqueira wrote:
>>>
>>>> FYI: http://developer.yahoo.com/blogs/ydn/posts/2012/11/bookkeeper-durability-at-scale/
>>>>
>>>> -Flavio
>>>
>
Re: BK Blog post
Posted by Uma Maheswara Rao G <ha...@gmail.com>.
On Thu, Nov 15, 2012 at 12:19 AM, Flavio Junqueira <fp...@yahoo-inc.com> wrote:
> Hi Uma, Thanks for the feedback.
>
> On Nov 14, 2012, at 7:30 PM, Uma Maheswara Rao G wrote:
>
>> Hi Flavio,
>>
>> Thanks a lot for the post. It is greatly explained how BK used with Hedwig.
>>
>> How about having some more info about BKJM? I think Ivan might have
>> written docs about it already.
>>
>
> I think Ivan should have written a blog post about BKJM a while back... but no pressure really. :-)
>
>> One question, currently BookKeeper does not have security feature
>> ready right? How you mange this in production cluster? is that systems
>> running in trusted zone and because running under push gateway?
>
> Great question. We have considered a number of options for BK:
>
> 1- Using something like iptables to block connection requests from non-listed hosts. Given that BK is actually an internal service (internal to a product), this solution works ok;
Oh, In our case also, we followed similar approach right now. That we
have made it configurable and provided for some customers (list of IPs
to allow). Still need Kerberos support for others
> 2- Authenticating through ZooKeeper, which means that a connection request only goes through the client requesting it is able to create a znode. We then rely upon ZooKeeper authentication to do the rest of the job;
This is also done from BKJM perspective in our internal version. Vinay
will post a patch soon in HDFS-3399 along with Rakesh patch in BK
about zk authentication.
> 3- We have developed an experimental authentication code that is able to use the authentication framework we use internally.
>
> Different properties opted for different solutions. Perhaps folks on the list from Yahoo! could comment more on the choices for their respective products.
>
It would be great if they also comment.
> -Flavio
>
>>
>> On Wed, Nov 14, 2012 at 9:08 PM, Flavio Junqueira <fp...@yahoo-inc.com> wrote:
>>> I haven't seen reactions from the community about the post, and I was wondering if people felt it was useful, what kind of detail was missing, etc. This kind of feedback would be great for a follow-up post perhaps.
>>>
>>> Let me know, please.
>>>
>>> -Flavio
>>>
>>> On Nov 12, 2012, at 6:10 PM, Flavio Junqueira wrote:
>>>
>>>> FYI: http://developer.yahoo.com/blogs/ydn/posts/2012/11/bookkeeper-durability-at-scale/
>>>>
>>>> -Flavio
>>>
>
Re: BK Blog post
Posted by Flavio Junqueira <fp...@yahoo-inc.com>.
Hi Uma, Thanks for the feedback.
On Nov 14, 2012, at 7:30 PM, Uma Maheswara Rao G wrote:
> Hi Flavio,
>
> Thanks a lot for the post. It is greatly explained how BK used with Hedwig.
>
> How about having some more info about BKJM? I think Ivan might have
> written docs about it already.
>
I think Ivan should have written a blog post about BKJM a while back... but no pressure really. :-)
> One question, currently BookKeeper does not have security feature
> ready right? How you mange this in production cluster? is that systems
> running in trusted zone and because running under push gateway?
Great question. We have considered a number of options for BK:
1- Using something like iptables to block connection requests from non-listed hosts. Given that BK is actually an internal service (internal to a product), this solution works ok;
2- Authenticating through ZooKeeper, which means that a connection request only goes through the client requesting it is able to create a znode. We then rely upon ZooKeeper authentication to do the rest of the job;
3- We have developed an experimental authentication code that is able to use the authentication framework we use internally.
Different properties opted for different solutions. Perhaps folks on the list from Yahoo! could comment more on the choices for their respective products.
-Flavio
>
> On Wed, Nov 14, 2012 at 9:08 PM, Flavio Junqueira <fp...@yahoo-inc.com> wrote:
>> I haven't seen reactions from the community about the post, and I was wondering if people felt it was useful, what kind of detail was missing, etc. This kind of feedback would be great for a follow-up post perhaps.
>>
>> Let me know, please.
>>
>> -Flavio
>>
>> On Nov 12, 2012, at 6:10 PM, Flavio Junqueira wrote:
>>
>>> FYI: http://developer.yahoo.com/blogs/ydn/posts/2012/11/bookkeeper-durability-at-scale/
>>>
>>> -Flavio
>>
Re: BK Blog post
Posted by Flavio Junqueira <fp...@yahoo-inc.com>.
Hi Uma, Thanks for the feedback.
On Nov 14, 2012, at 7:30 PM, Uma Maheswara Rao G wrote:
> Hi Flavio,
>
> Thanks a lot for the post. It is greatly explained how BK used with Hedwig.
>
> How about having some more info about BKJM? I think Ivan might have
> written docs about it already.
>
I think Ivan should have written a blog post about BKJM a while back... but no pressure really. :-)
> One question, currently BookKeeper does not have security feature
> ready right? How you mange this in production cluster? is that systems
> running in trusted zone and because running under push gateway?
Great question. We have considered a number of options for BK:
1- Using something like iptables to block connection requests from non-listed hosts. Given that BK is actually an internal service (internal to a product), this solution works ok;
2- Authenticating through ZooKeeper, which means that a connection request only goes through the client requesting it is able to create a znode. We then rely upon ZooKeeper authentication to do the rest of the job;
3- We have developed an experimental authentication code that is able to use the authentication framework we use internally.
Different properties opted for different solutions. Perhaps folks on the list from Yahoo! could comment more on the choices for their respective products.
-Flavio
>
> On Wed, Nov 14, 2012 at 9:08 PM, Flavio Junqueira <fp...@yahoo-inc.com> wrote:
>> I haven't seen reactions from the community about the post, and I was wondering if people felt it was useful, what kind of detail was missing, etc. This kind of feedback would be great for a follow-up post perhaps.
>>
>> Let me know, please.
>>
>> -Flavio
>>
>> On Nov 12, 2012, at 6:10 PM, Flavio Junqueira wrote:
>>
>>> FYI: http://developer.yahoo.com/blogs/ydn/posts/2012/11/bookkeeper-durability-at-scale/
>>>
>>> -Flavio
>>
Re: BK Blog post
Posted by Uma Maheswara Rao G <ha...@gmail.com>.
Hi Flavio,
Thanks a lot for the post. It is greatly explained how BK used with Hedwig.
How about having some more info about BKJM? I think Ivan might have
written docs about it already.
One question, currently BookKeeper does not have security feature
ready right? How you mange this in production cluster? is that systems
running in trusted zone and because running under push gateway?
Thanks,
Uma
On Wed, Nov 14, 2012 at 9:08 PM, Flavio Junqueira <fp...@yahoo-inc.com> wrote:
> I haven't seen reactions from the community about the post, and I was wondering if people felt it was useful, what kind of detail was missing, etc. This kind of feedback would be great for a follow-up post perhaps.
>
> Let me know, please.
>
> -Flavio
>
> On Nov 12, 2012, at 6:10 PM, Flavio Junqueira wrote:
>
>> FYI: http://developer.yahoo.com/blogs/ydn/posts/2012/11/bookkeeper-durability-at-scale/
>>
>> -Flavio
>
Re: BK Blog post
Posted by Uma Maheswara Rao G <ha...@gmail.com>.
Hi Flavio,
Thanks a lot for the post. It is greatly explained how BK used with Hedwig.
How about having some more info about BKJM? I think Ivan might have
written docs about it already.
One question, currently BookKeeper does not have security feature
ready right? How you mange this in production cluster? is that systems
running in trusted zone and because running under push gateway?
Thanks,
Uma
On Wed, Nov 14, 2012 at 9:08 PM, Flavio Junqueira <fp...@yahoo-inc.com> wrote:
> I haven't seen reactions from the community about the post, and I was wondering if people felt it was useful, what kind of detail was missing, etc. This kind of feedback would be great for a follow-up post perhaps.
>
> Let me know, please.
>
> -Flavio
>
> On Nov 12, 2012, at 6:10 PM, Flavio Junqueira wrote:
>
>> FYI: http://developer.yahoo.com/blogs/ydn/posts/2012/11/bookkeeper-durability-at-scale/
>>
>> -Flavio
>
Re: BK Blog post
Posted by Flavio Junqueira <fp...@yahoo-inc.com>.
I haven't seen reactions from the community about the post, and I was wondering if people felt it was useful, what kind of detail was missing, etc. This kind of feedback would be great for a follow-up post perhaps.
Let me know, please.
-Flavio
On Nov 12, 2012, at 6:10 PM, Flavio Junqueira wrote:
> FYI: http://developer.yahoo.com/blogs/ydn/posts/2012/11/bookkeeper-durability-at-scale/
>
> -Flavio
Re: BK Blog post
Posted by Flavio Junqueira <fp...@yahoo-inc.com>.
I haven't seen reactions from the community about the post, and I was wondering if people felt it was useful, what kind of detail was missing, etc. This kind of feedback would be great for a follow-up post perhaps.
Let me know, please.
-Flavio
On Nov 12, 2012, at 6:10 PM, Flavio Junqueira wrote:
> FYI: http://developer.yahoo.com/blogs/ydn/posts/2012/11/bookkeeper-durability-at-scale/
>
> -Flavio