You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@hadoop.apache.org by Brahma Reddy Battula <br...@apache.org> on 2021/08/06 17:20:23 UTC
Re: [E] Uber's story on running Apache Hadoop deployment in Docker
Thanks mithun, will block calendar based on your availability.
On Fri, Jul 23, 2021 at 10:26 PM Mithun Mathew <mm...@uber.com> wrote:
> Yes, we'd be happy to discuss this topic further.
>
>
>
>
> On Thu, Jul 22, 2021 at 11:43 PM Wei-Chiu Chuang <we...@cloudera.com>
> wrote:
>
>> Totally. That'll make a great topic
>>
>> On Fri, Jul 23, 2021 at 1:06 PM Brahma Reddy Battula <br...@apache.org>
>> wrote:
>>
>>> Thanks for sharing, Nice write up.
>>>
>>> @Wei-Chiu Chuang <we...@apache.org> how about having one syncup
>>> meeting with this topic if Mithun Mathew agrees?
>>>
>>> On Fri, Jul 23, 2021 at 5:08 AM Chao Sun <su...@apache.org> wrote:
>>>
>>>> Great to see the blog post and it's a really nice write-up. Thanks for
>>>> sharing it!
>>>>
>>>> Chao
>>>>
>>>> On Thu, Jul 22, 2021 at 3:05 PM Eric Badger
>>>> <eb...@verizonmedia.com.invalid> wrote:
>>>>
>>>> > Really interesting blog post. Thanks for writing this up!
>>>> >
>>>> >
>>>> >
>>>> https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/RuncContainers.html
>>>> >
>>>> > Just in case you don't know already, you might be interested in the
>>>> > RuncContainerRuntime as well. It has some advantages over
>>>> > DockerLinuxContainerRuntime in the fact that you don't have to deal
>>>> with
>>>> > the docker daemon (potential bottleneck and root escalation attack
>>>> vector)
>>>> > or have a dependency on Docker at all, images are distributed via the
>>>> YARN
>>>> > distributed cache, and you don't have to spend time decompressing
>>>> images up
>>>> > front, among some other smaller wins. If you have any questions, I'd
>>>> be
>>>> > happy to explain the work a little more
>>>> >
>>>> > Eric
>>>> >
>>>> > On Thu, Jul 22, 2021 at 2:59 PM Mithun Mathew
>>>> <mm...@uber.com.invalid>
>>>> > wrote:
>>>> >
>>>> > > Hi all
>>>> > >
>>>> > > We wanted to share our story with the community about migrating the
>>>> > > majority of the Apache Hadoop production fleet at Uber to run in
>>>> Docker
>>>> > > containers.
>>>> > >
>>>> > > Here's a link to our blog post that we published today:
>>>> > >
>>>> > >
>>>> >
>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__eng.uber.com_hadoop-2Dcontainer-2Dblog_&d=DwIBaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=KVdP1SUmHYb-tZP8tcigmw&m=ZCzEveVQ7BrTU-z0LhhshKNnBmVBEE-D2KqytQT_sdg&s=4Hd337BvWYh94n4APYFXMRuc4xP1bUqVMhJEm9kRzF8&e=
>>>> > > Hope our story helps the community in some way, as it has helped us
>>>> in
>>>> > > the past for scaling Uber's Hadoop deployment.
>>>> > >
>>>> > > On behalf of Uber's Hadoop team,
>>>> > > *Matt*
>>>> > >
>>>> >
>>>>
>>>
>>>
>>> --
>>>
>>>
>>>
>>> --Brahma Reddy Battula
>>>
>>
>
> --
> *Matt*
>
--
--Brahma Reddy Battula
Re: [E] Uber's story on running Apache Hadoop deployment in Docker
Posted by Brahma Reddy Battula <br...@apache.org>.
Hi All,
Update on here..
This will be on Sept 10 Or Sept 9, between 09:30 - 11:00 PDT.
Will send a meeting link....Please block your calendar.
On Fri, Aug 6, 2021 at 11:07 PM Mithun Mathew <mm...@uber.com> wrote:
> Sounds good
>
> On Fri, Aug 6, 2021 at 10:20 AM Brahma Reddy Battula <br...@apache.org>
> wrote:
>
>>
>> Thanks mithun, will block calendar based on your availability.
>>
>> On Fri, Jul 23, 2021 at 10:26 PM Mithun Mathew <mm...@uber.com> wrote:
>>
>>> Yes, we'd be happy to discuss this topic further.
>>>
>>>
>>>
>>>
>>> On Thu, Jul 22, 2021 at 11:43 PM Wei-Chiu Chuang <we...@cloudera.com>
>>> wrote:
>>>
>>>> Totally. That'll make a great topic
>>>>
>>>> On Fri, Jul 23, 2021 at 1:06 PM Brahma Reddy Battula <br...@apache.org>
>>>> wrote:
>>>>
>>>>> Thanks for sharing, Nice write up.
>>>>>
>>>>> @Wei-Chiu Chuang <we...@apache.org> how about having one syncup
>>>>> meeting with this topic if Mithun Mathew agrees?
>>>>>
>>>>> On Fri, Jul 23, 2021 at 5:08 AM Chao Sun <su...@apache.org> wrote:
>>>>>
>>>>>> Great to see the blog post and it's a really nice write-up. Thanks for
>>>>>> sharing it!
>>>>>>
>>>>>> Chao
>>>>>>
>>>>>> On Thu, Jul 22, 2021 at 3:05 PM Eric Badger
>>>>>> <eb...@verizonmedia.com.invalid> wrote:
>>>>>>
>>>>>> > Really interesting blog post. Thanks for writing this up!
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/RuncContainers.html
>>>>>> >
>>>>>> > Just in case you don't know already, you might be interested in the
>>>>>> > RuncContainerRuntime as well. It has some advantages over
>>>>>> > DockerLinuxContainerRuntime in the fact that you don't have to deal
>>>>>> with
>>>>>> > the docker daemon (potential bottleneck and root escalation attack
>>>>>> vector)
>>>>>> > or have a dependency on Docker at all, images are distributed via
>>>>>> the YARN
>>>>>> > distributed cache, and you don't have to spend time decompressing
>>>>>> images up
>>>>>> > front, among some other smaller wins. If you have any questions,
>>>>>> I'd be
>>>>>> > happy to explain the work a little more
>>>>>> >
>>>>>> > Eric
>>>>>> >
>>>>>> > On Thu, Jul 22, 2021 at 2:59 PM Mithun Mathew
>>>>>> <mm...@uber.com.invalid>
>>>>>> > wrote:
>>>>>> >
>>>>>> > > Hi all
>>>>>> > >
>>>>>> > > We wanted to share our story with the community about migrating
>>>>>> the
>>>>>> > > majority of the Apache Hadoop production fleet at Uber to run in
>>>>>> Docker
>>>>>> > > containers.
>>>>>> > >
>>>>>> > > Here's a link to our blog post that we published today:
>>>>>> > >
>>>>>> > >
>>>>>> >
>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__eng.uber.com_hadoop-2Dcontainer-2Dblog_&d=DwIBaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=KVdP1SUmHYb-tZP8tcigmw&m=ZCzEveVQ7BrTU-z0LhhshKNnBmVBEE-D2KqytQT_sdg&s=4Hd337BvWYh94n4APYFXMRuc4xP1bUqVMhJEm9kRzF8&e=
>>>>>> > > Hope our story helps the community in some way, as it has helped
>>>>>> us in
>>>>>> > > the past for scaling Uber's Hadoop deployment.
>>>>>> > >
>>>>>> > > On behalf of Uber's Hadoop team,
>>>>>> > > *Matt*
>>>>>> > >
>>>>>> >
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>>
>>>>>
>>>>> --Brahma Reddy Battula
>>>>>
>>>>
>>>
>>> --
>>> *Matt*
>>>
>>
>>
>> --
>>
>>
>>
>> --Brahma Reddy Battula
>>
>
>
> --
> *Matt*
>
--
--Brahma Reddy Battula
Re: [E] Uber's story on running Apache Hadoop deployment in Docker
Posted by Mithun Mathew <mm...@uber.com.INVALID>.
Sounds good
On Fri, Aug 6, 2021 at 10:20 AM Brahma Reddy Battula <br...@apache.org>
wrote:
>
> Thanks mithun, will block calendar based on your availability.
>
> On Fri, Jul 23, 2021 at 10:26 PM Mithun Mathew <mm...@uber.com> wrote:
>
>> Yes, we'd be happy to discuss this topic further.
>>
>>
>>
>>
>> On Thu, Jul 22, 2021 at 11:43 PM Wei-Chiu Chuang <we...@cloudera.com>
>> wrote:
>>
>>> Totally. That'll make a great topic
>>>
>>> On Fri, Jul 23, 2021 at 1:06 PM Brahma Reddy Battula <br...@apache.org>
>>> wrote:
>>>
>>>> Thanks for sharing, Nice write up.
>>>>
>>>> @Wei-Chiu Chuang <we...@apache.org> how about having one syncup
>>>> meeting with this topic if Mithun Mathew agrees?
>>>>
>>>> On Fri, Jul 23, 2021 at 5:08 AM Chao Sun <su...@apache.org> wrote:
>>>>
>>>>> Great to see the blog post and it's a really nice write-up. Thanks for
>>>>> sharing it!
>>>>>
>>>>> Chao
>>>>>
>>>>> On Thu, Jul 22, 2021 at 3:05 PM Eric Badger
>>>>> <eb...@verizonmedia.com.invalid> wrote:
>>>>>
>>>>> > Really interesting blog post. Thanks for writing this up!
>>>>> >
>>>>> >
>>>>> >
>>>>> https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/RuncContainers.html
>>>>> >
>>>>> > Just in case you don't know already, you might be interested in the
>>>>> > RuncContainerRuntime as well. It has some advantages over
>>>>> > DockerLinuxContainerRuntime in the fact that you don't have to deal
>>>>> with
>>>>> > the docker daemon (potential bottleneck and root escalation attack
>>>>> vector)
>>>>> > or have a dependency on Docker at all, images are distributed via
>>>>> the YARN
>>>>> > distributed cache, and you don't have to spend time decompressing
>>>>> images up
>>>>> > front, among some other smaller wins. If you have any questions, I'd
>>>>> be
>>>>> > happy to explain the work a little more
>>>>> >
>>>>> > Eric
>>>>> >
>>>>> > On Thu, Jul 22, 2021 at 2:59 PM Mithun Mathew
>>>>> <mm...@uber.com.invalid>
>>>>> > wrote:
>>>>> >
>>>>> > > Hi all
>>>>> > >
>>>>> > > We wanted to share our story with the community about migrating the
>>>>> > > majority of the Apache Hadoop production fleet at Uber to run in
>>>>> Docker
>>>>> > > containers.
>>>>> > >
>>>>> > > Here's a link to our blog post that we published today:
>>>>> > >
>>>>> > >
>>>>> >
>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__eng.uber.com_hadoop-2Dcontainer-2Dblog_&d=DwIBaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=KVdP1SUmHYb-tZP8tcigmw&m=ZCzEveVQ7BrTU-z0LhhshKNnBmVBEE-D2KqytQT_sdg&s=4Hd337BvWYh94n4APYFXMRuc4xP1bUqVMhJEm9kRzF8&e=
>>>>> > > Hope our story helps the community in some way, as it has helped
>>>>> us in
>>>>> > > the past for scaling Uber's Hadoop deployment.
>>>>> > >
>>>>> > > On behalf of Uber's Hadoop team,
>>>>> > > *Matt*
>>>>> > >
>>>>> >
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>>
>>>>
>>>> --Brahma Reddy Battula
>>>>
>>>
>>
>> --
>> *Matt*
>>
>
>
> --
>
>
>
> --Brahma Reddy Battula
>
--
*Matt*