You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by Joe Stein <jo...@stealth.ly> on 2014/08/08 16:50:49 UTC
Tablet Server locality
Hi, I have been looking into what it might take to get Accumulo to run on
Apache Mesos.
I am not sure if anyone has done this yet?
My biggest issue/concern is with the Tablet server and how it deals with
not being local to every data node. I feel like maybe I am missing
something here? Lets say you have 100 HDFS data nodes and you need only 10
tablet servers. How does this work best for Accumulo? I am guessing that
the 10 tablet servers just have to deal with data not always being local
and that is the trade off? It seems that a YARN deploy would have the same
issue.
Thanks in advance.
/*******************************************
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
********************************************/
Re: Tablet Server locality
Posted by Joe Stein <jo...@stealth.ly>.
That is what I figured but good to hear others thoughts, opinions and
experiences, thanks!
/*******************************************
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
********************************************/
On Fri, Aug 8, 2014 at 1:47 PM, Eric Newton <er...@gmail.com> wrote:
> The tablet servers will work. They will just fetch data from wherever it
> lives in HDFS. Writes will go local (and replicated out) if there's a
> local datanode. It may not be optimal, but it won't be terrible, either.
>
>
>
> On Fri, Aug 8, 2014 at 10:50 AM, Joe Stein <jo...@stealth.ly> wrote:
>
> > Hi, I have been looking into what it might take to get Accumulo to run on
> > Apache Mesos.
> >
> > I am not sure if anyone has done this yet?
> >
> > My biggest issue/concern is with the Tablet server and how it deals with
> > not being local to every data node. I feel like maybe I am missing
> > something here? Lets say you have 100 HDFS data nodes and you need only
> 10
> > tablet servers. How does this work best for Accumulo? I am guessing
> that
> > the 10 tablet servers just have to deal with data not always being local
> > and that is the trade off? It seems that a YARN deploy would have the
> same
> > issue.
> >
> > Thanks in advance.
> >
> > /*******************************************
> > Joe Stein
> > Founder, Principal Consultant
> > Big Data Open Source Security LLC
> > http://www.stealth.ly
> > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> > ********************************************/
> >
>
Re: Tablet Server locality
Posted by Eric Newton <er...@gmail.com>.
The tablet servers will work. They will just fetch data from wherever it
lives in HDFS. Writes will go local (and replicated out) if there's a
local datanode. It may not be optimal, but it won't be terrible, either.
On Fri, Aug 8, 2014 at 10:50 AM, Joe Stein <jo...@stealth.ly> wrote:
> Hi, I have been looking into what it might take to get Accumulo to run on
> Apache Mesos.
>
> I am not sure if anyone has done this yet?
>
> My biggest issue/concern is with the Tablet server and how it deals with
> not being local to every data node. I feel like maybe I am missing
> something here? Lets say you have 100 HDFS data nodes and you need only 10
> tablet servers. How does this work best for Accumulo? I am guessing that
> the 10 tablet servers just have to deal with data not always being local
> and that is the trade off? It seems that a YARN deploy would have the same
> issue.
>
> Thanks in advance.
>
> /*******************************************
> Joe Stein
> Founder, Principal Consultant
> Big Data Open Source Security LLC
> http://www.stealth.ly
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> ********************************************/
>
Re: Tablet Server locality
Posted by David Medinets <da...@gmail.com>.
Accumulo has worked (is working) on Mesos. Don't know that anyone has
shared more than that fact.
On Fri, Aug 8, 2014 at 10:50 AM, Joe Stein <jo...@stealth.ly> wrote:
> Hi, I have been looking into what it might take to get Accumulo to run on
> Apache Mesos.
>
> I am not sure if anyone has done this yet?
>
> My biggest issue/concern is with the Tablet server and how it deals with
> not being local to every data node. I feel like maybe I am missing
> something here? Lets say you have 100 HDFS data nodes and you need only 10
> tablet servers. How does this work best for Accumulo? I am guessing that
> the 10 tablet servers just have to deal with data not always being local
> and that is the trade off? It seems that a YARN deploy would have the same
> issue.
>
> Thanks in advance.
>
> /*******************************************
> Joe Stein
> Founder, Principal Consultant
> Big Data Open Source Security LLC
> http://www.stealth.ly
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> ********************************************/