You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@accumulo.apache.org by ameet kini <am...@gmail.com> on 2012/07/13 20:48:18 UTC
tablet distribution
Hi,
Is there a way to force a tablet to move to a particular tablet server?
Thanks,
Ameet
Re: tablet distribution
Posted by ameet kini <am...@gmail.com>.
Thanks, I was looking for something of the equivalent:
http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#move(byte[],
byte[])
The use case is common enough that there may be some other way to do what I
want. I have a table that doesn't have its tablets distributed evenly. At
the system (instance?) level, they are evenly distributed, but not at this
particular table level.
The docs seem to suggest that pre-splitting a table would automatically
guarantee that the tablets would be distributed evenly across the nodes of
the cluster. I see that pre-splitting guarantees that you have that many
tablets, but not necessarily that they are evenly distributed. Maybe if the
instance had only one table, then it would be forced to distribute its
tablets.
Thanks,
Ameet
PRE-SPLITTING TABLES
Accumulo will balance and distribute tables accross servers. Before a table
gets large, it will be maintained as a single tablet on a single server.
This limits the speed at which data can be added or queried to the speed of
a single node. To improve performance when the a table is new, or small,
you can add split points and generate new tablets.
In the shell:
root@myinstance> createtable newTableroot@myinstance> addsplits -t
newTable g n t
This will create a new table with 4 tablets. The table will be split on the
letters g'',n'', and ``t'' which will work nicely if the row data start
with lower-case alphabetic characters. If your row data includes binary
information or numeric information, or if the distribution of the row
information is not flat, then you would pick different split points. Now
ingest and query can proceed on 4 nodes which can improve performance.
On Fri, Jul 13, 2012 at 3:04 PM, Eric Newton <er...@gmail.com> wrote:
> Yes, you need to write your own tablet balancer.
>
> -Eric
>
> On Fri, Jul 13, 2012 at 2:48 PM, ameet kini <am...@gmail.com> wrote:
> >
> > Hi,
> >
> > Is there a way to force a tablet to move to a particular tablet server?
> >
> > Thanks,
> > Ameet
>
Re: tablet distribution
Posted by ameet kini <am...@gmail.com>.
Thanks Billie/John/Eric.
TableLoadBalancer does the trick.
Ameet
On Fri, Jul 13, 2012 at 5:21 PM, Billie J Rinaldi <billie.j.rinaldi@ugov.gov
> wrote:
> On Friday, July 13, 2012 5:14:53 PM, "John Vines" <jo...@ugov.gov>
> wrote:
> > Sounds like you want to use the TableLoadBalancer which will try to
> > even balance all tablets on a table-by-table basis.
>
> To use this, set "master.tablet.balancer" to
> "org.apache.accumulo.server.master.balancer.TableLoadBalancer". This has
> been made the default balancer as of 1.4.1.
>
> Billie
>
>
> > John
> >
> >
> > On Fri, Jul 13, 2012 at 5:09 PM, ameet kini < ameetkini@gmail.com >
> > wrote:
> >
> >
> > Thanks, I was looking for something of the equivalent:
> >
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#move(byte[]
> ,
> > byte[])
> >
> >
> > The use case is common enough that there may be some other way to do
> > what I want. I have a table that doesn't have its tablets distributed
> > evenly. At the system (instance?) level, they are evenly distributed,
> > but not at this particular table level.
> >
> >
> > The docs seem to suggest that pre-splitting a table would
> > automatically guarantee that the tablets would be distributed evenly
> > across the nodes of the cluster. I see that pre-splitting guarantees
> > that you have that many tablets, but not necessarily that they are
> > evenly distributed. Maybe if the instance had only one table, then it
> > would be forced to distribute its tablets.
> >
> >
> >
> > Thanks,
> > Ameet
> >
> >
> > PRE-SPLITTING TABLES
> >
> >
> > Accumulo will balance and distribute tables accross servers. Before a
> > table gets large, it will be maintained as a single tablet on a single
> > server. This limits the speed at which data can be added or queried to
> > the speed of a single node. To improve performance when the a table is
> > new, or small, you can add split points and generate new tablets.
> >
> > In the shell:
> > root @myinstance > createtable newTable root @myinstance > addsplits -
> > t newTable g n t
> >
> > This will create a new table with 4 tablets. The table will be split
> > on the letters g'', n'', and ``t'' which will work nicely if the row
> > data start with lower-case alphabetic characters. If your row data
> > includes binary information or numeric information, or if the
> > distribution of the row information is not flat, then you would pick
> > different split points. Now ingest and query can proceed on 4 nodes
> > which can improve performance.
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On Fri, Jul 13, 2012 at 3:04 PM, Eric Newton < eric.newton@gmail.com >
> > wrote:
> >
> >
> > Yes, you need to write your own tablet balancer.
> >
> > -Eric
> >
> >
> >
> > On Fri, Jul 13, 2012 at 2:48 PM, ameet kini < ameetkini@gmail.com >
> > wrote:
> > >
> > > Hi,
> > >
> > > Is there a way to force a tablet to move to a particular tablet
> > > server?
> > >
> > > Thanks,
> > > Ameet
>
Re: tablet distribution
Posted by Billie J Rinaldi <bi...@ugov.gov>.
On Friday, July 13, 2012 5:14:53 PM, "John Vines" <jo...@ugov.gov> wrote:
> Sounds like you want to use the TableLoadBalancer which will try to
> even balance all tablets on a table-by-table basis.
To use this, set "master.tablet.balancer" to "org.apache.accumulo.server.master.balancer.TableLoadBalancer". This has been made the default balancer as of 1.4.1.
Billie
> John
>
>
> On Fri, Jul 13, 2012 at 5:09 PM, ameet kini < ameetkini@gmail.com >
> wrote:
>
>
> Thanks, I was looking for something of the equivalent:
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#move(byte[],
> byte[])
>
>
> The use case is common enough that there may be some other way to do
> what I want. I have a table that doesn't have its tablets distributed
> evenly. At the system (instance?) level, they are evenly distributed,
> but not at this particular table level.
>
>
> The docs seem to suggest that pre-splitting a table would
> automatically guarantee that the tablets would be distributed evenly
> across the nodes of the cluster. I see that pre-splitting guarantees
> that you have that many tablets, but not necessarily that they are
> evenly distributed. Maybe if the instance had only one table, then it
> would be forced to distribute its tablets.
>
>
>
> Thanks,
> Ameet
>
>
> PRE-SPLITTING TABLES
>
>
> Accumulo will balance and distribute tables accross servers. Before a
> table gets large, it will be maintained as a single tablet on a single
> server. This limits the speed at which data can be added or queried to
> the speed of a single node. To improve performance when the a table is
> new, or small, you can add split points and generate new tablets.
>
> In the shell:
> root @myinstance > createtable newTable root @myinstance > addsplits -
> t newTable g n t
>
> This will create a new table with 4 tablets. The table will be split
> on the letters g'', n'', and ``t'' which will work nicely if the row
> data start with lower-case alphabetic characters. If your row data
> includes binary information or numeric information, or if the
> distribution of the row information is not flat, then you would pick
> different split points. Now ingest and query can proceed on 4 nodes
> which can improve performance.
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Fri, Jul 13, 2012 at 3:04 PM, Eric Newton < eric.newton@gmail.com >
> wrote:
>
>
> Yes, you need to write your own tablet balancer.
>
> -Eric
>
>
>
> On Fri, Jul 13, 2012 at 2:48 PM, ameet kini < ameetkini@gmail.com >
> wrote:
> >
> > Hi,
> >
> > Is there a way to force a tablet to move to a particular tablet
> > server?
> >
> > Thanks,
> > Ameet
Re: tablet distribution
Posted by John Vines <jo...@ugov.gov>.
Sounds like you want to use the TableLoadBalancer which will try to even
balance all tablets on a table-by-table basis.
John
On Fri, Jul 13, 2012 at 5:09 PM, ameet kini <am...@gmail.com> wrote:
> Thanks, I was looking for something of the equivalent:
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html#move(byte[],
> byte[])
>
> The use case is common enough that there may be some other way to do what
> I want. I have a table that doesn't have its tablets distributed evenly. At
> the system (instance?) level, they are evenly distributed, but not at this
> particular table level.
>
> The docs seem to suggest that pre-splitting a table would automatically
> guarantee that the tablets would be distributed evenly across the nodes of
> the cluster. I see that pre-splitting guarantees that you have that many
> tablets, but not necessarily that they are evenly distributed. Maybe if the
> instance had only one table, then it would be forced to distribute its
> tablets.
>
> Thanks,
> Ameet
>
> PRE-SPLITTING TABLES
>
> Accumulo will balance and distribute tables accross servers. Before a
> table gets large, it will be maintained as a single tablet on a single
> server. This limits the speed at which data can be added or queried to the
> speed of a single node. To improve performance when the a table is new, or
> small, you can add split points and generate new tablets.
>
> In the shell:
>
> root@myinstance> createtable newTableroot@myinstance> addsplits -t newTable g n t
>
> This will create a new table with 4 tablets. The table will be split on
> the letters g'',n'', and ``t'' which will work nicely if the row data
> start with lower-case alphabetic characters. If your row data includes
> binary information or numeric information, or if the distribution of the
> row information is not flat, then you would pick different split points.
> Now ingest and query can proceed on 4 nodes which can improve performance.
>
>
>
>
>
>
> On Fri, Jul 13, 2012 at 3:04 PM, Eric Newton <er...@gmail.com>wrote:
>
>> Yes, you need to write your own tablet balancer.
>>
>> -Eric
>>
>> On Fri, Jul 13, 2012 at 2:48 PM, ameet kini <am...@gmail.com> wrote:
>> >
>> > Hi,
>> >
>> > Is there a way to force a tablet to move to a particular tablet server?
>> >
>> > Thanks,
>> > Ameet
>>
>
>
Re: tablet distribution
Posted by Eric Newton <er...@gmail.com>.
Yes, you need to write your own tablet balancer.
-Eric
On Fri, Jul 13, 2012 at 2:48 PM, ameet kini <am...@gmail.com> wrote:
>
> Hi,
>
> Is there a way to force a tablet to move to a particular tablet server?
>
> Thanks,
> Ameet