Tablet count per table [SEC=UNOFFICIAL]

["Dickson, Matt MR" <ma...@defence.gov.au>]

UNOFFICIAL

Is there a way to query to the metadata table to quickly get a list of the number of tablets for all tables in an Accumulo instance?

I'd like to do this to integrate into existing monitoring tools rather than go to the Accumulo gui.

Thanks in advance

Re: Tablet count per table [SEC=UNOFFICIAL]

[Keith Turner <ke...@deenlo.com>]

There are two operations you can use in the public API to get these
counts.  The following code shows a sketch of what I am thinking,
however I have not tested it.

Connector conn = ....
long sum = 0;
for(String table : conn.tableOperations().tableIdMap().keys()){
   // the number of tablets in a table is the number of splits plus one
   sum += conn.tableOperations().listSplits(table, Integer.MAX_INT).size() + 1;
}

The nice thing about this approach is that it uses public API, so you
are not impacted by changes in the metadata table schema.  However, a
possible downside is that it reads all splits into memory.  So if a
tables splits would not fit into memory, thats not good.   The
scanning approaches suggested by others will not have this memory
problem, but you may have to change your code if the metadata schema
changes.

The Accumulo shell has analogues for listing tables and listing a tables splits.

[1]: http://accumulo.apache.org/1.8/apidocs/org/apache/accumulo/core/client/admin/TableOperations.html#locate%28java.lang.String,%20java.util.Collection%29

On Mon, Jan 23, 2017 at 11:30 PM, Dickson, Matt MR
<ma...@defence.gov.au> wrote:
> UNOFFICIAL
>
> Is there a way to query to the metadata table to quickly get a list of the
> number of tablets for all tables in an Accumulo instance?
>
> I'd like to do this to integrate into existing monitoring tools rather than
> go to the Accumulo gui.
>
> Thanks in advance

Re: Tablet count per table [SEC=UNOFFICIAL]

[Dave Marion <dl...@comcast.net>]

You could use the JSR-223 feature of the shell. I whipped up the javascript, has not been tested, but something like it should work. You can run it using the command:


script -e js -f /<script_location> -a table=<tableName>


Here is a script that counts files for a table. You could easily modify this to fetch a unique colf from the metadata table for any given tablet and if you count based on that, then you have the number of tablets.


importPackage(java.lang);
importPackage(java.util);
importPackage(org.apache.hadoop.io);
importPackage(org.apache.accumulo.core.data);
importPackage(org.apache.accumulo.core.security);


var tableId = connection.tableOperations().tableIdMap().get(table);
var auths = new Authorizations();
var scanner = connection.createScanner("accumulo.metadata", auths);
scanner.setRange(new Range(tableId+;", tableId+"<");
scanner.fetchColumnFamily(new Text("file"));

var iter = scanner.iterator();

var files = 0;
while (iter.hasNext()) {
   files = files + 1;
}

println(files);

> On January 24, 2017 at 9:34 AM Michael Wall <mj...@gmail.com> wrote:
> 
>     Matt,
> 
>     I typically run the following to see all the online tablets
> 
>     scan -t accumulo.metadata -c loc
> 
>     and the following to find all tablets, online and offline
> 
>     scan -t accumulo.metadata -c ~tab
> 
> 
> 
>     On Mon, Jan 23, 2017 at 11:35 PM Josh Elser <elserj@apache.org mailto:elserj@apache.org > wrote:
> 
>         > > I believe this would be the number of unique rows in the Accumulo
> >         metadata table (that don't fall in the "~del" prefix).
> > 
> >         Not sure if we have a class you could just invoke, but this would be
> >         pretty easy to script just using the Accumulo shell.
> > 
> >         Dickson, Matt MR wrote:
> >         > *UNOFFICIAL*
> >         >
> >         > Is there a way to query to the metadata table to quickly get a list of
> >         > the number of tablets for all tables in an Accumulo instance?
> >         > I'd like to do this to integrate into existing monitoring tools rather
> >         > than go to the Accumulo gui.
> >         > Thanks in advance
> > 
> >     > 

Re: Tablet count per table [SEC=UNOFFICIAL]

[Michael Wall <mj...@gmail.com>]

Matt,

I typically run the following to see all the online tablets

scan -t accumulo.metadata -c loc

and the following to find all tablets, online and offline

scan -t accumulo.metadata -c ~tab



On Mon, Jan 23, 2017 at 11:35 PM Josh Elser <el...@apache.org> wrote:

> I believe this would be the number of unique rows in the Accumulo
> metadata table (that don't fall in the "~del" prefix).
>
> Not sure if we have a class you could just invoke, but this would be
> pretty easy to script just using the Accumulo shell.
>
> Dickson, Matt MR wrote:
> > *UNOFFICIAL*
> >
> > Is there a way to query to the metadata table to quickly get a list of
> > the number of tablets for all tables in an Accumulo instance?
> > I'd like to do this to integrate into existing monitoring tools rather
> > than go to the Accumulo gui.
> > Thanks in advance
>

Re: Tablet count per table [SEC=UNOFFICIAL]

[Josh Elser <el...@apache.org>]

I believe this would be the number of unique rows in the Accumulo 
metadata table (that don't fall in the "~del" prefix).

Not sure if we have a class you could just invoke, but this would be 
pretty easy to script just using the Accumulo shell.

Dickson, Matt MR wrote:
> *UNOFFICIAL*
>
> Is there a way to query to the metadata table to quickly get a list of
> the number of tablets for all tables in an Accumulo instance?
> I'd like to do this to integrate into existing monitoring tools rather
> than go to the Accumulo gui.
> Thanks in advance