Fastest way to get count of records in huge hbase table?

[Ramasubramanian Narayanan <ra...@gmail.com>]

Dear All,

Is there any fastest way to get the count of records in a huge HBASE table
with billions of records?

The normal count command is running for a hour with this huge volume of
data..

regards,
Rams

Re: 答复: Fastest way to get count of records in huge hbase table?

[James Taylor <jt...@salesforce.com>]

Use Phoenix (https://github.com/forcedotcom/phoenix) by doing the following:
CREATE VIEW "myHTableName" (key VARBINARY NOT NULL PRIMARY KEY);
SELECT COUNT(*) FROM "myHTableName";

As fenghonghua@xiaomi.com said, you still need to scan the table, but
Phoenix will do it in parallel and use a coprocessor and an internal
scanner API to speed things up.

Thanks,
James
@JamesPlusPlus


On Tue, Sep 10, 2013 at 7:01 PM, 冯宏华 <fe...@xiaomi.com> wrote:

> No fast way to get the count of records of a table without scanning and
> counting, especially when you want to get the accurate count. By design the
> data/cells of a same record/row can scatter in many different HFiles and
> memstore, so even we can record the count of records of each HFile as meta
> in FileInfo, we still need to de-dup to get the accurate total count, which
> only can be achieved by scanning.
> ________________________________________
> 发件人: Ramasubramanian Narayanan [ramasubramanian.narayanan@gmail.com]
> 发送时间: 2013年9月10日 16:07
> 收件人: user@hbase.apache.org
> 主题: Fastest way to get count of records in huge hbase table?
>
> Dear All,
>
> Is there any fastest way to get the count of records in a huge HBASE table
> with billions of records?
>
> The normal count command is running for a hour with this huge volume of
> data..
>
> regards,
> Rams
>

答复: Fastest way to get count of records in huge hbase table?

[冯宏华 <fe...@xiaomi.com>]

No fast way to get the count of records of a table without scanning and counting, especially when you want to get the accurate count. By design the data/cells of a same record/row can scatter in many different HFiles and memstore, so even we can record the count of records of each HFile as meta in FileInfo, we still need to de-dup to get the accurate total count, which only can be achieved by scanning.
________________________________________
发件人: Ramasubramanian Narayanan [ramasubramanian.narayanan@gmail.com]
发送时间: 2013年9月10日 16:07
收件人: user@hbase.apache.org
主题: Fastest way to get count of records in huge hbase table?

Dear All,

Is there any fastest way to get the count of records in a huge HBASE table
with billions of records?

The normal count command is running for a hour with this huge volume of
data..

regards,
Rams

Re: Fastest way to get count of records in huge hbase table?

[Ashwanth Kumar <as...@googlemail.com>]

Try the RowCounter MR Job, that comes with HBase.

[1] http://hbase.apache.org/book/ops_mgt.html#rowcounter


On Tue, Sep 10, 2013 at 1:37 PM, Ramasubramanian Narayanan <
ramasubramanian.narayanan@gmail.com> wrote:

> Dear All,
>
> Is there any fastest way to get the count of records in a huge HBASE table
> with billions of records?
>
> The normal count command is running for a hour with this huge volume of
> data..
>
> regards,
> Rams
>



-- 

Ashwanth Kumar / ashwanthkumar.in