You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hbase.apache.org by Andrew Mains <an...@kontagent.com> on 2015/03/27 19:53:29 UTC

Multiple scans for mapreduce over snapshots

Hi all,

We're looking into using TableSnapshotInputFormat on a salted table, and 
we need to push down conditions on the rest of the rowkey to each bucket 
(using hive with my patch for 
https://issues.apache.org/jira/browse/HIVE-7805). MultiTableInputFormat 
allows us to do this on HBase proper, but it seems like this isn't yet 
supported on snapshots. There's nothing in either google or JIRA 
discussing such a feature afaict, so I thought I'd ask here:

Would it be reasonable for HBase to support an equivalent of 
`MultiTableInputFormat` over snapshots? Is there a better alternative 
that we should be using instead?

Thanks!

Andrew

Re: Multiple scans for mapreduce over snapshots

Posted by Andrew Mains <an...@kontagent.com>.

We'd certainly like it, and would be more than willing to take up the 
implementation :). I've filed a ticket here 
https://issues.apache.org/jira/browse/HBASE-13356 to track the feature.

Thanks for the response!

Andrew

On 3/27/15 11:02 PM, Ted Yu wrote:
> I think adding support for multi table scan over snapshots would be a nice feature to have.
>
>
>> On Mar 27, 2015, at 11:53 AM, Andrew Mains <an...@kontagent.com> wrote:
>>
>> Hi all,
>>
>> We're looking into using TableSnapshotInputFormat on a salted table, and we need to push down conditions on the rest of the rowkey to each bucket (using hive with my patch for https://issues.apache.org/jira/browse/HIVE-7805). MultiTableInputFormat allows us to do this on HBase proper, but it seems like this isn't yet supported on snapshots. There's nothing in either google or JIRA discussing such a feature afaict, so I thought I'd ask here:
>>
>> Would it be reasonable for HBase to support an equivalent of `MultiTableInputFormat` over snapshots? Is there a better alternative that we should be using instead?
>>
>> Thanks!
>>
>> Andrew

Re: Multiple scans for mapreduce over snapshots

Posted by Ted Yu <yu...@gmail.com>.

I think adding support for multi table scan over snapshots would be a nice feature to have. 


> On Mar 27, 2015, at 11:53 AM, Andrew Mains <an...@kontagent.com> wrote:
> 
> Hi all,
> 
> We're looking into using TableSnapshotInputFormat on a salted table, and we need to push down conditions on the rest of the rowkey to each bucket (using hive with my patch for https://issues.apache.org/jira/browse/HIVE-7805). MultiTableInputFormat allows us to do this on HBase proper, but it seems like this isn't yet supported on snapshots. There's nothing in either google or JIRA discussing such a feature afaict, so I thought I'd ask here:
> 
> Would it be reasonable for HBase to support an equivalent of `MultiTableInputFormat` over snapshots? Is there a better alternative that we should be using instead?
> 
> Thanks!
> 
> Andrew