You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Patrick Datko <pa...@ymc.ch> on 2010/05/18 18:25:46 UTC

HBase MR with Filter

Hey,

i'm building a Map Reduce Job which should get Data from a HBase Table
filter it an store the reduced data in another HBase Table. I used the
SingleColumnValueFilter to limit the Data that will be commit to the
Map-Process. The Problem is, the Filter doesn't reduce the Data but
commit all Data in the Table to the Map Process:

The code looks like this for the Filter:

Scan scan = new Scan();
String columns = "details";
String qualifier = "details:page";
String value = "5";
scan.setFilter(new SingleColumnValueFilter(Bytes.toBytes(columns),
Bytes.toBytes(qualifier), CompareOp.EQUAL, Bytes.toBytes("5")));
TableMapReduceUtil.initTableMapperJob("books", scan, mapper.class,
ImmutableBytesWritable.class, IntWritable.class, job);


And this was my Code for filling the Table:

Put put = new Put(rowkey);
put.add(Bytes.toBytes("details"), Bytes.toBytes("page"),
Bytes.toBytes(rand.nextInt(20)));

and I didn't understand why the filter doesn't work! I hope anybody can
help me.

Best regards,
Patrick

Re: HBase MR with Filter

Posted by Stack <st...@duboce.net>.
On Tue, May 18, 2010 at 9:25 AM, Patrick Datko <pa...@ymc.ch> wrote:
> Hey,
>
> i'm building a Map Reduce Job which should get Data from a HBase Table
> filter it an store the reduced data in another HBase Table. I used the
> SingleColumnValueFilter to limit the Data that will be commit to the
> Map-Process. The Problem is, the Filter doesn't reduce the Data but
> commit all Data in the Table to the Map Process:
>
> The code looks like this for the Filter:
>
> Scan scan = new Scan();
> String columns = "details";
> String qualifier = "details:page";
> String value = "5";
> scan.setFilter(new SingleColumnValueFilter(Bytes.toBytes(columns),
> Bytes.toBytes(qualifier), CompareOp.EQUAL, Bytes.toBytes("5")));
> TableMapReduceUtil.initTableMapperJob("books", scan, mapper.class,
> ImmutableBytesWritable.class, IntWritable.class, job);
>
>
> And this was my Code for filling the Table:
>
> Put put = new Put(rowkey);
> put.add(Bytes.toBytes("details"), Bytes.toBytes("page"),
> Bytes.toBytes(rand.nextInt(20)));
>
> and I didn't understand why the filter doesn't work! I hope anybody can
> help me.
>
> Best regards,
> Patrick
>
>
>
>

Re: HBase MR with Filter

Posted by Stack <st...@duboce.net>.
The below looks 'right'.

Maybe try uncommenting this:

      this.comparator.compareTo(Arrays.copyOfRange(data, offset,
offset + length));
//    if (LOG.isDebugEnabled()) {
//      LOG.debug("compareResult=" + compareResult + " " +
Bytes.toString(data, offset, length));
//    }

...in SingleColumnValueFilter.  It might give you a clue as to where
things are going awry.

St.Ack

On Tue, May 18, 2010 at 9:25 AM, Patrick Datko <pa...@ymc.ch> wrote:
> Hey,
>
> i'm building a Map Reduce Job which should get Data from a HBase Table
> filter it an store the reduced data in another HBase Table. I used the
> SingleColumnValueFilter to limit the Data that will be commit to the
> Map-Process. The Problem is, the Filter doesn't reduce the Data but
> commit all Data in the Table to the Map Process:
>
> The code looks like this for the Filter:
>
> Scan scan = new Scan();
> String columns = "details";
> String qualifier = "details:page";
> String value = "5";
> scan.setFilter(new SingleColumnValueFilter(Bytes.toBytes(columns),
> Bytes.toBytes(qualifier), CompareOp.EQUAL, Bytes.toBytes("5")));
> TableMapReduceUtil.initTableMapperJob("books", scan, mapper.class,
> ImmutableBytesWritable.class, IntWritable.class, job);
>
>
> And this was my Code for filling the Table:
>
> Put put = new Put(rowkey);
> put.add(Bytes.toBytes("details"), Bytes.toBytes("page"),
> Bytes.toBytes(rand.nextInt(20)));
>
> and I didn't understand why the filter doesn't work! I hope anybody can
> help me.
>
> Best regards,
> Patrick
>
>
>
>