You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Jason Rutherglen <ja...@gmail.com> on 2011/03/14 20:53:42 UTC
Getting the row with a Coprocessor
I've subclassed RegionObserver and am overriding postPut. How does
one obtain the row byte[] of the Put that generated the call? Is it
available via from the familyMap? What is the purpose of the info
family?
Re: Getting the row with a Coprocessor
Posted by Gary Helmling <gh...@gmail.com>.
Hi Jason,
Could be that the int is converting to a string as a non-printable
character? Did you convert it originally using Bytes.toBytes(int)? You
could use either Bytes.toInt() or Bytes.toStringBinary() to get either the
original int or a stringified representation.
Gary
On Mon, Mar 14, 2011 at 7:08 PM, Jason Rutherglen <
jason.rutherglen@gmail.com> wrote:
> Gary,
>
> Thanks, the example works.
>
> The next part is the row is not showing up in the KV, from the postPut
> method familyMap.
>
> row = kv.getRow();
> String rowStr = Bytes.toString(row);
>
> Where rowStr looks like 4 whitespace characters when debugging. The
> row is set in the Put object (as in the code below), eg, it's an int
> converted into a byte[].
>
> Jason
>
> On Mon, Mar 14, 2011 at 4:03 PM, Gary Helmling <gh...@gmail.com>
> wrote:
> >>
> >>
> >> HTable table = util.createTable(tableName, families);
> >> for (int x=0; x < 20; x++) {
> >> byte[] row = Bytes.toBytes(x);
> >> Put put = new Put(row);
> >> String s = "test hbase lucene";
> >> put.add(headersFam, to, Bytes.toBytes(s));
> >> table.put(put);
> >> }
> >>
> > ...
> >
> >> Maybe I need to set the Coprocessor on the table then.
> >>
> >>
> > So to load the cp from a table, you could do something like:
> >
> > HTableDescriptor desc = new HTableDescriptor(tableName);
> > for (String fam : families) {
> > desc.addFamily(new HColumnDescriptor(fam));
> > }
> > desc.setValue("COPROCESSOR$1",
> > String.format("/:%s:USER", LuceneCoprocessor.class.getName()));
> >
> > HBaseAdmin admin = util.getHBaseAdmin();
> > admin.createTable(desc);
> > HTable table = new HTable(util.getConfiguration(), tableName);
> >
> >
> > Then carry on with the puts, etc. And see how that works out. That
> should
> > limit the coprocessor to only load on your table.
> >
> > Let me know if you run into problems with this.
> >
> > Thanks,
> > Gary
> >
>
Re: Getting the row with a Coprocessor
Posted by Jason Rutherglen <ja...@gmail.com>.
Gary,
Thanks, the example works.
The next part is the row is not showing up in the KV, from the postPut
method familyMap.
row = kv.getRow();
String rowStr = Bytes.toString(row);
Where rowStr looks like 4 whitespace characters when debugging. The
row is set in the Put object (as in the code below), eg, it's an int
converted into a byte[].
Jason
On Mon, Mar 14, 2011 at 4:03 PM, Gary Helmling <gh...@gmail.com> wrote:
>>
>>
>> HTable table = util.createTable(tableName, families);
>> for (int x=0; x < 20; x++) {
>> byte[] row = Bytes.toBytes(x);
>> Put put = new Put(row);
>> String s = "test hbase lucene";
>> put.add(headersFam, to, Bytes.toBytes(s));
>> table.put(put);
>> }
>>
> ...
>
>> Maybe I need to set the Coprocessor on the table then.
>>
>>
> So to load the cp from a table, you could do something like:
>
> HTableDescriptor desc = new HTableDescriptor(tableName);
> for (String fam : families) {
> desc.addFamily(new HColumnDescriptor(fam));
> }
> desc.setValue("COPROCESSOR$1",
> String.format("/:%s:USER", LuceneCoprocessor.class.getName()));
>
> HBaseAdmin admin = util.getHBaseAdmin();
> admin.createTable(desc);
> HTable table = new HTable(util.getConfiguration(), tableName);
>
>
> Then carry on with the puts, etc. And see how that works out. That should
> limit the coprocessor to only load on your table.
>
> Let me know if you run into problems with this.
>
> Thanks,
> Gary
>
Re: Getting the row with a Coprocessor
Posted by Gary Helmling <gh...@gmail.com>.
>
>
> HTable table = util.createTable(tableName, families);
> for (int x=0; x < 20; x++) {
> byte[] row = Bytes.toBytes(x);
> Put put = new Put(row);
> String s = "test hbase lucene";
> put.add(headersFam, to, Bytes.toBytes(s));
> table.put(put);
> }
>
...
> Maybe I need to set the Coprocessor on the table then.
>
>
So to load the cp from a table, you could do something like:
HTableDescriptor desc = new HTableDescriptor(tableName);
for (String fam : families) {
desc.addFamily(new HColumnDescriptor(fam));
}
desc.setValue("COPROCESSOR$1",
String.format("/:%s:USER", LuceneCoprocessor.class.getName()));
HBaseAdmin admin = util.getHBaseAdmin();
admin.createTable(desc);
HTable table = new HTable(util.getConfiguration(), tableName);
Then carry on with the puts, etc. And see how that works out. That should
limit the coprocessor to only load on your table.
Let me know if you run into problems with this.
Thanks,
Gary
Re: Getting the row with a Coprocessor
Posted by Jason Rutherglen <ja...@gmail.com>.
Hi Gary,
Thanks for your help.
Using HBaseTestingUtility, then calling:
conf.set(CoprocessorHost.REGION_COPROCESSOR_CONF_KEY,
LuceneCoprocessor.class.getName());
Which is following the code in TestRegionObserverInterface.
Then:
HTable table = util.createTable(tableName, families);
for (int x=0; x < 20; x++) {
byte[] row = Bytes.toBytes(x);
Put put = new Put(row);
String s = "test hbase lucene";
put.add(headersFam, to, Bytes.toBytes(s));
table.put(put);
}
> How are you loading your coprocessor? Are you setting a HTableDescriptor
> value to load the cp on a specific table, or using the
> "hbase.coprocessor.region.classes" system-wide config?
Maybe I need to set the Coprocessor on the table then.
Jason
On Mon, Mar 14, 2011 at 1:58 PM, Gary Helmling <gh...@gmail.com> wrote:
> Hi Jason,
>
> How are you loading your coprocessor? Are you setting a HTableDescriptor
> value to load the cp on a specific table, or using the
> "hbase.coprocessor.region.classes" system-wide config?
>
> If the latter, I'm wondering if you're seeing puts to the .META. table. Have
> you checked the table name with
> RegionCoprocessorEnvironment.getRegion().getRegionInfo().getTableDesc().getName()?
>
>
>> Info seems to be a family that's given to the Coprocessor, at least
>> when the test case uses HBaseTestingUtility.
>>
>>
> Your cp should just be getting the contents of Put.getFamilyMap(). Can you
> share more info about your table schema, operations you're performing and
> testing, or even sample code?
>
> Thanks,
> Gary
>
Re: Getting the row with a Coprocessor
Posted by Gary Helmling <gh...@gmail.com>.
Hi Jason,
How are you loading your coprocessor? Are you setting a HTableDescriptor
value to load the cp on a specific table, or using the
"hbase.coprocessor.region.classes" system-wide config?
If the latter, I'm wondering if you're seeing puts to the .META. table. Have
you checked the table name with
RegionCoprocessorEnvironment.getRegion().getRegionInfo().getTableDesc().getName()?
> Info seems to be a family that's given to the Coprocessor, at least
> when the test case uses HBaseTestingUtility.
>
>
Your cp should just be getting the contents of Put.getFamilyMap(). Can you
share more info about your table schema, operations you're performing and
testing, or even sample code?
Thanks,
Gary
Re: Getting the row with a Coprocessor
Posted by Jason Rutherglen <ja...@gmail.com>.
Gary,
I've called KV.getRow() however there has been nothing returned, or
for the 'info' family, values other than what the Put object was
given.
> Sorry, not sure what you mean by the info family? You mean in the .META.
> table or somewhere else? In .META. it holds one row per region for each
> user table in HBase.
Info seems to be a family that's given to the Coprocessor, at least
when the test case uses HBaseTestingUtility.
Jason
On Mon, Mar 14, 2011 at 1:28 PM, Gary Helmling <gh...@gmail.com> wrote:
> You can get the row key from the KeyValue entries in the map. Just call
> KeyValue.getRow().
>
> Sorry, not sure what you mean by the info family? You mean in the .META.
> table or somewhere else? In .META. it holds one row per region for each
> user table in HBase.
>
>
> On Mon, Mar 14, 2011 at 12:53 PM, Jason Rutherglen <
> jason.rutherglen@gmail.com> wrote:
>
>> I've subclassed RegionObserver and am overriding postPut. How does
>> one obtain the row byte[] of the Put that generated the call? Is it
>> available via from the familyMap? What is the purpose of the info
>> family?
>>
>
Re: Getting the row with a Coprocessor
Posted by Gary Helmling <gh...@gmail.com>.
You can get the row key from the KeyValue entries in the map. Just call
KeyValue.getRow().
Sorry, not sure what you mean by the info family? You mean in the .META.
table or somewhere else? In .META. it holds one row per region for each
user table in HBase.
On Mon, Mar 14, 2011 at 12:53 PM, Jason Rutherglen <
jason.rutherglen@gmail.com> wrote:
> I've subclassed RegionObserver and am overriding postPut. How does
> one obtain the row byte[] of the Put that generated the call? Is it
> available via from the familyMap? What is the purpose of the info
> family?
>