You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Jean-Daniel Cryans <jd...@apache.org> on 2012/01/03 23:24:23 UTC
Re: HBase Custom Comparator
Is your comparator on the region server's classpath? Else it won't be
able to guess what that comparator looks like :)
Also your region server log should tell you it's not able to find
CustomComparator.
J-D
On Fri, Dec 30, 2011 at 12:56 AM, Nageswaran Surendhar
<na...@gmail.com> wrote:
> public class CustomComparator extends WritableByteArrayComparable
> {
>
> final CimKeyType cimKeyType;
> final byte[] value;
>
> public CustomComparator(byte[] value, CimKeyType cimKeyType)
> {
> super(value);
> this.value = value;
> this.cimKeyType = cimKeyType;
> }
>
> @Override
> public int compareTo(byte[] value)
> {
> int differenceValue;
>
> switch (cimKeyType)
> {
> case INTEGER:
> differenceValue = Bytes.toInt(this.value) - Bytes.toInt(value);
> break;
> case LONG:
> differenceValue = (int) (Bytes.toLong(this.value) -
> Bytes.toLong(value));
> break;
> case DOUBLE:
> differenceValue = (int) (Bytes.toDouble(this.value) -
> Bytes.toDouble(value));
> break;
> case STRING:
> default:
> differenceValue =
> Bytes.toString(this.value).compareTo(Bytes.toString(value));
> }
>
> System.out.println("differenceValue : " +differenceValue);
>
> return differenceValue;
> }
> }
>
>
>
> CustomComparator comparator = new CustomComparator(filterValue,
> lField.getCimKeyType());
> //BinaryComparator comparator = new BinaryComparator(filterValue);
>
> SingleColumnValueFilter singleColumnValueFilter = new
> SingleColumnValueFilter(Bytes.toBytes(familyAndCol[0]),
> Bytes.toBytes(familyAndCol[1]), compareOperator, comparator);
>
> if (myNumberOfVersions > 0)
> get.setMaxVersions(myNumberOfVersions);
>
>
> if (filterList != null)
> get.setFilter(filterList);
>
> Result result = hTable.get(get);
>
>
> I m using the above code inside hadoop-mapper. I had make it jar and placed
> it inside both hadoop/lib and hbase/lib.
>
> If i use BinaryComparator, i m not getting any exception, and error. But if
> I used CustomComparator, I m getting the follwoing Exception.
>
>
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
> at org.apache.hadoop.mapred.Child.main(Child.java:264)
> Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException:
> Trying to contact region server localhost:50682 for region
> mytable,,1325226129670.708fa93842d13d3448295d88906d7ab8., row '444',
> but failed after 10 attempts.
> Exceptions:
> java.io.IOException: Call to localhost/127.0.0.1:50682 failed on
> local exception: java.io.EOFException
> java.io.IOException: Call to localhost/127.0.0.1:50682 failed on
> local exception: java.io.EOFException
> java.io.IOException: Call to localhost/127.0.0.1:50682 failed on
> local exception: java.io.EOFException
> java.io.IOException: Call to localhost/127.0.0.1:50682 failed on
> local exception: java.io.EOFException
> java.io.IOException: Call to localhost/127.0.0.1:50682 failed on
> local exception: java.io.EOFException
> java.io.IOException: Call to localhost/127.0.0.1:50682 failed on
> local exception: java.io.EOFException
> java.io.IOException: Call to localhost/127.0.0.1:50682 failed on
> local exception: java.io.EOFException
> java.io.IOException: Call to localhost/127.0.0.1:50682 failed on
> local exception: java.io.EOFException
> java.io.IOException: Call to localhost/127.0.0.1:50682 failed on
> local exception: java.io.EOFException
> java.io.IOException: Call to localhost/127.0.0.1:50682 failed on
> local exception: java.io.EOFException
>
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1024)
> at org.apache.hadoop.hbase.client.HTable.get(HTable.java:555)
> at
> ... 12 more
> 2011-12-30 13:36:30,175 INFO org.apache.hadoop.mapred.Task:
> Runnning cleanup for the task
>
> command line jps output is
> 18667 HMaster
> 17929 NameNode
> 21942 Jps
> 18266 SecondaryNameNode
> 18100 DataNode
> 19143 Main
> 18523 TaskTracker
> 18362 JobTracker
> 8641
>
>
> here mytable is my hbase table name and 444 is rowkey
>
> Development environment : Ubuntu-11.10, Java-6, Hadoop and Hbase from
> Cloudera CDH3U2
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
>
> Regards,
> Nageswaran S
Re: HBase Custom Comparator
Posted by Nageswaran Surendhar <na...@gmail.com>.
Hi,
I have identified the problem, and fixed it. The problem is, I have to
provide default constructor for CustomComparator...
Regards,
Nageswaran. S
On Wed, Jan 4, 2012 at 3:54 AM, Jean-Daniel Cryans <jd...@apache.org>wrote:
> Is your comparator on the region server's classpath? Else it won't be
> able to guess what that comparator looks like :)
>
> Also your region server log should tell you it's not able to find
> CustomComparator.
>
> J-D
>
> On Fri, Dec 30, 2011 at 12:56 AM, Nageswaran Surendhar
> <na...@gmail.com> wrote:
> > public class CustomComparator extends WritableByteArrayComparable
> > {
> >
> > final CimKeyType cimKeyType;
> > final byte[] value;
> >
> > public CustomComparator(byte[] value, CimKeyType cimKeyType)
> > {
> > super(value);
> > this.value = value;
> > this.cimKeyType = cimKeyType;
> > }
> >
> > @Override
> > public int compareTo(byte[] value)
> > {
> > int differenceValue;
> >
> > switch (cimKeyType)
> > {
> > case INTEGER:
> > differenceValue = Bytes.toInt(this.value) -
> Bytes.toInt(value);
> > break;
> > case LONG:
> > differenceValue = (int) (Bytes.toLong(this.value) -
> > Bytes.toLong(value));
> > break;
> > case DOUBLE:
> > differenceValue = (int) (Bytes.toDouble(this.value) -
> > Bytes.toDouble(value));
> > break;
> > case STRING:
> > default:
> > differenceValue =
> > Bytes.toString(this.value).compareTo(Bytes.toString(value));
> > }
> >
> > System.out.println("differenceValue : " +differenceValue);
> >
> > return differenceValue;
> > }
> > }
> >
> >
> >
> > CustomComparator comparator = new CustomComparator(filterValue,
> > lField.getCimKeyType());
> > //BinaryComparator comparator = new BinaryComparator(filterValue);
> >
> > SingleColumnValueFilter singleColumnValueFilter = new
> > SingleColumnValueFilter(Bytes.toBytes(familyAndCol[0]),
> > Bytes.toBytes(familyAndCol[1]), compareOperator, comparator);
> >
> > if (myNumberOfVersions > 0)
> > get.setMaxVersions(myNumberOfVersions);
> >
> >
> > if (filterList != null)
> > get.setFilter(filterList);
> >
> > Result result = hTable.get(get);
> >
> >
> > I m using the above code inside hadoop-mapper. I had make it jar and
> placed
> > it inside both hadoop/lib and hbase/lib.
> >
> > If i use BinaryComparator, i m not getting any exception, and error. But
> if
> > I used CustomComparator, I m getting the follwoing Exception.
> >
> >
> > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391)
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
> > at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at javax.security.auth.Subject.doAs(Subject.java:396)
> > at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
> > at org.apache.hadoop.mapred.Child.main(Child.java:264)
> > Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException:
> > Trying to contact region server localhost:50682 for region
> > mytable,,1325226129670.708fa93842d13d3448295d88906d7ab8., row '444',
> > but failed after 10 attempts.
> > Exceptions:
> > java.io.IOException: Call to localhost/127.0.0.1:50682 failed on
> > local exception: java.io.EOFException
> > java.io.IOException: Call to localhost/127.0.0.1:50682 failed on
> > local exception: java.io.EOFException
> > java.io.IOException: Call to localhost/127.0.0.1:50682 failed on
> > local exception: java.io.EOFException
> > java.io.IOException: Call to localhost/127.0.0.1:50682 failed on
> > local exception: java.io.EOFException
> > java.io.IOException: Call to localhost/127.0.0.1:50682 failed on
> > local exception: java.io.EOFException
> > java.io.IOException: Call to localhost/127.0.0.1:50682 failed on
> > local exception: java.io.EOFException
> > java.io.IOException: Call to localhost/127.0.0.1:50682 failed on
> > local exception: java.io.EOFException
> > java.io.IOException: Call to localhost/127.0.0.1:50682 failed on
> > local exception: java.io.EOFException
> > java.io.IOException: Call to localhost/127.0.0.1:50682 failed on
> > local exception: java.io.EOFException
> > java.io.IOException: Call to localhost/127.0.0.1:50682 failed on
> > local exception: java.io.EOFException
> >
> > at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:1024)
> > at org.apache.hadoop.hbase.client.HTable.get(HTable.java:555)
> > at
> > ... 12 more
> > 2011-12-30 13:36:30,175 INFO org.apache.hadoop.mapred.Task:
> > Runnning cleanup for the task
> >
> > command line jps output is
> > 18667 HMaster
> > 17929 NameNode
> > 21942 Jps
> > 18266 SecondaryNameNode
> > 18100 DataNode
> > 19143 Main
> > 18523 TaskTracker
> > 18362 JobTracker
> > 8641
> >
> >
> > here mytable is my hbase table name and 444 is rowkey
> >
> > Development environment : Ubuntu-11.10, Java-6, Hadoop and Hbase from
> > Cloudera CDH3U2
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > --
> >
> > Regards,
> > Nageswaran S
>
--
Regards,
Nageswaran S
Ericsson
+91 95669 13981