You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Sandy Pratt <pr...@adobe.com> on 2012/02/03 00:05:34 UTC

OBOB in HMerge?

I've been reading through this a bit lately as I work on a merge tool for our tables, and it looks like there's an off by one bug in the OfflineMerger constructor in util.HMerge:

      InternalScanner rootScanner =
        root.getScanner(scan);

      try {
        List<KeyValue> results = new ArrayList<KeyValue>();
        while(rootScanner.next(results)) {
          for(KeyValue kv: results) {
            HRegionInfo info = Writables.getHRegionInfoOrNull(kv.getValue());
            if (info != null) {
              metaRegions.add(info);
            }
          }
        }
      } finally {
	...
      }

That call to InternalScanner.next() in the while condition returns true if there's another result *after* the one it just loaded into the out param.  That is, after it reads the last row into the 'results' collection, it returns false and the loop exits with that last row unread.  It probably wants to be structured more like this:

final InternalScanner metaScanner = meta.getScanner(scan);
List<KeyValue> results = Lists.newArrayList();

while (true) {

	boolean hasMore = metaScanner.next(results);

	for (KeyValue kv : results) {
		HRegionInfo hri = Writables.getHRegionInfoOrNull(kv.getValue());
		if (hri != null) {
			regionInfo.add(hri);
		}
	}
					
	if (!hasMore) {
		break;
	}
}

I get the impression this class doesn't get used much, but just thought I'd point it out.

Thanks,
Sandy

RE: OBOB in HMerge?

Posted by Sandy Pratt <pr...@adobe.com>.
Filed one, but looks like it's already fixed on trunk.

-----Original Message-----
From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
Sent: Thursday, February 02, 2012 9:08 PM
To: dev@hbase.apache.org
Subject: Re: OBOB in HMerge?

On Thu, Feb 2, 2012 at 3:05 PM, Sandy Pratt <pr...@adobe.com> wrote:
> I've been reading through this a bit lately as I work on a merge tool for our tables, and it looks like there's an off by one bug in the OfflineMerger constructor in util.HMerge:
>

Agree


> I get the impression this class doesn't get used much, but just thought I'd point it out.
>

Any chance of a JIRA and a patch Sandy?  (You are right that stuff doesn't get exercised much -- we need to finish off the online merge scripts -- are you working on somethink like this?)

St.Ack

Re: OBOB in HMerge?

Posted by Stack <st...@duboce.net>.
On Thu, Feb 2, 2012 at 3:05 PM, Sandy Pratt <pr...@adobe.com> wrote:
> I've been reading through this a bit lately as I work on a merge tool for our tables, and it looks like there's an off by one bug in the OfflineMerger constructor in util.HMerge:
>

Agree


> I get the impression this class doesn't get used much, but just thought I'd point it out.
>

Any chance of a JIRA and a patch Sandy?  (You are right that stuff
doesn't get exercised much -- we need to finish off the online merge
scripts -- are you working on somethink like this?)

St.Ack