[jira] [Comment Edited] (HBASE-21922) BloomContext#sanityCheck may failed when use ROWPREFIX_DELIMITED bloom filter

["Guanghao Zhang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-21922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16771060#comment-16771060 ] 

Guanghao Zhang edited comment on HBASE-21922 at 2/18/19 12:52 PM:
------------------------------------------------------------------

Write a ut for this case. But it has problem when read from bloom filter after remove the sanityCheck.

 
{code:java}
org.apache.hadoop.hbase.io.hfile.CompoundBloomFilter

@Override
public boolean contains(byte[] key, int keyOffset, int keyLength, ByteBuff bloom) {
  int block = index.rootBlockContainingKey(key, keyOffset, keyLength);

  // As the key "123" is small than the root key "1234", so this block is -1 and return false.  
  if (block < 0) {
    block = 0; // This key is not in the file.
  }

  boolean result;
  HFileBlock bloomBlock = getBloomBlock(block);
  try {
    ByteBuff bloomBuf = bloomBlock.getBufferReadOnly();
    result = BloomFilterUtil.contains(key, keyOffset, keyLength, bloomBuf,
        bloomBlock.headerSize(), bloomBlock.getUncompressedSizeWithoutHeader(), hash, hashCount);
  } finally {
    // After the use return back the block if it was served from a cache.
    reader.returnBlock(bloomBlock);
  }
  if (numPositivesPerChunk != null && result) {
    // Update statistics. Only used in unit tests.
    ++numPositivesPerChunk[block];
  }
  return result;
}
{code}
 


was (Author: zghaobac):
Write a ut for this case. But it has problem when read from bloom filter after remove the sanityCheck.

 
{code:java}
org.apache.hadoop.hbase.io.hfile.CompoundBloomFilter

@Override
public boolean contains(byte[] key, int keyOffset, int keyLength, ByteBuff bloom) {
  int block = index.rootBlockContainingKey(key, keyOffset, keyLength);

  // As the key "123" is small than the root key "1234", so this block is -1 and return false.  
  if (block < 0) {
    block = 0; // This key is not in the file.
  }

  boolean result;
  HFileBlock bloomBlock = getBloomBlock(block);
  try {
    ByteBuff bloomBuf = bloomBlock.getBufferReadOnly();
    result = BloomFilterUtil.contains(key, keyOffset, keyLength, bloomBuf,
        bloomBlock.headerSize(), bloomBlock.getUncompressedSizeWithoutHeader(), hash, hashCount);
  } finally {
    // After the use return back the block if it was served from a cache.
    reader.returnBlock(bloomBlock);
  }
  if (numPositivesPerChunk != null && result) {
    // Update statistics. Only used in unit tests.
    ++numPositivesPerChunk[block];
  }
  return result;
}
{code}
 

> BloomContext#sanityCheck may failed when use ROWPREFIX_DELIMITED bloom filter
> -----------------------------------------------------------------------------
>
>                 Key: HBASE-21922
>                 URL: https://issues.apache.org/jira/browse/HBASE-21922
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Guanghao Zhang
>            Priority: Major
>
> Assume we use '5' as the delimiter, there are rowkeys
> {code:java}
> 12345xxx
> 1235xxxx{code}
> When use ROWPREFIX_DELIMITED bloom filter, the rowkey write to bloom filter are
> {code:java}
> 1234
> 123{code}
> Then BloomContext#sanityCheck will failed.
> {code:java}
> private void sanityCheck(Cell cell) throws IOException {
>   if (this.getLastCell() != null) {
>     LOG.debug("Current cell " + cell + ", prevCell = " + this.getLastCell());
>     if (comparator.compare(cell, this.getLastCell()) <= 0) {
>       throw new IOException("Added a key not lexically larger than" + " previous. Current cell = "
>           + cell + ", prevCell = " + this.getLastCell());
>     }
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)