[jira] [Created] (HBASE-8464) FastDiffEncoder - valueLength calculation is incorrect

["ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>]

ramkrishna.s.vasudevan created HBASE-8464:
---------------------------------------------

             Summary: FastDiffEncoder - valueLength calculation is incorrect
                 Key: HBASE-8464
                 URL: https://issues.apache.org/jira/browse/HBASE-8464
             Project: HBase
          Issue Type: Bug
    Affects Versions: 0.95.0, 0.94.7
            Reporter: ramkrishna.s.vasudevan
            Assignee: ramkrishna.s.vasudevan
             Fix For: 0.98.0, 0.94.8, 0.95.1


In FastDiffEncoder
Inside compressSingleKeyValue()
{code}
 currentState.prevOffset = in.position();
    int keyLength = in.getInt();
    int valueOffset =
        currentState.prevOffset + keyLength + KeyValue.ROW_OFFSET;
    int valueLength = in.getInt();
    byte flag = 0;
{code}
Before seeing the bug, whenever we write something into encoders, we take the ByteBuffer that is created by Writer.append().
This basically writes
keyLength, valueLength, keyarray, valuearray, <memstoreTS> 
Now consider a case where the keyarray size is 20 and valuearray size is 20.
As per the above code for the first KV
Read keyLength (4  bytes), value length (4 bytes).
First time the prevOffset is 0 so our value Offset is - 0+20+8 =28.
This is correct.
After the first KV is read when we take up the next KV,
Now the currentState.prevOffset => 28+20 = 48 (the value is also read)
The above calculation will give us
28+20+8 = 56.
But the bytebuffer has only 48 bytes in it.

Why our testcases did not catch this bug?
========================================
It is because in the TestDataBlockEncoders we create a ByteBuffer directly from the KVs and we do not create the way the HFileWriterV2 does it.
See RedundantKVGenerator.convertKvToByteBuffer().

Pls correct me if am wrong.  I can provide a patch for the same if my above analysis is correct.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira