[jira] [Created] (LUCENE-8533) Incorrect readVInt: negative number could be returned

["Vladimir Dolzhenko (JIRA)" <ji...@apache.org>]

Vladimir Dolzhenko created LUCENE-8533:
------------------------------------------

             Summary: Incorrect readVInt: negative number could be returned
                 Key: LUCENE-8533
                 URL: https://issues.apache.org/jira/browse/LUCENE-8533
             Project: Lucene - Core
          Issue Type: Improvement
            Reporter: Vladimir Dolzhenko
         Attachments: readVInt.patch

readVInt has to return positive numbers (and zero), throw some exception in case of negative numbers.

While for the sequence of bytes {{[-1, -1, -1, -1, 15]}} it returns {{-1}}.

simplifying [readVInt|https://github.com/apache/lucene-solr/blob/1d85cd783863f75cea133fb9c452302214165a4d/lucene/core/src/java/org/apache/lucene/store/DataInput.java#L113] up to last readByte (exclusive):

{code:java}
int i = ((byte)-1) & 0x7F;
i |= (((byte)-1) & 0x7F) << 7;
i |= (((byte)-1) & 0x7F) << 14;
i |= (((byte)-1) & 0x7F) << 21;
{code}

Here {{i = 268435455}} or in binary format is {{00001111_11111111_11111111_11111111}}

Keeping in mind that int is signed type we have only 3 more bits before overflow happens or in another words {{(Integer.MAX_VALUE - i) >> 28 == 7}}

that's max value could be stored in 5th byte to avoid overflow.

Instead of 

{code:java}
i |= (b & 0x0F) << 28;
if ((b & 0xF0) == 0) return i;
{code}

has to be

{code:java}
i |= (b & 0x07) << 28;
if ((b & 0xF8) == 0) return i;
{code}




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org