Index file format documentation

[Shai Erera <se...@gmail.com>]

Hi

I've read the index file format documentation (
http://lucene.apache.org/java/3_0_2/fileformats.html) trying to understand
something, and I spent quite a lot of time on the skip data documentation.
The definition that puzzled me is this:

SkipData --> <<SkipLevelLength, SkipLevel> NumSkipLevels-1, SkipLevel>
<SkipDatum>

First, can someone please tell me how to read this? :)

Second, I think it's incorrect (unless I misunderstood how to read it).
After reading the code, it feels to me like the definition should have been:

SkipData --> <SkipLevelLength, SkipDatum> NumSkipLevels-1> <SkipDatum>

That is, all levels > 0, record their skip level data length followed by the
SkipDatum entries while level = 0 records the same besides the skip level
length ('cause it's never cached, therefore there's no need to record the
length).

Am I correct in my understanding?

Shai

Re: Index file format documentation

[Shai Erera <se...@gmail.com>]

Ahh I've missed SkipLevel below it! Now it makes more sense.

I agree then the SkipDatum in the end is redundant … the last
SkipLevel in the definition refers to level 0.

Thanks !

Shai

On Sunday, July 18, 2010, Michael McCandless <lu...@mikemccandless.com> wrote:
> Actually I think the docs are [close to] correct?
>
> SkipLevel is a sequence of SkipDatum.
>
> Though I'm not sure why that last <SkipDatum> is attached; that seems odd.
>
> Mike
>
> On Sun, Jul 18, 2010 at 6:07 AM, Shai Erera <se...@gmail.com> wrote:
>> Hi
>>
>> I've read the index file format documentation
>> (http://lucene.apache.org/java/3_0_2/fileformats.html) trying to understand
>> something, and I spent quite a lot of time on the skip data documentation.
>> The definition that puzzled me is this:
>>
>> SkipData --> <<SkipLevelLength, SkipLevel> NumSkipLevels-1, SkipLevel>
>> <SkipDatum>
>>
>> First, can someone please tell me how to read this? :)
>>
>> Second, I think it's incorrect (unless I misunderstood how to read it).
>> After reading the code, it feels to me like the definition should have been:
>>
>> SkipData --> <SkipLevelLength, SkipDatum> NumSkipLevels-1> <SkipDatum>
>>
>> That is, all levels > 0, record their skip level data length followed by the
>> SkipDatum entries while level = 0 records the same besides the skip level
>> length ('cause it's never cached, therefore there's no need to record the
>> length).
>>
>> Am I correct in my understanding?
>>
>> Shai
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: Index file format documentation

[Michael McCandless <lu...@mikemccandless.com>]

Actually I think the docs are [close to] correct?

SkipLevel is a sequence of SkipDatum.

Though I'm not sure why that last <SkipDatum> is attached; that seems odd.

Mike

On Sun, Jul 18, 2010 at 6:07 AM, Shai Erera <se...@gmail.com> wrote:
> Hi
>
> I've read the index file format documentation
> (http://lucene.apache.org/java/3_0_2/fileformats.html) trying to understand
> something, and I spent quite a lot of time on the skip data documentation.
> The definition that puzzled me is this:
>
> SkipData --> <<SkipLevelLength, SkipLevel> NumSkipLevels-1, SkipLevel>
> <SkipDatum>
>
> First, can someone please tell me how to read this? :)
>
> Second, I think it's incorrect (unless I misunderstood how to read it).
> After reading the code, it feels to me like the definition should have been:
>
> SkipData --> <SkipLevelLength, SkipDatum> NumSkipLevels-1> <SkipDatum>
>
> That is, all levels > 0, record their skip level data length followed by the
> SkipDatum entries while level = 0 records the same besides the skip level
> length ('cause it's never cached, therefore there's no need to record the
> length).
>
> Am I correct in my understanding?
>
> Shai
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org