[jira] Commented: (LUCENE-2529) always apply position increment gap between values

["Michael McCandless (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LUCENE-2529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917177#action_12917177 ] 

Michael McCandless commented on LUCENE-2529:
--------------------------------------------

I like the idea of disregarding the posIncrGap of the first token.

Maybe, instead of that +1 inside IW, we change the default posIncrGap
to 1?

Can you spell out examples of how the indexed positions will change w/
this patch -- I'm having trouble visualizing this.  EG for a single
valued field, multi-valued, etc.

For one, because we now ignore the token's posIncrGap if it's the
first token (per value in the field), any tokenizer that alters the
posIncr from the default (1) will now see different positions indexed,
right?

Man I really want to get this logic out of indexer and into the
analysis chain (LUCENE-2450 enables this).  How multi-valued streams
should handle the transition from one value to another shouldn't be
inside the indexer... and maybe (someday) tokens should store their
position (not the gap) so we don't have this cryptic logic inside the
indexer...


> always apply position increment gap between values
> --------------------------------------------------
>
>                 Key: LUCENE-2529
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2529
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: 2.9.3, 3.0.2, 3.1, 4.0
>         Environment: (I don't know which version to say this affects since it's some quasi trunk release and the new versioning scheme confuses me.)
>            Reporter: David Smiley
>            Assignee: Koji Sekiguchi
>            Priority: Minor
>             Fix For: 3.1, 4.0
>
>         Attachments: LUCENE-2529_always_apply_position_increment_gap_between_values.patch, LUCENE-2529_skip_posIncr_for_1st_token.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> I'm doing some fancy stuff with span queries that is very sensitive to term positions.  I discovered that the position increment gap on indexing is only applied between values when there are existing terms indexed for the document.  I suspect this logic wasn't deliberate, it's just how its always been for no particular reason.  I think it should always apply the gap between fields.  Reference DocInverterPerField.java line 82:
> if (fieldState.length > 0)
>           fieldState.position += docState.analyzer.getPositionIncrementGap(fieldInfo.name);
> This is checking fieldState.length.  I think the condition should simply be:  if (i > 0).
> I don't think this change will affect anyone at all but it will certainly help me.  Presently, I can either change this line in Lucene, or I can put in a hack so that the first value for the document is some dummy value which is wasteful.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org