You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Jibo John (JIRA)" <ji...@apache.org> on 2011/07/14 20:06:00 UTC
[jira] [Created] (LUCENE-3315) Omit termFreq but keep Positions
Omit termFreq but keep Positions
--------------------------------
Key: LUCENE-3315
URL: https://issues.apache.org/jira/browse/LUCENE-3315
Project: Lucene - Java
Issue Type: Improvement
Components: core/index
Affects Versions: 3.1
Reporter: Jibo John
It would be useful to have an option to discard term frequency information but still keep the positions. Currently, setOmitTermFreqAndPositions discards both. The recent commit LUCENE-2048 adds the flexibility to discard only the positions. With the new option to discard frequency, position-dependent queries can continue to work, and whoever can afford to ignore the scoring based on tf can save a lot on the index size.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3315) Omit termFreq but keep Positions
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13065473#comment-13065473 ]
Michael McCandless commented on LUCENE-3315:
--------------------------------------------
I don't think this is realistically possible: in order to record the positions we need to know the tf for that term/doc because that tells us how many positions there are.
> Omit termFreq but keep Positions
> --------------------------------
>
> Key: LUCENE-3315
> URL: https://issues.apache.org/jira/browse/LUCENE-3315
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index
> Affects Versions: 3.1
> Reporter: Jibo John
>
> It would be useful to have an option to discard term frequency information but still keep the positions. Currently, setOmitTermFreqAndPositions discards both. The recent commit LUCENE-2048 adds the flexibility to discard only the positions. With the new option to discard frequency, position-dependent queries can continue to work, and whoever can afford to ignore the scoring based on tf can save a lot on the index size.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3315) Omit termFreq but keep Positions
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13065476#comment-13065476 ]
Robert Muir commented on LUCENE-3315:
-------------------------------------
simpletext used to not record its freq but instead count its positions and return that for freq :)
> Omit termFreq but keep Positions
> --------------------------------
>
> Key: LUCENE-3315
> URL: https://issues.apache.org/jira/browse/LUCENE-3315
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index
> Affects Versions: 3.1
> Reporter: Jibo John
>
> It would be useful to have an option to discard term frequency information but still keep the positions. Currently, setOmitTermFreqAndPositions discards both. The recent commit LUCENE-2048 adds the flexibility to discard only the positions. With the new option to discard frequency, position-dependent queries can continue to work, and whoever can afford to ignore the scoring based on tf can save a lot on the index size.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (LUCENE-3315) Omit termFreq but keep Positions
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir resolved LUCENE-3315.
---------------------------------
Resolution: Won't Fix
All jokes aside, marking as won't fix.
Your freqs probably arent taking up as much as you think, you may be misled by ".frq" extension, but its actually mostly document id deltas.
> Omit termFreq but keep Positions
> --------------------------------
>
> Key: LUCENE-3315
> URL: https://issues.apache.org/jira/browse/LUCENE-3315
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index
> Affects Versions: 3.1
> Reporter: Jibo John
>
> It would be useful to have an option to discard term frequency information but still keep the positions. Currently, setOmitTermFreqAndPositions discards both. The recent commit LUCENE-2048 adds the flexibility to discard only the positions. With the new option to discard frequency, position-dependent queries can continue to work, and whoever can afford to ignore the scoring based on tf can save a lot on the index size.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org