You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Greg Miller (Jira)" <ji...@apache.org> on 2021/03/17 23:06:00 UTC
[jira] [Commented] (LUCENE-9850) Explore PFOR for Doc ID delta
encoding (instead of FOR)
[ https://issues.apache.org/jira/browse/LUCENE-9850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17303762#comment-17303762 ]
Greg Miller commented on LUCENE-9850:
-------------------------------------
I've got a (somewhat hacky) tool in the works that will show bit-per-value distribution over an index using FOR vs. PFOR for doc ID delta encoding. I'll see if I can put that somewhere sharable in the next day or two along with some results.
> Explore PFOR for Doc ID delta encoding (instead of FOR)
> -------------------------------------------------------
>
> Key: LUCENE-9850
> URL: https://issues.apache.org/jira/browse/LUCENE-9850
> Project: Lucene - Core
> Issue Type: Task
> Components: core/codecs
> Affects Versions: main (9.0)
> Reporter: Greg Miller
> Priority: Minor
>
> It'd be interesting to explore using PFOR instead of FOR for doc ID encoding. Right now PFOR is used for positions, frequencies and payloads, but FOR is used for doc ID deltas. From a recent [conversation|http://mail-archives.apache.org/mod_mbox/lucene-dev/202103.mbox/%3CCAPsWd%2BOp7d_GxNosB5r%3DQMPA-v0SteHWjXUmG3gwQot4gkubWw%40mail.gmail.com%3E] on the dev mailing list, it sounds like this decision was made based on the optimization possible when expanding the deltas.
> I'd be interesting in measuring the index size reduction possible with switching to PFOR compared to the performance reduction we might see by no longer being able to apply the deltas in as optimal a way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org