You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "stefanvodita (via GitHub)" <gi...@apache.org> on 2026/09/14 21:05:15 UTC
[I] Umbra-style strings for Lucene [lucene]
stefanvodita opened a new issue, #16672:
URL: https://github.com/apache/lucene/issues/16672
### Description
I was reading the [Umbra DB paper](https://www.google.com/url?sa=t&source=web&rct=j&opi=89978449&url=https://db.in.tum.de/~freitag/papers/p29-neumann-cidr20.pdf&ved=2ahUKEwjwg9f--e6WAxWqYEEAHWBdBwwQFnoECBEQAQ&usg=AOvVaw3m6CLTrajFRkbnORIPDXNs) and they describe an approach to storing strings (heard them called German Strings elsewhere). They use 16 bytes. Short strings fit in the 16 bytes. Long strings keep a prefix for fast equality comparisons and a pointer to the full string.
I was thinking if these have a place in Lucene. Maybe for binary doc values? It would be a use-case with high-cardinality mostly short strings, e.g. symbols on the stock exchange.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Umbra-style strings for Lucene [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on issue #16672:
URL: https://github.com/apache/lucene/issues/16672#issuecomment-5705115640
Well otherwise, the short-string optimizations are usually a way to avoid heap allocations.
But for java, I'm just not seeing any benefits, and I worry about only resulting in more pointer-chasing and GC pressure, not less.
For your listed use-case, I think binarydocvalues is the wrong thing to use at all. If you want to efficiently compare the strings then use sorted/sortedset?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Umbra-style strings for Lucene [lucene]
Posted by "stefanvodita (via GitHub)" <gi...@apache.org>.
stefanvodita commented on issue #16672:
URL: https://github.com/apache/lucene/issues/16672#issuecomment-5720252713
You're probably right. I'm not very optimistic either, but thought I'd at least bring up the idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Umbra-style strings for Lucene [lucene]
Posted by "stefanvodita (via GitHub)" <gi...@apache.org>.
stefanvodita closed issue #16672: Umbra-style strings for Lucene
URL: https://github.com/apache/lucene/issues/16672
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Umbra-style strings for Lucene [lucene]
Posted by "stefanvodita (via GitHub)" <gi...@apache.org>.
stefanvodita commented on issue #16672:
URL: https://github.com/apache/lucene/issues/16672#issuecomment-5703056610
I'm not sure to what extent the advantage is inlining strings in stack space. There's also a consideration of fitting the string on two registers. The first register would hold the length of the string and its prefix, for fast comparison, and the second register would be a pointer to the string or the rest of the string if it's short enough. Then there's the benefit of having a fixed stride in a list of strings, being able to iterate through them and jump to a string by index without dereferencing pointers. I think these benefits hold in Java too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] Umbra-style strings for Lucene [lucene]
Posted by "rmuir (via GitHub)" <gi...@apache.org>.
rmuir commented on issue #16672:
URL: https://github.com/apache/lucene/issues/16672#issuecomment-5685256367
isn't the point of these optimizations usually to try to inline small strings into stack space? does it apply to java?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org