You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by "Kevin Liang (Jira)" <ji...@apache.org> on 2026/09/22 19:08:00 UTC
[jira] [Commented] (SOLR-18476) Support file suffix based MMapDirectory preload
[ https://issues.apache.org/jira/browse/SOLR-18476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18118122#comment-18118122 ]
Kevin Liang commented on SOLR-18476:
------------------------------------
https://lucene.apache.org/core/9_7_0/core/org/apache/lucene/store/MMapDirectory.html#setPreload(java.util.function.BiPredicate)
> Support file suffix based MMapDirectory preload
> -----------------------------------------------
>
> Key: SOLR-18476
> URL: https://issues.apache.org/jira/browse/SOLR-18476
> Project: Solr
> Issue Type: Improvement
> Reporter: Kevin Liang
> Priority: Major
>
> *Context:*
> Solr uses various DirectoryFactory implementations to interface with the raw index on disk. When Solr requests index files, it often is actually going to MMap'ed off-heap memory rather than disk itself (disk reads being slow). By default, Solr starts up "cold" however there is existing support for preload=true as a configuration param of MMapDirectoryFactory which will attempt to load as much of the index directory into off-heap memory page cache as possible. This preload is done both at startup time, but also after merges, and would avoid issues such as first queries after merge being much slower.
> The issue with this is that when our index is large, some files should be given priority rather than the indiscriminate preload=true. An example being that dense vector search is highly dependent on having the graph and vectors in-memory for traversal and scoring (.vex, .vem, .vec). The inverse can also be true that we may not want to populate the page cache with postings or stored fields, as would be done in an all-or-nothing preload. Lucene provides this ability to choose files in their API, however Solr is not currently leveraging it
> *Proposed Change:*
> Add a <str name="preloadExtensions">...</str> property to MMapDirectoryFactory (or if we want to be more cautious, could consider implementing a new "SelectiveMMapDirectoryFactory"). The provided ordered list of file extensions would be attempted to be preloaded into page cache in that order and take precedent over other index files.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org