You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-issues@jackrabbit.apache.org by "Julian Reschke (Jira)" <ji...@apache.org> on 2026/05/18 14:09:00 UTC
[jira] (OAK-12193) Large numbers of deleted Lucene documents cause incremental indexing to consume too much memory
[ https://issues.apache.org/jira/browse/OAK-12193 ]
Julian Reschke deleted comment on OAK-12193:
--------------------------------------
was (Author: reschke):
trunk: [57a2b05bfd|https://github.com/apache/jackrabbit-oak/commit/57a2b05bfdc2bff7370dc91ab448d27ed8610dbe] [cf0f60da1e|https://github.com/apache/jackrabbit-oak/commit/cf0f60da1e420f4c0aa360fedf941b5d765a394e]
> Large numbers of deleted Lucene documents cause incremental indexing to consume too much memory
> -----------------------------------------------------------------------------------------------
>
> Key: OAK-12193
> URL: https://issues.apache.org/jira/browse/OAK-12193
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: indexing, lucene
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Priority: Major
> Fix For: 2.2.0
>
>
> When Oak performs an incremental indexing cycle which contains a lot of deletions, it can consume a lot of memory.
> The issue is caused by accumulated Lucene document deletions, which are sent to each of the Lucene Index writers for all the indexes that cover the path of the deleted node (regardless of whether they match the nodetype of it). Since there can be numerous indexes the cumulative memory use can easily overwhelm the JVM.
> {noformat}
> Example thread:
> org.apache.jackrabbit.oak.plugins.index.lucene.writer.MultiplexingIndexWriter
> org.apache.jackrabbit.oak.plugins.index.search.CompositePropertyUpdateCallback
> org.apache.jackrabbit.oak.plugins.index.search.spi.binary.FulltextBinaryTextExtractor
> org.apache.jackrabbit.oak.plugins.index.lucene.directory.DefaultDirectoryFactory
> org.apache.jackrabbit.oak.plugins.index.lucene.writer.DefaultIndexWriterFactory
> org.apache.jackrabbit.oak.plugins.index.lucene.LucenelndexEditorContext
> Thread name: async-index-update-async
> memory bytes percentage class @ instance
> 2,516,736,256 69.87% java.lang.Thread @ 0x6a52565d0 async-index-update-async Thread
> 879,048,256 24.41% org.apache.jackrabbit.oak.plugins.index.lucene.LucenelndexEditorContext @ 0x6be46e860
> 879,047,904 24.41% org.apache.jackrabbit.oak.plugins.index.lucene.writer.MultiplexingIndexWriter @ 0x6be46e8e0
> 879,047,864 24.41% java.util.HashMap @ 0x6be46e908
> 879,047,816 24.41% • java.util.HashMap$Node[16] @ 0x6be46e938
> 879,047,736 24.41% java.util.HashMap$Node @ 0x6be46e988
> 879,047,704 24.41% org.apache.jackrabbit.oak.plugins.index.lucene.writer.DefaultIndexWriter @ 0x6be46e9a8
> 879,047,648 24.41% org.apache.lucene.index.IndexWriter @ 0x6be46e9e0
> 870,569,576 24.17% org.apache.lucene.index.ThreadAffinityDocumentsWriterThreadPool @ 0x6be477680
> 870,568,880 24.17% org.apache.lucene.index.DocumentsWriterPerThreadPool$ThreadState @ 0x6be477888
> 870,568,816 24.17% org.apache.lucene.index.DocumentsWriterPerThread @ 0x6be4778c8
> 863,858,200 23.98% org.apache.lucene.index.DocumentsWriterDeleteQueue$DeleteSlice @ 0x6be4f4020
> 4,381,272 0.12% org.apache.lucene.index.BufferedUpdates @ 0x6be4b3978
> 2,325,704 0.06% org.apache.lucene.index.DocFieldProcessor @ 0x6be477a50
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)