You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Mike Klaas (JIRA)" <ji...@apache.org> on 2007/07/19 02:16:05 UTC
[jira] Created: (SOLR-310) Deleting pending handling can cause OOM
Deleting pending handling can cause OOM
---------------------------------------
Key: SOLR-310
URL: https://issues.apache.org/jira/browse/SOLR-310
Project: Solr
Issue Type: Bug
Reporter: Mike Klaas
Assignee: Mike Klaas
Fix For: 1.3
> Mike:
>> Could happen when doDeleting the pending docs too. James: try
>> sending commit every 500k docs or so.
Yonik:
> Hmmm, right... some of the memory usage will be related to the treemap
> keeping track of deleted items, and some of it will be related to the
> reader itself (the term index, and the norms).
> Perhaps we should have some sort of limit on the number of deletions
> we cache so people don't have to worry about that part.
Good idea. Perhaps a solrconfig.xml setting, <maxPendingDocs>? Default to 100k?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Work started: (SOLR-310) Deleting pending handling can cause
OOM
Posted by "Mike Klaas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on SOLR-310 started by Mike Klaas.
> Deleting pending handling can cause OOM
> ---------------------------------------
>
> Key: SOLR-310
> URL: https://issues.apache.org/jira/browse/SOLR-310
> Project: Solr
> Issue Type: Bug
> Reporter: Mike Klaas
> Assignee: Mike Klaas
> Fix For: 1.3
>
>
> > Mike:
> >> Could happen when doDeleting the pending docs too. James: try
> >> sending commit every 500k docs or so.
> Yonik:
> > Hmmm, right... some of the memory usage will be related to the treemap
> > keeping track of deleted items, and some of it will be related to the
> > reader itself (the term index, and the norms).
> > Perhaps we should have some sort of limit on the number of deletions
> > we cache so people don't have to worry about that part.
> Good idea. Perhaps a solrconfig.xml setting, <maxPendingDocs>? Default to 100k?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-310) Deleting pending handling can cause OOM
Posted by "Mike Klaas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mike Klaas resolved SOLR-310.
-----------------------------
Resolution: Fixed
committed in r572831
> Deleting pending handling can cause OOM
> ---------------------------------------
>
> Key: SOLR-310
> URL: https://issues.apache.org/jira/browse/SOLR-310
> Project: Solr
> Issue Type: Bug
> Reporter: Mike Klaas
> Assignee: Mike Klaas
> Fix For: 1.3
>
> Attachments: maxPendingDeletes.patch
>
>
> > Mike:
> >> Could happen when doDeleting the pending docs too. James: try
> >> sending commit every 500k docs or so.
> Yonik:
> > Hmmm, right... some of the memory usage will be related to the treemap
> > keeping track of deleted items, and some of it will be related to the
> > reader itself (the term index, and the norms).
> > Perhaps we should have some sort of limit on the number of deletions
> > we cache so people don't have to worry about that part.
> Good idea. Perhaps a solrconfig.xml setting, <maxPendingDocs>? Default to 100k?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-310) Deleting pending handling can cause OOM
Posted by "Mike Klaas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mike Klaas updated SOLR-310:
----------------------------
Attachment: maxPendingDeletes.patch
q&d patch that implements the above. Called it maxPendingDeletes, since the complexity/memory being bounded is determined by that, not pending docs.
> Deleting pending handling can cause OOM
> ---------------------------------------
>
> Key: SOLR-310
> URL: https://issues.apache.org/jira/browse/SOLR-310
> Project: Solr
> Issue Type: Bug
> Reporter: Mike Klaas
> Assignee: Mike Klaas
> Fix For: 1.3
>
> Attachments: maxPendingDeletes.patch
>
>
> > Mike:
> >> Could happen when doDeleting the pending docs too. James: try
> >> sending commit every 500k docs or so.
> Yonik:
> > Hmmm, right... some of the memory usage will be related to the treemap
> > keeping track of deleted items, and some of it will be related to the
> > reader itself (the term index, and the norms).
> > Perhaps we should have some sort of limit on the number of deletions
> > we cache so people don't have to worry about that part.
> Good idea. Perhaps a solrconfig.xml setting, <maxPendingDocs>? Default to 100k?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.