You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jackrabbit.apache.org by "Thomas Mueller (JIRA)" <ji...@apache.org> on 2009/11/23 17:02:39 UTC
[jira] Created: (JCR-2402) Observation: avoid running out of memory
Observation: avoid running out of memory
----------------------------------------
Key: JCR-2402
URL: https://issues.apache.org/jira/browse/JCR-2402
Project: Jackrabbit Content Repository
Issue Type: Improvement
Components: jackrabbit-core, observation
Reporter: Thomas Mueller
Assignee: Thomas Mueller
Jackrabbit uses an unbounded observation queue for event listeners (for asynchronous listeners, which are the default). If an observation listener is very slow, the observation queue gets larger and larger, and the JVM will eventually run out of memory.
I suggest to use a maximum queue size of 100'000 by default. Adding new events to the queue will block until the observation listeners removed an item. I'm not sure if we need a way to configure this option; probably a system property is enough as a start (we can still add a better way to configure this setting if it turns out somebody actually needs a different value).
A special case is observation listeners that themselves write to the repository and therefore cause new events. In this case, it doesn't make sense to block adding an event, because that would block the whole system. However a warning should be written to the log file.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (JCR-2402) Observation: avoid running out of memory
Posted by "Thomas Mueller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-2402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Mueller updated JCR-2402:
--------------------------------
Status: Open (was: Patch Available)
> Observation: avoid running out of memory
> ----------------------------------------
>
> Key: JCR-2402
> URL: https://issues.apache.org/jira/browse/JCR-2402
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core, observation
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Priority: Minor
>
> Jackrabbit uses an unbounded observation queue for event listeners (for asynchronous listeners, which are the default). If an observation listener is very slow, the observation queue gets larger and larger, and the JVM will eventually run out of memory.
> I suggest to use a maximum queue size of 100'000 by default. Adding new events to the queue will block until the observation listeners removed an item. I'm not sure if we need a way to configure this option; probably a system property is enough as a start (we can still add a better way to configure this setting if it turns out somebody actually needs a different value).
> A special case is observation listeners that themselves write to the repository and therefore cause new events. In this case, it doesn't make sense to block adding an event, because that would block the whole system. However a warning should be written to the log file.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (JCR-2402) Observation: avoid running out of memory
Posted by "Thomas Mueller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-2402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Mueller updated JCR-2402:
--------------------------------
Attachment: jcr-2402.patch
This patch uses a default max queue size of 100'000. A warning is logged at most every 5 seconds, together with the stack trace (to find problems with recursive event listeners).
> Observation: avoid running out of memory
> ----------------------------------------
>
> Key: JCR-2402
> URL: https://issues.apache.org/jira/browse/JCR-2402
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core, observation
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Priority: Minor
> Attachments: jcr-2402.patch
>
>
> Jackrabbit uses an unbounded observation queue for event listeners (for asynchronous listeners, which are the default). If an observation listener is very slow, the observation queue gets larger and larger, and the JVM will eventually run out of memory.
> I suggest to use a maximum queue size of 100'000 by default. Adding new events to the queue will block until the observation listeners removed an item. I'm not sure if we need a way to configure this option; probably a system property is enough as a start (we can still add a better way to configure this setting if it turns out somebody actually needs a different value).
> A special case is observation listeners that themselves write to the repository and therefore cause new events. In this case, it doesn't make sense to block adding an event, because that would block the whole system. However a warning should be written to the log file.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (JCR-2402) Observation: avoid running out of memory
Posted by "Thomas Mueller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-2402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Mueller updated JCR-2402:
--------------------------------
Status: Patch Available (was: Open)
> Observation: avoid running out of memory
> ----------------------------------------
>
> Key: JCR-2402
> URL: https://issues.apache.org/jira/browse/JCR-2402
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core, observation
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Priority: Minor
>
> Jackrabbit uses an unbounded observation queue for event listeners (for asynchronous listeners, which are the default). If an observation listener is very slow, the observation queue gets larger and larger, and the JVM will eventually run out of memory.
> I suggest to use a maximum queue size of 100'000 by default. Adding new events to the queue will block until the observation listeners removed an item. I'm not sure if we need a way to configure this option; probably a system property is enough as a start (we can still add a better way to configure this setting if it turns out somebody actually needs a different value).
> A special case is observation listeners that themselves write to the repository and therefore cause new events. In this case, it doesn't make sense to block adding an event, because that would block the whole system. However a warning should be written to the log file.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (JCR-2402) Observation: avoid running out of memory
Posted by "Thomas Mueller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-2402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Mueller updated JCR-2402:
--------------------------------
Attachment: jcr-2402-2.patch
jcr-2402-2.patch will calculate the number of event states, not the number of DispatchAction objects. The limit is set to 200'000, which is about 200 MB (according to my test). The thread waits after adding the DispatchAction, otherwise actions with many events would wait forever.
> Observation: avoid running out of memory
> ----------------------------------------
>
> Key: JCR-2402
> URL: https://issues.apache.org/jira/browse/JCR-2402
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core, observation
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Priority: Minor
> Attachments: jcr-2402-2.patch, jcr-2402.patch
>
>
> Jackrabbit uses an unbounded observation queue for event listeners (for asynchronous listeners, which are the default). If an observation listener is very slow, the observation queue gets larger and larger, and the JVM will eventually run out of memory.
> I suggest to use a maximum queue size of 100'000 by default. Adding new events to the queue will block until the observation listeners removed an item. I'm not sure if we need a way to configure this option; probably a system property is enough as a start (we can still add a better way to configure this setting if it turns out somebody actually needs a different value).
> A special case is observation listeners that themselves write to the repository and therefore cause new events. In this case, it doesn't make sense to block adding an event, because that would block the whole system. However a warning should be written to the log file.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (JCR-2402) Observation: avoid running out of memory
Posted by "Thomas Mueller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-2402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Mueller updated JCR-2402:
--------------------------------
Priority: Minor (was: Major)
> Observation: avoid running out of memory
> ----------------------------------------
>
> Key: JCR-2402
> URL: https://issues.apache.org/jira/browse/JCR-2402
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core, observation
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Priority: Minor
>
> Jackrabbit uses an unbounded observation queue for event listeners (for asynchronous listeners, which are the default). If an observation listener is very slow, the observation queue gets larger and larger, and the JVM will eventually run out of memory.
> I suggest to use a maximum queue size of 100'000 by default. Adding new events to the queue will block until the observation listeners removed an item. I'm not sure if we need a way to configure this option; probably a system property is enough as a start (we can still add a better way to configure this setting if it turns out somebody actually needs a different value).
> A special case is observation listeners that themselves write to the repository and therefore cause new events. In this case, it doesn't make sense to block adding an event, because that would block the whole system. However a warning should be written to the log file.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (JCR-2402) Observation: avoid running out of
memory
Posted by "Thomas Mueller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-2402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Mueller resolved JCR-2402.
---------------------------------
Resolution: Fixed
Fix Version/s: 2.0-beta4
> Observation: avoid running out of memory
> ----------------------------------------
>
> Key: JCR-2402
> URL: https://issues.apache.org/jira/browse/JCR-2402
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core, observation
> Reporter: Thomas Mueller
> Assignee: Thomas Mueller
> Priority: Minor
> Fix For: 2.0-beta4
>
> Attachments: jcr-2402-2.patch, jcr-2402.patch
>
>
> Jackrabbit uses an unbounded observation queue for event listeners (for asynchronous listeners, which are the default). If an observation listener is very slow, the observation queue gets larger and larger, and the JVM will eventually run out of memory.
> I suggest to use a maximum queue size of 100'000 by default. Adding new events to the queue will block until the observation listeners removed an item. I'm not sure if we need a way to configure this option; probably a system property is enough as a start (we can still add a better way to configure this setting if it turns out somebody actually needs a different value).
> A special case is observation listeners that themselves write to the repository and therefore cause new events. In this case, it doesn't make sense to block adding an event, because that would block the whole system. However a warning should be written to the log file.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.