You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@groovy.apache.org by "Paul King (Jira)" <ji...@apache.org> on 2026/08/28 07:32:00 UTC
[jira] [Updated] (GROOVY-12316) groovy.antlr4.cache.threshold silently disables pressure-based DFA clearing
[ https://issues.apache.org/jira/browse/GROOVY-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul King updated GROOVY-12316:
-------------------------------
Description:
h2. Problem
{{AtnManager}} has two mechanisms for dropping the shared parser DFA cache, but they are mutually exclusive rather than composable:
* a *GC canary* — the softly referenced {{AtnWrapper}}. Its collection signals memory pressure, so the cache is dropped when the next parse observes the cleared reference.
* a *parse counter* — clears every {{groovy.antlr4.cache.threshold}} parses.
{{isSmartCleanupEnabled()}} is defined as {{DFA_CACHE_THRESHOLD == 0}}, and the canary clear is guarded by {{shouldClearDfaCache() && isSmartCleanupEnabled()}}. Setting *any* positive threshold therefore switches the canary off.
Since the threshold is the documented knob for bounding DFA cache growth, a user reaching for it to reduce memory use silently removes the only mechanism that responds to actual memory pressure — and can end up worse off than before.
This was hit in practice: a user chasing an OOM on 5.1.1 under Maven at {{-Xmx128m}} set {{-Dgroovy.antlr4.cache.threshold=200}} as a mitigation, which made *5.1.0* start OOMing too, because it disabled the valve that had been keeping it alive.
h2. Reproduction
In a forked JVM with {{-Dgroovy.antlr4.cache.threshold=25}}, parse 12 sources (fewer than the threshold, so the counter cannot fire), force the JVM to clear soft references, then parse once more and count live states in {{GroovyParser._ATN.decisionToDFA}}:
|| || DFA states populated || after soft refs cleared ||
| current | 478 | 486 — not reclaimed |
| composed | 478 | drops — reclaimed by the canary |
h2. Fix
Derive two independent switches from the raw property instead of overloading one:
{code:java}
GC_CANARY_ENABLED = gcCanaryEnabled(t); // t >= 0
DFA_CACHE_THRESHOLD = counterThreshold(t); // max(t, 0)
{code}
The canary clear is then guarded by {{GC_CANARY_ENABLED}} and the parse counter by {{isThresholdCleanupEnabled()}}, so both can be active at once.
|| threshold || before || after ||
| {{0}} (default) | canary on, counter off | unchanged |
| {{> 0}} | canary *off*, counter on | canary *on*, counter on |
| {{< 0}} | never clear | unchanged (never clear) |
Behaviour changes only in the {{> 0}} case. The negative value remains the explicit "never clear" escape hatch — it now switches off both mechanisms, since "canary on" is no longer implied by a zero threshold.
h2. Notes
* The default configuration is unchanged, so this carries no throughput cost for users who do not set the property.
* This makes the knob usable; it does not on its own close the 5.1.1 memory regression. At the default the canary is still only observed on the parse path, so under a tight heap the cache can go unreclaimed until an OOM that arrives mid-parse. Closing that needs either a non-zero default threshold or holding the DFA cache softly in the ANTLR runtime.
> groovy.antlr4.cache.threshold silently disables pressure-based DFA clearing
> ---------------------------------------------------------------------------
>
> Key: GROOVY-12316
> URL: https://issues.apache.org/jira/browse/GROOVY-12316
> Project: Groovy
> Issue Type: Improvement
> Reporter: Paul King
> Assignee: Paul King
> Priority: Major
>
> h2. Problem
> {{AtnManager}} has two mechanisms for dropping the shared parser DFA cache, but they are mutually exclusive rather than composable:
> * a *GC canary* — the softly referenced {{AtnWrapper}}. Its collection signals memory pressure, so the cache is dropped when the next parse observes the cleared reference.
> * a *parse counter* — clears every {{groovy.antlr4.cache.threshold}} parses.
> {{isSmartCleanupEnabled()}} is defined as {{DFA_CACHE_THRESHOLD == 0}}, and the canary clear is guarded by {{shouldClearDfaCache() && isSmartCleanupEnabled()}}. Setting *any* positive threshold therefore switches the canary off.
> Since the threshold is the documented knob for bounding DFA cache growth, a user reaching for it to reduce memory use silently removes the only mechanism that responds to actual memory pressure — and can end up worse off than before.
> This was hit in practice: a user chasing an OOM on 5.1.1 under Maven at {{-Xmx128m}} set {{-Dgroovy.antlr4.cache.threshold=200}} as a mitigation, which made *5.1.0* start OOMing too, because it disabled the valve that had been keeping it alive.
> h2. Reproduction
> In a forked JVM with {{-Dgroovy.antlr4.cache.threshold=25}}, parse 12 sources (fewer than the threshold, so the counter cannot fire), force the JVM to clear soft references, then parse once more and count live states in {{GroovyParser._ATN.decisionToDFA}}:
> || || DFA states populated || after soft refs cleared ||
> | current | 478 | 486 — not reclaimed |
> | composed | 478 | drops — reclaimed by the canary |
> h2. Fix
> Derive two independent switches from the raw property instead of overloading one:
> {code:java}
> GC_CANARY_ENABLED = gcCanaryEnabled(t); // t >= 0
> DFA_CACHE_THRESHOLD = counterThreshold(t); // max(t, 0)
> {code}
> The canary clear is then guarded by {{GC_CANARY_ENABLED}} and the parse counter by {{isThresholdCleanupEnabled()}}, so both can be active at once.
> || threshold || before || after ||
> | {{0}} (default) | canary on, counter off | unchanged |
> | {{> 0}} | canary *off*, counter on | canary *on*, counter on |
> | {{< 0}} | never clear | unchanged (never clear) |
> Behaviour changes only in the {{> 0}} case. The negative value remains the explicit "never clear" escape hatch — it now switches off both mechanisms, since "canary on" is no longer implied by a zero threshold.
> h2. Notes
> * The default configuration is unchanged, so this carries no throughput cost for users who do not set the property.
> * This makes the knob usable; it does not on its own close the 5.1.1 memory regression. At the default the canary is still only observed on the parse path, so under a tight heap the cache can go unreclaimed until an OOM that arrives mid-parse. Closing that needs either a non-zero default threshold or holding the DFA cache softly in the ANTLR runtime.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)