You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tapestry.apache.org by "Fernando (JIRA)" <de...@tapestry.apache.org> on 2008/06/23 17:08:45 UTC
[jira] Created: (TAPESTRY-2473) Thread Lock Ups, unkonwn cause
Thread Lock Ups, unkonwn cause
------------------------------
Key: TAPESTRY-2473
URL: https://issues.apache.org/jira/browse/TAPESTRY-2473
Project: Tapestry
Issue Type: Bug
Components: Framework
Affects Versions: 5.0.13, 5.0.12, 5.0.11
Environment: Red Hat, Tomcat, java version "1.5.0_10"
Reporter: Fernando
Priority: Critical
We are running in production, and pretty regularly the system seems to hang up. After I do a jstack it looks like all the threads are tied up against some lock.
Can you help me figure out the root cause?
I will be attaching the full jstack the the stuck threads all look like:
Thread 25014: (state = BLOCKED)
- sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise)
- java.util.concurrent.locks.LockSupport.park() @bci=5, line=118 (Compiled frame)
- java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(int) @bci=10, line=1124 (Compiled frame)
- org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=27, line=70 (Compiled frame)
- $RequestHandler_11aa7f96e6c.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response) @bci=10 (Compiled frame)
- org.apache.tapestry.services.TapestryModule$12.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=40, line=924 (Compiled frame)
- $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
- org.apache.tapestry.internal.services.IgnoredPathsFilter.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci
=95, line=62 (Compiled frame)
- $HttpServletRequestFilter_11aa7f96e63.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci=7 (Compiled frame)
- $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
- $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
- org.apache.tapestry.TapestryFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=12, line=168 (Compiled frame)
......
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Updated: (TAPESTRY-2473) Thread Lock Ups, unkonwn cause
Posted by "Fernando (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Fernando updated TAPESTRY-2473:
-------------------------------
Attachment: js.txt
a jstack when the server is stuck.
> Thread Lock Ups, unkonwn cause
> ------------------------------
>
> Key: TAPESTRY-2473
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2473
> Project: Tapestry
> Issue Type: Bug
> Components: Framework
> Affects Versions: 5.0.11, 5.0.12, 5.0.13
> Environment: Red Hat, Tomcat, java version "1.5.0_10"
> Reporter: Fernando
> Priority: Critical
> Attachments: js.txt, js2.txt
>
>
> We are running in production, and pretty regularly the system seems to hang up. After I do a jstack it looks like all the threads are tied up against some lock.
> Can you help me figure out the root cause?
> I will be attaching the full jstack the the stuck threads all look like:
> Thread 25014: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise)
> - java.util.concurrent.locks.LockSupport.park() @bci=5, line=118 (Compiled frame)
> - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(int) @bci=10, line=1124 (Compiled frame)
> - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=27, line=70 (Compiled frame)
> - $RequestHandler_11aa7f96e6c.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response) @bci=10 (Compiled frame)
> - org.apache.tapestry.services.TapestryModule$12.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=40, line=924 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.internal.services.IgnoredPathsFilter.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci
> =95, line=62 (Compiled frame)
> - $HttpServletRequestFilter_11aa7f96e63.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci=7 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.TapestryFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=12, line=168 (Compiled frame)
> ......
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Commented: (TAPESTRY-2473) Application locks up, with all
threads waiting for read lock inside ConcurrentBarrier
Posted by "Fernando (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607313#action_12607313 ]
Fernando commented on TAPESTRY-2473:
------------------------------------
Yes, CheckForUpdatesFilter is partially responsible, but I think mostly because it is the only user of the tryWithRead function.
At the moment the code in ConcurrentBarrier just looks too complicated for me.. I would propose for you to put some thought reviewing that code and see if you still want locking code that is so complicated ( interactions between read lock, write lock, read lock Flag and a read to write lock upgrade support ). I bet you've already put lots of thought on it and it's most likely correct, but it might just be brittle for long term support.
here is some food for thought:
Does the checking for updates really need to be done inline for every request (and upgrading a read lock to a write lock)? Could it be launched as a side thread with it's own locking, then only block requests while it's rebuilding caches or while it's reseting page caches...
> Application locks up, with all threads waiting for read lock inside ConcurrentBarrier
> -------------------------------------------------------------------------------------
>
> Key: TAPESTRY-2473
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2473
> Project: Tapestry
> Issue Type: Bug
> Components: Framework
> Affects Versions: 5.0.11, 5.0.12, 5.0.13
> Environment: Red Hat, Tomcat, java version "1.5.0_10"
> Reporter: Fernando
> Assignee: Howard M. Lewis Ship
> Priority: Critical
> Attachments: js.txt, js2.txt
>
>
> We are running in production, and pretty regularly the system seems to hang up. After I do a jstack it looks like all the threads are tied up against some lock.
> Can you help me figure out the root cause?
> I will be attaching the full jstack the the stuck threads all look like:
> Thread 25014: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise)
> - java.util.concurrent.locks.LockSupport.park() @bci=5, line=118 (Compiled frame)
> - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(int) @bci=10, line=1124 (Compiled frame)
> - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=27, line=70 (Compiled frame)
> - $RequestHandler_11aa7f96e6c.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response) @bci=10 (Compiled frame)
> - org.apache.tapestry.services.TapestryModule$12.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=40, line=924 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.internal.services.IgnoredPathsFilter.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci
> =95, line=62 (Compiled frame)
> - $HttpServletRequestFilter_11aa7f96e63.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci=7 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.TapestryFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=12, line=168 (Compiled frame)
> ......
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Commented: (TAPESTRY-2473) Application locks up, with all
threads waiting for read lock inside ConcurrentBarrier
Posted by "Howard M. Lewis Ship (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607325#action_12607325 ]
Howard M. Lewis Ship commented on TAPESTRY-2473:
------------------------------------------------
This message (about a year old) looks related:
http://markmail.org/message/gzx5akfhcjcpis3z
Which leads to this bug:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6571733
which claims not to exist in JDK 1.5, just in 1.6 and above.
> Application locks up, with all threads waiting for read lock inside ConcurrentBarrier
> -------------------------------------------------------------------------------------
>
> Key: TAPESTRY-2473
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2473
> Project: Tapestry
> Issue Type: Bug
> Components: Framework
> Affects Versions: 5.0.11, 5.0.12, 5.0.13
> Environment: Red Hat, Tomcat, java version "1.5.0_10"
> Reporter: Fernando
> Assignee: Howard M. Lewis Ship
> Priority: Critical
> Attachments: js.txt, js2.txt
>
>
> We are running in production, and pretty regularly the system seems to hang up. After I do a jstack it looks like all the threads are tied up against some lock.
> Can you help me figure out the root cause?
> I will be attaching the full jstack the the stuck threads all look like:
> Thread 25014: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise)
> - java.util.concurrent.locks.LockSupport.park() @bci=5, line=118 (Compiled frame)
> - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(int) @bci=10, line=1124 (Compiled frame)
> - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=27, line=70 (Compiled frame)
> - $RequestHandler_11aa7f96e6c.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response) @bci=10 (Compiled frame)
> - org.apache.tapestry.services.TapestryModule$12.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=40, line=924 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.internal.services.IgnoredPathsFilter.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci
> =95, line=62 (Compiled frame)
> - $HttpServletRequestFilter_11aa7f96e63.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci=7 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.TapestryFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=12, line=168 (Compiled frame)
> ......
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Commented: (TAPESTRY-2473) Application locks up, with all
threads waiting for read lock inside ConcurrentBarrier
Posted by "Howard M. Lewis Ship (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607336#action_12607336 ]
Howard M. Lewis Ship commented on TAPESTRY-2473:
------------------------------------------------
I've checked in an attempt at a fix that synchronized access to the ThreadLocal.
The file, tapestry-ioc-5.0.14-SNAPSHOT.jar is available here: http://senduit.com/767cd1
Please give it a try and get back to us about whether it resolves the problem.
As to your question about the complexity: I believe a later JDK understands the need to upgrade a read-lock to a write-lock, which would simplify this code a bit. Having a single filter than can reduce the system momentarily to a single thread (one that checks for underlying file updates and, as necessary, clears caches) is very important: it simplifies the rest of the code base and ensures that the application operates in a predictable way.
> Application locks up, with all threads waiting for read lock inside ConcurrentBarrier
> -------------------------------------------------------------------------------------
>
> Key: TAPESTRY-2473
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2473
> Project: Tapestry
> Issue Type: Bug
> Components: Framework
> Affects Versions: 5.0.11, 5.0.12, 5.0.13
> Environment: Red Hat, Tomcat, java version "1.5.0_10"
> Reporter: Fernando
> Assignee: Howard M. Lewis Ship
> Priority: Critical
> Attachments: js.txt, js2.txt
>
>
> We are running in production, and pretty regularly the system seems to hang up. After I do a jstack it looks like all the threads are tied up against some lock.
> Can you help me figure out the root cause?
> I will be attaching the full jstack the the stuck threads all look like:
> Thread 25014: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise)
> - java.util.concurrent.locks.LockSupport.park() @bci=5, line=118 (Compiled frame)
> - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(int) @bci=10, line=1124 (Compiled frame)
> - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=27, line=70 (Compiled frame)
> - $RequestHandler_11aa7f96e6c.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response) @bci=10 (Compiled frame)
> - org.apache.tapestry.services.TapestryModule$12.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=40, line=924 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.internal.services.IgnoredPathsFilter.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci
> =95, line=62 (Compiled frame)
> - $HttpServletRequestFilter_11aa7f96e63.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci=7 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.TapestryFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=12, line=168 (Compiled frame)
> ......
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Updated: (TAPESTRY-2473) Application locks up, with all
threads waiting for read lock inside ConcurrentBarrier
Posted by "Howard M. Lewis Ship (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Howard M. Lewis Ship updated TAPESTRY-2473:
-------------------------------------------
Summary: Application locks up, with all threads waiting for read lock inside ConcurrentBarrier (was: Thread Lock Ups, unkonwn cause)
> Application locks up, with all threads waiting for read lock inside ConcurrentBarrier
> -------------------------------------------------------------------------------------
>
> Key: TAPESTRY-2473
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2473
> Project: Tapestry
> Issue Type: Bug
> Components: Framework
> Affects Versions: 5.0.11, 5.0.12, 5.0.13
> Environment: Red Hat, Tomcat, java version "1.5.0_10"
> Reporter: Fernando
> Assignee: Howard M. Lewis Ship
> Priority: Critical
> Attachments: js.txt, js2.txt
>
>
> We are running in production, and pretty regularly the system seems to hang up. After I do a jstack it looks like all the threads are tied up against some lock.
> Can you help me figure out the root cause?
> I will be attaching the full jstack the the stuck threads all look like:
> Thread 25014: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise)
> - java.util.concurrent.locks.LockSupport.park() @bci=5, line=118 (Compiled frame)
> - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(int) @bci=10, line=1124 (Compiled frame)
> - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=27, line=70 (Compiled frame)
> - $RequestHandler_11aa7f96e6c.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response) @bci=10 (Compiled frame)
> - org.apache.tapestry.services.TapestryModule$12.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=40, line=924 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.internal.services.IgnoredPathsFilter.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci
> =95, line=62 (Compiled frame)
> - $HttpServletRequestFilter_11aa7f96e63.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci=7 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.TapestryFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=12, line=168 (Compiled frame)
> ......
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Commented: (TAPESTRY-2473) Thread Lock Ups, unkonwn cause
Posted by "Howard M. Lewis Ship (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607295#action_12607295 ]
Howard M. Lewis Ship commented on TAPESTRY-2473:
------------------------------------------------
What's odd here is that all the threads are blocked in withRead() (i.e., acquiring the shared read lock). That's very odd because I don't see any thread inside withWrite() (i.e., with the exclusive write lock). I remember there was a earlier hickup where Tapestry's use of ReentrantReadWriteLock tripped across a JDK bug.
> Thread Lock Ups, unkonwn cause
> ------------------------------
>
> Key: TAPESTRY-2473
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2473
> Project: Tapestry
> Issue Type: Bug
> Components: Framework
> Affects Versions: 5.0.11, 5.0.12, 5.0.13
> Environment: Red Hat, Tomcat, java version "1.5.0_10"
> Reporter: Fernando
> Priority: Critical
> Attachments: js.txt, js2.txt
>
>
> We are running in production, and pretty regularly the system seems to hang up. After I do a jstack it looks like all the threads are tied up against some lock.
> Can you help me figure out the root cause?
> I will be attaching the full jstack the the stuck threads all look like:
> Thread 25014: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise)
> - java.util.concurrent.locks.LockSupport.park() @bci=5, line=118 (Compiled frame)
> - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(int) @bci=10, line=1124 (Compiled frame)
> - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=27, line=70 (Compiled frame)
> - $RequestHandler_11aa7f96e6c.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response) @bci=10 (Compiled frame)
> - org.apache.tapestry.services.TapestryModule$12.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=40, line=924 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.internal.services.IgnoredPathsFilter.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci
> =95, line=62 (Compiled frame)
> - $HttpServletRequestFilter_11aa7f96e63.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci=7 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.TapestryFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=12, line=168 (Compiled frame)
> ......
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Commented: (TAPESTRY-2473) Application locks up, with all
threads waiting for read lock inside ConcurrentBarrier
Posted by "Fernando (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607310#action_12607310 ]
Fernando commented on TAPESTRY-2473:
------------------------------------
I did a simple grep/sort/count on the two files:
It does look like they are also stuck on a few restoreReadLock, so that means that the writeLocks aren't releasing properly??
hala:~> grep ConcurrentBarrier js.txt | sort | uniq -c
2 - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.restoreReadLock(boolean) @bci=13, line=158 (Compiled frame)
354 - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=27, line=70 (Compiled frame)
3 - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=44, line=77 (Compiled frame)
hala:~> grep ConcurrentBarrier js2.txt | sort | uniq -c
5 - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.restoreReadLock(boolean) @bci=13, line=158 (Compiled frame)
393 - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=27, line=70 (Compiled frame)
6 - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=44, line=77 (Compiled frame)
> Application locks up, with all threads waiting for read lock inside ConcurrentBarrier
> -------------------------------------------------------------------------------------
>
> Key: TAPESTRY-2473
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2473
> Project: Tapestry
> Issue Type: Bug
> Components: Framework
> Affects Versions: 5.0.11, 5.0.12, 5.0.13
> Environment: Red Hat, Tomcat, java version "1.5.0_10"
> Reporter: Fernando
> Assignee: Howard M. Lewis Ship
> Priority: Critical
> Attachments: js.txt, js2.txt
>
>
> We are running in production, and pretty regularly the system seems to hang up. After I do a jstack it looks like all the threads are tied up against some lock.
> Can you help me figure out the root cause?
> I will be attaching the full jstack the the stuck threads all look like:
> Thread 25014: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise)
> - java.util.concurrent.locks.LockSupport.park() @bci=5, line=118 (Compiled frame)
> - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(int) @bci=10, line=1124 (Compiled frame)
> - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=27, line=70 (Compiled frame)
> - $RequestHandler_11aa7f96e6c.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response) @bci=10 (Compiled frame)
> - org.apache.tapestry.services.TapestryModule$12.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=40, line=924 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.internal.services.IgnoredPathsFilter.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci
> =95, line=62 (Compiled frame)
> - $HttpServletRequestFilter_11aa7f96e63.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci=7 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.TapestryFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=12, line=168 (Compiled frame)
> ......
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Commented: (TAPESTRY-2473) Application locks up, with all
threads waiting for read lock inside ConcurrentBarrier
Posted by "Fernando (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607314#action_12607314 ]
Fernando commented on TAPESTRY-2473:
------------------------------------
but if synchronizing access to the threadLocal fixes it, and you're happy with that. then we're all set!! :) :)
> Application locks up, with all threads waiting for read lock inside ConcurrentBarrier
> -------------------------------------------------------------------------------------
>
> Key: TAPESTRY-2473
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2473
> Project: Tapestry
> Issue Type: Bug
> Components: Framework
> Affects Versions: 5.0.11, 5.0.12, 5.0.13
> Environment: Red Hat, Tomcat, java version "1.5.0_10"
> Reporter: Fernando
> Assignee: Howard M. Lewis Ship
> Priority: Critical
> Attachments: js.txt, js2.txt
>
>
> We are running in production, and pretty regularly the system seems to hang up. After I do a jstack it looks like all the threads are tied up against some lock.
> Can you help me figure out the root cause?
> I will be attaching the full jstack the the stuck threads all look like:
> Thread 25014: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise)
> - java.util.concurrent.locks.LockSupport.park() @bci=5, line=118 (Compiled frame)
> - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(int) @bci=10, line=1124 (Compiled frame)
> - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=27, line=70 (Compiled frame)
> - $RequestHandler_11aa7f96e6c.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response) @bci=10 (Compiled frame)
> - org.apache.tapestry.services.TapestryModule$12.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=40, line=924 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.internal.services.IgnoredPathsFilter.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci
> =95, line=62 (Compiled frame)
> - $HttpServletRequestFilter_11aa7f96e63.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci=7 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.TapestryFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=12, line=168 (Compiled frame)
> ......
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Commented: (TAPESTRY-2473) Thread Lock Ups, unkonwn cause
Posted by "Howard M. Lewis Ship (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607305#action_12607305 ]
Howard M. Lewis Ship commented on TAPESTRY-2473:
------------------------------------------------
Another possibility is the ThreadLocal bug (in the JDK 1.5). May have to synchronize access to the ThreadLocal object inside ConcurrentBarrier.
> Thread Lock Ups, unkonwn cause
> ------------------------------
>
> Key: TAPESTRY-2473
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2473
> Project: Tapestry
> Issue Type: Bug
> Components: Framework
> Affects Versions: 5.0.11, 5.0.12, 5.0.13
> Environment: Red Hat, Tomcat, java version "1.5.0_10"
> Reporter: Fernando
> Priority: Critical
> Attachments: js.txt, js2.txt
>
>
> We are running in production, and pretty regularly the system seems to hang up. After I do a jstack it looks like all the threads are tied up against some lock.
> Can you help me figure out the root cause?
> I will be attaching the full jstack the the stuck threads all look like:
> Thread 25014: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise)
> - java.util.concurrent.locks.LockSupport.park() @bci=5, line=118 (Compiled frame)
> - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(int) @bci=10, line=1124 (Compiled frame)
> - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=27, line=70 (Compiled frame)
> - $RequestHandler_11aa7f96e6c.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response) @bci=10 (Compiled frame)
> - org.apache.tapestry.services.TapestryModule$12.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=40, line=924 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.internal.services.IgnoredPathsFilter.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci
> =95, line=62 (Compiled frame)
> - $HttpServletRequestFilter_11aa7f96e63.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci=7 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.TapestryFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=12, line=168 (Compiled frame)
> ......
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Updated: (TAPESTRY-2473) Thread Lock Ups, unkonwn cause
Posted by "Fernando (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Fernando updated TAPESTRY-2473:
-------------------------------
Attachment: js2.txt
another stack trace that just happened
> Thread Lock Ups, unkonwn cause
> ------------------------------
>
> Key: TAPESTRY-2473
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2473
> Project: Tapestry
> Issue Type: Bug
> Components: Framework
> Affects Versions: 5.0.11, 5.0.12, 5.0.13
> Environment: Red Hat, Tomcat, java version "1.5.0_10"
> Reporter: Fernando
> Priority: Critical
> Attachments: js.txt, js2.txt
>
>
> We are running in production, and pretty regularly the system seems to hang up. After I do a jstack it looks like all the threads are tied up against some lock.
> Can you help me figure out the root cause?
> I will be attaching the full jstack the the stuck threads all look like:
> Thread 25014: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise)
> - java.util.concurrent.locks.LockSupport.park() @bci=5, line=118 (Compiled frame)
> - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(int) @bci=10, line=1124 (Compiled frame)
> - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=27, line=70 (Compiled frame)
> - $RequestHandler_11aa7f96e6c.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response) @bci=10 (Compiled frame)
> - org.apache.tapestry.services.TapestryModule$12.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=40, line=924 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.internal.services.IgnoredPathsFilter.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci
> =95, line=62 (Compiled frame)
> - $HttpServletRequestFilter_11aa7f96e63.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci=7 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.TapestryFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=12, line=168 (Compiled frame)
> ......
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Closed: (TAPESTRY-2473) Application locks up, with all
threads waiting for read lock inside ConcurrentBarrier
Posted by "Howard M. Lewis Ship (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Howard M. Lewis Ship closed TAPESTRY-2473.
------------------------------------------
Resolution: Fixed
Fix Version/s: 5.0.14
Let's just assume this *is* the fix, and re-open or create a new bug if it is not.
> Application locks up, with all threads waiting for read lock inside ConcurrentBarrier
> -------------------------------------------------------------------------------------
>
> Key: TAPESTRY-2473
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2473
> Project: Tapestry
> Issue Type: Bug
> Components: Framework
> Affects Versions: 5.0.11, 5.0.12, 5.0.13
> Environment: Red Hat, Tomcat, java version "1.5.0_10"
> Reporter: Fernando
> Assignee: Howard M. Lewis Ship
> Priority: Critical
> Fix For: 5.0.14
>
> Attachments: js.txt, js2.txt
>
>
> We are running in production, and pretty regularly the system seems to hang up. After I do a jstack it looks like all the threads are tied up against some lock.
> Can you help me figure out the root cause?
> I will be attaching the full jstack the the stuck threads all look like:
> Thread 25014: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise)
> - java.util.concurrent.locks.LockSupport.park() @bci=5, line=118 (Compiled frame)
> - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(int) @bci=10, line=1124 (Compiled frame)
> - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=27, line=70 (Compiled frame)
> - $RequestHandler_11aa7f96e6c.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response) @bci=10 (Compiled frame)
> - org.apache.tapestry.services.TapestryModule$12.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=40, line=924 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.internal.services.IgnoredPathsFilter.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci
> =95, line=62 (Compiled frame)
> - $HttpServletRequestFilter_11aa7f96e63.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci=7 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.TapestryFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=12, line=168 (Compiled frame)
> ......
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
Re: [jira] Commented: (TAPESTRY-2473) Application locks up, with
all threads waiting for read lock inside ConcurrentBarrier
Posted by Fernando Padilla <fe...@alum.mit.edu>.
I am trying to disable the CheckForUpdates filter..
contributeRequestHandler(........
configuration.add("CheckForUpdates", new
CheckForUpdatesFilter(_updateListenerHub, checkInterval, updateTimeout),
"before:*");
.........
but I can't figure out how to override a filter through the
configuration object.. since you can't seem to be able to delete a
configuration entry, nor override it (with same name), and the filter
wasn't instantiated through as a service ( no decoration available ).
Any other thoughts??
I also tried to have our system change the checkInterval to be different
when production-mode=true, but then tapestry complained about a cyclic
loop..
Howard M. Lewis Ship (JIRA) wrote:
> [ https://issues.apache.org/jira/browse/TAPESTRY-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607309#action_12607309 ]
>
> Howard M. Lewis Ship commented on TAPESTRY-2473:
> ------------------------------------------------
>
> The above trace omits a key detail:
>
> Thread 20716: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise)
> - java.util.concurrent.locks.LockSupport.park() @bci=5, line=118 (Compiled frame)
> - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=27, line=70 (Compiled frame)
> - org.apache.tapestry.internal.services.CheckForUpdatesFilter.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response, org.apache.tapestry.services.RequestHandler) @bci=30, line=106 (Compiled frame)
> - $RequestHandler_11aa3e523f4.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response) @bci=10 (Compiled frame)
> - $RequestHandler_11aa3e523ec.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response) @bci=6 (Compiled frame)
> - org.apache.tapestry.services.TapestryModule$12.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=40, line=924 (Compiled frame)
>
>
> The problem is certainly inside CheckForUpdatesFilter.
>
>> Application locks up, with all threads waiting for read lock inside ConcurrentBarrier
>> -------------------------------------------------------------------------------------
>>
>> Key: TAPESTRY-2473
>> URL: https://issues.apache.org/jira/browse/TAPESTRY-2473
>> Project: Tapestry
>> Issue Type: Bug
>> Components: Framework
>> Affects Versions: 5.0.11, 5.0.12, 5.0.13
>> Environment: Red Hat, Tomcat, java version "1.5.0_10"
>> Reporter: Fernando
>> Assignee: Howard M. Lewis Ship
>> Priority: Critical
>> Attachments: js.txt, js2.txt
>>
>>
>> We are running in production, and pretty regularly the system seems to hang up. After I do a jstack it looks like all the threads are tied up against some lock.
>> Can you help me figure out the root cause?
>> I will be attaching the full jstack the the stuck threads all look like:
>> Thread 25014: (state = BLOCKED)
>> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise)
>> - java.util.concurrent.locks.LockSupport.park() @bci=5, line=118 (Compiled frame)
>> - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(int) @bci=10, line=1124 (Compiled frame)
>> - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=27, line=70 (Compiled frame)
>> - $RequestHandler_11aa7f96e6c.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response) @bci=10 (Compiled frame)
>> - org.apache.tapestry.services.TapestryModule$12.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=40, line=924 (Compiled frame)
>> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
>> - org.apache.tapestry.internal.services.IgnoredPathsFilter.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci
>> =95, line=62 (Compiled frame)
>> - $HttpServletRequestFilter_11aa7f96e63.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci=7 (Compiled frame)
>> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
>> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
>> - org.apache.tapestry.TapestryFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=12, line=168 (Compiled frame)
>> ......
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Commented: (TAPESTRY-2473) Application locks up, with all
threads waiting for read lock inside ConcurrentBarrier
Posted by "Howard M. Lewis Ship (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607309#action_12607309 ]
Howard M. Lewis Ship commented on TAPESTRY-2473:
------------------------------------------------
The above trace omits a key detail:
Thread 20716: (state = BLOCKED)
- sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise)
- java.util.concurrent.locks.LockSupport.park() @bci=5, line=118 (Compiled frame)
- org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=27, line=70 (Compiled frame)
- org.apache.tapestry.internal.services.CheckForUpdatesFilter.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response, org.apache.tapestry.services.RequestHandler) @bci=30, line=106 (Compiled frame)
- $RequestHandler_11aa3e523f4.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response) @bci=10 (Compiled frame)
- $RequestHandler_11aa3e523ec.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response) @bci=6 (Compiled frame)
- org.apache.tapestry.services.TapestryModule$12.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=40, line=924 (Compiled frame)
The problem is certainly inside CheckForUpdatesFilter.
> Application locks up, with all threads waiting for read lock inside ConcurrentBarrier
> -------------------------------------------------------------------------------------
>
> Key: TAPESTRY-2473
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2473
> Project: Tapestry
> Issue Type: Bug
> Components: Framework
> Affects Versions: 5.0.11, 5.0.12, 5.0.13
> Environment: Red Hat, Tomcat, java version "1.5.0_10"
> Reporter: Fernando
> Assignee: Howard M. Lewis Ship
> Priority: Critical
> Attachments: js.txt, js2.txt
>
>
> We are running in production, and pretty regularly the system seems to hang up. After I do a jstack it looks like all the threads are tied up against some lock.
> Can you help me figure out the root cause?
> I will be attaching the full jstack the the stuck threads all look like:
> Thread 25014: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise)
> - java.util.concurrent.locks.LockSupport.park() @bci=5, line=118 (Compiled frame)
> - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(int) @bci=10, line=1124 (Compiled frame)
> - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=27, line=70 (Compiled frame)
> - $RequestHandler_11aa7f96e6c.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response) @bci=10 (Compiled frame)
> - org.apache.tapestry.services.TapestryModule$12.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=40, line=924 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.internal.services.IgnoredPathsFilter.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci
> =95, line=62 (Compiled frame)
> - $HttpServletRequestFilter_11aa7f96e63.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci=7 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.TapestryFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=12, line=168 (Compiled frame)
> ......
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Commented: (TAPESTRY-2473) Application locks up, with all
threads waiting for read lock inside ConcurrentBarrier
Posted by "Fernando (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607334#action_12607334 ]
Fernando commented on TAPESTRY-2473:
------------------------------------
hmm. you know your test cases better, could you somehow create a test case in tapestry to try to catch this? I know if it's VM related you might not hit it, but maybe someone will..
> Application locks up, with all threads waiting for read lock inside ConcurrentBarrier
> -------------------------------------------------------------------------------------
>
> Key: TAPESTRY-2473
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2473
> Project: Tapestry
> Issue Type: Bug
> Components: Framework
> Affects Versions: 5.0.11, 5.0.12, 5.0.13
> Environment: Red Hat, Tomcat, java version "1.5.0_10"
> Reporter: Fernando
> Assignee: Howard M. Lewis Ship
> Priority: Critical
> Attachments: js.txt, js2.txt
>
>
> We are running in production, and pretty regularly the system seems to hang up. After I do a jstack it looks like all the threads are tied up against some lock.
> Can you help me figure out the root cause?
> I will be attaching the full jstack the the stuck threads all look like:
> Thread 25014: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise)
> - java.util.concurrent.locks.LockSupport.park() @bci=5, line=118 (Compiled frame)
> - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(int) @bci=10, line=1124 (Compiled frame)
> - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=27, line=70 (Compiled frame)
> - $RequestHandler_11aa7f96e6c.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response) @bci=10 (Compiled frame)
> - org.apache.tapestry.services.TapestryModule$12.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=40, line=924 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.internal.services.IgnoredPathsFilter.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci
> =95, line=62 (Compiled frame)
> - $HttpServletRequestFilter_11aa7f96e63.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci=7 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.TapestryFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=12, line=168 (Compiled frame)
> ......
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Assigned: (TAPESTRY-2473) Thread Lock Ups, unkonwn cause
Posted by "Howard M. Lewis Ship (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Howard M. Lewis Ship reassigned TAPESTRY-2473:
----------------------------------------------
Assignee: Howard M. Lewis Ship
> Thread Lock Ups, unkonwn cause
> ------------------------------
>
> Key: TAPESTRY-2473
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2473
> Project: Tapestry
> Issue Type: Bug
> Components: Framework
> Affects Versions: 5.0.11, 5.0.12, 5.0.13
> Environment: Red Hat, Tomcat, java version "1.5.0_10"
> Reporter: Fernando
> Assignee: Howard M. Lewis Ship
> Priority: Critical
> Attachments: js.txt, js2.txt
>
>
> We are running in production, and pretty regularly the system seems to hang up. After I do a jstack it looks like all the threads are tied up against some lock.
> Can you help me figure out the root cause?
> I will be attaching the full jstack the the stuck threads all look like:
> Thread 25014: (state = BLOCKED)
> - sun.misc.Unsafe.park(boolean, long) @bci=0 (Compiled frame; information may be imprecise)
> - java.util.concurrent.locks.LockSupport.park() @bci=5, line=118 (Compiled frame)
> - java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(int) @bci=10, line=1124 (Compiled frame)
> - org.apache.tapestry.ioc.internal.util.ConcurrentBarrier.withRead(org.apache.tapestry.ioc.internal.util.Invokable) @bci=27, line=70 (Compiled frame)
> - $RequestHandler_11aa7f96e6c.service(org.apache.tapestry.services.Request, org.apache.tapestry.services.Response) @bci=10 (Compiled frame)
> - org.apache.tapestry.services.TapestryModule$12.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=40, line=924 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.internal.services.IgnoredPathsFilter.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci
> =95, line=62 (Compiled frame)
> - $HttpServletRequestFilter_11aa7f96e63.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse, org.apache.tapestry.services.HttpServletRequestHandler) @bci=7 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - $HttpServletRequestHandler_11aa7f96e65.service(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse) @bci=10 (Compiled frame)
> - org.apache.tapestry.TapestryFilter.doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse, javax.servlet.FilterChain) @bci=12, line=168 (Compiled frame)
> ......
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org