You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ofbiz.apache.org by "Jacques Le Roux (JIRA)" <ji...@apache.org> on 2010/12/05 15:36:12 UTC

[jira] Created: (OFBIZ-4043) Memory leaks in (at least) stable demo

Memory leaks in (at least) stable demo
--------------------------------------

                 Key: OFBIZ-4043
                 URL: https://issues.apache.org/jira/browse/OFBIZ-4043
             Project: OFBiz
          Issue Type: Bug
    Affects Versions: Release Branch 09.04
         Environment: On OFBiz demo stable
            Reporter: Jacques Le Roux


Using [Mat|http://www.eclipse.org/mat/]

I generated these report

110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"

Keywords
java.net.URLClassLoader @ 0x7f3681c111f0
org.apache.derby.impl.jdbc.EmbedConnection40
java.util.WeakHashMap$Entry[]


It's not quite clear to me how this happends, but I'm quirte sure it happens often in stable demo and maybe in trunk also. I have also attached a screen-copy with more information

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (OFBIZ-4043) Memory leaks in (at least) stable demo

Posted by "Jacques Le Roux (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/OFBIZ-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12972023#action_12972023 ] 

Jacques Le Roux commented on OFBIZ-4043:
----------------------------------------

Thanks to Gavin and the infra team, we have now 4GB (instead of 2.5GB). This let us approximatly 3.6GB available for our 2 instances. I think if we fix the current issues we should even be able to run 3 instances.

I have set this config for now, we will see...

stable (branch9): -Xms128M -Xmx1024M -XX:MaxPermSize=128m 
trunk: -Xms128M -Xmx1536M -XX:MaxPermSize=256m




> Memory leaks in (at least) stable demo
> --------------------------------------
>
>                 Key: OFBIZ-4043
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-4043
>             Project: OFBiz
>          Issue Type: Bug
>    Affects Versions: Release Branch 09.04
>         Environment: On OFBiz demo stable
>            Reporter: Jacques Le Roux
>         Attachments: Leaks Suspect .jpg, screenshot-2 (Timer thread).jpg
>
>
> Using [Mat|http://www.eclipse.org/mat/]
> I generated these report
> 110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"
> Keywords
> java.net.URLClassLoader @ 0x7f3681c111f0
> org.apache.derby.impl.jdbc.EmbedConnection40
> java.util.WeakHashMap$Entry[]
> It's not quite clear to me how this happends, but I'm quite sure it happens in stable demo and maybe also in trunk. I have also attached a screen-copy with more information

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (OFBIZ-4043) Memory leaks in (at least) stable demo

Posted by "Jacopo Cappellato (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/OFBIZ-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12972004#action_12972004 ] 

Jacopo Cappellato commented on OFBIZ-4043:
------------------------------------------

what if we do the following:

1) officially vote and release 10.04RC1
2) shutdown the 09.04 demo and replace it with the 10.04(RC1) demo

?


> Memory leaks in (at least) stable demo
> --------------------------------------
>
>                 Key: OFBIZ-4043
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-4043
>             Project: OFBiz
>          Issue Type: Bug
>    Affects Versions: Release Branch 09.04
>         Environment: On OFBiz demo stable
>            Reporter: Jacques Le Roux
>         Attachments: Leaks Suspect .jpg, screenshot-2 (Timer thread).jpg
>
>
> Using [Mat|http://www.eclipse.org/mat/]
> I generated these report
> 110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"
> Keywords
> java.net.URLClassLoader @ 0x7f3681c111f0
> org.apache.derby.impl.jdbc.EmbedConnection40
> java.util.WeakHashMap$Entry[]
> It's not quite clear to me how this happends, but I'm quite sure it happens in stable demo and maybe also in trunk. I have also attached a screen-copy with more information

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (OFBIZ-4043) Memory leaks in (at least) stable demo

Posted by "Jacques Le Roux (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/OFBIZ-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jacques Le Roux updated OFBIZ-4043:
-----------------------------------

    Description: 
Using [Mat|http://www.eclipse.org/mat/]

I generated these report

110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"

Keywords
java.net.URLClassLoader @ 0x7f3681c111f0
org.apache.derby.impl.jdbc.EmbedConnection40
java.util.WeakHashMap$Entry[]


It's not quite clear to me how this happends, but I'm quite sure it happens often in stable demo and maybe also in trunk. I have also attached a screen-copy with more information

  was:
Using [Mat|http://www.eclipse.org/mat/]

I generated these report

110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"

Keywords
java.net.URLClassLoader @ 0x7f3681c111f0
org.apache.derby.impl.jdbc.EmbedConnection40
java.util.WeakHashMap$Entry[]


It's not quite clear to me how this happends, but I'm quirte sure it happens often in stable demo and maybe in trunk also. I have also attached a screen-copy with more information


> Memory leaks in (at least) stable demo
> --------------------------------------
>
>                 Key: OFBIZ-4043
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-4043
>             Project: OFBiz
>          Issue Type: Bug
>    Affects Versions: Release Branch 09.04
>         Environment: On OFBiz demo stable
>            Reporter: Jacques Le Roux
>         Attachments: Leaks Suspect .jpg
>
>
> Using [Mat|http://www.eclipse.org/mat/]
> I generated these report
> 110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"
> Keywords
> java.net.URLClassLoader @ 0x7f3681c111f0
> org.apache.derby.impl.jdbc.EmbedConnection40
> java.util.WeakHashMap$Entry[]
> It's not quite clear to me how this happends, but I'm quite sure it happens often in stable demo and maybe also in trunk. I have also attached a screen-copy with more information

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (OFBIZ-4043) Memory leaks in (at least) stable demo

Posted by "BJ Freeman (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/OFBIZ-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12971849#action_12971849 ] 

BJ Freeman commented on OFBIZ-4043:
-----------------------------------

since the local host work on my machine and the trunk has run for a month without failing on my server, we need to think  of the OS or activity as the triggers for this.


> Memory leaks in (at least) stable demo
> --------------------------------------
>
>                 Key: OFBIZ-4043
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-4043
>             Project: OFBiz
>          Issue Type: Bug
>    Affects Versions: Release Branch 09.04
>         Environment: On OFBiz demo stable
>            Reporter: Jacques Le Roux
>         Attachments: Leaks Suspect .jpg, screenshot-2 (Timer thread).jpg
>
>
> Using [Mat|http://www.eclipse.org/mat/]
> I generated these report
> 110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"
> Keywords
> java.net.URLClassLoader @ 0x7f3681c111f0
> org.apache.derby.impl.jdbc.EmbedConnection40
> java.util.WeakHashMap$Entry[]
> It's not quite clear to me how this happends, but I'm quite sure it happens in stable demo and maybe also in trunk. I have also attached a screen-copy with more information

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (OFBIZ-4043) Memory leaks in (at least) stable demo

Posted by "Jacques Le Roux (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/OFBIZ-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12971670#action_12971670 ] 

Jacques Le Roux commented on OFBIZ-4043:
----------------------------------------

Another one today:

108 639 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7fc419071150" occupy 408 594 832 (78,54%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"

Keywords
org.apache.derby.impl.jdbc.EmbedConnection40
java.net.URLClassLoader @ 0x7fc419071150
java.util.WeakHashMap$Entry[]


See also 2d screenshoot, this time it comes from a Timer, so I guess a job polling. It seems we don't cross such issues in trunk. The problem there seems mostly related to blocked threads. Not sure why yet, but at least here is a stack taken from a blocked state:
{code}
$ ps aux | grep ofbiz
ofbiz    29724 83.6 42.4 1468468 1092996 ?     Sl   Dec14 1111:26 java -Xms128M -Xmx768M -XX:MaxPermSize=192m -XX:+HeapDumpOnOutOfMemoryError -Dofbiz.admin.port=10523 -Dofbiz.admin.key=so3du5kasd5dn -jar ofbiz.jar

Trunk
$ jstack 29724
2010-12-15 07:25:15
Full thread dump Java HotSpot(TM) 64-Bit Server VM (17.1-b03 mixed mode):

"GenericXaResource-Thread" daemon prio=10 tid=0x00007f9a54107000 nid=0x1daa runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Attach Listener" daemon prio=10 tid=0x0000000041684000 nid=0x1da8 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"default-invoker-Thread-102545" prio=10 tid=0x00007f9a546ea800 nid=0x5ce0 waiting on condition [0x00007f9a6360f000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at org.ofbiz.service.job.JobInvoker.run(JobInvoker.java:226)
        - locked <0x00007f9a84464668> (a org.ofbiz.service.job.JobInvoker)
        at java.lang.Thread.run(Thread.java:662)

"default-invoker-Thread-102544" prio=10 tid=0x0000000042099000 nid=0x5cdf waiting on condition [0x00007f9a62ff9000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at org.ofbiz.service.job.JobInvoker.run(JobInvoker.java:226)
        - locked <0x00007f9a84464768> (a org.ofbiz.service.job.JobInvoker)
        at java.lang.Thread.run(Thread.java:662)

"default-invoker-Thread-102543" prio=10 tid=0x0000000042098800 nid=0x5cde waiting on condition [0x00007f9a58e59000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at org.ofbiz.service.job.JobInvoker.run(JobInvoker.java:226)
        - locked <0x00007f9a84464868> (a org.ofbiz.service.job.JobInvoker)
        at java.lang.Thread.run(Thread.java:662)

"default-invoker-Thread-102542" prio=10 tid=0x0000000041dec000 nid=0x5cdd waiting on condition [0x00007f9a58d58000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at org.ofbiz.service.job.JobInvoker.run(JobInvoker.java:226)
        - locked <0x00007f9a84464968> (a org.ofbiz.service.job.JobInvoker)
        at java.lang.Thread.run(Thread.java:662)

"default-invoker-Thread-102541" prio=10 tid=0x0000000042424000 nid=0x5cdc waiting on condition [0x00007f9a62ef8000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at org.ofbiz.service.job.JobInvoker.run(JobInvoker.java:226)
        - locked <0x00007f9a84464338> (a org.ofbiz.service.job.JobInvoker)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8080-9" daemon prio=10 tid=0x000000004363a000 nid=0x3e88 in Object.wait() [0x00007f9a630fa000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a84225258> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a84225258> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"TP-Processor35" daemon prio=10 tid=0x0000000042584800 nid=0x6ccb runnable [0x00007f9a5864f000]
   java.lang.Thread.State: RUNNABLE
        at java.lang.StringBuffer.append(StringBuffer.java:224)
        - locked <0x00007f9a828e7ed8> (a java.lang.StringBuffer)
        at java.io.StringWriter.write(StringWriter.java:84)
        at java.io.StringWriter.append(StringWriter.java:126)
        at java.io.StringWriter.append(StringWriter.java:24)
        at org.ofbiz.base.util.UtilIO.copy(UtilIO.java:111)
        at org.ofbiz.content.data.DataResourceWorker.renderFile(DataResourceWorker.java:899)
        at org.ofbiz.content.data.DataResourceWorker.writeDataResourceText(DataResourceWorker.java:820)
        at org.ofbiz.content.data.DataResourceWorker.renderDataResourceAsText(DataResourceWorker.java:626)
        at org.ofbiz.content.content.ContentWorker.renderContentAsText(ContentWorker.java:273)
        at org.ofbiz.content.content.ContentWorker.renderContentAsText(ContentWorker.java:330)
        at org.ofbiz.content.cms.CmsEvents.cms(CmsEvents.java:289)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:92)
        at org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:78)
        at org.ofbiz.webapp.control.RequestHandler.runEvent(RequestHandler.java:635)
        at org.ofbiz.webapp.control.RequestHandler.doRequest(RequestHandler.java:381)
        at org.ofbiz.webapp.control.ControlServlet.doGet(ControlServlet.java:230)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:338)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
        at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
        at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
        at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:774)
        at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
        at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:896)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8080-8" daemon prio=10 tid=0x0000000042343000 nid=0x6b1a in Object.wait() [0x00007f9a59560000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a82646da8> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a82646da8> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8080-7" daemon prio=10 tid=0x0000000042ab5800 nid=0x6b19 in Object.wait() [0x00007f9a58550000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a82648040> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a82648040> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8443-14" daemon prio=10 tid=0x00000000417cd000 nid=0x7ab0 in Object.wait() [0x00007f9a59762000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a81b95590> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a81b95590> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8443-13" daemon prio=10 tid=0x00000000435c3800 nid=0x7aaf in Object.wait() [0x00007f9a59661000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a81b999d8> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a81b999d8> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8443-12" daemon prio=10 tid=0x00000000417d3000 nid=0x7aae in Object.wait() [0x00007f9a5935e000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a81b9de20> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a81b9de20> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8443-11" daemon prio=10 tid=0x00000000428fc000 nid=0x7aad in Object.wait() [0x00007f9a59863000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a81b9df00> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a81b9df00> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8443-10" daemon prio=10 tid=0x00000000414be000 nid=0x7aac in Object.wait() [0x00007f9a5a3e2000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a81b9dfe0> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a81b9dfe0> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8443-9" daemon prio=10 tid=0x0000000041a18000 nid=0x7a3a in Object.wait() [0x00007f9a5a6e5000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a81b6e760> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a81b6e760> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8443-8" daemon prio=10 tid=0x0000000042900000 nid=0x79a2 in Object.wait() [0x00007f9a59a65000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a81b3c418> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a81b3c418> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"TP-Processor31" daemon prio=10 tid=0x0000000041f30800 nid=0x49e5 runnable [0x00007f9a59b64000]
   java.lang.Thread.State: RUNNABLE
        at java.lang.StringBuffer.append(StringBuffer.java:224)
        - locked <0x00007f9a843a95e0> (a java.lang.StringBuffer)
        at java.io.StringWriter.write(StringWriter.java:84)
        at java.io.StringWriter.append(StringWriter.java:126)
        at java.io.StringWriter.append(StringWriter.java:24)
        at org.ofbiz.base.util.UtilIO.copy(UtilIO.java:111)
        at org.ofbiz.content.data.DataResourceWorker.renderFile(DataResourceWorker.java:899)
        at org.ofbiz.content.data.DataResourceWorker.writeDataResourceText(DataResourceWorker.java:820)
        at org.ofbiz.content.data.DataResourceWorker.renderDataResourceAsText(DataResourceWorker.java:626)
        at org.ofbiz.content.content.ContentWorker.renderContentAsText(ContentWorker.java:273)
        at org.ofbiz.content.content.ContentWorker.renderContentAsText(ContentWorker.java:330)
        at org.ofbiz.content.cms.CmsEvents.cms(CmsEvents.java:289)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:92)
        at org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:78)
        at org.ofbiz.webapp.control.RequestHandler.runEvent(RequestHandler.java:635)
        at org.ofbiz.webapp.control.RequestHandler.doRequest(RequestHandler.java:381)
        at org.ofbiz.webapp.control.ControlServlet.doGet(ControlServlet.java:230)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:338)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
        at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
        at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
        at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:774)
        at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
        at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:896)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8080-6" daemon prio=10 tid=0x0000000041624000 nid=0x4651 in Object.wait() [0x00007f9a5b4f3000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a81395698> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a81395698> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8080-5" daemon prio=10 tid=0x00000000415d1000 nid=0x4650 in Object.wait() [0x00007f9a5a4e3000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a81396930> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a81396930> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8080-4" daemon prio=10 tid=0x0000000041f34800 nid=0x464f in Object.wait() [0x00007f9a5adec000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a81399108> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a81399108> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8080-3" daemon prio=10 tid=0x0000000041c5f000 nid=0x464e in Object.wait() [0x00007f9a5a5e4000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a8139a3a0> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a8139a3a0> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8443-7" daemon prio=10 tid=0x0000000042077000 nid=0x32a6 in Object.wait() [0x00007f9a5a8e7000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a8112a680> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a8112a680> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8080-2" daemon prio=10 tid=0x0000000042fe0000 nid=0xe5d in Object.wait() [0x00007f9a5a9e8000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a800ff450> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a800ff450> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"TP-Processor28" daemon prio=10 tid=0x0000000043664800 nid=0x22b runnable [0x00007f9a5b5f2000]
   java.lang.Thread.State: RUNNABLE
        at java.io.Reader.read(Reader.java:83)
        at org.ofbiz.base.util.UtilIO.copy(UtilIO.java:109)
        at org.ofbiz.content.data.DataResourceWorker.renderFile(DataResourceWorker.java:899)
        at org.ofbiz.content.data.DataResourceWorker.writeDataResourceText(DataResourceWorker.java:820)
        at org.ofbiz.content.data.DataResourceWorker.renderDataResourceAsText(DataResourceWorker.java:626)
        at org.ofbiz.content.content.ContentWorker.renderContentAsText(ContentWorker.java:273)
        at org.ofbiz.content.content.ContentWorker.renderContentAsText(ContentWorker.java:330)
        at org.ofbiz.content.cms.CmsEvents.cms(CmsEvents.java:289)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:92)
        at org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:78)
        at org.ofbiz.webapp.control.RequestHandler.runEvent(RequestHandler.java:635)
        at org.ofbiz.webapp.control.RequestHandler.doRequest(RequestHandler.java:381)
        at org.ofbiz.webapp.control.ControlServlet.doGet(ControlServlet.java:230)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:338)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
        at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
        at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
        at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:774)
        at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
        at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:896)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
        at java.lang.Thread.run(Thread.java:662)

"TP-Processor23" daemon prio=10 tid=0x000000004222e800 nid=0x157 runnable [0x00007f9a5ace9000]
   java.lang.Thread.State: RUNNABLE
        at java.lang.StringBuffer.append(StringBuffer.java:224)
        - locked <0x00007f9a80f38480> (a java.lang.StringBuffer)
        at java.io.StringWriter.write(StringWriter.java:84)
        at java.io.StringWriter.append(StringWriter.java:126)
        at java.io.StringWriter.append(StringWriter.java:24)
        at org.ofbiz.base.util.UtilIO.copy(UtilIO.java:111)
        at org.ofbiz.content.data.DataResourceWorker.renderFile(DataResourceWorker.java:899)
        at org.ofbiz.content.data.DataResourceWorker.writeDataResourceText(DataResourceWorker.java:820)
        at org.ofbiz.content.data.DataResourceWorker.renderDataResourceAsText(DataResourceWorker.java:626)
        at org.ofbiz.content.content.ContentWorker.renderContentAsText(ContentWorker.java:273)
        at org.ofbiz.content.content.ContentWorker.renderContentAsText(ContentWorker.java:330)
        at org.ofbiz.content.cms.CmsEvents.cms(CmsEvents.java:289)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:92)
        at org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:78)
        at org.ofbiz.webapp.control.RequestHandler.runEvent(RequestHandler.java:635)
        at org.ofbiz.webapp.control.RequestHandler.doRequest(RequestHandler.java:381)
        at org.ofbiz.webapp.control.ControlServlet.doGet(ControlServlet.java:230)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:338)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
        at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
        at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
        at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:774)
        at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
        at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:896)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
        at java.lang.Thread.run(Thread.java:662)

"TP-Processor22" daemon prio=10 tid=0x000000004222d800 nid=0x156 runnable [0x00007f9a5bcf9000]
   java.lang.Thread.State: RUNNABLE
        at java.lang.StringBuffer.append(StringBuffer.java:224)
        - locked <0x00007f9a7ffad3f8> (a java.lang.StringBuffer)
        at java.io.StringWriter.write(StringWriter.java:84)
        at java.io.StringWriter.append(StringWriter.java:126)
        at java.io.StringWriter.append(StringWriter.java:24)
        at org.ofbiz.base.util.UtilIO.copy(UtilIO.java:111)
        at org.ofbiz.content.data.DataResourceWorker.renderFile(DataResourceWorker.java:899)
        at org.ofbiz.content.data.DataResourceWorker.writeDataResourceText(DataResourceWorker.java:820)
        at org.ofbiz.content.data.DataResourceWorker.renderDataResourceAsText(DataResourceWorker.java:626)
        at org.ofbiz.content.content.ContentWorker.renderContentAsText(ContentWorker.java:273)
        at org.ofbiz.content.content.ContentWorker.renderContentAsText(ContentWorker.java:330)
        at org.ofbiz.content.cms.CmsEvents.cms(CmsEvents.java:289)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:92)
        at org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:78)
        at org.ofbiz.webapp.control.RequestHandler.runEvent(RequestHandler.java:635)
        at org.ofbiz.webapp.control.RequestHandler.doRequest(RequestHandler.java:381)
        at org.ofbiz.webapp.control.ControlServlet.doGet(ControlServlet.java:230)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:338)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
        at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
        at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
        at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:774)
        at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
        at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:896)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8443-6" daemon prio=10 tid=0x0000000042dbc000 nid=0x7970 in Object.wait() [0x00007f9a5b7f6000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7f2bc138> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a7f2bc138> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8443-4" daemon prio=10 tid=0x00000000418e2000 nid=0x796e in Object.wait() [0x00007f9a5b9f8000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7f2bd4b0> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a7f2bd4b0> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8443-3" daemon prio=10 tid=0x0000000042db9000 nid=0x796d in Object.wait() [0x00007f9a5bdfc000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7f2be748> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a7f2be748> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8443-2" daemon prio=10 tid=0x00000000418df800 nid=0x796c in Object.wait() [0x00007f9a5b6f5000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7f2bf9e0> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a7f2bf9e0> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"TP-Processor16" daemon prio=10 tid=0x000000004149f800 nid=0x7932 runnable [0x00007f9a60129000]
   java.lang.Thread.State: RUNNABLE
        at java.lang.StringBuffer.append(StringBuffer.java:224)
        - locked <0x00007f9a8071f7d8> (a java.lang.StringBuffer)
        at java.io.StringWriter.write(StringWriter.java:84)
        at java.io.StringWriter.append(StringWriter.java:126)
        at java.io.StringWriter.append(StringWriter.java:24)
        at org.ofbiz.base.util.UtilIO.copy(UtilIO.java:111)
        at org.ofbiz.content.data.DataResourceWorker.renderFile(DataResourceWorker.java:899)
        at org.ofbiz.content.data.DataResourceWorker.writeDataResourceText(DataResourceWorker.java:820)
        at org.ofbiz.content.data.DataResourceWorker.renderDataResourceAsText(DataResourceWorker.java:626)
        at org.ofbiz.content.content.ContentWorker.renderContentAsText(ContentWorker.java:273)
        at org.ofbiz.content.content.ContentWorker.renderContentAsText(ContentWorker.java:330)
        at org.ofbiz.content.cms.CmsEvents.cms(CmsEvents.java:289)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:92)
        at org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:78)
        at org.ofbiz.webapp.control.RequestHandler.runEvent(RequestHandler.java:635)
        at org.ofbiz.webapp.control.RequestHandler.doRequest(RequestHandler.java:381)
        at org.ofbiz.webapp.control.ControlServlet.doGet(ControlServlet.java:230)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:338)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
        at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
        at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
        at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:774)
        at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
        at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:896)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
        at java.lang.Thread.run(Thread.java:662)

"TP-Processor13" daemon prio=10 tid=0x0000000041616000 nid=0x792f runnable [0x00007f9a5bbf8000]
   java.lang.Thread.State: RUNNABLE
        at java.lang.StringBuffer.append(StringBuffer.java:224)
        - locked <0x00007f9a8083eef8> (a java.lang.StringBuffer)
        at java.io.StringWriter.write(StringWriter.java:84)
        at java.io.StringWriter.append(StringWriter.java:126)
        at java.io.StringWriter.append(StringWriter.java:24)
        at org.ofbiz.base.util.UtilIO.copy(UtilIO.java:111)
        at org.ofbiz.content.data.DataResourceWorker.renderFile(DataResourceWorker.java:899)
        at org.ofbiz.content.data.DataResourceWorker.writeDataResourceText(DataResourceWorker.java:820)
        at org.ofbiz.content.data.DataResourceWorker.renderDataResourceAsText(DataResourceWorker.java:626)
        at org.ofbiz.content.content.ContentWorker.renderContentAsText(ContentWorker.java:273)
        at org.ofbiz.content.content.ContentWorker.renderContentAsText(ContentWorker.java:330)
        at org.ofbiz.content.cms.CmsEvents.cms(CmsEvents.java:289)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:92)
        at org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:78)
        at org.ofbiz.webapp.control.RequestHandler.runEvent(RequestHandler.java:635)
        at org.ofbiz.webapp.control.RequestHandler.doRequest(RequestHandler.java:381)
        at org.ofbiz.webapp.control.ControlServlet.doGet(ControlServlet.java:230)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:338)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
        at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
        at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
        at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:774)
        at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
        at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:896)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
        at java.lang.Thread.run(Thread.java:662)

"TP-Processor10" daemon prio=10 tid=0x0000000041dd7000 nid=0x77ff runnable [0x00007f9a60bd3000]
   java.lang.Thread.State: RUNNABLE
        at java.lang.StringBuffer.append(StringBuffer.java:224)
        - locked <0x00007f9a82819a20> (a java.lang.StringBuffer)
        at java.io.StringWriter.write(StringWriter.java:84)
        at java.io.StringWriter.append(StringWriter.java:126)
        at java.io.StringWriter.append(StringWriter.java:24)
        at org.ofbiz.base.util.UtilIO.copy(UtilIO.java:111)
        at org.ofbiz.content.data.DataResourceWorker.renderFile(DataResourceWorker.java:899)
        at org.ofbiz.content.data.DataResourceWorker.writeDataResourceText(DataResourceWorker.java:820)
        at org.ofbiz.content.data.DataResourceWorker.renderDataResourceAsText(DataResourceWorker.java:626)
        at org.ofbiz.content.content.ContentWorker.renderContentAsText(ContentWorker.java:273)
        at org.ofbiz.content.content.ContentWorker.renderContentAsText(ContentWorker.java:330)
        at org.ofbiz.content.cms.CmsEvents.cms(CmsEvents.java:289)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:92)
        at org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:78)
        at org.ofbiz.webapp.control.RequestHandler.runEvent(RequestHandler.java:635)
        at org.ofbiz.webapp.control.RequestHandler.doRequest(RequestHandler.java:381)
        at org.ofbiz.webapp.control.ControlServlet.doGet(ControlServlet.java:230)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:338)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
        at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
        at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
        at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:774)
        at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
        at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:896)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
        at java.lang.Thread.run(Thread.java:662)

"Java2D Disposer" daemon prio=10 tid=0x00007f9a6608b800 nid=0x7765 in Object.wait() [0x00007f9a629f3000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7ec9dfa0> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
        - locked <0x00007f9a7ec9dfa0> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
        at sun.java2d.Disposer.run(Disposer.java:127)
        at java.lang.Thread.run(Thread.java:662)

"http-0.0.0.0-8443-1" daemon prio=10 tid=0x000000004197b800 nid=0x76e5 in Object.wait() [0x00007f9a60dd7000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7ec3a260> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:458)
        - locked <0x00007f9a7ec3a260> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:484)
        at java.lang.Thread.run(Thread.java:662)

"TP-Processor7" daemon prio=10 tid=0x0000000042c42000 nid=0x76cf runnable [0x00007f9a625ed000]
   java.lang.Thread.State: RUNNABLE
        at java.lang.StringBuffer.append(StringBuffer.java:224)
        - locked <0x00007f9a82a058f8> (a java.lang.StringBuffer)
        at java.io.StringWriter.write(StringWriter.java:84)
        at java.io.StringWriter.append(StringWriter.java:126)
        at java.io.StringWriter.append(StringWriter.java:24)
        at org.ofbiz.base.util.UtilIO.copy(UtilIO.java:111)
        at org.ofbiz.content.data.DataResourceWorker.renderFile(DataResourceWorker.java:899)
        at org.ofbiz.content.data.DataResourceWorker.writeDataResourceText(DataResourceWorker.java:820)
        at org.ofbiz.content.data.DataResourceWorker.renderDataResourceAsText(DataResourceWorker.java:626)
        at org.ofbiz.content.content.ContentWorker.renderContentAsText(ContentWorker.java:273)
        at org.ofbiz.content.content.ContentWorker.renderContentAsText(ContentWorker.java:330)
        at org.ofbiz.content.cms.CmsEvents.cms(CmsEvents.java:289)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:92)
        at org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:78)
        at org.ofbiz.webapp.control.RequestHandler.runEvent(RequestHandler.java:635)
        at org.ofbiz.webapp.control.RequestHandler.doRequest(RequestHandler.java:381)
        at org.ofbiz.webapp.control.ControlServlet.doGet(ControlServlet.java:230)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:338)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
        at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
        at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
        at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:774)
        at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
        at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:896)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
        at java.lang.Thread.run(Thread.java:662)

"DestroyJavaVM" prio=10 tid=0x00007f9a661e8000 nid=0x741d waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Thread-205" prio=10 tid=0x00007f9a66543800 nid=0x7546 runnable [0x00007f9a60fd9000]
   java.lang.Thread.State: RUNNABLE
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
        - locked <0x00007f9a7e2f3600> (a java.net.SocksSocketImpl)
        at java.net.ServerSocket.implAccept(ServerSocket.java:453)
        at java.net.ServerSocket.accept(ServerSocket.java:421)
        at bsh.util.Sessiond.run(Unknown Source)
        at java.lang.Thread.run(Thread.java:662)

"Thread-203" prio=10 tid=0x00007f9a678d4800 nid=0x7545 runnable [0x00007f9a610da000]
   java.lang.Thread.State: RUNNABLE
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
        - locked <0x00007f9a7e2f3ab0> (a java.net.SocksSocketImpl)
        at java.net.ServerSocket.implAccept(ServerSocket.java:453)
        at java.net.ServerSocket.accept(ServerSocket.java:421)
        at bsh.util.Httpd.run(Unknown Source)
        at java.lang.Thread.run(Thread.java:662)

"Worker-JM" prio=10 tid=0x00000000424d5800 nid=0x7534 in Object.wait() [0x00007f9a611db000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7e1bbc28> (a java.util.ArrayList)
        at org.eclipse.core.internal.jobs.InternalWorker.run(InternalWorker.java:58)
        - locked <0x00007f9a7e1bbc28> (a java.util.ArrayList)

"Bundle File Closer" daemon prio=10 tid=0x0000000041fc2000 nid=0x7532 in Object.wait() [0x00007f9a612dc000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7e182818> (a org.eclipse.osgi.framework.eventmgr.EventManager$EventThread)
        at java.lang.Object.wait(Object.java:485)
        at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.getNextEvent(EventManager.java:397)
        - locked <0x00007f9a7e182818> (a org.eclipse.osgi.framework.eventmgr.EventManager$EventThread)
        at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:333)

"Start Level Event Dispatcher" daemon prio=10 tid=0x00007f9a64620000 nid=0x752e in Object.wait() [0x00007f9a613dd000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7e1c5830> (a org.eclipse.osgi.framework.eventmgr.EventManager$EventThread)
        at java.lang.Object.wait(Object.java:485)
        at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.getNextEvent(EventManager.java:397)
        - locked <0x00007f9a7e1c5830> (a org.eclipse.osgi.framework.eventmgr.EventManager$EventThread)
        at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:333)

"Framework Event Dispatcher" daemon prio=10 tid=0x00007f9a5c617800 nid=0x752d in Object.wait() [0x00007f9a614de000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7e1c5b28> (a org.eclipse.osgi.framework.eventmgr.EventManager$EventThread)
        at java.lang.Object.wait(Object.java:485)
        at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.getNextEvent(EventManager.java:397)
        - locked <0x00007f9a7e1c5b28> (a org.eclipse.osgi.framework.eventmgr.EventManager$EventThread)
        at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:333)

"TP-Processor2" daemon prio=10 tid=0x00007f9a66744000 nid=0x7515 runnable [0x00007f9a6350c000]
   java.lang.Thread.State: RUNNABLE
        at java.io.Reader.read(Reader.java:83)
        at org.ofbiz.base.util.UtilIO.copy(UtilIO.java:109)
        at org.ofbiz.content.data.DataResourceWorker.renderFile(DataResourceWorker.java:899)
        at org.ofbiz.content.data.DataResourceWorker.writeDataResourceText(DataResourceWorker.java:820)
        at org.ofbiz.content.data.DataResourceWorker.renderDataResourceAsText(DataResourceWorker.java:626)
        at org.ofbiz.content.content.ContentWorker.renderContentAsText(ContentWorker.java:273)
        at org.ofbiz.content.content.ContentWorker.renderContentAsText(ContentWorker.java:330)
        at org.ofbiz.content.cms.CmsEvents.cms(CmsEvents.java:289)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:92)
        at org.ofbiz.webapp.event.JavaEventHandler.invoke(JavaEventHandler.java:78)
        at org.ofbiz.webapp.control.RequestHandler.runEvent(RequestHandler.java:635)
        at org.ofbiz.webapp.control.RequestHandler.doRequest(RequestHandler.java:381)
        at org.ofbiz.webapp.control.ControlServlet.doGet(ControlServlet.java:230)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:338)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
        at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:190)
        at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
        at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:774)
        at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
        at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:896)
        at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
        at java.lang.Thread.run(Thread.java:662)

"TTLObject(async-update)-1" daemon prio=10 tid=0x00007f9a67327000 nid=0x7497 waiting on condition [0x00007f9a61de7000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00007f9a7db5d4a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.DelayQueue.take(DelayQueue.java:160)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:662)

"TTLObject(async-update)-0" daemon prio=10 tid=0x00007f9a65cf1800 nid=0x7496 waiting on condition [0x00007f9a61ce6000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00007f9a7db5d4a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.DelayQueue.take(DelayQueue.java:160)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:662)

"ThreadPool-1" daemon prio=10 tid=0x00000000418da000 nid=0x7495 waiting on condition [0x00007f9a621eb000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00007f9a7db5d7d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.DelayQueue.take(DelayQueue.java:160)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:662)

"ThreadPool-0" daemon prio=10 tid=0x00000000415dc800 nid=0x7494 waiting on condition [0x00007f9a61be5000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00007f9a7db5d7d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.DelayQueue.take(DelayQueue.java:160)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:662)

"ReferenceCleaner" daemon prio=10 tid=0x0000000041c58800 nid=0x748e in Object.wait() [0x00007f9a620ea000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7db5db00> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
        - locked <0x00007f9a7db5db00> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
        at org.webslinger.lang.ReferenceCleaner.run(ReferenceCleaner.java:93)
        at org.webslinger.lang.ReferenceCleaner.access$100(ReferenceCleaner.java:10)
        at org.webslinger.lang.ReferenceCleaner$ReferenceCleanerBootstrapper$1.run(ReferenceCleaner.java:74)
        at java.lang.Thread.run(Thread.java:662)

"TTLObject(0) pulse" daemon prio=10 tid=0x00000000423a6800 nid=0x748a waiting on condition [0x00007f9a623ed000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00007f9a7db5dc00> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.DelayQueue.take(DelayQueue.java:160)
        at org.webslinger.concurrent.TTLObject$TTLObjectWorker.run(TTLObject.java:27)
        at java.lang.Thread.run(Thread.java:662)

"ReferenceCleaner" daemon prio=10 tid=0x00000000428f1000 nid=0x7473 in Object.wait() [0x00007f9a622ec000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7da453f0> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
        - locked <0x00007f9a7da453f0> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
        at org.ofbiz.base.util.ReferenceCleaner$CleanerThread.run(ReferenceCleaner.java:45)

"ExecutionPoolPulseWorker(0)" daemon prio=10 tid=0x00007f9a646d5800 nid=0x745a waiting on condition [0x00007f9a624ee000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00007f9a7d2034b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at java.util.concurrent.DelayQueue.take(DelayQueue.java:160)
        at org.ofbiz.base.concurrent.ExecutionPool$ExecutionPoolPulseWorker.run(ExecutionPool.java:106)
        at java.lang.Thread.run(Thread.java:662)

"RMI RenewClean-[140.211.11.41:42426,org.ofbiz.service.rmi.socket.ssl.SSLClientSocketFactory@2ebf13d5]" daemon prio=10 tid=0x0000000041c75000 nid=0x7457 in Object.wait() [0x00007f9a626f0000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7d0f7788> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
        - locked <0x00007f9a7d0f7788> (a java.lang.ref.ReferenceQueue$Lock)
        at sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:516)
        at java.lang.Thread.run(Thread.java:662)

"RMI Scheduler(0)" daemon prio=10 tid=0x00007f9a653c7800 nid=0x7456 waiting on condition [0x00007f9a627f1000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00007f9a7d0f78f8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025)
        at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:662)

"GC Daemon" daemon prio=10 tid=0x00007f9a5c52a800 nid=0x7454 in Object.wait() [0x00007f9a62af4000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7d0f7b38> (a sun.misc.GC$LatencyLock)
        at sun.misc.GC$Daemon.run(GC.java:100)
        - locked <0x00007f9a7d0f7b38> (a sun.misc.GC$LatencyLock)

"RMI Reaper" prio=10 tid=0x00007f9a66dde800 nid=0x7453 in Object.wait() [0x00007f9a62bf5000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7d0f7bf8> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
        - locked <0x00007f9a7d0f7bf8> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
        at sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:333)
        at java.lang.Thread.run(Thread.java:662)

"RMI TCP Accept-0" daemon prio=10 tid=0x00007f9a65717000 nid=0x7452 runnable [0x00007f9a62cf6000]
   java.lang.Thread.State: RUNNABLE
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
        - locked <0x00007f9a7d0f7d08> (a java.net.SocksSocketImpl)
        at java.net.ServerSocket.implAccept(ServerSocket.java:453)
        at com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:261)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
        at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:341)
        at java.lang.Thread.run(Thread.java:662)

"org.ofbiz.service.job.JobPoller@6c1c19c6" prio=10 tid=0x0000000042be7000 nid=0x744e runnable [0x00007f9a62df7000]
   java.lang.Thread.State: RUNNABLE
        at org.apache.log4j.AsyncAppender.append(AsyncAppender.java:223)
        at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
        - locked <0x00007f9a797edb80> (a org.apache.log4j.AsyncAppender)
        at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
        at org.apache.log4j.Category.callAppenders(Category.java:206)
        - locked <0x00007f9a797406e0> (a org.apache.log4j.spi.RootLogger)
        at org.apache.log4j.Category.forcedLog(Category.java:391)
        at org.apache.log4j.Category.log(Category.java:856)
        at org.ofbiz.base.util.Debug.log(Debug.java:166)
        at org.ofbiz.base.util.Debug.log(Debug.java:149)
        at org.ofbiz.base.util.Debug.logError(Debug.java:314)
        at org.ofbiz.entity.GenericDelegator.storeByCondition(GenericDelegator.java:1293)
        at org.ofbiz.entity.GenericDelegator.storeByCondition(GenericDelegator.java:1257)
        at org.ofbiz.service.job.JobManager.poll(JobManager.java:157)
        - locked <0x00007f9a7b864520> (a org.ofbiz.service.job.JobManager)
        - locked <0x00007f9a7b864520> (a org.ofbiz.service.job.JobManager)
        at org.ofbiz.service.job.JobPoller.run(JobPoller.java:90)
        - locked <0x00007f9a7b85c368> (a org.ofbiz.service.job.JobPoller)
        at java.lang.Thread.run(Thread.java:662)

"Thread-5" daemon prio=10 tid=0x0000000041553000 nid=0x7445 waiting on condition [0x00007f9a63710000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at org.apache.geronimo.transaction.manager.TransactionTimer$CurrentTime.run(TransactionTimer.java:38)

"derby.rawStoreDaemon" daemon prio=10 tid=0x00000000419e7800 nid=0x7443 in Object.wait() [0x00007f9a639f2000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7b575950> (a org.apache.derby.impl.services.daemon.BasicDaemon)
        at org.apache.derby.impl.services.daemon.BasicDaemon.rest(Unknown Source)
        - locked <0x00007f9a7b575950> (a org.apache.derby.impl.services.daemon.BasicDaemon)
        at org.apache.derby.impl.services.daemon.BasicDaemon.run(Unknown Source)
        at java.lang.Thread.run(Thread.java:662)

"derby.rawStoreDaemon" daemon prio=10 tid=0x0000000041a87000 nid=0x7441 in Object.wait() [0x00007f9a63af3000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7b575a58> (a org.apache.derby.impl.services.daemon.BasicDaemon)
        at org.apache.derby.impl.services.daemon.BasicDaemon.rest(Unknown Source)
        - locked <0x00007f9a7b575a58> (a org.apache.derby.impl.services.daemon.BasicDaemon)
        at org.apache.derby.impl.services.daemon.BasicDaemon.run(Unknown Source)
        at java.lang.Thread.run(Thread.java:662)

"derby.rawStoreDaemon" daemon prio=10 tid=0x00000000419ef800 nid=0x743b in Object.wait() [0x00007f9a63bf4000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7b33d748> (a org.apache.derby.impl.services.daemon.BasicDaemon)
        at org.apache.derby.impl.services.daemon.BasicDaemon.rest(Unknown Source)
        - locked <0x00007f9a7b33d748> (a org.apache.derby.impl.services.daemon.BasicDaemon)
        at org.apache.derby.impl.services.daemon.BasicDaemon.run(Unknown Source)
        at java.lang.Thread.run(Thread.java:662)

"Timer-1" daemon prio=10 tid=0x00000000417a2800 nid=0x743a in Object.wait() [0x00007f9a63cf5000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7b33d860> (a java.util.TaskQueue)
        at java.util.TimerThread.mainLoop(Timer.java:509)
        - locked <0x00007f9a7b33d860> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:462)

"Timer-0" daemon prio=10 tid=0x0000000041871800 nid=0x7439 in Object.wait() [0x00007f9a63efd000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7b33d988> (a java.util.TaskQueue)
        at java.lang.Object.wait(Object.java:485)
        at java.util.TimerThread.mainLoop(Timer.java:483)
        - locked <0x00007f9a7b33d988> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:462)

"derby.antiGC" daemon prio=10 tid=0x00000000416fd000 nid=0x7437 in Object.wait() [0x00007f9a63ffe000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a7b33da58> (a org.apache.derby.impl.services.monitor.AntiGC)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.derby.impl.services.monitor.AntiGC.run(Unknown Source)
        - locked <0x00007f9a7b33da58> (a org.apache.derby.impl.services.monitor.AntiGC)
        at java.lang.Thread.run(Thread.java:662)

"Dispatcher-Thread-2" daemon prio=10 tid=0x00007f9a64490800 nid=0x7428 in Object.wait() [0x00007f9a684c5000]
   java.lang.Thread.State: BLOCKED (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a797201b8> (a java.util.ArrayList)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.log4j.AsyncAppender$Dispatcher.run(AsyncAppender.java:545)
        - locked <0x00007f9a797201b8> (a java.util.ArrayList)
        at java.lang.Thread.run(Thread.java:662)

"org.ofbiz.base.start.Start@576a9d42" prio=10 tid=0x00007f9a64004000 nid=0x7426 runnable [0x00007f9a688ab000]
   java.lang.Thread.State: RUNNABLE
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
        - locked <0x00007f9a797202d8> (a java.net.SocksSocketImpl)
        at java.net.ServerSocket.implAccept(ServerSocket.java:453)
        at java.net.ServerSocket.accept(ServerSocket.java:421)
        at org.ofbiz.base.start.Start.run(Start.java:114)
        at java.lang.Thread.run(Thread.java:662)

"Low Memory Detector" daemon prio=10 tid=0x0000000041415800 nid=0x7424 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"CompilerThread1" daemon prio=10 tid=0x0000000041410000 nid=0x7423 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"CompilerThread0" daemon prio=10 tid=0x000000004140e000 nid=0x7422 waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=10 tid=0x000000004140c000 nid=0x7421 runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=10 tid=0x00000000413e6800 nid=0x7420 in Object.wait() [0x00007f9a6900e000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a79720920> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
        - locked <0x00007f9a79720920> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=10 tid=0x00000000413e4800 nid=0x741f in Object.wait() [0x00007f9a6910f000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00007f9a79720a08> (a java.lang.ref.Reference$Lock)
        at java.lang.Object.wait(Object.java:485)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
        - locked <0x00007f9a79720a08> (a java.lang.ref.Reference$Lock)

"VM Thread" prio=10 tid=0x00000000413e0000 nid=0x741e runnable

"VM Periodic Task Thread" prio=10 tid=0x0000000041420800 nid=0x7425 waiting on condition

JNI global references: 1821
{code}


> Memory leaks in (at least) stable demo
> --------------------------------------
>
>                 Key: OFBIZ-4043
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-4043
>             Project: OFBiz
>          Issue Type: Bug
>    Affects Versions: Release Branch 09.04
>         Environment: On OFBiz demo stable
>            Reporter: Jacques Le Roux
>         Attachments: Leaks Suspect .jpg
>
>
> Using [Mat|http://www.eclipse.org/mat/]
> I generated these report
> 110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"
> Keywords
> java.net.URLClassLoader @ 0x7f3681c111f0
> org.apache.derby.impl.jdbc.EmbedConnection40
> java.util.WeakHashMap$Entry[]
> It's not quite clear to me how this happends, but I'm quite sure it happens in stable demo and maybe also in trunk. I have also attached a screen-copy with more information

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (OFBIZ-4043) Memory leaks in (at least) stable demo

Posted by "Jacques Le Roux (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/OFBIZ-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jacques Le Roux updated OFBIZ-4043:
-----------------------------------

    Attachment: screenshot-2 (Timer thread).jpg

> Memory leaks in (at least) stable demo
> --------------------------------------
>
>                 Key: OFBIZ-4043
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-4043
>             Project: OFBiz
>          Issue Type: Bug
>    Affects Versions: Release Branch 09.04
>         Environment: On OFBiz demo stable
>            Reporter: Jacques Le Roux
>         Attachments: Leaks Suspect .jpg, screenshot-2 (Timer thread).jpg
>
>
> Using [Mat|http://www.eclipse.org/mat/]
> I generated these report
> 110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"
> Keywords
> java.net.URLClassLoader @ 0x7f3681c111f0
> org.apache.derby.impl.jdbc.EmbedConnection40
> java.util.WeakHashMap$Entry[]
> It's not quite clear to me how this happends, but I'm quite sure it happens in stable demo and maybe also in trunk. I have also attached a screen-copy with more information

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (OFBIZ-4043) Memory leaks in stable demo, and locked threads in trunk demo

Posted by "Jacques Le Roux (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/OFBIZ-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jacques Le Roux updated OFBIZ-4043:
-----------------------------------

          Description: 
# Using [Mat|http://www.eclipse.org/mat/] on a stable heap dump
I generated these report
110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"
Keywords
java.net.URLClassLoader @ 0x7f3681c111f0
org.apache.derby.impl.jdbc.EmbedConnection40
java.util.WeakHashMap$Entry[]
It's not quite clear to me how this happends, but I'm quite sure it happens in stable demo and maybe also in trunk. I have also attached a screen-copy with more information
# Using jstack on trunk I found that org.ofbiz.content.cms.CmsEvents.cms(CmsEvents.java:289) seems a deadlock culprit


  was:
Using [Mat|http://www.eclipse.org/mat/]

I generated these report

110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"

Keywords
java.net.URLClassLoader @ 0x7f3681c111f0
org.apache.derby.impl.jdbc.EmbedConnection40
java.util.WeakHashMap$Entry[]


It's not quite clear to me how this happends, but I'm quite sure it happens in stable demo and maybe also in trunk. I have also attached a screen-copy with more information

          Environment: On demo stable and trunk demos  (was: On OFBiz demo stable)
    Affects Version/s: SVN trunk
              Summary: Memory leaks in stable demo, and locked threads in trunk demo  (was: Memory leaks in (at least) stable demo)

> Memory leaks in stable demo, and locked threads in trunk demo
> -------------------------------------------------------------
>
>                 Key: OFBIZ-4043
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-4043
>             Project: OFBiz
>          Issue Type: Bug
>    Affects Versions: Release Branch 09.04, SVN trunk
>         Environment: On demo stable and trunk demos
>            Reporter: Jacques Le Roux
>         Attachments: Leaks Suspect .jpg, screenshot-2 (Timer thread).jpg
>
>
> # Using [Mat|http://www.eclipse.org/mat/] on a stable heap dump
> I generated these report
> 110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"
> Keywords
> java.net.URLClassLoader @ 0x7f3681c111f0
> org.apache.derby.impl.jdbc.EmbedConnection40
> java.util.WeakHashMap$Entry[]
> It's not quite clear to me how this happends, but I'm quite sure it happens in stable demo and maybe also in trunk. I have also attached a screen-copy with more information
> # Using jstack on trunk I found that org.ofbiz.content.cms.CmsEvents.cms(CmsEvents.java:289) seems a deadlock culprit

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Issue Comment Edited: (OFBIZ-4043) Memory leaks in (at least) stable demo

Posted by "Jacques Le Roux (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/OFBIZ-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12972029#action_12972029 ] 

Jacques Le Roux edited comment on OFBIZ-4043 at 12/16/10 4:59 AM:
------------------------------------------------------------------

=======TYPO========
Jacopo, Erwan,

Sorry I did not see your posts (while editing). 

*Officially vote and release 10.04RC1:* +1
*Shutdown the 09.04 demo and replace it with the 10.04(RC1) demo:* -1. Rather fix issues there and keep alive. It can run with -Xms128M -Xmx512M (with default perm size hence 64m so at most 512+64=576MB. I think we can afford that now. Stable is much more stable at the moment than the trunk (only the leaks above which happen not so often), I don't know for 10.04. I guess we would need a little help from infra if we want to run 3 instances

      was (Author: jacques.le.roux):
    Jacopo, Erwan,

Sorry I did not see your posts (while editing). 

*Officially vote and release 10.04RC1:* +1
*Shutdown the 09.04 demo and replace it with the 10.04(RC1) demo:* -1. Rather fix issues there and keep alive. It can run with -Xms128M -Xmx512M (with default perm size hence 64m so at most 512+64=576MB. I think we can afford that now. Stable is much more stable at the moment than the trunk (only the leals above which happens not so often), I don't know for 10.04. I guess we would need a little help from infra if we want to run 3 instances
  
> Memory leaks in (at least) stable demo
> --------------------------------------
>
>                 Key: OFBIZ-4043
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-4043
>             Project: OFBiz
>          Issue Type: Bug
>    Affects Versions: Release Branch 09.04
>         Environment: On OFBiz demo stable
>            Reporter: Jacques Le Roux
>         Attachments: Leaks Suspect .jpg, screenshot-2 (Timer thread).jpg
>
>
> Using [Mat|http://www.eclipse.org/mat/]
> I generated these report
> 110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"
> Keywords
> java.net.URLClassLoader @ 0x7f3681c111f0
> org.apache.derby.impl.jdbc.EmbedConnection40
> java.util.WeakHashMap$Entry[]
> It's not quite clear to me how this happends, but I'm quite sure it happens in stable demo and maybe also in trunk. I have also attached a screen-copy with more information

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (OFBIZ-4043) Memory leaks in (at least) stable demo

Posted by "Erwan de FERRIERES (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/OFBIZ-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12972007#action_12972007 ] 

Erwan de FERRIERES commented on OFBIZ-4043:
-------------------------------------------

And officially make avalaible on the website the 10.04 and not anymore the 904?
http://ofbiz.apache.org/download.html

> Memory leaks in (at least) stable demo
> --------------------------------------
>
>                 Key: OFBIZ-4043
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-4043
>             Project: OFBiz
>          Issue Type: Bug
>    Affects Versions: Release Branch 09.04
>         Environment: On OFBiz demo stable
>            Reporter: Jacques Le Roux
>         Attachments: Leaks Suspect .jpg, screenshot-2 (Timer thread).jpg
>
>
> Using [Mat|http://www.eclipse.org/mat/]
> I generated these report
> 110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"
> Keywords
> java.net.URLClassLoader @ 0x7f3681c111f0
> org.apache.derby.impl.jdbc.EmbedConnection40
> java.util.WeakHashMap$Entry[]
> It's not quite clear to me how this happends, but I'm quite sure it happens in stable demo and maybe also in trunk. I have also attached a screen-copy with more information

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (OFBIZ-4043) Memory leaks in (at least) stable demo

Posted by "Jacques Le Roux (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/OFBIZ-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12971990#action_12971990 ] 

Jacques Le Roux commented on OFBIZ-4043:
----------------------------------------

Thanks Erwan,

You upgraded only the trunk and the "Derby issue" appeared only in the stable branch9 so far. Anyway, though we see org.apache.derby.impl.jdbc.EmbedConnection40 is implied I'm not sure that Derby is the reason. Actually I'm even inclined to think that Derby it's not the culprit, but calls to it.

> Memory leaks in (at least) stable demo
> --------------------------------------
>
>                 Key: OFBIZ-4043
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-4043
>             Project: OFBiz
>          Issue Type: Bug
>    Affects Versions: Release Branch 09.04
>         Environment: On OFBiz demo stable
>            Reporter: Jacques Le Roux
>         Attachments: Leaks Suspect .jpg, screenshot-2 (Timer thread).jpg
>
>
> Using [Mat|http://www.eclipse.org/mat/]
> I generated these report
> 110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"
> Keywords
> java.net.URLClassLoader @ 0x7f3681c111f0
> org.apache.derby.impl.jdbc.EmbedConnection40
> java.util.WeakHashMap$Entry[]
> It's not quite clear to me how this happends, but I'm quite sure it happens in stable demo and maybe also in trunk. I have also attached a screen-copy with more information

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (OFBIZ-4043) Memory leaks in (at least) stable demo

Posted by "Jacques Le Roux (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/OFBIZ-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jacques Le Roux updated OFBIZ-4043:
-----------------------------------

    Description: 
Using [Mat|http://www.eclipse.org/mat/]

I generated these report

110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"

Keywords
java.net.URLClassLoader @ 0x7f3681c111f0
org.apache.derby.impl.jdbc.EmbedConnection40
java.util.WeakHashMap$Entry[]


It's not quite clear to me how this happends, but I'm quite sure it happens in stable demo and maybe also in trunk. I have also attached a screen-copy with more information

  was:
Using [Mat|http://www.eclipse.org/mat/]

I generated these report

110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"

Keywords
java.net.URLClassLoader @ 0x7f3681c111f0
org.apache.derby.impl.jdbc.EmbedConnection40
java.util.WeakHashMap$Entry[]


It's not quite clear to me how this happends, but I'm quite sure it happens often in stable demo and maybe also in trunk. I have also attached a screen-copy with more information


> Memory leaks in (at least) stable demo
> --------------------------------------
>
>                 Key: OFBIZ-4043
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-4043
>             Project: OFBiz
>          Issue Type: Bug
>    Affects Versions: Release Branch 09.04
>         Environment: On OFBiz demo stable
>            Reporter: Jacques Le Roux
>         Attachments: Leaks Suspect .jpg
>
>
> Using [Mat|http://www.eclipse.org/mat/]
> I generated these report
> 110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"
> Keywords
> java.net.URLClassLoader @ 0x7f3681c111f0
> org.apache.derby.impl.jdbc.EmbedConnection40
> java.util.WeakHashMap$Entry[]
> It's not quite clear to me how this happends, but I'm quite sure it happens in stable demo and maybe also in trunk. I have also attached a screen-copy with more information

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (OFBIZ-4043) Memory leaks in (at least) stable demo

Posted by "Jacopo Cappellato (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/OFBIZ-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12972008#action_12972008 ] 

Jacopo Cappellato commented on OFBIZ-4043:
------------------------------------------

All the official releases will be always available for download and listed there; however the current (or latest) official release will be 10.04RC1 and will be at the top of the download page (or 10.04 if we will decide to name it as 'stable').


> Memory leaks in (at least) stable demo
> --------------------------------------
>
>                 Key: OFBIZ-4043
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-4043
>             Project: OFBiz
>          Issue Type: Bug
>    Affects Versions: Release Branch 09.04
>         Environment: On OFBiz demo stable
>            Reporter: Jacques Le Roux
>         Attachments: Leaks Suspect .jpg, screenshot-2 (Timer thread).jpg
>
>
> Using [Mat|http://www.eclipse.org/mat/]
> I generated these report
> 110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"
> Keywords
> java.net.URLClassLoader @ 0x7f3681c111f0
> org.apache.derby.impl.jdbc.EmbedConnection40
> java.util.WeakHashMap$Entry[]
> It's not quite clear to me how this happends, but I'm quite sure it happens in stable demo and maybe also in trunk. I have also attached a screen-copy with more information

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (OFBIZ-4043) Memory leaks in (at least) stable demo

Posted by "Jacques Le Roux (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/OFBIZ-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jacques Le Roux updated OFBIZ-4043:
-----------------------------------

    Attachment: Leaks Suspect .jpg

> Memory leaks in (at least) stable demo
> --------------------------------------
>
>                 Key: OFBIZ-4043
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-4043
>             Project: OFBiz
>          Issue Type: Bug
>    Affects Versions: Release Branch 09.04
>         Environment: On OFBiz demo stable
>            Reporter: Jacques Le Roux
>         Attachments: Leaks Suspect .jpg
>
>
> Using [Mat|http://www.eclipse.org/mat/]
> I generated these report
> 110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"
> Keywords
> java.net.URLClassLoader @ 0x7f3681c111f0
> org.apache.derby.impl.jdbc.EmbedConnection40
> java.util.WeakHashMap$Entry[]
> It's not quite clear to me how this happends, but I'm quirte sure it happens often in stable demo and maybe in trunk also. I have also attached a screen-copy with more information

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (OFBIZ-4043) Memory leaks in (at least) stable demo

Posted by "Jacques Le Roux (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/OFBIZ-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12972029#action_12972029 ] 

Jacques Le Roux commented on OFBIZ-4043:
----------------------------------------

Jacopo, Erwan,

Sorry I did not see your posts (while editing). 

*Officially vote and release 10.04RC1:* +1
*Shutdown the 09.04 demo and replace it with the 10.04(RC1) demo:* -1. Rather fix issues there and keep alive. It can run with -Xms128M -Xmx512M (with default perm size hence 64m so at most 512+64=576MB. I think we can afford that now. Stable is much more stable at the moment than the trunk (only the leals above which happens not so often), I don't know for 10.04. I guess we would need a little help from infra if we want to run 3 instances

> Memory leaks in (at least) stable demo
> --------------------------------------
>
>                 Key: OFBIZ-4043
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-4043
>             Project: OFBiz
>          Issue Type: Bug
>    Affects Versions: Release Branch 09.04
>         Environment: On OFBiz demo stable
>            Reporter: Jacques Le Roux
>         Attachments: Leaks Suspect .jpg, screenshot-2 (Timer thread).jpg
>
>
> Using [Mat|http://www.eclipse.org/mat/]
> I generated these report
> 110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"
> Keywords
> java.net.URLClassLoader @ 0x7f3681c111f0
> org.apache.derby.impl.jdbc.EmbedConnection40
> java.util.WeakHashMap$Entry[]
> It's not quite clear to me how this happends, but I'm quite sure it happens in stable demo and maybe also in trunk. I have also attached a screen-copy with more information

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (OFBIZ-4043) Memory leaks in stable demo, and locked threads in trunk demo

Posted by "Jacques Le Roux (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/OFBIZ-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12972469#action_12972469 ] 

Jacques Le Roux commented on OFBIZ-4043:
----------------------------------------

The deadlock issue in trunk is fixed at r1050364 see OFBIZ-4065


> Memory leaks in stable demo, and locked threads in trunk demo
> -------------------------------------------------------------
>
>                 Key: OFBIZ-4043
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-4043
>             Project: OFBiz
>          Issue Type: Bug
>    Affects Versions: Release Branch 09.04, SVN trunk
>         Environment: On demo stable and trunk demos
>            Reporter: Jacques Le Roux
>         Attachments: Leaks Suspect .jpg, screenshot-2 (Timer thread).jpg
>
>
> # Using [Mat|http://www.eclipse.org/mat/] on a stable heap dump
> I generated these report
> 110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"
> Keywords
> java.net.URLClassLoader @ 0x7f3681c111f0
> org.apache.derby.impl.jdbc.EmbedConnection40
> java.util.WeakHashMap$Entry[]
> It's not quite clear to me how this happends, but I'm quite sure it happens in stable demo and maybe also in trunk. I have also attached a screen-copy with more information
> # Using jstack on trunk I found that org.ofbiz.content.cms.CmsEvents.cms(CmsEvents.java:289) seems a deadlock culprit

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (OFBIZ-4043) Memory leaks in (at least) stable demo

Posted by "Erwan de FERRIERES (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/OFBIZ-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12971842#action_12971842 ] 

Erwan de FERRIERES commented on OFBIZ-4043:
-------------------------------------------

Jacques,

I just upgraded the Derby driver. Maybe this could help ?

> Memory leaks in (at least) stable demo
> --------------------------------------
>
>                 Key: OFBIZ-4043
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-4043
>             Project: OFBiz
>          Issue Type: Bug
>    Affects Versions: Release Branch 09.04
>         Environment: On OFBiz demo stable
>            Reporter: Jacques Le Roux
>         Attachments: Leaks Suspect .jpg, screenshot-2 (Timer thread).jpg
>
>
> Using [Mat|http://www.eclipse.org/mat/]
> I generated these report
> 110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"
> Keywords
> java.net.URLClassLoader @ 0x7f3681c111f0
> org.apache.derby.impl.jdbc.EmbedConnection40
> java.util.WeakHashMap$Entry[]
> It's not quite clear to me how this happends, but I'm quite sure it happens in stable demo and maybe also in trunk. I have also attached a screen-copy with more information

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Closed: (OFBIZ-4043) Memory leaks in stable demo, and locked threads in trunk demo

Posted by "Jacques Le Roux (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/OFBIZ-4043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jacques Le Roux closed OFBIZ-4043.
----------------------------------

       Resolution: Fixed
    Fix Version/s: SVN trunk
         Assignee: Jacques Le Roux

I have fixed the locked threads in trunk demo at r1050452 and it seems the memory leaks in stable demo was due to the JVM version: http://markmail.org/message/4qv336blri6chlxb

> Memory leaks in stable demo, and locked threads in trunk demo
> -------------------------------------------------------------
>
>                 Key: OFBIZ-4043
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-4043
>             Project: OFBiz
>          Issue Type: Bug
>    Affects Versions: Release Branch 09.04, SVN trunk
>         Environment: On demo stable and trunk demos
>            Reporter: Jacques Le Roux
>            Assignee: Jacques Le Roux
>             Fix For: SVN trunk
>
>         Attachments: Leaks Suspect .jpg, screenshot-2 (Timer thread).jpg
>
>
> # Using [Mat|http://www.eclipse.org/mat/] on a stable heap dump
> I generated these report
> 110 542 instances of "org.apache.derby.impl.jdbc.EmbedConnection40", loaded by "java.net.URLClassLoader @ 0x7f3681c111f0" occupy 415 680 440 (79,89%) bytes. These instances are referenced from one instance of "java.util.WeakHashMap$Entry[]", loaded by "<system class loader>"
> Keywords
> java.net.URLClassLoader @ 0x7f3681c111f0
> org.apache.derby.impl.jdbc.EmbedConnection40
> java.util.WeakHashMap$Entry[]
> It's not quite clear to me how this happends, but I'm quite sure it happens in stable demo and maybe also in trunk. I have also attached a screen-copy with more information
> # Using jstack on trunk I found that org.ofbiz.content.cms.CmsEvents.cms(CmsEvents.java:289) seems a deadlock culprit

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.