You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@velocity.apache.org by Terence Kent <te...@gmail.com> on 2014/02/05 19:05:52 UTC
ResourceNotFound errors start after several days using the ClasspathResourceLoader
Hello,
We just starting using Velocity 1.7 in our production environment and encountered the following error after our JVMs had been running for about 5 days:
ERROR ResourceManager : unable to find resource '/path/to/template.vm' in any resource loader.
We are getting no other errors in any of our code and this is the only velocity message in our logs (we captured WARN and higher messages). Prior to the error occurring, velocity was working as expected and able to find our template. After the first time we received this error, all following attempts to use the template failed. We had two separate systems running the same code and they both experienced the issue within about 10 minutes of each other. Restarting the JVM fixed the issue in both cases.
This appears to be a very similar issue to the one described Peter over a year ago using Velocity 1.6.2. Unfortunately, the thread appears to trail off before an issue was diagnosed/fixed. (see http://velocity.10973.n7.nabble.com/ResourceNotFoundException-Unable-to-find-resource-td10186.html)
I'm a bit stuck since I've been unable to reproduce the issue or find out more information. I've tried some load tests at a concurrency of 50 (far larger than we receive on production), and this had no failures after 45 minutes. Has anybody had success in diagnosing/resolving this issue? In case it's relevant, here are the details of our deployment environment:
* We use Tomcat 7.0.42, with 2 webapps deployed, A and B. A is fairly large and complex, providing a web services api and a client API. B just contains a tiny servlet that calls velocity on a single template. Velocity is only used in webapp B.
* We use Velocity 1.7, configured to use the ClasspathResourceLoader. We configure Velocity programmatically.
* Our JVM is OpenJDK build 20.0-b12 (java version "1.6.0_27")
Thanks in advance!
Terence
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
For additional commands, e-mail: dev-help@velocity.apache.org
Re: ResourceNotFound errors start after several days using the ClasspathResourceLoader
Posted by Sergiu Dumitriu <se...@gmail.com>.
Since not all servers have the same load (I guess), I don't believe it's
related to the number of times the file is requested. Could it be
something unrelated to Velocity, but to the underlying system? Are there
automated upgrades happening? Any cron tasks running?
Can you try to enable debug logging?
On 02/13/2014 10:00 PM, Terence Kent wrote:
> As a follow up, the issue just occurred again - about 5 days after the last JVM restart. Just like last time, it occurred across multiple systems within about 10 minutes of each other. There was very low usage at the time and nothing else running in the JVM had any issues. The exception in the stack trace below was thrown each time the velocity code was hit until the JVM was restarted.
>
> org.apache.velocity.exception.ResourceNotFoundException: Unable to find resource '/templates/XetusOne.hta'
> at org.apache.velocity.runtime.resource.ResourceManagerImpl.loadResource(ResourceManagerImpl.java:474)
> at org.apache.velocity.runtime.resource.ResourceManagerImpl.getResource(ResourceManagerImpl.java:352)
> at org.apache.velocity.runtime.RuntimeInstance.getTemplate(RuntimeInstance.java:1533)
> at org.apache.velocity.app.VelocityEngine.mergeTemplate(VelocityEngine.java:343)
> at com.xetus.one.client.hta.HTAServlet.doGet(HTAServlet.java:118)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:770)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
> at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
> at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
> at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
> at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:679)
>
> On Feb 5, 2014, at 3:10 PM, Terence Kent <te...@gmail.com> wrote:
>
>> On each of the systems that had the issue, there is currently a "normal" amount of open files (1200-1300 total, 700-800 are regular files). The JVMs account for about 90 of the regular open files and the max for the operating system is 793674.
>>
>> Of course, I can't tell you what the lsof output was at the time of the failure, but I can say that I cross checked the system logs (Ubuntu 10.04). There was no other process/daemon issues and no indication of I/O failures or even high activity.
>>
>> On Feb 5, 2014, at 2:59 PM, Sergiu Dumitriu <se...@gmail.com> wrote:
>>
>>> Can you check how many open file handles are there? Try using 'lsof'
>>>
>>> On 02/05/2014 01:05 PM, Terence Kent wrote:
>>>> Hello,
>>>>
>>>> We just starting using Velocity 1.7 in our production environment and encountered the following error after our JVMs had been running for about 5 days:
>>>>
>>>> ERROR ResourceManager : unable to find resource '/path/to/template.vm' in any resource loader.
>>>>
>>>> We are getting no other errors in any of our code and this is the only velocity message in our logs (we captured WARN and higher messages). Prior to the error occurring, velocity was working as expected and able to find our template. After the first time we received this error, all following attempts to use the template failed. We had two separate systems running the same code and they both experienced the issue within about 10 minutes of each other. Restarting the JVM fixed the issue in both cases.
>>>>
>>>> This appears to be a very similar issue to the one described Peter over a year ago using Velocity 1.6.2. Unfortunately, the thread appears to trail off before an issue was diagnosed/fixed. (see http://velocity.10973.n7.nabble.com/ResourceNotFoundException-Unable-to-find-resource-td10186.html)
>>>>
>>>> I'm a bit stuck since I've been unable to reproduce the issue or find out more information. I've tried some load tests at a concurrency of 50 (far larger than we receive on production), and this had no failures after 45 minutes. Has anybody had success in diagnosing/resolving this issue? In case it's relevant, here are the details of our deployment environment:
>>>>
>>>> * We use Tomcat 7.0.42, with 2 webapps deployed, A and B. A is fairly large and complex, providing a web services api and a client API. B just contains a tiny servlet that calls velocity on a single template. Velocity is only used in webapp B.
>>>> * We use Velocity 1.7, configured to use the ClasspathResourceLoader. We configure Velocity programmatically.
>>>> * Our JVM is OpenJDK build 20.0-b12 (java version "1.6.0_27")
--
Sergiu Dumitriu
http://purl.org/net/sergiu
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
For additional commands, e-mail: dev-help@velocity.apache.org
Re: ResourceNotFound errors start after several days using the ClasspathResourceLoader
Posted by Terence Kent <te...@gmail.com>.
As a follow up, the issue just occurred again - about 5 days after the last JVM restart. Just like last time, it occurred across multiple systems within about 10 minutes of each other. There was very low usage at the time and nothing else running in the JVM had any issues. The exception in the stack trace below was thrown each time the velocity code was hit until the JVM was restarted.
org.apache.velocity.exception.ResourceNotFoundException: Unable to find resource '/templates/XetusOne.hta'
at org.apache.velocity.runtime.resource.ResourceManagerImpl.loadResource(ResourceManagerImpl.java:474)
at org.apache.velocity.runtime.resource.ResourceManagerImpl.getResource(ResourceManagerImpl.java:352)
at org.apache.velocity.runtime.RuntimeInstance.getTemplate(RuntimeInstance.java:1533)
at org.apache.velocity.app.VelocityEngine.mergeTemplate(VelocityEngine.java:343)
at com.xetus.one.client.hta.HTAServlet.doGet(HTAServlet.java:118)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:668)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:770)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
On Feb 5, 2014, at 3:10 PM, Terence Kent <te...@gmail.com> wrote:
> On each of the systems that had the issue, there is currently a "normal" amount of open files (1200-1300 total, 700-800 are regular files). The JVMs account for about 90 of the regular open files and the max for the operating system is 793674.
>
> Of course, I can't tell you what the lsof output was at the time of the failure, but I can say that I cross checked the system logs (Ubuntu 10.04). There was no other process/daemon issues and no indication of I/O failures or even high activity.
>
> On Feb 5, 2014, at 2:59 PM, Sergiu Dumitriu <se...@gmail.com> wrote:
>
>> Can you check how many open file handles are there? Try using 'lsof'
>>
>> On 02/05/2014 01:05 PM, Terence Kent wrote:
>>> Hello,
>>>
>>> We just starting using Velocity 1.7 in our production environment and encountered the following error after our JVMs had been running for about 5 days:
>>>
>>> ERROR ResourceManager : unable to find resource '/path/to/template.vm' in any resource loader.
>>>
>>> We are getting no other errors in any of our code and this is the only velocity message in our logs (we captured WARN and higher messages). Prior to the error occurring, velocity was working as expected and able to find our template. After the first time we received this error, all following attempts to use the template failed. We had two separate systems running the same code and they both experienced the issue within about 10 minutes of each other. Restarting the JVM fixed the issue in both cases.
>>>
>>> This appears to be a very similar issue to the one described Peter over a year ago using Velocity 1.6.2. Unfortunately, the thread appears to trail off before an issue was diagnosed/fixed. (see http://velocity.10973.n7.nabble.com/ResourceNotFoundException-Unable-to-find-resource-td10186.html)
>>>
>>> I'm a bit stuck since I've been unable to reproduce the issue or find out more information. I've tried some load tests at a concurrency of 50 (far larger than we receive on production), and this had no failures after 45 minutes. Has anybody had success in diagnosing/resolving this issue? In case it's relevant, here are the details of our deployment environment:
>>>
>>> * We use Tomcat 7.0.42, with 2 webapps deployed, A and B. A is fairly large and complex, providing a web services api and a client API. B just contains a tiny servlet that calls velocity on a single template. Velocity is only used in webapp B.
>>> * We use Velocity 1.7, configured to use the ClasspathResourceLoader. We configure Velocity programmatically.
>>> * Our JVM is OpenJDK build 20.0-b12 (java version "1.6.0_27")
>>
>> --
>> Sergiu Dumitriu
>> http://purl.org/net/sergiu
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
>> For additional commands, e-mail: dev-help@velocity.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
For additional commands, e-mail: dev-help@velocity.apache.org
Re: ResourceNotFound errors start after several days using the ClasspathResourceLoader
Posted by Terence Kent <te...@gmail.com>.
On each of the systems that had the issue, there is currently a "normal" amount of open files (1200-1300 total, 700-800 are regular files). The JVMs account for about 90 of the regular open files and the max for the operating system is 793674.
Of course, I can't tell you what the lsof output was at the time of the failure, but I can say that I cross checked the system logs (Ubuntu 10.04). There was no other process/daemon issues and no indication of I/O failures or even high activity.
On Feb 5, 2014, at 2:59 PM, Sergiu Dumitriu <se...@gmail.com> wrote:
> Can you check how many open file handles are there? Try using 'lsof'
>
> On 02/05/2014 01:05 PM, Terence Kent wrote:
>> Hello,
>>
>> We just starting using Velocity 1.7 in our production environment and encountered the following error after our JVMs had been running for about 5 days:
>>
>> ERROR ResourceManager : unable to find resource '/path/to/template.vm' in any resource loader.
>>
>> We are getting no other errors in any of our code and this is the only velocity message in our logs (we captured WARN and higher messages). Prior to the error occurring, velocity was working as expected and able to find our template. After the first time we received this error, all following attempts to use the template failed. We had two separate systems running the same code and they both experienced the issue within about 10 minutes of each other. Restarting the JVM fixed the issue in both cases.
>>
>> This appears to be a very similar issue to the one described Peter over a year ago using Velocity 1.6.2. Unfortunately, the thread appears to trail off before an issue was diagnosed/fixed. (see http://velocity.10973.n7.nabble.com/ResourceNotFoundException-Unable-to-find-resource-td10186.html)
>>
>> I'm a bit stuck since I've been unable to reproduce the issue or find out more information. I've tried some load tests at a concurrency of 50 (far larger than we receive on production), and this had no failures after 45 minutes. Has anybody had success in diagnosing/resolving this issue? In case it's relevant, here are the details of our deployment environment:
>>
>> * We use Tomcat 7.0.42, with 2 webapps deployed, A and B. A is fairly large and complex, providing a web services api and a client API. B just contains a tiny servlet that calls velocity on a single template. Velocity is only used in webapp B.
>> * We use Velocity 1.7, configured to use the ClasspathResourceLoader. We configure Velocity programmatically.
>> * Our JVM is OpenJDK build 20.0-b12 (java version "1.6.0_27")
>
> --
> Sergiu Dumitriu
> http://purl.org/net/sergiu
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
> For additional commands, e-mail: dev-help@velocity.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
For additional commands, e-mail: dev-help@velocity.apache.org
Re: ResourceNotFound errors start after several days using the ClasspathResourceLoader
Posted by Sergiu Dumitriu <se...@gmail.com>.
Can you check how many open file handles are there? Try using 'lsof'
On 02/05/2014 01:05 PM, Terence Kent wrote:
> Hello,
>
> We just starting using Velocity 1.7 in our production environment and encountered the following error after our JVMs had been running for about 5 days:
>
> ERROR ResourceManager : unable to find resource '/path/to/template.vm' in any resource loader.
>
> We are getting no other errors in any of our code and this is the only velocity message in our logs (we captured WARN and higher messages). Prior to the error occurring, velocity was working as expected and able to find our template. After the first time we received this error, all following attempts to use the template failed. We had two separate systems running the same code and they both experienced the issue within about 10 minutes of each other. Restarting the JVM fixed the issue in both cases.
>
> This appears to be a very similar issue to the one described Peter over a year ago using Velocity 1.6.2. Unfortunately, the thread appears to trail off before an issue was diagnosed/fixed. (see http://velocity.10973.n7.nabble.com/ResourceNotFoundException-Unable-to-find-resource-td10186.html)
>
> I'm a bit stuck since I've been unable to reproduce the issue or find out more information. I've tried some load tests at a concurrency of 50 (far larger than we receive on production), and this had no failures after 45 minutes. Has anybody had success in diagnosing/resolving this issue? In case it's relevant, here are the details of our deployment environment:
>
> * We use Tomcat 7.0.42, with 2 webapps deployed, A and B. A is fairly large and complex, providing a web services api and a client API. B just contains a tiny servlet that calls velocity on a single template. Velocity is only used in webapp B.
> * We use Velocity 1.7, configured to use the ClasspathResourceLoader. We configure Velocity programmatically.
> * Our JVM is OpenJDK build 20.0-b12 (java version "1.6.0_27")
--
Sergiu Dumitriu
http://purl.org/net/sergiu
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
For additional commands, e-mail: dev-help@velocity.apache.org