High CPU usage by ofbiz process.

[Ganesh Bawne <ga...@gmail.com>]

I'm running into a strange issue whereby my ofbiz will run just fine for up
to several hours. At some point though the CPU utilization slowly begins to
creep higher. Eventually everything locks and the program hangs. I can see
the transaction timeout exceptions in the logs for most of our scheduled
services. At some point--anywhere from 30 minutes to several hours
later--CPU utilization drops to normal and the program resumes operation as
if everything were fine. This goes on for up to several hours, then the
utilization issue repeats. So it's a very odd issue I've run into.

Additionally, when I view "top" I generally see a dozen or so  "idle"
postgres processes but most of them are 0%.  Not surem if its connected to
high cup usage.

Any assistance is greatly appreciated.

-- 
Thanks & Regards
Ganesh Bawne
Enterprise Software Developer
Viithiisys Pvt. Ltd.
http://www.viithiisys.com

Re: High CPU usage by ofbiz process.

[Ganesh Bawne <ga...@gmail.com>]

We are using ofbiz Release 12.04.
Sorry not able to get the exact revision as we added this release to our
svn and then after lots of custom changes made.
We have around 45 scheduled jobs running (some of them scheduled to run for
every 12 mins). There is no customized code in running instance and usually
the jobs won't Crash unless the server is stopped in between the job is
running. But, when high CPU happens then most of scheduled services get
failed due to transaction time out.

I see below warning in the logs, so may be some services are called by
deprecated method but still they work fine.
PersistedServiceJob.java:137:WARN ] Persisted Job [12812626] references a
RecurrenceInfo, recommend using TemporalExpression instead

Do you think, it could be related to max_connection limit. The postgres
server has max_connection limit set to 200 and ofbiz has 256?

I will see, if I can mange to find some useful result with 'YourKit'. We
can't test it on dumps because the scheduled services can be tested only
with live environment.




On Sat, Feb 8, 2014 at 8:19 PM, Jacques Le Roux <
jacques.le.roux@les7arts.com> wrote:

> Looks like a concurrent issue to me, related to scheduled job? That's
> really just a bet.
> Which Release.revision are you using? I guess you have customised code in
> your running instance?
> How many jobs have you scheduled?
> You instance does not crash, right? Else you can try using
> -XX:+HeapDumpOnOutOfMemoryError
>
> I will recommend to use YourKit http://yourkit.com/ or a similar tool to
> check your instance.
> You might also take dumps at any moment and use Mat
> https://www.eclipse.org/mat/ (free). The big advantage of YourKit: all is
> dynamic...
>
> HTH
>
> Jacques
>
> On Saturday, February 08, 2014 11:55 AM, ganesh4ofbiz@gmail.com wrote
> > I'm running into a strange issue whereby my ofbiz will run just fine for
> up
> > to several hours. At some point though the CPU utilization slowly begins
> to
> > creep higher. Eventually everything locks and the program hangs. I can
> see
> > the transaction timeout exceptions in the logs for most of our scheduled
> > services. At some point--anywhere from 30 minutes to several hours
> > later--CPU utilization drops to normal and the program resumes operation
> as
> > if everything were fine. This goes on for up to several hours, then the
> > utilization issue repeats. So it's a very odd issue I've run into.
> >
> > Additionally, when I view "top" I generally see a dozen or so  "idle"
> > postgres processes but most of them are 0%.  Not surem if its connected
> to
> > high cup usage.
> >
> > Any assistance is greatly appreciated.
>



-- 
Thanks & Regards
Ganesh Bawne
Enterprise Software Developer
Viithiisys Pvt. Ltd.
http://www.viithiisys.com

Re: High CPU usage by ofbiz process.

[Jacques Le Roux <ja...@les7arts.com>]

Looks like a concurrent issue to me, related to scheduled job? That's really just a bet.
Which Release.revision are you using? I guess you have customised code in your running instance?
How many jobs have you scheduled?
You instance does not crash, right? Else you can try using -XX:+HeapDumpOnOutOfMemoryError

I will recommend to use YourKit http://yourkit.com/ or a similar tool to check your instance.
You might also take dumps at any moment and use Mat https://www.eclipse.org/mat/ (free). The big advantage of YourKit: all is dynamic...

HTH

Jacques

On Saturday, February 08, 2014 11:55 AM, ganesh4ofbiz@gmail.com wrote
> I'm running into a strange issue whereby my ofbiz will run just fine for up
> to several hours. At some point though the CPU utilization slowly begins to
> creep higher. Eventually everything locks and the program hangs. I can see
> the transaction timeout exceptions in the logs for most of our scheduled
> services. At some point--anywhere from 30 minutes to several hours
> later--CPU utilization drops to normal and the program resumes operation as
> if everything were fine. This goes on for up to several hours, then the
> utilization issue repeats. So it's a very odd issue I've run into.
> 
> Additionally, when I view "top" I generally see a dozen or so  "idle"
> postgres processes but most of them are 0%.  Not surem if its connected to
> high cup usage.
> 
> Any assistance is greatly appreciated.