Output of investigation into memory usage on BPEL Redeployments

[Ciaran <ci...@gmail.com>]

Hi Folks,

I think I've completed my investigation into memory leaks when doing
re-deployments of in-memory processes.  At the start of this process I was
leaking several tens, to hundreds of megabytes (depending on the workflow
size,) I was unable to deploy the largest of my flows and couldn't re-deploy
more than twice my medium size flow, after these changes I can only find one
visible 'leak' (see below) and several other instances hanging about with
weak references, I can re-deploy arbritary sized workflows (if they deploy
once, then they will un-deploy and re-deploy again successfully as many
times as I need tbc. I've done several hundred deployments of a medium
workflow, currently testing the large flow deployments.)

* The outstanding 'leak' is of jpa.VersionDAOImpl objects, as far as I can
tell this 'leak' is deliberate, can I ask why we don't purge old unused
versions?  (There are various othiner DAOImpl objects that are not
explicitly deleted from memory, but they seem to be held by weak references
so the GC will reap them eventually)

The list of JIRA issues (and associated patches) follows:

   1. https://issues.apache.org/jira/browse/ODE-574  - Memory leak when
   Un-deploying processes that contain XSL stylesheets
   2. https://issues.apache.org/jira/browse/ODE-575  -  oAssign$Literal
   seems to consume an excessive volume of memory
   3. https://issues.apache.org/jira/browse/ODE-576  - ProcessDaoImpl
   objects get leaked when using in-memory-processes
   4. https://issues.apache.org/jira/browse/ODE-580  - DeploymentUnitDir
   object gets leaked after un-deployment

Fwiw, I suspect the reason I'm having such major memory issues is because I
use both in-memory processes and XSL stylesheets, I guess those aren't
common scenarios for consumers of the engine.

In terms of 'contetious-ness' ;) Issues 3 + 4 are the most;)

Issue 3 I've re-opened after it was invalid-closed, my patch fixes a large
issue for me, but I suspect I've fixed something in-advertantly (based on
Karthick's comments) however without it I can't use ODE, so I would
definately appreciate someone taking another look at the issue + the patch

Issue 4 is actually an Axis2 bug that has been resolved in Axis2 1.4.1, and
unfortunately my fix uses reflection to clear out a private field ( a map,)
I suspect this isn't acceptable, the leak this causes is fairly minimal tbh,
and pales into insignificance when compared to 2, then 1 then 3 (in that
order !)

Next I'll be looking at whether memory is leaked when in-memory processes
are under continuous and reasonably heavy load (my initial testing suggests
that it is)   The purpose of this e-mail is to make  others aware of these
changes and gather feedback.

Hope this mail finds you all well ;)
- cj.


<https://issues.apache.org/jira/browse/ODE-574>