buildup of files in var/cache

[Kevan Miller <ke...@gmail.com>]

A while back we had a problem with files for undeployed applications were building up in a servers repository. This problem was fixed. We still, however, have files that will build up in var/cache until the server is restarted. I ran a deploy/undeploy test of a simple HelloWorld jsp app. Each deploy/undeploy cycle will consume 2 megabytes of disk space, until the server is restarted. In some long running tests (but not *that* long running tests), I've seen the server consume 12 GB of hard drive space.

Would really like to see this addressed. Any ideas?

--kevan

Re: buildup of files in var/cache

[Ivan <xh...@gmail.com>]

Hmm, the exception I mentioned seems also fixed with that entry, so the
problem is just when a new 3.7.1 could be released, or we might package a
customized equonix.

2011/7/27 Ivan <xh...@gmail.com>

> Just find the it is caused by store manager, it did not remove the old
> managed files while it switches to a new one. Although there is a property
> could be used to clean up the temp file, it is required to restart the
> framework and there is an issue while the cache folder does not exist.
> After googled, there is already a bug opened for this
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=259981. And seems that this
> is a long known issue,  luckily, it is fixed recently, but it is not
> included in the equonix version used in Geronimo now. Also, there might be
> an issue if the cleanOnOpen property is configured without cache folder, I
> have replied on the issue, and hope that someone could check it.
>
>
> 2011/7/23 Ivan <xh...@gmail.com>
>
>> I just did a simple check, and it seems that the bundles are correctly
>> removed after the target application is undeployed, at least on the
>> successful scenarios. The extra 2M size is caused by a .lazy.* file in the
>> cache folder. and that file should be related to the store manager. Will
>> check it further later ...
>>
>> 2011/7/23 Kevan Miller <ke...@gmail.com>
>>
>>>
>>> On Jul 22, 2011, at 10:45 PM, Ivan wrote:
>>>
>>> > Suppose you mean the OSGi cache directory ? On the server runtime, the
>>> size of this directory should be almost have the same size with the
>>> repository folder, as all the files would be copied.
>>>
>>> Right. I'm not worried about relatively static content. I am worried
>>> about consuming more and more disk space and never freeing it...
>>>
>>> > For the 12 GB var/cache directory size, there should be a leak issue
>>> somewhere,
>>>
>>> There's definitely a leak. As mentioned, I see two megabytes leaked for
>>> every deploy/undeploy of a very simple JSP web app.
>>>
>>> > a. In the deployment process, the deployer will deploy a temp bundle in
>>> the cache for builder analysis, maybe it is not uninstalled correctly due to
>>> some exceptions.
>>> > b. In the undeployment process, Geronimo did not invoke uninstall for
>>> the undeployed application due to unknown reason ?
>>> > c. Some exported classes of the undeployed application are wired with
>>> other bundles, so the OSGi runtime will keep it there until the framework is
>>> restarted. Considering the current package generation mechanism, should not
>>> happen.
>>> > Think that if we could reproduce the issue, it should be easy to fix.
>>> Also, I am also thinking that there is no need to package the temp bundle,
>>> just use the target directory. which mentioned in another thread in the mail
>>> list.
>>>
>>> It's extremely easy to recreate. Choose and application and
>>> deploy/undeploy it a few times... 'du -h var/cache' or similar to monitor
>>> disk space between iterations...
>>>
>>> --kevan
>>
>>
>>
>>
>> --
>> Ivan
>>
>
>
>
> --
> Ivan
>



-- 
Ivan

Re: buildup of files in var/cache

[Ivan <xh...@gmail.com>]

Just find the it is caused by store manager, it did not remove the old
managed files while it switches to a new one. Although there is a property
could be used to clean up the temp file, it is required to restart the
framework and there is an issue while the cache folder does not exist.
After googled, there is already a bug opened for this
https://bugs.eclipse.org/bugs/show_bug.cgi?id=259981. And seems that this is
a long known issue,  luckily, it is fixed recently, but it is not included
in the equonix version used in Geronimo now. Also, there might be an issue
if the cleanOnOpen property is configured without cache folder, I have
replied on the issue, and hope that someone could check it.

2011/7/23 Ivan <xh...@gmail.com>

> I just did a simple check, and it seems that the bundles are correctly
> removed after the target application is undeployed, at least on the
> successful scenarios. The extra 2M size is caused by a .lazy.* file in the
> cache folder. and that file should be related to the store manager. Will
> check it further later ...
>
> 2011/7/23 Kevan Miller <ke...@gmail.com>
>
>>
>> On Jul 22, 2011, at 10:45 PM, Ivan wrote:
>>
>> > Suppose you mean the OSGi cache directory ? On the server runtime, the
>> size of this directory should be almost have the same size with the
>> repository folder, as all the files would be copied.
>>
>> Right. I'm not worried about relatively static content. I am worried about
>> consuming more and more disk space and never freeing it...
>>
>> > For the 12 GB var/cache directory size, there should be a leak issue
>> somewhere,
>>
>> There's definitely a leak. As mentioned, I see two megabytes leaked for
>> every deploy/undeploy of a very simple JSP web app.
>>
>> > a. In the deployment process, the deployer will deploy a temp bundle in
>> the cache for builder analysis, maybe it is not uninstalled correctly due to
>> some exceptions.
>> > b. In the undeployment process, Geronimo did not invoke uninstall for
>> the undeployed application due to unknown reason ?
>> > c. Some exported classes of the undeployed application are wired with
>> other bundles, so the OSGi runtime will keep it there until the framework is
>> restarted. Considering the current package generation mechanism, should not
>> happen.
>> > Think that if we could reproduce the issue, it should be easy to fix.
>> Also, I am also thinking that there is no need to package the temp bundle,
>> just use the target directory. which mentioned in another thread in the mail
>> list.
>>
>> It's extremely easy to recreate. Choose and application and
>> deploy/undeploy it a few times... 'du -h var/cache' or similar to monitor
>> disk space between iterations...
>>
>> --kevan
>
>
>
>
> --
> Ivan
>



-- 
Ivan

Re: buildup of files in var/cache

[Ivan <xh...@gmail.com>]

I just did a simple check, and it seems that the bundles are correctly
removed after the target application is undeployed, at least on the
successful scenarios. The extra 2M size is caused by a .lazy.* file in the
cache folder. and that file should be related to the store manager. Will
check it further later ...

2011/7/23 Kevan Miller <ke...@gmail.com>

>
> On Jul 22, 2011, at 10:45 PM, Ivan wrote:
>
> > Suppose you mean the OSGi cache directory ? On the server runtime, the
> size of this directory should be almost have the same size with the
> repository folder, as all the files would be copied.
>
> Right. I'm not worried about relatively static content. I am worried about
> consuming more and more disk space and never freeing it...
>
> > For the 12 GB var/cache directory size, there should be a leak issue
> somewhere,
>
> There's definitely a leak. As mentioned, I see two megabytes leaked for
> every deploy/undeploy of a very simple JSP web app.
>
> > a. In the deployment process, the deployer will deploy a temp bundle in
> the cache for builder analysis, maybe it is not uninstalled correctly due to
> some exceptions.
> > b. In the undeployment process, Geronimo did not invoke uninstall for the
> undeployed application due to unknown reason ?
> > c. Some exported classes of the undeployed application are wired with
> other bundles, so the OSGi runtime will keep it there until the framework is
> restarted. Considering the current package generation mechanism, should not
> happen.
> > Think that if we could reproduce the issue, it should be easy to fix.
> Also, I am also thinking that there is no need to package the temp bundle,
> just use the target directory. which mentioned in another thread in the mail
> list.
>
> It's extremely easy to recreate. Choose and application and deploy/undeploy
> it a few times... 'du -h var/cache' or similar to monitor disk space between
> iterations...
>
> --kevan




-- 
Ivan

Re: buildup of files in var/cache

[Kevan Miller <ke...@gmail.com>]

On Jul 22, 2011, at 10:45 PM, Ivan wrote:

> Suppose you mean the OSGi cache directory ? On the server runtime, the size of this directory should be almost have the same size with the repository folder, as all the files would be copied.

Right. I'm not worried about relatively static content. I am worried about consuming more and more disk space and never freeing it...

> For the 12 GB var/cache directory size, there should be a leak issue somewhere, 

There's definitely a leak. As mentioned, I see two megabytes leaked for every deploy/undeploy of a very simple JSP web app.

> a. In the deployment process, the deployer will deploy a temp bundle in the cache for builder analysis, maybe it is not uninstalled correctly due to some exceptions.
> b. In the undeployment process, Geronimo did not invoke uninstall for the undeployed application due to unknown reason ?
> c. Some exported classes of the undeployed application are wired with other bundles, so the OSGi runtime will keep it there until the framework is restarted. Considering the current package generation mechanism, should not happen.
> Think that if we could reproduce the issue, it should be easy to fix. Also, I am also thinking that there is no need to package the temp bundle, just use the target directory. which mentioned in another thread in the mail list.

It's extremely easy to recreate. Choose and application and deploy/undeploy it a few times... 'du -h var/cache' or similar to monitor disk space between iterations...

--kevan

Re: buildup of files in var/cache

[Ivan <xh...@gmail.com>]

Suppose you mean the OSGi cache directory ? On the server runtime, the size
of this directory should be almost have the same size with the repository
folder, as all the files would be copied. For the 12 GB var/cache directory
size, there should be a leak issue somewhere,
a. In the deployment process, the deployer will deploy a temp bundle in the
cache for builder analysis, maybe it is not uninstalled correctly due to
some exceptions.
b. In the undeployment process, Geronimo did not invoke uninstall for the
undeployed application due to unknown reason ?
c. Some exported classes of the undeployed application are wired with other
bundles, so the OSGi runtime will keep it there until the framework is
restarted. Considering the current package generation mechanism, should not
happen.
Think that if we could reproduce the issue, it should be easy to fix. Also,
I am also thinking that there is no need to package the temp bundle, just
use the target directory. which mentioned in another thread in the mail
list.


2011/7/23 Kevan Miller <ke...@gmail.com>

> A while back we had a problem with files for undeployed applications were
> building up in a servers repository. This problem was fixed. We still,
> however, have files that will build up in var/cache until the server is
> restarted. I ran a deploy/undeploy test of a simple HelloWorld jsp app. Each
> deploy/undeploy cycle will consume 2 megabytes of disk space, until the
> server is restarted. In some long running tests (but not *that* long running
> tests), I've seen the server consume 12 GB of hard drive space.
>
> Would really like to see this addressed. Any ideas?
>
> --kevan




-- 
Ivan