You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@geronimo.apache.org by Patrick Kranz <pa...@immobilienscout24.de> on 2009/04/30 12:13:19 UTC
Deployment takes forever with many files
Hello list,
I´ve been working with Geronimo for some days now and I have a question
regarding the deployment process, especially about what happens behind
the scenes.
I am working on a project that has a lot of content (mainly JSPs but
also images, pdfs and so on). For the development environment this
content is reduced to a minimum, that is something around 800MB. The
live system has more data because customers can upload images and even
small videos. This data is placed on a file server and mounted via NFS
on the development machines, where the content is linked from the
docroot using symbolic links.
If I start this scenario with Tomcat, the application start takes about
3,5 minutes (really application startup, no copying of data). If I try
the same with Geronimo startup takes about 30 minutes with the system
almost being idle and enormous network traffic. If I copy all the static
content to my system and start geronimo it takes about 5 minutes.
So, my question is, what does Geronimo do in the background that causes
this startup time if the content is on a network share and can I prevent
this from happening?
Thanks in advance for every help!
Greets,
Patrick
Geronimo 2.1.4
System: Linux CentOS 5
Re: Deployment takes forever with many files
Posted by Patrick Kranz <pa...@immobilienscout24.de>.
Hi Kevan,
I tried the deployment a few more times with different settings. I also
reduced the content, so we are talking now only about JSP files and a
few GIFs (in numbers something about 12000 files with 900 being GIFs)
Basically, the deployment with all the content on my local disk took
4:42 minutes which is about one minute longer than in our working tomcat
environment.
Then I tried again with the content being on the NFS share. With the
network monitor applet in Gnome I can see that a lot of network traffic
is going on. I also configured log4j to be in debug mode for the server
and the deployer (don´t know actually, why I did not do this before :( )
but there was not very much more information. But two things got my
attention:
Right after the startup of the server I can see the following line in
the geronimo.log:
2009-05-06 07:55:49,240 DEBUG [DirectoryHotDeployer] Initialization
complete; directory scanner entering normal scan mode
I guess, this is the task that is checking the deploy folder on a
regular basis for new applications?
After starting the deployment I saw the network traffic and almost no
cpu usage. Something around 1GB went over the wire until something more
happened. Then the deploy.log showed this line:
2009-05-06 08:16:48,560 DEBUG [JspModuleBuilderExtension] scanModule(
de.is24/Scout3/1.0/car ): Entry
And again, about 300MG network traffic was done. After done the
application started. The overall time was 33:04 minutes.
After that I tried without the inPlace option. This took even longer.
After 44:30 minutes the deployment stopped reporting that the sum file
already exists. Maybe there was something left from the previous
deployment but you can clearly see, that it takes much longer and the
network traffic was somewhere beyond 2GB.
I am currently evaluating if Geronimo could be our next application
container and this is the blocker since we need to be able to integrate
it into the development process and obviously it is not acceptable for a
developer to wait half an hour for the deployment and start of the
application.
Greets,
Patrick
Kevan Miller schrieb:
>
> On May 5, 2009, at 3:19 AM, Patrick Kranz wrote:
>
>> No, it´s not only images and videos. There are also a lot of JSPs in
>> the structure. The deepest directory structure should be something
>> around 5 or 6 levels but find -type d returns 8904 folders. So, I
>> guess you could say it is a rather complex structure.
>
> Interesting... So, what happens if you reduce the amount of static
> content? e.g. fewer video/pdf files? Does the behavior change if you
> don't use the --inplace option?
>
> --kevan
Re: Deployment takes forever with many files
Posted by Kevan Miller <ke...@gmail.com>.
On May 5, 2009, at 3:19 AM, Patrick Kranz wrote:
> No, it´s not only images and videos. There are also a lot of JSPs in
> the structure. The deepest directory structure should be something
> around 5 or 6 levels but find -type d returns 8904 folders. So, I
> guess you could say it is a rather complex structure.
Interesting... So, what happens if you reduce the amount of static
content? e.g. fewer video/pdf files? Does the behavior change if you
don't use the --inplace option?
--kevan
Re: Deployment takes forever with many files
Posted by Patrick Kranz <pa...@immobilienscout24.de>.
No, it´s not only images and videos. There are also a lot of JSPs in the
structure. The deepest directory structure should be something around 5
or 6 levels but find -type d returns 8904 folders. So, I guess you could
say it is a rather complex structure.
Jack Cai schrieb:
> Are the data all static contents such as images, videos etc.? Is there
> a deep directory structure?
>
> -Jack
>
> 2009/5/4 Patrick Kranz <patrick.kranz@immobilienscout24.de
> <ma...@immobilienscout24.de>>
>
> Hi Ivan,
>
> sorry it took me so long to answer.
>
> No, the log files don“t give any hint about what is going on. It
> only prints messages coming from our application and those
> messages are the same for both cases (started with all data on the
> local disk and started with data located on an NFS share).
>
> Patrick
>
>
>
> Ivan schrieb:
>
> Does Geronimo output anything in the deployment process, could
> you please show us those logs ?
> Ivan
>
> 2009/4/30 Patrick Kranz <patrick.kranz@immobilienscout24.de
> <ma...@immobilienscout24.de>
> <mailto:patrick.kranz@immobilienscout24.de
> <ma...@immobilienscout24.de>>>
>
>
> Sorry, I forgot to mention: I am doing a deployment with the
> --inPlace parameter, so the application is not being copied
> to the
> Geronimo repository.
>
> Greets,
> Patrick
>
> Patrick Kranz schrieb:
>
> Hello list,
>
> I“ve been working with Geronimo for some days now and I
> have a
> question regarding the deployment process, especially about
> what happens behind the scenes.
>
> I am working on a project that has a lot of content (mainly
> JSPs but also images, pdfs and so on). For the development
> environment this content is reduced to a minimum, that is
> something around 800MB. The live system has more data
> because
> customers can upload images and even small videos. This
> data
> is placed on a file server and mounted via NFS on the
> development machines, where the content is linked from the
> docroot using symbolic links.
>
> If I start this scenario with Tomcat, the application start
> takes about 3,5 minutes (really application startup, no
> copying of data). If I try the same with Geronimo startup
> takes about 30 minutes with the system almost being
> idle and
> enormous network traffic. If I copy all the static
> content to
> my system and start geronimo it takes about 5 minutes.
>
> So, my question is, what does Geronimo do in the background
> that causes this startup time if the content is on a
> network
> share and can I prevent this from happening?
>
> Thanks in advance for every help!
>
> Greets,
> Patrick
>
>
> Geronimo 2.1.4
> System: Linux CentOS 5
>
>
>
>
>
> --
> Ivan
>
>
>
Re: Deployment takes forever with many files
Posted by Jack Cai <gr...@gmail.com>.
Are the data all static contents such as images, videos etc.? Is there a
deep directory structure?
-Jack
2009/5/4 Patrick Kranz <pa...@immobilienscout24.de>
> Hi Ivan,
>
> sorry it took me so long to answer.
>
> No, the log files don“t give any hint about what is going on. It only
> prints messages coming from our application and those messages are the same
> for both cases (started with all data on the local disk and started with
> data located on an NFS share).
>
> Patrick
>
>
>
> Ivan schrieb:
>
>> Does Geronimo output anything in the deployment process, could you please
>> show us those logs ?
>> Ivan
>>
>> 2009/4/30 Patrick Kranz <patrick.kranz@immobilienscout24.de <mailto:
>> patrick.kranz@immobilienscout24.de>>
>>
>>
>> Sorry, I forgot to mention: I am doing a deployment with the
>> --inPlace parameter, so the application is not being copied to the
>> Geronimo repository.
>>
>> Greets,
>> Patrick
>>
>> Patrick Kranz schrieb:
>>
>> Hello list,
>>
>> I“ve been working with Geronimo for some days now and I have a
>> question regarding the deployment process, especially about
>> what happens behind the scenes.
>>
>> I am working on a project that has a lot of content (mainly
>> JSPs but also images, pdfs and so on). For the development
>> environment this content is reduced to a minimum, that is
>> something around 800MB. The live system has more data because
>> customers can upload images and even small videos. This data
>> is placed on a file server and mounted via NFS on the
>> development machines, where the content is linked from the
>> docroot using symbolic links.
>>
>> If I start this scenario with Tomcat, the application start
>> takes about 3,5 minutes (really application startup, no
>> copying of data). If I try the same with Geronimo startup
>> takes about 30 minutes with the system almost being idle and
>> enormous network traffic. If I copy all the static content to
>> my system and start geronimo it takes about 5 minutes.
>>
>> So, my question is, what does Geronimo do in the background
>> that causes this startup time if the content is on a network
>> share and can I prevent this from happening?
>>
>> Thanks in advance for every help!
>>
>> Greets,
>> Patrick
>>
>>
>> Geronimo 2.1.4
>> System: Linux CentOS 5
>>
>>
>>
>>
>>
>> --
>> Ivan
>>
>
>
Re: Deployment takes forever with many files
Posted by Patrick Kranz <pa...@immobilienscout24.de>.
Hi Ivan,
sorry it took me so long to answer.
No, the log files don´t give any hint about what is going on. It only
prints messages coming from our application and those messages are the
same for both cases (started with all data on the local disk and started
with data located on an NFS share).
Patrick
Ivan schrieb:
> Does Geronimo output anything in the deployment process, could you
> please show us those logs ?
> Ivan
>
> 2009/4/30 Patrick Kranz <patrick.kranz@immobilienscout24.de
> <ma...@immobilienscout24.de>>
>
> Sorry, I forgot to mention: I am doing a deployment with the
> --inPlace parameter, so the application is not being copied to the
> Geronimo repository.
>
> Greets,
> Patrick
>
> Patrick Kranz schrieb:
>
> Hello list,
>
> I´ve been working with Geronimo for some days now and I have a
> question regarding the deployment process, especially about
> what happens behind the scenes.
>
> I am working on a project that has a lot of content (mainly
> JSPs but also images, pdfs and so on). For the development
> environment this content is reduced to a minimum, that is
> something around 800MB. The live system has more data because
> customers can upload images and even small videos. This data
> is placed on a file server and mounted via NFS on the
> development machines, where the content is linked from the
> docroot using symbolic links.
>
> If I start this scenario with Tomcat, the application start
> takes about 3,5 minutes (really application startup, no
> copying of data). If I try the same with Geronimo startup
> takes about 30 minutes with the system almost being idle and
> enormous network traffic. If I copy all the static content to
> my system and start geronimo it takes about 5 minutes.
>
> So, my question is, what does Geronimo do in the background
> that causes this startup time if the content is on a network
> share and can I prevent this from happening?
>
> Thanks in advance for every help!
>
> Greets,
> Patrick
>
>
> Geronimo 2.1.4
> System: Linux CentOS 5
>
>
>
>
>
> --
> Ivan
Re: Deployment takes forever with many files
Posted by Ivan <xh...@gmail.com>.
Does Geronimo output anything in the deployment process, could you please
show us those logs ?Ivan
2009/4/30 Patrick Kranz <pa...@immobilienscout24.de>
> Sorry, I forgot to mention: I am doing a deployment with the --inPlace
> parameter, so the application is not being copied to the Geronimo
> repository.
>
> Greets,
> Patrick
>
> Patrick Kranz schrieb:
>
> Hello list,
>>
>> I´ve been working with Geronimo for some days now and I have a question
>> regarding the deployment process, especially about what happens behind the
>> scenes.
>>
>> I am working on a project that has a lot of content (mainly JSPs but also
>> images, pdfs and so on). For the development environment this content is
>> reduced to a minimum, that is something around 800MB. The live system has
>> more data because customers can upload images and even small videos. This
>> data is placed on a file server and mounted via NFS on the development
>> machines, where the content is linked from the docroot using symbolic links.
>>
>> If I start this scenario with Tomcat, the application start takes about
>> 3,5 minutes (really application startup, no copying of data). If I try the
>> same with Geronimo startup takes about 30 minutes with the system almost
>> being idle and enormous network traffic. If I copy all the static content to
>> my system and start geronimo it takes about 5 minutes.
>>
>> So, my question is, what does Geronimo do in the background that causes
>> this startup time if the content is on a network share and can I prevent
>> this from happening?
>>
>> Thanks in advance for every help!
>>
>> Greets,
>> Patrick
>>
>>
>> Geronimo 2.1.4
>> System: Linux CentOS 5
>>
>
>
--
Ivan
Re: Deployment takes forever with many files
Posted by Patrick Kranz <pa...@immobilienscout24.de>.
Sorry, I forgot to mention: I am doing a deployment with the --inPlace
parameter, so the application is not being copied to the Geronimo
repository.
Greets,
Patrick
Patrick Kranz schrieb:
> Hello list,
>
> I´ve been working with Geronimo for some days now and I have a
> question regarding the deployment process, especially about what
> happens behind the scenes.
>
> I am working on a project that has a lot of content (mainly JSPs but
> also images, pdfs and so on). For the development environment this
> content is reduced to a minimum, that is something around 800MB. The
> live system has more data because customers can upload images and even
> small videos. This data is placed on a file server and mounted via NFS
> on the development machines, where the content is linked from the
> docroot using symbolic links.
>
> If I start this scenario with Tomcat, the application start takes
> about 3,5 minutes (really application startup, no copying of data). If
> I try the same with Geronimo startup takes about 30 minutes with the
> system almost being idle and enormous network traffic. If I copy all
> the static content to my system and start geronimo it takes about 5
> minutes.
>
> So, my question is, what does Geronimo do in the background that
> causes this startup time if the content is on a network share and can
> I prevent this from happening?
>
> Thanks in advance for every help!
>
> Greets,
> Patrick
>
>
> Geronimo 2.1.4
> System: Linux CentOS 5