Tomcat clustering and nightly processes

[Rizwan Merchant <ri...@sysconet.ca>]

Hi,
We have a server in production currently and are thinking about 
implementing clustering to achieve load balancing and scalability.
Currently we have some processes that run nightly, for example

1. Users upload files that are processed later at night.
2. order updates (we retrieve all orders from db to determine if they 
have been delivered using Fedex's web service, etc).
3. Manifest submissions - generate a report from the orders of the day 
to submit to various companies via FTP

The above are just some examples. As you can see, we have a fair amount 
of stuff going on in the background during off peak hours.

My question is this. Once we implement clustering, how can the above be 
handled. The above processes cannot be running on all instances of the 
tomcat servers. For example, if a file is uploaded for processing later, 
then that file should be processed only once, but as many times as there 
are server instances (since they are all running same copies of the 
application code).

I hope my concern makes sense, and would appreciate some insight into 
this before we consider this cluster implementation further.

Thanks,
-Rizwan Merchant.

Re: Tomcat clustering and nightly processes

[Len Popp <le...@gmail.com>]

A couple of possibilities:

The nightly jobs aren't really related to serving web pages, so they
could be separate programs that are each installed on one server -
thus no synchronization needed. These programs could be implemented as
web apps if you want, so they can be administered through the web, but
each program (OrderUpdate, ReportGen, etc.) is only installed on one
server.

Alternatively, if one of the nightly jobs needs to run on several
servers in order to handle its load, you will have to implement some
sort of distributed processing. Again this doesn't necessarily mean
that the job will be implemented as part of the web cluster - it could
be a separate program running on the same (or different) servers. How
you manage the distributed processing would depend on your
requirements and your OS - that part isn't really Tomcat related.
-- 
Len

On 7/26/07, Rizwan Merchant <ri...@sysconet.ca> wrote:
>
> Hi,
> We have a server in production currently and are thinking about
> implementing clustering to achieve load balancing and scalability.
> Currently we have some processes that run nightly, for example
>
> 1. Users upload files that are processed later at night.
> 2. order updates (we retrieve all orders from db to determine if they
> have been delivered using Fedex's web service, etc).
> 3. Manifest submissions - generate a report from the orders of the day
> to submit to various companies via FTP
>
> The above are just some examples. As you can see, we have a fair amount
> of stuff going on in the background during off peak hours.
>
> My question is this. Once we implement clustering, how can the above be
> handled. The above processes cannot be running on all instances of the
> tomcat servers. For example, if a file is uploaded for processing later,
> then that file should be processed only once, but as many times as there
> are server instances (since they are all running same copies of the
> application code).
>
> I hope my concern makes sense, and would appreciate some insight into
> this before we consider this cluster implementation further.
>
> Thanks,
> -Rizwan Merchant.
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org