Re: Can tomcat serve MPI (parallel) applications?

[john Matlock <jo...@gmail.com>]

As I said earlier, it has been a long time since I worked with MPI (1991 to
be exact) so I am not quite understanding your application or the software
topology.  Let me ask a few more questions.  It seems you say that a
request for processing comes in, the task it split among many hosts using
MPI, probably with mpi_put and mpi-get to handle the communications between
hosts.  In this kind of system each host only works on one part of the
problem, yet you say that the subsidiary host is to send the "final
presentation" over the web, back to presumably whoever asked for the
processing to be done, wherever in the world it is needed.  If the
subsidiary host is only working on part of the problem, how does it get the
data results from how ever many other hosts are doing their part of the
processing?  If a single host can produce the results, what is MPI being
used to do?  Why not just send the request to the server that is going to
process the data?

Again going back over the years, the system I worked on had a front end
machine that was sent the problem/application to be processed.  It broke up
the processing task to get more CPU power onto the problem.  Then the
results from the individual tasks were sent back to the front end which
assembled the independent or intermediate results into the final
presentation.  The whole system was not connected to the web (1991
remember), but I see no reason that the front end machine couldn't take in
requests from the web and then use the web to distribute these results for
which Tomcat would work fine.  But I don't understand why you would want to
use Tomcat on each individual processing node.  MPI communicators would
seem to handle this internal communication better than trying to fit these
communications onto some kind of intranet.

I don't think I am helping you very much.

John Matlock

On Mon, Nov 30, 2015 at 2:25 AM, Stefan Mayr <st...@mayr-stefan.de> wrote:

> Am 29.11.2015 um 19:24 schrieb Martijn Slouter:
>
>> Thanks for your reply, comments below:
>> ...
>> Any suggestion how I can accomplish the configuration, if I start
>> tomcat with the MPI web application using "mpirun -n 2 java ..." so
>> that only the first MPI process opens the tomcat communication ports,
>> while all other MPI processes disable their communicators?
>>
>> As an alternative I can run the MPI application as a separate server
>> (tested across 16 hosts already), and use tomcat as a (serial) client
>> to this parallel server. The disadvantage is that huge amounts of data
>> need to be processed another time instead of being served directly
>> from the MPI application.
>>
>>
> How does mpirun communicate to the started java process that it is the
> first process? Maybe it is easier to write a wrapper that can decide which
> tomcat configuration to use depending if this is your master process or
> not. If you write this wrapper in java you could use an embedded tomcat or
> jetty to startup a servlet container where needed.
>
> Regards,
>
>    Stefan Mayr
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>

Re: Can tomcat serve MPI (parallel) applications?

[Christopher Schultz <ch...@christopherschultz.net>]

John and Martijn,

On 11/30/15 7:57 PM, john Matlock wrote:
> As I said earlier, it has been a long time since I worked with MPI (1991 to
> be exact) so I am not quite understanding your application or the software
> topology.  Let me ask a few more questions.  It seems you say that a
> request for processing comes in, the task it split among many hosts using
> MPI, probably with mpi_put and mpi-get to handle the communications between
> hosts.  In this kind of system each host only works on one part of the
> problem, yet you say that the subsidiary host is to send the "final
> presentation" over the web, back to presumably whoever asked for the
> processing to be done, wherever in the world it is needed.  If the
> subsidiary host is only working on part of the problem, how does it get the
> data results from how ever many other hosts are doing their part of the
> processing?  If a single host can produce the results, what is MPI being
> used to do?  Why not just send the request to the server that is going to
> process the data?
> 
> Again going back over the years, the system I worked on had a front end
> machine that was sent the problem/application to be processed.  It broke up
> the processing task to get more CPU power onto the problem.  Then the
> results from the individual tasks were sent back to the front end which
> assembled the independent or intermediate results into the final
> presentation.  The whole system was not connected to the web (1991
> remember), but I see no reason that the front end machine couldn't take in
> requests from the web and then use the web to distribute these results for
> which Tomcat would work fine.  But I don't understand why you would want to
> use Tomcat on each individual processing node.  MPI communicators would
> seem to handle this internal communication better than trying to fit these
> communications onto some kind of intranet.
> 
> I don't think I am helping you very much.

I'm not sure why it's important to start Tomcat itself using "mpirun".
Isn't Tomcat just the interface through which clients submit jobs to the
MPI cluster? Just have your Tomcat start up normally, accept an HTTP
request, and then send that request off to the MPI server(s) using
whatever mechanism is typically used (socket, shared memory, etc.)

Tomcat itself doesn't have to participate in the whole MPI party, does it?

-chris

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org