Architectural advice requested

[Adam Duston <ad...@gmail.com>]

We are a couple of smart and educated but
large/concurrent-application-inexperienced developers, and we've found
ourselves handling a large project of our own design without much
external assistance. We have a general architectural question, and
we're hoping that some kind experts out there could at least point us
in some direction, or tell us about resources (e.g. books) we could
consult, or else work with us for a few hours a week on a (paid)
consulting basis.

We have an application that we hope will ultimately support thousands
of persistent connections from clients. The clients are flash apps in
web browsers and the connections use RTMP, a flash protocol built on
tcp/ip. We're using bits and pieces from the Red5 project (which uses
MINA) to handle these connections -- specifically all the parts that
translate Actionscript Message Format messages to and from Java calls.

On the server we have hundreds of stateful objects which, for for the
sake of discussion, we'll call "Programs". Each Program will be
associated with hundreds of clients (connected through the Red5 setup
just described) at any given point in time. Each Program receives RPCs
from its clients, which a component translates from AMF into method
calls on the Programs. They also issue events, where a server-side
observer translates the events to AMF calls back to the clients.

We are particularly worried about scalability and some other
architectural issues. For example, how do we ensure that our
application, given this architecture, can support enormous numbers of
clients? Meaning, how can we run such an application on a server
cluster? I mentioned that we're inexperienced with creating large
concurrent applications but, more importantly, we're very imaginitive
and aggressive when it comes to implementing our ideas. If anyone
could please direct us to any resources that might help us, we would
be very grateful. Alternatively (or, in addition, rather) we would be
interested in hiring any architecture/J2EE experts on a consulting
basis for a few hours a week to give us advice and direction. Please
contact me (aduston@uchicago.edu) if you're interested.

-- 
aduston@uchicago.edu
312-375-9879

Re: Architectural advice requested

[Maarten Bosteels <mb...@gmail.com>]

On 9/9/06, John E. Conlon <jc...@verticon.com> wrote:
>
> Greetings Adam,
>
>
> Sounds like an interesting project.
>
> On Fri, 2006-09-08 at 14:25 -0500, Adam Duston wrote:
> > We are a couple of smart and educated but
> > large/concurrent-application-inexperienced developers, and we've found
> > ourselves handling a large project of our own design without much
> > external assistance. We have a general architectural question, and
> > we're hoping that some kind experts out there could at least point us
> > in some direction, or tell us about resources (e.g. books) we could
> > consult,
> Just about done reading "Java Concurrency in Practice" by Brian Goetz,
> Tim Peierls, Joshua Bloch, Joseph Bowbeer, David Holmes, and Doug Lea.
>
> This book is great. Buy one for both of you! (The authors are the
> primary members of JSR-166. Doug Lea the main author of the Java
> Concurrency Utilities.)
>
> you find what you need in there,



It is indeed a great book, I really enjoyed reading it.
It is very precise but never boring.
I would recommend it to everyone doing serious java development,
because sooner or later you will run into threading issues.

Maarten

John
>
>

Re: Architectural advice requested

["John E. Conlon" <jc...@verticon.com>]

Greetings Adam,


Sounds like an interesting project.

On Fri, 2006-09-08 at 14:25 -0500, Adam Duston wrote:
> We are a couple of smart and educated but
> large/concurrent-application-inexperienced developers, and we've found
> ourselves handling a large project of our own design without much
> external assistance. We have a general architectural question, and
> we're hoping that some kind experts out there could at least point us
> in some direction, or tell us about resources (e.g. books) we could
> consult, 
Just about done reading "Java Concurrency in Practice" by Brian Goetz,
Tim Peierls, Joshua Bloch, Joseph Bowbeer, David Holmes, and Doug Lea.

This book is great. Buy one for both of you! (The authors are the
primary members of JSR-166. Doug Lea the main author of the Java
Concurrency Utilities.)

you find what you need in there,

John

Re: Architectural advice requested

[Trustin Lee <tr...@gmail.com>]

MINA has a very general architecture for nearly all kinds of network
applications.  We usually start to understand the characteristics of the
application or service while we test it.  There's no golden rule AFAIK.  You
will have to stress-test, profile and optimize your application step by
step.  Writing a realistic regression test application is a primary concern.

Trustin

On 9/9/06, Adam Duston <ad...@gmail.com> wrote:
>
> We are a couple of smart and educated but
> large/concurrent-application-inexperienced developers, and we've found
> ourselves handling a large project of our own design without much
> external assistance. We have a general architectural question, and
> we're hoping that some kind experts out there could at least point us
> in some direction, or tell us about resources (e.g. books) we could
> consult, or else work with us for a few hours a week on a (paid)
> consulting basis.
>
> We have an application that we hope will ultimately support thousands
> of persistent connections from clients. The clients are flash apps in
> web browsers and the connections use RTMP, a flash protocol built on
> tcp/ip. We're using bits and pieces from the Red5 project (which uses
> MINA) to handle these connections -- specifically all the parts that
> translate Actionscript Message Format messages to and from Java calls.
>
> On the server we have hundreds of stateful objects which, for for the
> sake of discussion, we'll call "Programs". Each Program will be
> associated with hundreds of clients (connected through the Red5 setup
> just described) at any given point in time. Each Program receives RPCs
> from its clients, which a component translates from AMF into method
> calls on the Programs. They also issue events, where a server-side
> observer translates the events to AMF calls back to the clients.
>
> We are particularly worried about scalability and some other
> architectural issues. For example, how do we ensure that our
> application, given this architecture, can support enormous numbers of
> clients? Meaning, how can we run such an application on a server
> cluster? I mentioned that we're inexperienced with creating large
> concurrent applications but, more importantly, we're very imaginitive
> and aggressive when it comes to implementing our ideas. If anyone
> could please direct us to any resources that might help us, we would
> be very grateful. Alternatively (or, in addition, rather) we would be
> interested in hiring any architecture/J2EE experts on a consulting
> basis for a few hours a week to give us advice and direction. Please
> contact me (aduston@uchicago.edu) if you're interested.
>
> --
> aduston@uchicago.edu
> 312-375-9879
>



-- 
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP key fingerprints:
* E167 E6AF E73A CBCE EE41  4A29 544D DE48 FE95 4E7E
* B693 628E 6047 4F8F CFA4  455E 1C62 A7DC 0255 ECA6