You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Andrew Ho <an...@tellme.com> on 2001/10/19 04:11:04 UTC
Excellent article on Apache/mod_perl at eToys
Hello,
I checked the list archives and it didn't look like this had been posted yet.
For those of you who haven't seen it yet... a great read on perl.com about
the Apache/mod_perl setup at eToys, co-authored by our own mod_perl
regular contributer Perrin Harkins.
http://www.perl.com/pub/a/2001/10/17/etoys.html
Humbly,
Andrew
----------------------------------------------------------------------
Andrew Ho http://www.tellme.com/ andrew@tellme.com
Engineer info@tellme.com Voice 650-930-9062
Tellme Networks, Inc. 1-800-555-TELL Fax 650-930-9101
----------------------------------------------------------------------
Re: [OT] Excellent article on Apache/mod_perl at eToys
Posted by Jeremy Howard <jh...@fastmail.fm>.
> By the way, I noticed he documented still another gotcha due to a sneaky
> closure. Does anyone know off-hand whether the Perl 6 folks plan to
> change the closure syntax so they don't sneak into your code this way?
>
Closures, references, et al are being thoroughly revised, such that there
will be far fewer gotchas. However, these kinds of memory leaks will be less
of an issue anyway because Perl 6 will be garbage collected. Also, proper
exception handling will be built into the language.
PS: I haven't seen many of the regulars here on the Perl 6 lists. There's
both language design and internals streams currently happening, so
everyone's interests and skills should be catered for. Please have a look at
http://dev.perl.org and consider getting involved. It's interesting work and
there's a similarly positive and intellectual spirit on the P6 lists to what
we experience here.
Re: Excellent article on Apache/mod_perl at eToys
Posted by "Keith G. Murphy" <ke...@mindspring.com>.
Andrew Ho wrote:
>
> Hello,
>
> I checked the list archives and it didn't look like this had been posted yet.
> For those of you who haven't seen it yet... a great read on perl.com about
> the Apache/mod_perl setup at eToys, co-authored by our own mod_perl
> regular contributer Perrin Harkins.
>
> http://www.perl.com/pub/a/2001/10/17/etoys.html
>
That's so awesome, the single best piece of technical writing on the web
I can remember in a long time. Thanks for sharing.
By the way, I noticed he documented still another gotcha due to a sneaky
closure. Does anyone know off-hand whether the Perl 6 folks plan to
change the closure syntax so they don't sneak into your code this way?
I wonder how many mod_perl authors have never been burned by one?
Re: Excellent article on Apache/mod_perl at eToys
Posted by Nathan Torkington <gn...@oreilly.com>.
I suppose I should point out that perl.com is always interested in
mod_perl articles. If you've learned lessons that others could
benefit from, contact the perl.com editor, Simon Cozens
<si...@netthink.co.uk>.
Nat
Re: Excellent article on Apache/mod_perl at eToys
Posted by Tatsuhiko Miyagawa <mi...@edge.co.jp>.
On Thu, 18 Oct 2001 19:11:04 -0700 (PDT)
Andrew Ho <an...@tellme.com> wrote:
> I checked the list archives and it didn't look like this had been posted yet.
> For those of you who haven't seen it yet... a great read on perl.com about
> the Apache/mod_perl setup at eToys, co-authored by our own mod_perl
> regular contributer Perrin Harkins.
>
> http://www.perl.com/pub/a/2001/10/17/etoys.html
I've translated this article into Japanese, and you can get it on
http://bulknews.net/lib/doc-ja/etoys.ja.html
http://bulknews.net/lib/doc-ja/etoys.ja.pod
Just wanted to let you know. Thanks for the excellent article, Perrin!
--
Tatsuhiko Miyagawa <mi...@bulknews.net>
Re: [OT] Re: Excellent article on Apache/mod_perl at eToys
Posted by Perrin Harkins <pe...@elem.com>.
> Wasn't this "seasonal rush" at least partly caused by the so-called toywar
> (www.toywar.com) between eToys.com (the online retailer) and etoy.com
> (the art group)?
There were a lot of DoS attacks (and some even uglier, nastier ones) in 1999
as a result of that. Most of that was dealt with through standard access
control stuff, plus the throttling code that was mentioned in the article.
The 2000 rush, which we built this new system for, was real customer
traffic, although there's always some joker with a bot trying to buy all the
PlayStation 2 units.
- Perrin
[OT] Re: Excellent article on Apache/mod_perl at eToys
Posted by Thomas Klausner <do...@zsi.at>.
Hi!
On Fri, Oct 19, 2001 at 09:59:18AM -0400, Drew Taylor wrote:
> What I found most interesting was the detail of the extensive caching which
> was implemented to survive the seasonal rush.
Wasn't this "seasonal rush" at least partly caused by the so-called toywar
(www.toywar.com) between eToys.com (the online retailer) and etoy.com
(the art group)?
As far as I remember this incident, eToys.com sued etoy.com (who were
holding this domain since 1994) because of trademark delusion etc., which
caused one of the first distrubuted DOS attacks against etoys.com.
see:
http://rtmark.com/etoymain.html
--
D_OMM
O_xyderkes
M_echanen http://domm.zsi.at
M_asteuei
Re: [Templates] Re: Excellent article on Apache/mod_perl at
eToys
Posted by Drew Taylor <dr...@drewtaylor.com>.
What I found most interesting was the detail of the extensive caching which
was implemented to survive the seasonal rush. I look forward to working on
a project one day that is big enough to warrant such a system. All in all,
a most excellent and informative read.
Thanks again for everything you've personally done for the community! I
look forward to seeing those graphics. :-)
At 09:49 AM 10/19/01 -0400, Perrin Harkins wrote:
>Thanks to all for the kind words. This article actually went up a little
>bit before it was supposed to, and there should be a revision going up soon
>with some grammatical fixes and a set of graphics to illustrate parts of it.
>I'll post a follow-up when that happens in case anyone wants to go and look
>at the pretty pictures.
>
>While we're on the subject, thanks to everyone who contributed to the many
>open source projects that we used. We couldn't have done it without you.
Drew Taylor JA[P|m_p|SQL]H
http://www.drewtaylor.com/ Just Another Perl|mod_perl|SQL Hacker
mailto:drew@drewtaylor.com *** God bless America! ***
ICQ: 135298242
Re: Excellent article on Apache/mod_perl at eToys
Posted by Perrin Harkins <pe...@elem.com>.
Thanks to all for the kind words. This article actually went up a little
bit before it was supposed to, and there should be a revision going up soon
with some grammatical fixes and a set of graphics to illustrate parts of it.
I'll post a follow-up when that happens in case anyone wants to go and look
at the pretty pictures.
While we're on the subject, thanks to everyone who contributed to the many
open source projects that we used. We couldn't have done it without you.
- Perrin
Re: Excellent article on Apache/mod_perl at eToys
Posted by Gunther Birznieks <gu...@extropia.com>.
Yeah we really enjoyed it over here. I think it's really excellent to get
great advocacy articles about Perl and surrounding products (mod_perl, TT,
etc) powering real businesses with real high powered needs out there like
this one.
:)
At 04:40 PM 10/19/2001, Greg Cope wrote:
>Andrew Ho wrote:
> >
> > Hello,
> >
> > I checked the list archives and it didn't look like this had been
> posted yet.
> > For those of you who haven't seen it yet... a great read on perl.com about
> > the Apache/mod_perl setup at eToys, co-authored by our own mod_perl
> > regular contributer Perrin Harkins.
> >
> > http://www.perl.com/pub/a/2001/10/17/etoys.html
> >
>
>Yup, this is an excellent read. Thanks Perrin.
>
>Greg
__________________________________________________
Gunther Birznieks (gunther.birznieks@eXtropia.com)
eXtropia - The Open Web Technology Company
http://www.eXtropia.com/
Re: Excellent article on Apache/mod_perl at eToys
Posted by Greg Cope <gj...@rubberplant.freeserve.co.uk>.
Andrew Ho wrote:
>
> Hello,
>
> I checked the list archives and it didn't look like this had been posted yet.
> For those of you who haven't seen it yet... a great read on perl.com about
> the Apache/mod_perl setup at eToys, co-authored by our own mod_perl
> regular contributer Perrin Harkins.
>
> http://www.perl.com/pub/a/2001/10/17/etoys.html
>
Yup, this is an excellent read. Thanks Perrin.
Greg
Re: Excellent article on Apache/mod_perl at eToys
Posted by Steven Lembark <le...@wrkhors.com>.
> Exactly- the statically typed languages I am familiar with have a casting
> mechanism to utterly subvert compile-time type checks. While static typing
> allows better compile-time optimization, it's value as a debugging
> mechanism is near the bottom of the list of advantages for engineering
> a large project. If interface guarantees are important between Perl
> objects, I'd have a look at Class::Contract.
One nice thing about which is that the "contract" portion can be
essentially turned off in production. Gives you the best of both
worlds: strong checking during development w/ good speed in productin.
>
>> > Compile time checking can definitely be a friend of yours especially
>> > when dealing with large systems. But it's also a friend that's
>> > judgemental (strongly typed) so he's a pain to drag along to a
>> > party....
>>
>> To me, strongly vs weakly typed is less descriptive than statically vs
>> dynamically typed.
>
> To me, it is utterly nondescriptive in a PHB buzzwordy way, whereas
> static vs. dynamic typing is meaningful (and what I think most people
> really mean by the former).
Comparing Perl to other OO languages runs into the fact that
only Perl uses a fairly high-level object as its basic storage
unit (scalar). The first thing most people write/buy for the
strongly typed languages is a library of contaner classes: things
that subvert the strong checking of low-level types. The second
thing that usually happens is that the low-level types get their
base operations overloaded via container objects.
Net result is minimal -- at best -- type checking.
The problem isn't really that scalars are weakly typed, since
all you can deal with are scalars. If programmers bothered to
write checks in their code (e.g., $foo =~ /^\d$/ or croak "NAN")
then Perl will do as well or better than most languages. Coders
are too used to having the low-level type checking around as
a crutch, so they don't. Implementing code that uses Class::Contract
forces you to declare [perhaps even analyze?] not only data types
but expectations, and ends up causing people to do a much better
job of checking then most other languages even allow.
--
Steven Lembark 2930 W. Palmer
Workhorse Computing Chicago, IL 60647
+1 800 762 1582
Re: Neo-Classical Transaction Processing
Posted by Rob Nagler <na...@bivio.net>.
Perrin Harkins writes:
> The trouble here should be obvious: sooner or later it becomes hard to scale
> the database. You can cache the read-only data, but the read/write data
> isn't so simple.
Good point. Fortunately, the problem isn't new.
> Theoretically, the big players like Oracle and DB2 offer clustering
> solutions to deal with this, but they don't seem to get used very
> often.
Oracle was built on an SMP assumption. They added clustering later.
It doesn't scale well, which is probably why you haven't heard of
people using their parallel server solutions. I don't know much about
DB2, but I'm pretty sure it assumes shared memory. Tandem's Non-Stop
SQL is a shared nothing architecture. It scales well, but isn't cheap
to walk in the door.
> Other sites find ways to divide their traffic up (users 1 - n go to
> this database, n - m go to that one, etc.)
Partitioning is a great way to get scalability, if you can do it.
> However, you can usually scale up enough just by getting a bigger
> box to run your database on until you reach the reach the realm of
> Yahoo and Amazon, so this doesn't become an issue for most sites.
I agree. This is why I think Apache/mod_perl is a great solution for
the majority of web apps. The scaling issues supposedly being solved
by J2EE don't exist.
On another note, one of the ways to make sure your database scales
better is to keep the database as simple as possible. I've seen a lot
of solutions which rely on stored procedures to "get performance".
All this does is make the database slower and more of a bottleneck.
> But how can you actually make a shared nothing system for a commerce web
> site? They may not be sharing local memory, but you'll need read/write
> access to the same data, which means shared locking and waiting somewhere
> along the line.
I meant "shared nothing" in the sense of multiprocessor architectures.
SMP (symmetric multiprocessing) relies on shared memory. This is the
J2EE/E10K model. "shared nothing" is the Neo Classical model. Really
these are NUMAs (non-uniform memory architecture), because most
servers are SMPs. Here's a classic from Stonebraker on the subject:
http://db.cs.berkeley.edu/papers/hpts85-nothing.pdf
DeWitt has a lot of papers on parallelism and distributed db design:
http://www.cs.wisc.edu/~dewitt/
Cheers,
Rob
Re: Neo-Classical Transaction Processing (was Re: Excellent article...)
Posted by Perrin Harkins <pe...@elem.com>.
> The mod_perl servers are the work horses, just like the custom
> servers. In a classical OLTP system, the customer servers are
> stateless, that is, if a server goes down, the TM/mod_proxy server
> routes around it. (The TM rollsback any transactions and restarts the
> entire request, which is interesting but irrelevant for this
> discussion.) If the work servers are fully loaded, you simply add
> more hardware. If all the servers are stateless, the system scales
> linearly, i.e. the number of servers is directly proportional to the
> number of users that can be served.
The mod_perl servers in the eToys setup were essentially stateless. They
had some local caching of user-specific data which made it desirable to go
back to the same one if possible for the same user, but it would still work
if the user switched to another server, since all state was stored in the
central database.
The trouble here should be obvious: sooner or later it becomes hard to scale
the database. You can cache the read-only data, but the read/write data
isn't so simple. Theoretically, the big players like Oracle and DB2 offer
clustering solutions to deal with this, but they don't seem to get used very
often. Other sites find ways to divide their traffic up (users 1 - n go to
this database, n - m go to that one, etc.) However, you can usually scale
up enough just by getting a bigger box to run your database on until you
reach the reach the realm of Yahoo and Amazon, so this doesn't become an
issue for most sites.
> If the whole multi-threaded server ever has to wait on
> a single shared resource (e.g. the garbage collector), all
> simultaneous requests in progress lose. This doesn't happen if the
> work servers are single threaded, i.e. "shared nothing". You can
> easily model the resources required to process a request and no
> request has to wait on another, except to get CPU.
But how can you actually make a shared nothing system for a commerce web
site? They may not be sharing local memory, but you'll need read/write
access to the same data, which means shared locking and waiting somewhere
along the line.
> The front-end servers are different. They do no work. Their job is
> switching requests/responses. Some of the requests are going directly
> to the OS (e.g. get an icon) and others are going to the work servers.
> The front-end servers "wait" most of the time. Here I buy into the
> the asynchronous I/O model of serving multiple requests and not the
> lightweight process model.
I do feel that servers built on non-blocking I/O would have been much more
efficient on the eToys front end. I've seen how well thttpd can do. But
since these boxes already scaled well beyond our needs with basic
Apache/mod_proxy, we didn't bother to look for something more complicated.
If I ran into problems with this now, I would probably look at Red Hat's Tux
server.
- Perrin
Neo-Classical Transaction Processing (was Re: Excellent article...)
Posted by Rob Nagler <na...@bivio.net>.
Joe Schaefer writes:
> > experience, the only way to build large scale systems is with
> > stateless, single-threaded servers.
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> Could you say some more about what you mean by this? Do you mean
> something like
>
> "use a functional language (like Haskell or Scheme), rather
> than an imperative language (like C, Java, Perl ...)",
Not exactly, but this is an interesting topic.
> or are you talking more about the application's platform and design
> (e.g. http://www.kegel.com/c10k.html )?
This article addresses path length, which is the "single-threaded"
part. Scalability is not addressed. Both parts are important to
understand when you build enterprise systems.
I changed the subject to Neo-Classical Transaction Processing which is
the way I look at web applications. If you'll bear with me, I can
explain the Neo and Classical parts with a picture. Here's a
classical transaction processing system:
T ______
e +----------------+ +--------------+ / \
r -----| |------| |-----------\______/
m | Transaction | | +--------------+ | |
i -----| Monitor |---------| |--------| |
n | | | | +--------------+ | DB |
a -----| |------------| |-----| |
l | | | | | Custom | | |
s +----------------+ +--| | Servers | | |
| | | | |
+--| | | |
| | \______/
+--------------+
Now here's a a typical (large) Apache/mod_perl setup (Neo-Classical):
______
B +----------------+ +--------------+ / \
r -----| |------| |-----------\______/
o | Apache | | +--------------+ | |
w -----| mod_proxy |---------| |--------| |
s | | | | +--------------+ | DB |
e -----| |------------| |-----| |
r | | | | | Apache | | |
s +----------------+ +--| | mod_perl | | |
| | | | |
+--| | | |
| | \______/
+--------------+
The browsers are connected to a fast IP router(s), equivalent to
yesteryears I/O processor. The mod_proxy servers are simple switches,
just like the TM. (Unlike the TM, front-ends don't manage the
transactions.) It's usually a given that the front-ends are
stateless. Their job is dynamically routing for load-balancing and
reliability. They also serve icons and other stateless files. If a
front-end crashes, the IP router ignores it and goes to another
front-end. No harm done. The IP router also balances the load,
something that isn't provided by classical I/O processors.
The mod_perl servers are the work horses, just like the custom
servers. In a classical OLTP system, the customer servers are
stateless, that is, if a server goes down, the TM/mod_proxy server
routes around it. (The TM rollsback any transactions and restarts the
entire request, which is interesting but irrelevant for this
discussion.) If the work servers are fully loaded, you simply add
more hardware. If all the servers are stateless, the system scales
linearly, i.e. the number of servers is directly proportional to the
number of users that can be served.
That's the stateless part. Threading is the other issue. Should the
servers (mod_proxy or mod_perl) be threaded. In classical OLTPs, the
work servers are single threaded (as in one request at a time) and the
TM handles multiple simultaneous requests, but isn't "multi-threaded"
(in the Java sense).
The work server can be thought of as a resource unit. Usually it
represents a fair bit of code and takes up a chunk of memory. It can
only process so many requests per unit time. If the work server is
multi-threaded, it is harder to manage resources and configure for
peak load. In the single threaded model, each work server (process)
is a reservation for the resources it needs for one request. In a
multi-threaded model, the resource reservations are less clear. It
might have a shared database connection pool or it might have two
simultaneous requests which need more memory. The meaning of capacity
becomes fuzzy. If the whole multi-threaded server ever has to wait on
a single shared resource (e.g. the garbage collector), all
simultaneous requests in progress lose. This doesn't happen if the
work servers are single threaded, i.e. "shared nothing". You can
easily model the resources required to process a request and no
request has to wait on another, except to get CPU.
The front-end servers are different. They do no work. Their job is
switching requests/responses. Some of the requests are going directly
to the OS (e.g. get an icon) and others are going to the work servers.
The front-end servers "wait" most of the time. Here I buy into the
the asynchronous I/O model of serving multiple requests and not the
lightweight process model.
The asynch I/O model (variantions of which are discussed in the c10k
paper) is a way to achieve asynchrony much more efficiently than
threads. Instead of reserving a stack and some OS space for each
client, you reserve a block of memory in a single-threaded process.
The OS is not involved. The main loop evaluates which I/O ports are
ready and processes them serially. With preemptable threads, as in
Java, the processor is used inefficiently, because there's an
unnecessary and unpredictable process switching cost. The asynch I/O
model does not incur this overhead, which can be significant wrt to
the cost of processing each I/O.
Preemptable threading is an "easy" solution to the asynchrony problem,
but it is resource intensive. What's scary about modern distributed
application servers is that they hide the asynchrony problem. This
works fine until system must scale to hundreds and thousands of users.
It becomes very hard to determine how the server will behave at and
above peak load.
Rob
Re: Excellent article on Apache/mod_perl at eToys
Posted by Joe Schaefer <jo...@sunstarsys.com>.
Rob Nagler <na...@bivio.net> writes:
> Gunther wrote:
> > If you do not have a strongly typed system, then when you break
> > apart and rebuild another part of the system, Perl may very well not
> > complain when a subtle bug comes up because of the fact that it is
> > not strongly typed. Whereas Java will complain quite often and
> > usually early with compile time checking.
>
> I don't think there's an "objective" view about this. I also think
> the "it compiles, so it works" attitude is dangerous. You don't know
> it works until your unit and acceptance tests pass. I've been in too
> many shops where the "nightly build" was the extent of the quality
> assurance program.
Exactly- the statically typed languages I am familiar with have a casting
mechanism to utterly subvert compile-time type checks. While static typing
allows better compile-time optimization, it's value as a debugging
mechanism is near the bottom of the list of advantages for engineering
a large project. If interface guarantees are important between Perl
objects, I'd have a look at Class::Contract.
> > Compile time checking can definitely be a friend of yours especially
> > when dealing with large systems. But it's also a friend that's
> > judgemental (strongly typed) so he's a pain to drag along to a party....
>
> To me, strongly vs weakly typed is less descriptive than statically vs
> dynamically typed.
To me, it is utterly nondescriptive in a PHB buzzwordy way, whereas
static vs. dynamic typing is meaningful (and what I think most people
really mean by the former).
[...]
> Here's a strong statement: Threads have no place in information
> systems. The NYSE is run on Tandem boxes. Tandem's OS does not have
> threads. The NYSE can process over a billion stock transactions a
> day. The EJB spec says you can't fire off threads in a bean. I think
> there's a reason for the way these systems have been architected.
>
> Threads are a false economy for systems which have to scale. As some
> people have joked, Java is Sun's way of moving E10K servers. SMP
> doesn't scale. As soon as you outgrow your box, you are hosed. A
> shared memory cache doesn't work well over the wire. In my
> experience, the only way to build large scale systems is with
> stateless, single-threaded servers.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Could you say some more about what you mean by this? Do you mean
something like
"use a functional language (like Haskell or Scheme), rather
than an imperative language (like C, Java, Perl ...)",
or are you talking more about the application's platform and design
(e.g. http://www.kegel.com/c10k.html )?
--
Joe Schaefer
Re: Excellent article on Apache/mod_perl at eToys
Posted by Rob Nagler <na...@bivio.net>.
Gunther wrote:
> If you do not have a strongly typed system, then when you break apart and
> rebuild another part of the system, Perl may very well not complain when a
> subtle bug comes up because of the fact that it is not strongly typed.
> Whereas Java will complain quite often and usually early with compile time
> checking.
I don't think there's an "objective" view about this. I also think
the "it compiles, so it works" attitude is dangerous. You don't know
it works until your unit and acceptance tests pass. I've been in too
many shops where the "nightly build" was the extent of the quality
assurance program.
> Compile time checking can definitely be a friend of yours especially when
> dealing with large systems. But it's also a friend that's judgemental
> (strongly typed) so he's a pain to drag along to a party....
To me, strongly vs weakly typed is less descriptive than statically vs
dynamically typed. For example, Java is missing "undef". It has NULL
for pointers, but not "undef" for ints, chars, booleans, etc. Large
systems often have unexpected initialization order problems which are
not handled well by Java due to this missing feature.
> Java's support for multi-threading makes writing servers feel fairly
> trivial with no jumping through IPC::Shared memory stuff hoops to get
> shared memory caches and the like.. you just synchronize on global data
> structures.
It's important to define the problem space for this discussion. I
think Perl is really good for information systems, be they enterprise
or not. I probably wouldn't program a real-time system in Perl. I
might program it in Java.
Here's a strong statement: Threads have no place in information
systems. The NYSE is run on Tandem boxes. Tandem's OS does not have
threads. The NYSE can process over a billion stock transactions a
day. The EJB spec says you can't fire off threads in a bean. I think
there's a reason for the way these systems have been architected.
Threads are a false economy for systems which have to scale. As some
people have joked, Java is Sun's way of moving E10K servers. SMP
doesn't scale. As soon as you outgrow your box, you are hosed. A
shared memory cache doesn't work well over the wire. In my
experience, the only way to build large scale systems is with
stateless, single-threaded servers.
Rob
Re: Excellent article on Apache/mod_perl at eToys
Posted by Perrin Harkins <pe...@elem.com>.
Matthew Kennedy wrote
>Is there a Perl equivalent to Struts? Mason seems to come close if you
>keep yourself disciplined somewhat.
>
I mentioned a couple of tools in the article that are specifically aimed
at MVC: OpenInteract, and Apache::PageKit.
Actually, I think it's pretty easy to replace Struts. Think about what
it offers. First, there's a set of taglibs for writing templates
without embedding Java code. Template Toolkit blows it away in this
area. (I once suggested to one of the authors of Struts that
tags like <logic:greaterThan parameter="number" value="7"> are a bit
verbose. He told me to buy DreamWeaver.)
It doesn't add much beyond advice in the model area. The controller is
basically just a dispatch table which can easily be replaced with
Apache::Dispatch or even a well-written httpd.conf.
To be fair, it does some things to try and make forms easier to deal
with. You'd need to get OpenInteract or PageKit to replace that stuff,
or write some glue code for the form validators and such on CPAN.
Of course you could use Mason, but most Mason users aren't aiming for a
strict MVC separation and it's very easy to cheat when your templates
are written in in-line perl.
- Perrin
Re: Excellent article on Apache/mod_perl at eToys
Posted by Matthew Kennedy <mk...@opushealthcare.com>.
On Tue, 2001-10-23 at 10:09, Gunther Birznieks wrote:
> At 08:46 PM 10/23/2001, Rob Nagler wrote:
> >I don't quite understand the difference between worflow in the front-end and
> >workflow in the back-end. They both change. The danger of making one part
> >of the system easier to change is that people tend to "cheat". They won't
> >put the business logic in the back-end if it takes twice as long.
>
> Yes, people can cheat. But generally the workflow that is most likely to
> change is the UI based workflow. UI isn't just about templates, it's a lot
> about how you go from one screen to the next, or not.
>
> Actually business logic is a bit of a misnomer because there is plenty of
> business logic in the front-end from your choice of fields to the
> javascript to the UI workflow.
I agree. Basically there's front end business logic which defined the
work flow of the client's end (web server/web browser etc). And then
business logic in the middle tier. I've been using Apache/Jakarta Struts
for the front end business stuff, a row of "session beans" (java speak
for purely process related, not data related objects) behind the client
to model work flow and then another row of "entity beans" (java speak
for data object persistence) to get at a rdbms.
This arrangement has worked well for me. This approach makes "cheating"
code look immediately out-of-place, and just downright embarrassing :)
Is there a Perl equivalent to Struts? Mason seems to come close if you
keep yourself disciplined somewhat.
--
pgp: http://www.geocities.com/matthewbk/pubkeyw.txt
Re: Excellent article on Apache/mod_perl at eToys
Posted by Gunther Birznieks <gu...@extropia.com>.
At 08:46 PM 10/23/2001, Rob Nagler wrote:
> > is easier and more standardized, and well documented. But I "feel" like
> > coding front-end web applications is much easier in Perl where the workflow
> > bits change all the time. For this, I like using SOAP on the backend Java
> > server and SOAP on the front-end Perl.
>
>I don't quite understand the difference between worflow in the front-end and
>workflow in the back-end. They both change. The danger of making one part
>of the system easier to change is that people tend to "cheat". They won't
>put the business logic in the back-end if it takes twice as long.
Yes, people can cheat. But generally the workflow that is most likely to
change is the UI based workflow. UI isn't just about templates, it's a lot
about how you go from one screen to the next, or not.
Actually business logic is a bit of a misnomer because there is plenty of
business logic in the front-end from your choice of fields to the
javascript to the UI workflow.
>To me, the major issue in Perl vs Java is dynamic vs static typing. Building
>large scale systems in Perl is much like building them in Smalltalk or Lisp.
>It takes a certain mindset. The lack of compiled interfaces means you need
>much more discipline (e.g. unit testing). The payoff is big with Perl,
>because
>you can refactor more easily and quickly than in Java.
I don't think refactoring in Perl is necessarily faster. Actually, it can
be quite hard to refactor in Perl as well but for the opposite reason.
If you do not have a strongly typed system, then when you break apart and
rebuild another part of the system, Perl may very well not complain when a
subtle bug comes up because of the fact that it is not strongly typed.
Whereas Java will complain quite often and usually early with compile time
checking.
Compile time checking can definitely be a friend of yours especially when
dealing with large systems. But it's also a friend that's judgemental
(strongly typed) so he's a pain to drag along to a party....
In practice, I still think using a good framework, it's about as long to
develop in Perl as it is in Java for a medium sized system. My only beef I
have with Java and I still have it is that the debugging support is simply
atrocious for web apps. Perl has many more debugging utilities and allows
much greater introspection. When I develop in Java, the only reason it
takes me about 20% longer is that I am restarting the server much more
often than I do in Perl.
>The libraries aren't much an issue. A good example is SOAP. SOAP is
>middleware. It is standardized, documented, and the rest of it. You like
>it for connecting Perl to Java, but why can't it be the other way around?
>If it can be the other way around, why aren't Perl and Java equally adapted
>to building middleware applications?
Java's support for multi-threading makes writing servers feel fairly
trivial with no jumping through IPC::Shared memory stuff hoops to get
shared memory caches and the like.. you just synchronize on global data
structures.
Arguably, an overuse of threads has problems in itself, but in general, I
think it is easier. It is also easier to find programmers who know how to
code middleware that do Java and harder to find people who have coded
middleware that are Perl programmers... so maybe the language would support
it, but there is also a personnel issue.
Later,
Gunther
Re: Excellent article on Apache/mod_perl at eToys
Posted by Rob Nagler <na...@bivio.net>.
> is easier and more standardized, and well documented. But I "feel" like
> coding front-end web applications is much easier in Perl where the workflow
> bits change all the time. For this, I like using SOAP on the backend Java
> server and SOAP on the front-end Perl.
I don't quite understand the difference between worflow in the front-end and
workflow in the back-end. They both change. The danger of making one part
of the system easier to change is that people tend to "cheat". They won't
put the business logic in the back-end if it takes twice as long.
To me, the major issue in Perl vs Java is dynamic vs static typing. Building
large scale systems in Perl is much like building them in Smalltalk or Lisp.
It takes a certain mindset. The lack of compiled interfaces means you need
much more discipline (e.g. unit testing). The payoff is big with Perl, because
you can refactor more easily and quickly than in Java.
The libraries aren't much an issue. A good example is SOAP. SOAP is
middleware. It is standardized, documented, and the rest of it. You like
it for connecting Perl to Java, but why can't it be the other way around?
If it can be the other way around, why aren't Perl and Java equally adapted
to building middleware applications?
Rob
RE: Excellent article on Apache/mod_perl at eToys
Posted by Gunther Birznieks <gu...@extropia.com>.
At 09:14 AM 10/23/2001, Robert Koberg wrote:
> > For comparions, a nice aspect of j2ee applications IMHO is the
> > "application server" tends to be more general. ie. the application
> > server is not just the web server (as it is with mod_perl). I've found
> > j2ee features such as message beans, queues and such especially useful
> > for back-end work. For these reasons, I personally don't buy the
> > argument that mod_perl makes an effective application server for a good
> > sized task (your mileage will vary no doubt ;).> >
>
>What is stopping you from using both, if you want?
It's generally quite tough to convince management that maintaining two sets
of knowledge -- Java and Perl is cost effective. I suppose it depends on
the size of the organization but "consolidation" of "standards" in an
organization is an age-old thing (eg move all servers to NT, move everyone
to Java, etc...)
Whether this actually saves costs is a bit of an arguement probably not
best for this mailing list.
However, I would have to say that I "feel" like coding middleware in Java
is easier and more standardized, and well documented. But I "feel" like
coding front-end web applications is much easier in Perl where the workflow
bits change all the time. For this, I like using SOAP on the backend Java
server and SOAP on the front-end Perl.
YMMV. And I don't know that many organizations which will allow such a
heterogeneous environment unless a vendor installs it as a turnkey solution
and is willing to deal with all the support.
__________________________________________________
Gunther Birznieks (gunther.birznieks@eXtropia.com)
eXtropia - The Open Web Technology Company
http://www.eXtropia.com/
RE: Excellent article on Apache/mod_perl at eToys
Posted by Matthew Kennedy <mk...@opushealthcare.com>.
On Mon, 2001-10-22 at 20:14, Robert Koberg wrote:
>
>
> > For comparions, a nice aspect of j2ee applications IMHO is the
> > "application server" tends to be more general. ie. the application
> > server is not just the web server (as it is with mod_perl). I've found
> > j2ee features such as message beans, queues and such especially useful
> > for back-end work. For these reasons, I personally don't buy the
> > argument that mod_perl makes an effective application server for a good
> > sized task (your mileage will vary no doubt ;).> >
>
> What is stopping you from using both, if you want?
Actually, I already do -- but not at the same time. The criteria I
personally use is basically this: if the task doesn't involve more than
one or two disparate system to be talked to, and a two tier model
(apache and a rdbms for example) fits the problem well, then I'll chose
Perl (mod_perl, Mason and *sometimes* ORBit's Perl bindings). If the app
is to talk to a number of disparate systems and a variety of client
types (stand alone app, web, wireless etc.) then generally I'll use the
a J2EE app server etc. Of course there are will be exceptions.
ORBit is a corba orb, BTW.
Matt
--
pgp: http://www.geocities.com/matthewbk/pubkeyw.txt
RE: Excellent article on Apache/mod_perl at eToys
Posted by Robert Koberg <ro...@koberg.com>.
> For comparions, a nice aspect of j2ee applications IMHO is the
> "application server" tends to be more general. ie. the application
> server is not just the web server (as it is with mod_perl). I've found
> j2ee features such as message beans, queues and such especially useful
> for back-end work. For these reasons, I personally don't buy the
> argument that mod_perl makes an effective application server for a good
> sized task (your mileage will vary no doubt ;).> >
What is stopping you from using both, if you want?
Re: Excellent article on Apache/mod_perl at eToys
Posted by Ask Bjoern Hansen <as...@perl.org>.
On 22 Oct 2001, Matthew Kennedy wrote:
> Why was Berkeley DB chosen for caching when a RDBMS (mysql in
> this case) was already being used?
For speed.
You want to hit the RDBMS as little as possibly; Berkeley DB makes a
good cache.
- ask
--
ask bjoern hansen, http://ask.netcetera.dk/ !try; do();
Re: Excellent article on Apache/mod_perl at eToys
Posted by Perrin Harkins <pe...@elem.com>.
Matthew Kennedy wrote:
> On Mon, 2001-10-22 at 21:27, Perrin Harkins wrote:
>
>>It sounds like the limitation there is that you're interfacing with systems
>>that can't notify you when something new happens. That's not Perl's fault.
>>If you wrote your daemons in Java alpahabet soup, they'd still have to poll
>>the pop3 server.
>>
>
> Well, I'd more likely be using a standard javamail API within the
> context of a EJB server. There is a fairly rich set of call backs there
> which mean I generally don't have to poll/sleep etc. Also true of the
> java message service.
I suppose the Java stuff does make it more obvious how to do that kind
of thing. There are dozens of ways to have some perl code execute every
time a message arrives at a mail server without polling (Mail::Filter,
procmail, Perl pop3 server, maybe PerlMX, etc.), but there isn't any One
True Perl Mail API.
Pointing newbies in the right direction is the strength of standardized
APIs, and it does seem to have helped Java gain acceptance. There are
commercial JMS implementations that simply poll database tables, but
most people don't seem to mind as long as that's all hidden behind the
standard JMS API. So, maybe a version in Perl would be a useful thing.
- Perrin
Re: Excellent article on Apache/mod_perl at eToys
Posted by Matthew Kennedy <mk...@opushealthcare.com>.
On Mon, 2001-10-22 at 21:27, Perrin Harkins wrote:
>
> It sounds like the limitation there is that you're interfacing with systems
> that can't notify you when something new happens. That's not Perl's fault.
> If you wrote your daemons in Java alpahabet soup, they'd still have to poll
> the pop3 server.
Well, I'd more likely be using a standard javamail API within the
context of a EJB server. There is a fairly rich set of call backs there
which mean I generally don't have to poll/sleep etc. Also true of the
java message service.
--
pgp: http://www.geocities.com/matthewbk/pubkeyw.txt
Re: Excellent article on Apache/mod_perl at eToys
Posted by Perrin Harkins <pe...@elem.com>.
on 10/22/01 11:13 AM, Matthew Kennedy at mkennedy@opushealthcare.com wrote:
> Why was Berkeley DB chosen for caching when a RDBMS (mysql in this case)
> was already being used?
It's faster and less resource-intensive for this kind of thing. We just
wanted a really fast persistent hash, and didn't need SQL or relational
concepts.
> Secondly, I've worked on a good-sized commerce site with
> mod_perl+HTML::Mason. One of the more dirty secrets is that the back-end
> of the site involves several standalone perl programs running as
> daemons. What's even worse is; most of them have to sit in poll/wait
> loops waiting on message from external systems (such as pop3 servers,
> ccvs etc.)
It sounds like the limitation there is that you're interfacing with systems
that can't notify you when something new happens. That's not Perl's fault.
If you wrote your daemons in Java alpahabet soup, they'd still have to poll
the pop3 server.
> For comparions, a nice aspect of j2ee applications IMHO is the
> "application server" tends to be more general. ie. the application
> server is not just the web server (as it is with mod_perl). I've found
> j2ee features such as message beans, queues and such especially useful
> for back-end work. For these reasons, I personally don't buy the
> argument that mod_perl makes an effective application server for a good
> sized task (your mileage will vary no doubt ;).
I guess it all depends on what you think "application server" means. In our
case, it was a server that ran code persistently, which supported an HTTP
interface. It wasn't our front-end web server, and we could run anything we
wanted to in it.
Lots of people implement reliable queues in Perl. They generally use
something like an RDBMS or Berkeley DB (which has a queue option) for the
backing store. Many JMS implementations use an RDBMS for all the hard stuff
as well.
> So again, I'm curious, what does the eToys.com site back-end stuff look
> like?
The web site part that was discussed in the article ended at the Oracle
database. There was a system built on Oracle's queuing technology that
replicated orders to a backend system for processing. I believe the last
revision of that was largely in PL/SQL. I don't know much about it, since
there was an entirely different team working on it.
- Perrin
Re: Excellent article on Apache/mod_perl at eToys
Posted by Matthew Kennedy <mk...@opushealthcare.com>.
Firstly, I am curious...
Why was Berkeley DB chosen for caching when a RDBMS (mysql in this case)
was already being used?
Secondly, I've worked on a good-sized commerce site with
mod_perl+HTML::Mason. One of the more dirty secrets is that the back-end
of the site involves several standalone perl programs running as
daemons. What's even worse is; most of them have to sit in poll/wait
loops waiting on message from external systems (such as pop3 servers,
ccvs etc.)
For comparions, a nice aspect of j2ee applications IMHO is the
"application server" tends to be more general. ie. the application
server is not just the web server (as it is with mod_perl). I've found
j2ee features such as message beans, queues and such especially useful
for back-end work. For these reasons, I personally don't buy the
argument that mod_perl makes an effective application server for a good
sized task (your mileage will vary no doubt ;).
So again, I'm curious, what does the eToys.com site back-end stuff look
like?
Great article BTW, Bill & Perrin,
Matt
On Thu, 2001-10-18 at 21:11, Andrew Ho wrote:
> Hello,
>
> I checked the list archives and it didn't look like this had been posted yet.
> For those of you who haven't seen it yet... a great read on perl.com about
> the Apache/mod_perl setup at eToys, co-authored by our own mod_perl
> regular contributer Perrin Harkins.
>
> http://www.perl.com/pub/a/2001/10/17/etoys.html
>
> Humbly,
>
> Andrew
>
> ----------------------------------------------------------------------
> Andrew Ho http://www.tellme.com/ andrew@tellme.com
> Engineer info@tellme.com Voice 650-930-9062
> Tellme Networks, Inc. 1-800-555-TELL Fax 650-930-9101
> ----------------------------------------------------------------------
>
>
>
--
pgp: http://www.geocities.com/matthewbk/pubkeyw.txt