You are viewing a plain text version of this content. The canonical link for it is here.
Posted to community@apache.org by "Noel J. Bergman" <no...@devtech.com> on 2003/06/26 08:03:10 UTC
WORA Considered Evil ;-)
> [WORA] is the reason why we basically we have mod_* where * is any
> programming language, but not mod_java
> for apache 2.0, a JNI-based mod_java is perfectly valid
> architecturarely, but nobody works on it
Well, there seems to be a JNI (in-process) worker on some platforms, but
personally I prefer to have workers outside of httpd for purposes of
security, load balancing, etc., which seems to be the approach you prefer as
well (below).
> Also, java programmers tend *not* to have any knowledge of things like
> "how to link a library in a native environment".
Do you really know only such cloistered Java programmers despite all of the
ones you meet? The majority of Java programmers I know have rich and varied
backgrounds. I highly recommmend the www.softwaresummit.com conference as
the place to meet Java programmers who are both serious about Java and well
versed in other areas.
> basically they are totally isolated, which leads to concepts such as
server
> microkernel architectures (avalon Phoenix) which look cool from a purely
> architectural perspective, but are totally weak from a security and
> stability perspective, because they use one JVM for the entire thing, a
> very weak setup.
However, having something like httpd front-end lots of backend JVMs on
multiple machines is nice. And what prevents mod_jk2 from using a launcher
to control and monitor JVM processes? It would be nice to be able to do
something like:
<virtualhost ...>
ServerName RingBearer
JkUserID frodo FoTR
Jk...
...
</virtualhost>
and have Coyote communicate to a "jkLauncher" a connection request for a JVM
running as that user (configuration of the JVM not being the launcher's
responsibility). It would certainly make my configuration easier, and more
robust. Currently, I do something similar by hand.
A problem with multiple JVM instances is the lack of sharing between
multiple instances. If you run multiple instances of tomcat, one per
virtual host for example, your memory overhead is considerable since none of
the class code is shared. I would like to see the JVM/JIT generate and
share common class code on platforms that support it.
Sun appears to prefer intra-process threading or inter-process communication
over network interfaces. You might find:
http://www.iee.org/Publish/Journals/ProfJourn/Proc/SEN/vinter.pdf
http://www.cs.berkeley.edu/~szewczyk/cs267/Final.html
http://matsu-www.is.titech.ac.jp/~sohda/research/javag2001.pdf
interesting. I didn't say practical. :-) Although JPS looks interesting.
True, Java is not a systems programming language. But without WORA, I do
not believe that Java would have the success that it has on the server.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Stefano Mazzocchi <st...@apache.org>.
on 6/26/03 11:28 AM Stefano Mazzocchi wrote:
> So, we created the Mailet API and started JAMES, later we had Federico
> involved that did most of the coding.
The above is not painting the picture correctly. Federico did the POP3
server and the first Avalon integration, while Serge did the SMTP part
(including DNS lookup and all that).
Serge has been the real driver of the JAMES project since the beginning.
Pier, Federico and I were just instrumental in getting the concept in
java.apache.org and inject a little vision on the mailet concept.
But James has been a real community development and we have pretty much
nothing to do with what James is right now.
Serge infinite much more credits than we do.
Apologies for not having been precise.
--
Stefano.
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
sendmail (was: Re: WORA Considered Evil ;-) )
Posted by Henning Schmiedehausen <hp...@intermeta.de>.
On Sat, 2003-06-28 at 21:06, Stefano Mazzocchi wrote:
> Look: everybody on this planet hates sendmail,
That's pretty much a misconception. Sendmail might have a large number
of people that don't like the "monolitic" approach, but you will find
just as well loads of people that consider sendmail 'Just Another
Mailer".
The hate-sendmail/love-qmail reflex is as much ideology as the vi/emacs
debate.
Regards
Henning
--
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen INTERMETA GmbH
hps@intermeta.de +49 9131 50 654 0 http://www.intermeta.de/
Java, perl, Solaris, Linux, xSP Consulting, Web Services
freelance consultant -- Jakarta Turbine Development -- hero for hire
--- Quote of the week: "It is pointless to tell people anything when
you know that they won't process the message." --- Jonathan Revusky
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
sendmail (was: Re: WORA Considered Evil ;-) )
Posted by Henning Schmiedehausen <hp...@intermeta.de>.
On Sat, 2003-06-28 at 21:06, Stefano Mazzocchi wrote:
> Look: everybody on this planet hates sendmail,
That's pretty much a misconception. Sendmail might have a large number
of people that don't like the "monolitic" approach, but you will find
just as well loads of people that consider sendmail 'Just Another
Mailer".
The hate-sendmail/love-qmail reflex is as much ideology as the vi/emacs
debate.
Regards
Henning
--
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen INTERMETA GmbH
hps@intermeta.de +49 9131 50 654 0 http://www.intermeta.de/
Java, perl, Solaris, Linux, xSP Consulting, Web Services
freelance consultant -- Jakarta Turbine Development -- hero for hire
--- Quote of the week: "It is pointless to tell people anything when
you know that they won't process the message." --- Jonathan Revusky
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Pier Fumagalli <pi...@betaversion.org>.
On 29/6/03 20:45 "Noel J. Bergman" <no...@devtech.com> wrote:
>> James, then, is addressing concerns different from mine: you're playing
>> the Notes/iPlanet/Exchange game, a sector of the market I don't care about.
>
> I would not put Notes and James in the same category. James and Exchange
> maybe, in the sense of James wanting to offer an IMAP service, which would
> support IMAP-based groupware. In many respects, James and qmail seem more
> similar: they are built from modular services, a pipeline and message
> stores. Other than feature differences (James lacks ezmlm's functionality
> right now), the significant difference is that currently James has its
> analogues to qmail, ezmlm and qmail-pop3d all in one JVM. That is the only
> similarity to Notes that I see, and it is superficial, in my view.
It's not a matter of functionality, it's a matter of architecture.
All that you name (ezmlm, imap, pop3d) are things external to Qmail, they
don't touch the core distribution, you can strip them out and Qmail will
work... All Qmail can be summed up in the following:
- qmail-queue/clean (the queue)
- qmail-lspawn/local (delivery agent, local)
- qmail-rspawn/remote (delivery agent, remote via SMTP)
- qmail-inject (injector, local)
- qmail-smtpd (injector, remote via SMTP)
And with this I'm already including a number of things not "required" to run
qmail, just usual stuff you see in the distribution and you're going to use.
And note that those 8 binaries are completely independent in regard to their
application space and security model.
Qmail is _designed_ to be a set of different little tools running in
different processes with different users, relying on the OS for security,
Notes/Exchange/iPlanet follow another model... In my dialect "FASO TOTO ME",
meaning, "I take care of the whole kit and kaboodle"...
James does that, it's design is absolutely orthogonal to Qmail and Postfix.
Ok, it's "customizable" like Qmail (you can create pipelines more or less
like you do w/ Qmail) but it ain't the same thing architecturally.
>> Turns out that our bandwidth is saturated by Sobig/E messages,
>> our front-exchanger sendmails and qmails have no whatsoever
>> problem in handling that ...
>
> What allows them to keep up? Are they fast enough to rejecting messages in
> real-time, or is it simply a much better pipeline, with rejection happening
> on the backend(s) independent of the frontend?
>
> Notes wasn't designed to be a high throughput MTA. A lot of newer Notes
> features (we were using Notes 4.x and 5, if I recall correctly), always felt
> like misplaced hacks bolted onto the core. I'm amazed that Notes still
> holds up as well as it does. It ought to have been ditched in the original
> Domino timeframe, re-architected from the ground up, and built using a real
> database, real MTA, real web server, etc., in a modular fashion. Which is
> why I would not want to add Notes features to James. In my view, it would
> be better to facilitate integration between James, web app servers, and
> other applications; and let James do what it does best.
On one thing we agree... Notes is a piece of crap... Let's stop talking
about it because it gets on my nerves big times...
Pier
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
>> I'll tell you one thing. SHIT happens, whether you use Qmail, Notes
>> or James. And unfortunately YOU are not the ones who get paged at
>> 11:30 pm out on a dinner date because the friggin's server has
>> gone down...
No ... but I have had my cell phone called from the data center when a
system failed, and have to decide about abandoning my vacation and driving
back 300 miles to deal with it.
So far the only break-in I've ever had on a system came courtesy of a PHP 3
vulnerability caused by the one site I allowed to use it. Fortunately, it
was limited by other security precautions, and the exploit has since been
patched by both belts and suspenders.
> > IMO Don't use James in critical heavy traffic scenarios. Do use james
where
> > having a java mail API in the server gives your business a real
advantage.
> > In part I agree this discussion addresses James' competitor in one
field, but
> > don't forget the whole other half of James' uphill struggle is our other
> > competitor, M$ Exchange.
> James, then, is addressing concerns different from mine: you're playing
the
> Notes/iPlanet/Exchange game, a sector of the market I don't care about.
I would not put Notes and James in the same category. James and Exchange
maybe, in the sense of James wanting to offer an IMAP service, which would
support IMAP-based groupware. In many respects, James and qmail seem more
similar: they are built from modular services, a pipeline and message
stores. Other than feature differences (James lacks ezmlm's functionality
right now), the significant difference is that currently James has its
analogues to qmail, ezmlm and qmail-pop3d all in one JVM. That is the only
similarity to Notes that I see, and it is superficial, in my view.
>> Now, I _know_ that Qmail gives me a very good chance to finish my dinner,
>> get back home and have a good night of sleep. Will James do the
>> same thing?
It does for me. Don't use it until it you feel that it would for you, too.
> Turns out that our bandwidth is saturated by Sobig/E messages,
> our front-exchanger sendmails and qmails have no whatsoever
> problem in handling that ...
What allows them to keep up? Are they fast enough to rejecting messages in
real-time, or is it simply a much better pipeline, with rejection happening
on the backend(s) independent of the frontend?
Notes wasn't designed to be a high throughput MTA. A lot of newer Notes
features (we were using Notes 4.x and 5, if I recall correctly), always felt
like misplaced hacks bolted onto the core. I'm amazed that Notes still
holds up as well as it does. It ought to have been ditched in the original
Domino timeframe, re-architected from the ground up, and built using a real
database, real MTA, real web server, etc., in a modular fashion. Which is
why I would not want to add Notes features to James. In my view, it would
be better to facilitate integration between James, web app servers, and
other applications; and let James do what it does best.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: *Possible Spam *Re: WORA Considered Evil ;-)
Posted by Danny Angus <da...@apache.org>.
> James, then, is addressing concerns different from mine: you're
> playing the
> Notes/iPlanet/Exchange game, a sector of the market I don't care about.
>
> Frankly I use Notes every day, and I'm happy that Brian at VNU is taking
> care of that thing. I'm talking about MTAs, you're talking about "mail
> servers"...
Yeah. Although I can't speak for anyone else my own opinion is that James addresses a real need for an accessable all round mail server.
All the competition seem to require either access to a lot of sysadmin savvy or to $$$, in windows particularly it is hard to find mail servers where you get more than support for two or three acounts without having to spend-spend-spend.
You or Stefano spoke earlier about the mailet API being seen from the start as converging mail and web for the benefit of people providing services using both mediums. I find that James is in a class of its own when it provides the first or last hop for mail into/out of an organisation and represents the first or last opportunity to process that mail. By also providing POP (and IMAP) and full SMTP functionality (not just forwarding to a single gateway) it makes exploiting that opportunity into an achieveable reality, and allows people with java skills and resources to take full advantage of them in the domain of mail handling.
I hope that James *does* become capable of fulfilling the apmail role(s) but I don't think that this is necessary in order to consider James to have been a sucess.
d.
Re: WORA Considered Evil ;-)
Posted by Pier Fumagalli <pi...@betaversion.org>.
On 29/6/03 12:31 "Danny Angus" <da...@apache.org> wrote:
> Pier,
>
>> Yeah... And I believe that now I'm finally on the other side... I'll tell
>> you one thing. SHIT happens, whether you use Qmail, Notes or James. And
>> unfortunately YOU are not the ones who get paged at 11:30 pm out
>> on a dinner
>> date because the friggin's server has gone down...
>
> Actually I am. I agree that there are roles for James and roles for
> $not-james.
>
> IMO Don't use James in critical heavy traffic scenarios. Do use james where
> having a java mail API in the server gives your business a real advantage.
Yes... Absolutely I see your point now. As you said in this other email:
> In part I agree this discussion addresses James' competitor in one field, but
> don't forget the whole other half of James' uphill struggle is our other
> competitor, M$ Exchange.
James, then, is addressing concerns different from mine: you're playing the
Notes/iPlanet/Exchange game, a sector of the market I don't care about.
Frankly I use Notes every day, and I'm happy that Brian at VNU is taking
care of that thing. I'm talking about MTAs, you're talking about "mail
servers"...
>> Now, I _know_ that Qmail gives me a very good chance to finish my dinner,
>> get back home and have a good night of sleep. Will James do the
>> same thing?
>
> I refuse to guarantee it (but i'd make a small wager). :-)
As the Qmail author refuses to guarantee that Qmail is not going to fall
over, but at the same time (take what happened yesterday night), we had a
problem w/ our mail @ VNU. Turns out that our bandwidth is saturated by
Sobig/E messages, our front-exchanger sendmails and qmails have no
whatsoever problem in handling that, but notes on the back is struggling to
mark-and-delete everything that norton identifies as a virus.
Brian is fixing Notes, I am @ home replying to your email... I still like my
solution better :-) :-) :-)
Pier
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: WORA Considered Evil ;-)
Posted by Danny Angus <da...@apache.org>.
Pier,
> Yeah... And I believe that now I'm finally on the other side... I'll tell
> you one thing. SHIT happens, whether you use Qmail, Notes or James. And
> unfortunately YOU are not the ones who get paged at 11:30 pm out
> on a dinner
> date because the friggin's server has gone down...
Actually I am. I agree that there are roles for James and roles for $not-james.
IMO Don't use James in critical heavy traffic scenarios. Do use james where having a java mail API in the server gives your business a real advantage.
>
> Now, I _know_ that Qmail gives me a very good chance to finish my dinner,
> get back home and have a good night of sleep. Will James do the
> same thing?
I refuse to guarantee it (but i'd make a small wager). :-)
d.
RE: WORA Considered Evil ;-)
Posted by Danny Angus <da...@apache.org>.
Pier,
> Yeah... And I believe that now I'm finally on the other side... I'll tell
> you one thing. SHIT happens, whether you use Qmail, Notes or James. And
> unfortunately YOU are not the ones who get paged at 11:30 pm out
> on a dinner
> date because the friggin's server has gone down...
Actually I am. I agree that there are roles for James and roles for $not-james.
IMO Don't use James in critical heavy traffic scenarios. Do use james where having a java mail API in the server gives your business a real advantage.
>
> Now, I _know_ that Qmail gives me a very good chance to finish my dinner,
> get back home and have a good night of sleep. Will James do the
> same thing?
I refuse to guarantee it (but i'd make a small wager). :-)
d.
Re: WORA Considered Evil ;-)
Posted by Pier Fumagalli <pi...@betaversion.org>.
On 28/6/03 8:06 pm, "Stefano Mazzocchi" <st...@apache.org> wrote:
> Reading your post that dismiss the UNIX sysadm fears as a "think of the
> past" [...]
Yeah... And I believe that now I'm finally on the other side... I'll tell
you one thing. SHIT happens, whether you use Qmail, Notes or James. And
unfortunately YOU are not the ones who get paged at 11:30 pm out on a dinner
date because the friggin's server has gone down...
Now, I _know_ that Qmail gives me a very good chance to finish my dinner,
get back home and have a good night of sleep. Will James do the same thing?
Start living with a pager as your best friend, life will change for you as
well...
Pier (paged tonight as Sobig/E is killing our bandwidth)
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: WORA Considered Evil ;-)
Posted by Steve Brewin <sb...@synsys.com>.
Stefano,
> -----Original Message-----
> From: Stefano Mazzocchi [mailto:stefano@apache.org]
> Sent: 28 June 2003 20:07
> To: James Developers List
> Cc: community@apache.org
> Subject: Re: WORA Considered Evil ;-)
>
> Reading your post that dismiss the UNIX sysadm fears as a
> "think of the
> past",
and
> "get modern and stop the crap"
These are unrepresentative paraphrases of what I said. I don't believe that
the post actually said or implied either of these things.
I did say "I'm sure that everyone is in favour of hardening James as much as
possible. Its just that we should approach it from a Java perspective, not a
C on Unix one. The issues are different."
Why? Because the potential vulnerabilities and the optimum measures to
eliminate them are not the same for the two environments. To take an
approach that assumes that they are the same will result in a less hardend
solution. Examining the risks as they exist in James' Java environment
deployed on physical OSes such as Unix will achieve a maximally hardened
solution.
Achieving sysadm trust is not the same as achieving a maximally hardened
solution. Perhaps James could achieve a level of trust from some Unix
sysadms by making it possible to mirror the deployment environments that
they trust. Fine. But as developers we shouldn't be blind to the fact this
is a minor detour on the road to a maximally hardened solution.
I don't view this as an appeal to ignore anyone. I don't make a habit of
considering anyones views as "crap".
-- Steve
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: WORA Considered Evil ;-)
Posted by Steve Brewin <sb...@synsys.com>.
Stefano,
> -----Original Message-----
> From: Stefano Mazzocchi [mailto:stefano@apache.org]
> Sent: 28 June 2003 20:07
> To: James Developers List
> Cc: community@apache.org
> Subject: Re: WORA Considered Evil ;-)
>
> Reading your post that dismiss the UNIX sysadm fears as a
> "think of the
> past",
and
> "get modern and stop the crap"
These are unrepresentative paraphrases of what I said. I don't believe that
the post actually said or implied either of these things.
I did say "I'm sure that everyone is in favour of hardening James as much as
possible. Its just that we should approach it from a Java perspective, not a
C on Unix one. The issues are different."
Why? Because the potential vulnerabilities and the optimum measures to
eliminate them are not the same for the two environments. To take an
approach that assumes that they are the same will result in a less hardend
solution. Examining the risks as they exist in James' Java environment
deployed on physical OSes such as Unix will achieve a maximally hardened
solution.
Achieving sysadm trust is not the same as achieving a maximally hardened
solution. Perhaps James could achieve a level of trust from some Unix
sysadms by making it possible to mirror the deployment environments that
they trust. Fine. But as developers we shouldn't be blind to the fact this
is a minor detour on the road to a maximally hardened solution.
I don't view this as an appeal to ignore anyone. I don't make a habit of
considering anyones views as "crap".
-- Steve
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Santiago Gala <sg...@hisitech.com>.
Serge Knystautas escribió:
> Santiago Gala wrote:
>
>> I think a good equilibrium point between the "marketing" view of
>> security (making sysadms trust) and purist java technical view would
>> be to allow James not having to run as root under Unix (to handle
>> protected ports like 25, 110, etc.) and then securing the rest of the
>> processing through java security declarations.
>
>
> Since people here know qmail and sendmail a lot better than I do... how
> do they bind to those ports without running as root?
>
It is done, AFAIK, having a small program running as root, which just
opens the server socket(s), listens to them. Every time a connection is
accepted, this driver forks and spawns a different program under lesser
privileges, passing it the socket as file descriptor. (Don't take this
as a precise description)
A small auxiliary process (a minimalistic and security conscious C
program) doing this and using some kind of IPC to communicate with a
James+JNI process could do the job in a way that is both portable and
can be trusted by sysadms. Please correct me if I'm wrong, as I'm not a
POSIX wizard at all. I don't know how much of this applies to Windows,
although sandboxing Windows services does not look like a bad idea.
Similar to what Costin and Pier discussed about some days ago re:
communicating Apache with Tomcat, in a thread named "How ASF membership
works and what it means".
P.S.) ASF membership means you can speak about those difficult issues
and (some) people will actually listen :-P (For those blunt enough, this
is a disclaimer)
Regards
--
Santiago Gala
High Sierra Technology, S.L. (http://hisitech.com)
http://memojo.com?page=SantiagoGalaBlog
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Ben Hyde <bh...@pobox.com>.
On Wednesday, July 2, 2003, at 10:46 AM, Serge Knystautas wrote:
> Santiago Gala wrote:
>> I think a good equilibrium point between the "marketing" view of
>> security (making sysadms trust) and purist java technical view would
>> be to allow James not having to run as root under Unix (to handle
>> protected ports like 25, 110, etc.) and then securing the rest of the
>> processing through java security declarations.
>
> Since people here know qmail and sendmail a lot better than I do...
> how do they bind to those ports without running as root?
A guiding principal of qmail is no process should do more than the
absolute minimum and it certainly shouldn't have any more rights than
the absolute minimum. Living up to those principals demands having
lots of user accounts, each with exactly the minimum number of rights,
and then using fork to create safe bubbles - you setting the user
before forking to create the bubbles, and then cleverly passing just
the right stuff to these processes to give them the right capabilities.
So for example a process that's root has the right to create a
listener, but that process can then fork a process passing it that
listener to give the capability to process/user with much more
restricted rights.
We used to have a patch for HTTP that allowed a short lived root
process to create the HTTP listener and then pass that open file handle
to the server (notifying it were to find it via a parameter). That
enabled the entire server to run in user space. It fell off the wagon
at some point.
I know nothing about sendmail, makes you jealous doesn't it?
- ben
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Thom May <th...@apache.org>.
* Ask Bjoern Hansen (ask@perl.org) wrote :
> On Wed, 2 Jul 2003, James Duncan Davidson wrote:
>
> > By changing their id after they launch as root. setuid. Pretty common
> > thing to do. See man setuid.
>
> With qmail it's even more separated. There's a small program that
> opens the port and then drops root. The smtpd itself never has any
> special access.
>
> The qmail mail system is one of the most beautiful pieces of
> software around; making another mail system without looking at qmail
> carefully is silly.
>
To some extent I'd agree. However, there is a distinct lack of defensive
programming in qmail *itself*. So while the security aspects are totally
laudable, and there are definitely a lot of good ideas, you shouldn't go
looking at qmail as the be all and end all of mail servers.
(For people wanting specific pointers, /var/qmail/queue/lock/trigger is an
accident waiting to happen; and we've seen qmail-send just silently stop
delivering mail a few times. Also, having individual mails tied to inodes is
an absolute nightmare for disaster recovery. Oh, and the way the remote
queues can get hung up waiting because a site has fallen off the net is a
pain. (Yes, I can keep going like this for some time ;-) ))
And the fact it isn't Free really really bugs me.
Mind you, we do run qmail on ~300 servers, so I guess we are a limit case
for seeing qmail break in weird and wonderful ways.
So I guess this email is "take all the paranoia from qmail you can, but then
look at postfix too." :-)
Cheers
-Thom
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Ask Bjoern Hansen <as...@perl.org>.
On Wed, 2 Jul 2003, James Duncan Davidson wrote:
> By changing their id after they launch as root. setuid. Pretty common
> thing to do. See man setuid.
With qmail it's even more separated. There's a small program that
opens the port and then drops root. The smtpd itself never has any
special access.
The qmail mail system is one of the most beautiful pieces of
software around; making another mail system without looking at qmail
carefully is silly.
- ask
--
http://www.askbjoernhansen.com/ - http://develooper.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by James Duncan Davidson <du...@x180.net>.
On Wednesday, July 2, 2003, at 07:46 AM, Serge Knystautas wrote:
> Santiago Gala wrote:
>> I think a good equilibrium point between the "marketing" view of
>> security (making sysadms trust) and purist java technical view would
>> be to allow James not having to run as root under Unix (to handle
>> protected ports like 25, 110, etc.) and then securing the rest of the
>> processing through java security declarations.
>
> Since people here know qmail and sendmail a lot better than I do...
> how do they bind to those ports without running as root?
By changing their id after they launch as root. setuid. Pretty common
thing to do. See man setuid. There's some source code floating around
the net to compile a native library for Java that will do a setuid for
you... We wrote it back in the Java Web Server days so that we could
start up on port 80 and then bounce to nobody (or whatever user you
wanted).
It's f'ing insane that it's not a "standard" thing in the platform.
There's zillions of audio things in there but not a good setuid. Feh.
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Serge Knystautas <se...@lokitech.com>.
Santiago Gala wrote:
> I think a good equilibrium point between the "marketing" view of
> security (making sysadms trust) and purist java technical view would be
> to allow James not having to run as root under Unix (to handle protected
> ports like 25, 110, etc.) and then securing the rest of the processing
> through java security declarations.
Since people here know qmail and sendmail a lot better than I do... how
do they bind to those ports without running as root?
--
Serge Knystautas
President
Lokitech >> software . strategy . design >> http://www.lokitech.com
p. 301.656.5501
e. sergek@lokitech.com
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Santiago Gala <sg...@hisitech.com>.
Steve Brewin escribió:
> Achieving sysadm trust is not the same as achieving a maximally hardened
> solution. Perhaps James could achieve a level of trust from some Unix
> sysadms by making it possible to mirror the deployment environments that
> they trust. Fine. But as developers we shouldn't be blind to the fact this
> is a minor detour on the road to a maximally hardened solution.
>
I think a good equilibrium point between the "marketing" view of
security (making sysadms trust) and purist java technical view would be
to allow James not having to run as root under Unix (to handle protected
ports like 25, 110, etc.) and then securing the rest of the processing
through java security declarations.
I think having to run a big program as root looks always dangerous, as
it makes far more difficult to know what it is doing or to scan it for
security problems. Specially if it is extensible or takes third party
extension modules (mailets, etc.)
This (and better) hardening could be achieved at the "pure" java level
by granting specific permissions to different codebases/jars, using
doPrivileged() when needed and running under a security manager, but I
doubt most sysadms would be convinced if they don't understand java
security. I, for one, could "buy" a pure java securiy solution, but I'm
not a typical sysadm.
Regards
--
Santiago Gala
High Sierra Technology, S.L. (http://hisitech.com)
http://memojo.com?page=SantiagoGalaBlog
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by James Duncan Davidson <du...@x180.net>.
Some random thoughts--aside from the personal defense against a
perceived attack which I'm not wanting to get involved with. :)
On Wednesday, July 2, 2003, at 12:53 AM, Steve Brewin wrote:
> I did say "I'm sure that everyone is in favour of hardening James as
> much as
> possible. Its just that we should approach it from a Java perspective,
> not a
> C on Unix one. The issues are different."
The only real issue that is different is that you don't have to worry
about memory overflows since the VM doesn't allow it. Other than that,
all of the basic security worries are the same no matter what language
the system is written in. And since quite a few security holes are
exploits of the code paths of a program rather than buffer overflows...
it follows that the issues aren't necessarily different.
> Why? Because the potential vulnerabilities and the optimum measures to
> eliminate them are not the same for the two environments. To take an
> approach that assumes that they are the same will result in a less
> hardend
> solution. Examining the risks as they exist in James' Java environment
> deployed on physical OSes such as Unix will achieve a maximally
> hardened
> solution.
The problem is that you can't really examine the risks all the way. A
large part of being a security weenie is protecting against what you
can't predict. And that's where the Unix BOFH mentality on security
comes from--decades of seeing what works and what doesn't. Trying to
say that the problem doesn't exist or is different using Java isn't
really helpful.
> Achieving sysadm trust is not the same as achieving a maximally
> hardened
> solution.
The latter is something that you can't prove, you can only show good
results with time. The former at least means that you play by rules
that are known to work well.
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by Steve Brewin <sb...@synsys.com>.
Stefano,
> -----Original Message-----
> From: Stefano Mazzocchi [mailto:stefano@apache.org]
> Sent: 28 June 2003 20:07
> To: James Developers List
> Cc: community@apache.org
> Subject: Re: WORA Considered Evil ;-)
>
> Reading your post that dismiss the UNIX sysadm fears as a
> "think of the
> past",
and
> "get modern and stop the crap"
These are unrepresentative paraphrases of what I said. I don't believe that
the post actually said or implied either of these things.
I did say "I'm sure that everyone is in favour of hardening James as much as
possible. Its just that we should approach it from a Java perspective, not a
C on Unix one. The issues are different."
Why? Because the potential vulnerabilities and the optimum measures to
eliminate them are not the same for the two environments. To take an
approach that assumes that they are the same will result in a less hardend
solution. Examining the risks as they exist in James' Java environment
deployed on physical OSes such as Unix will achieve a maximally hardened
solution.
Achieving sysadm trust is not the same as achieving a maximally hardened
solution. Perhaps James could achieve a level of trust from some Unix
sysadms by making it possible to mirror the deployment environments that
they trust. Fine. But as developers we shouldn't be blind to the fact this
is a minor detour on the road to a maximally hardened solution.
I don't view this as an appeal to ignore anyone. I don't make a habit of
considering anyones views as "crap".
-- Steve
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Nicola Ken Barozzi <ni...@apache.org>.
Danny Angus wrote, On 29/06/2003 13.27:
...
> James has been described by someone as Apache's "best kept secret". I met some guys (ASF members) a year or so ago and some of them were unaware that the ASF had a mailserver project.
James is a killer project, based on solid ideas and in my company
intranet it simply works (TM).
The cool thing about it is that it's pipeline based, so that it's very
very easy, I'd say near to trivial, to make messages do what you want by
using mailets, and usually the ones you need are already there.
My net admin loves the baby: MySql as a backend (she loves to be able to
do queries and simple backups), a simple config file with spam blocking,
the thing installed as a service, and nothing to touch :-)
Oh, and memory usage is really stable, with no noticeable load problems
on the server. Ok, the load is light (50 users), even if we have
multi-mega attachments for the technical drawings, but it can do all and
it's easy and stable.
I love it :-)
--
Nicola Ken Barozzi nicolaken@apache.org
- verba volant, scripta manent -
(discussions get forgotten, just code remains)
---------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Danny Angus <da...@apache.org>.
> You can choose to ignore them, but in doing so, James will just never
> exit the "cool concept" stage it has been in since its creation.
In part I agree this discussion addresses James' competitor in one field, but don't forget the whole other half of James' uphill struggle is our other competitor, M$ Exchange.
We've taken steps in that direction too, offering to try to make James meet Open Office Groupware's requirements for a groupware server. (OOogw are developing a client only)
> I believe there is critical mass for this project to exit that stage and
> enter the next phase where it really starts eroding marketshares of
> other mail servers, but the mindset of its developers must exit the 'go
> pure and screw their stupid fears' because it's not the right approach.
I don't believe that any of the active James developers have that approach, we're pretty much all here because James does the job, or because we have a job we want to make James do, or true to the Apache way "James didn't do the job I wanted, no none fixed it for me so I did it myself".
> Look: everybody on this planet hates sendmail, qmail users like the
> software very much but strongly dislike the fact there is no community
> around it, postfix is gaining momentum not because of technical
> excellence but because of its real open source nature.
>
> Combine technical and architectural excellence with an apache-style open
> development community and you have the potential to become one of the
> big players of this market.
>
> But as yourself: why this is not happening?
James has been described by someone as Apache's "best kept secret". I met some guys (ASF members) a year or so ago and some of them were unaware that the ASF had a mailserver project.
As a developer I've quite liked that out-of-the-spotlight ability to press ahead with development in peace and quiet.
However we're now getting more attention, helped no doubt by promotion to TLP, but also a renaisance of developer interest.
> Today, UNIX sysadm understand that java is not *that* unsafe, if you can
> run a those huge appservers without that many security problems. So,
> they will give you at least one chance of proving your point.
>
> So, IMO, the time has come for james to show what java can do for email.
+1
> They are willing to make a step toward you, are you willing to make a
> step toward them?
I wouldn't be happy to encourage Pier to gamble on James. Sooner or later James will be easily capable of allaying Piers fears, and *that* is the time he should use it not now.
If _anyone_ was encouraged to take a chance and try James and it let them down that would be infinitely more harmful to our reputation than having people shy away from James.
We'll fight for Piers attention when we know that James really *is* the right product for his need.
d.
Re: WORA Considered Evil ;-)
Posted by Pier Fumagalli <pi...@betaversion.org>.
On 28/6/03 8:06 pm, "Stefano Mazzocchi" <st...@apache.org> wrote:
> Reading your post that dismiss the UNIX sysadm fears as a "think of the
> past" [...]
Yeah... And I believe that now I'm finally on the other side... I'll tell
you one thing. SHIT happens, whether you use Qmail, Notes or James. And
unfortunately YOU are not the ones who get paged at 11:30 pm out on a dinner
date because the friggin's server has gone down...
Now, I _know_ that Qmail gives me a very good chance to finish my dinner,
get back home and have a good night of sleep. Will James do the same thing?
Start living with a pager as your best friend, life will change for you as
well...
Pier (paged tonight as Sobig/E is killing our bandwidth)
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Danny Angus <da...@apache.org>.
> You can choose to ignore them, but in doing so, James will just never
> exit the "cool concept" stage it has been in since its creation.
In part I agree this discussion addresses James' competitor in one field, but don't forget the whole other half of James' uphill struggle is our other competitor, M$ Exchange.
We've taken steps in that direction too, offering to try to make James meet Open Office Groupware's requirements for a groupware server. (OOogw are developing a client only)
> I believe there is critical mass for this project to exit that stage and
> enter the next phase where it really starts eroding marketshares of
> other mail servers, but the mindset of its developers must exit the 'go
> pure and screw their stupid fears' because it's not the right approach.
I don't believe that any of the active James developers have that approach, we're pretty much all here because James does the job, or because we have a job we want to make James do, or true to the Apache way "James didn't do the job I wanted, no none fixed it for me so I did it myself".
> Look: everybody on this planet hates sendmail, qmail users like the
> software very much but strongly dislike the fact there is no community
> around it, postfix is gaining momentum not because of technical
> excellence but because of its real open source nature.
>
> Combine technical and architectural excellence with an apache-style open
> development community and you have the potential to become one of the
> big players of this market.
>
> But as yourself: why this is not happening?
James has been described by someone as Apache's "best kept secret". I met some guys (ASF members) a year or so ago and some of them were unaware that the ASF had a mailserver project.
As a developer I've quite liked that out-of-the-spotlight ability to press ahead with development in peace and quiet.
However we're now getting more attention, helped no doubt by promotion to TLP, but also a renaisance of developer interest.
> Today, UNIX sysadm understand that java is not *that* unsafe, if you can
> run a those huge appservers without that many security problems. So,
> they will give you at least one chance of proving your point.
>
> So, IMO, the time has come for james to show what java can do for email.
+1
> They are willing to make a step toward you, are you willing to make a
> step toward them?
I wouldn't be happy to encourage Pier to gamble on James. Sooner or later James will be easily capable of allaying Piers fears, and *that* is the time he should use it not now.
If _anyone_ was encouraged to take a chance and try James and it let them down that would be infinitely more harmful to our reputation than having people shy away from James.
We'll fight for Piers attention when we know that James really *is* the right product for his need.
d.
Re: WORA Considered Evil ;-)
Posted by Stefano Mazzocchi <st...@apache.org>.
on 6/27/03 5:37 PM Steve Brewin wrote:
>> - the chance of a JVM exploit.
>> - potential exploits via native code in
>> a JDBC driver.
>> - the use of native code in matchers/mailets,
>> e.g., the anti-virus matcher.
>> ---------------------------------------------------
>> - the use of third party matchers/mailets.
>> - the use of user-defined scripting matchers/mailets
>> - support for SOAP
>> - one pipeline being extra busy or big
>> performing lots of processing and/or
>> handling large messages, should not
>> deny service to other users.
>>
>
> I remain unconvinced that all of the issues are potential security risks,
> but as I tried to say in an earlier posting, its a matter of trust.
Many java programmers are used to think that being so abstract from
metal, a java JVM is instrinsically more secure.
It is true that the usual types of attack like buffer overflows just
don't make any sense (unless they attack the native libraries
underneath, but even that is hard) and that the JVM security model is
very well designed (they choose security over performance and it's a
choice that, IMO, paid well, .NET does the other way around and we'll
see what happens)
At the same time, imagine taking all sendmails of the world and
substitute them with what James right now, how long would it take hack
into it?
I would suspect a few weeks.
Yeah, it's a matter of trust and it's because of paranoia that trust is
created because it's a catch-22 cycle:
- sysadm are paranoid (irrationally so, so forget about changing that)
- sysadm with big load and with a system that works, won't change it
because of architectural elegance, you need functionality: they look at
security, performance and ease of configuration/use (in that order) and
99% of them stop there
- performance can be effectively measured, ease of configuration/use
can be tried and appreciated (although irrationally as well, so be
prepared to do stuff that *they* are used to, not stuff that you think
they should be learning to do! this is the key to usability: adapt the
system to users, not the other way around) but...
- security cannot be measured, it's a matter of trust.
I'll tell you a story: Apache JServ is still considered today one of the
most solid, scalable and lightweight servlet engines on the market. So
much so that Oracle still ships it in their AppServer.
Well, we *never* were able to convince the Apache sysadms to install it
on apache machines. Why? at the end, I think, it was lack of trust. On
java, on java on freeBSD, on the load consumption, on the memory
consumption, I can't really tell you what it was, but, reality is that
it never happened.
is there an *real* reason for that? I don't think so. but we are humans,
we have feelings and we have fears. If you don't design a technical
system taking ease of use and the users' fears into consideration, you
are doomed to failure in aquiring a good and diverse community of users.
Reading your post that dismiss the UNIX sysadm fears as a "think of the
past", remind me of the discussion Pier and I had when we thought about
the exact same thing of Brian's (for us, excessive) concerns on java.
Today, after being involved in infrastructure@ for a while (just
lurking, I'm not even close to be decent enough as a sysadm to help
them), I can tell you that it's exactly this "get modern and stop the
crap" approach that is hurting java and stopping it from becoming
mainstream in main fields.
I might be wrong, but I blame it on WORA.
I'm really happy to hear Noel having a much more "down to earth"
approach to the problems, it will give James much more chances of being
appreciated by a much wider audience.
And this was the reason why I started talking about this.
You can argue all day about the technical reasons why those fears are
unjustified, but fears are not objective: there are aereonautic
engineers who are afraid of flying. From a purely rational point of
view, it should be impossible, but it happens.
Pier is a world-class java programmer and earned this merit by
participating in writing some of the best java software available on the
market. Still, and maybe because of this, he doesn't trust it as much as
it trusts other software or, let's put it this way, doesn't trust the
WORA-injected syndrome that everything should be run inside the JVM.
I heard rumors that many of the Solaris engineers in Sun think exactly
the same about java, but they are silenced by the company. Go figure.
So, at the end, while I agree with you that some of those fears are
technically hard to justify, the only objective part is that they exist.
You can choose to ignore them, but in doing so, James will just never
exit the "cool concept" stage it has been in since its creation.
I believe there is critical mass for this project to exit that stage and
enter the next phase where it really starts eroding marketshares of
other mail servers, but the mindset of its developers must exit the 'go
pure and screw their stupid fears' because it's not the right approach.
Look: everybody on this planet hates sendmail, qmail users like the
software very much but strongly dislike the fact there is no community
around it, postfix is gaining momentum not because of technical
excellence but because of its real open source nature.
Combine technical and architectural excellence with an apache-style open
development community and you have the potential to become one of the
big players of this market.
But as yourself: why this is not happening?
Today, UNIX sysadm understand that java is not *that* unsafe, if you can
run a those huge appservers without that many security problems. So,
they will give you at least one chance of proving your point.
So, IMO, the time has come for james to show what java can do for email.
They are willing to make a step toward you, are you willing to make a
step toward them?
--
Stefano.
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: WORA Considered Evil ;-)
Posted by Stefano Mazzocchi <st...@apache.org>.
on 6/27/03 5:37 PM Steve Brewin wrote:
>> - the chance of a JVM exploit.
>> - potential exploits via native code in
>> a JDBC driver.
>> - the use of native code in matchers/mailets,
>> e.g., the anti-virus matcher.
>> ---------------------------------------------------
>> - the use of third party matchers/mailets.
>> - the use of user-defined scripting matchers/mailets
>> - support for SOAP
>> - one pipeline being extra busy or big
>> performing lots of processing and/or
>> handling large messages, should not
>> deny service to other users.
>>
>
> I remain unconvinced that all of the issues are potential security risks,
> but as I tried to say in an earlier posting, its a matter of trust.
Many java programmers are used to think that being so abstract from
metal, a java JVM is instrinsically more secure.
It is true that the usual types of attack like buffer overflows just
don't make any sense (unless they attack the native libraries
underneath, but even that is hard) and that the JVM security model is
very well designed (they choose security over performance and it's a
choice that, IMO, paid well, .NET does the other way around and we'll
see what happens)
At the same time, imagine taking all sendmails of the world and
substitute them with what James right now, how long would it take hack
into it?
I would suspect a few weeks.
Yeah, it's a matter of trust and it's because of paranoia that trust is
created because it's a catch-22 cycle:
- sysadm are paranoid (irrationally so, so forget about changing that)
- sysadm with big load and with a system that works, won't change it
because of architectural elegance, you need functionality: they look at
security, performance and ease of configuration/use (in that order) and
99% of them stop there
- performance can be effectively measured, ease of configuration/use
can be tried and appreciated (although irrationally as well, so be
prepared to do stuff that *they* are used to, not stuff that you think
they should be learning to do! this is the key to usability: adapt the
system to users, not the other way around) but...
- security cannot be measured, it's a matter of trust.
I'll tell you a story: Apache JServ is still considered today one of the
most solid, scalable and lightweight servlet engines on the market. So
much so that Oracle still ships it in their AppServer.
Well, we *never* were able to convince the Apache sysadms to install it
on apache machines. Why? at the end, I think, it was lack of trust. On
java, on java on freeBSD, on the load consumption, on the memory
consumption, I can't really tell you what it was, but, reality is that
it never happened.
is there an *real* reason for that? I don't think so. but we are humans,
we have feelings and we have fears. If you don't design a technical
system taking ease of use and the users' fears into consideration, you
are doomed to failure in aquiring a good and diverse community of users.
Reading your post that dismiss the UNIX sysadm fears as a "think of the
past", remind me of the discussion Pier and I had when we thought about
the exact same thing of Brian's (for us, excessive) concerns on java.
Today, after being involved in infrastructure@ for a while (just
lurking, I'm not even close to be decent enough as a sysadm to help
them), I can tell you that it's exactly this "get modern and stop the
crap" approach that is hurting java and stopping it from becoming
mainstream in main fields.
I might be wrong, but I blame it on WORA.
I'm really happy to hear Noel having a much more "down to earth"
approach to the problems, it will give James much more chances of being
appreciated by a much wider audience.
And this was the reason why I started talking about this.
You can argue all day about the technical reasons why those fears are
unjustified, but fears are not objective: there are aereonautic
engineers who are afraid of flying. From a purely rational point of
view, it should be impossible, but it happens.
Pier is a world-class java programmer and earned this merit by
participating in writing some of the best java software available on the
market. Still, and maybe because of this, he doesn't trust it as much as
it trusts other software or, let's put it this way, doesn't trust the
WORA-injected syndrome that everything should be run inside the JVM.
I heard rumors that many of the Solaris engineers in Sun think exactly
the same about java, but they are silenced by the company. Go figure.
So, at the end, while I agree with you that some of those fears are
technically hard to justify, the only objective part is that they exist.
You can choose to ignore them, but in doing so, James will just never
exit the "cool concept" stage it has been in since its creation.
I believe there is critical mass for this project to exit that stage and
enter the next phase where it really starts eroding marketshares of
other mail servers, but the mindset of its developers must exit the 'go
pure and screw their stupid fears' because it's not the right approach.
Look: everybody on this planet hates sendmail, qmail users like the
software very much but strongly dislike the fact there is no community
around it, postfix is gaining momentum not because of technical
excellence but because of its real open source nature.
Combine technical and architectural excellence with an apache-style open
development community and you have the potential to become one of the
big players of this market.
But as yourself: why this is not happening?
Today, UNIX sysadm understand that java is not *that* unsafe, if you can
run a those huge appservers without that many security problems. So,
they will give you at least one chance of proving your point.
So, IMO, the time has come for james to show what java can do for email.
They are willing to make a step toward you, are you willing to make a
step toward them?
--
Stefano.
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> I'm sure that everyone is in favour of hardening James as much as
possible.
> Its just that we should approach it from a Java perspective, not a C on
> Unix one. The issues are different.
Not so much. We seem to all be in agreement except on the specific case of
how many JVMs per machine, and why. We can discuss that in more detail on
james-dev@, and take Community@ off the distribution. But we should keep
Pier on the list (and add Stefano?) for this discussion.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> I'm sure that everyone is in favour of hardening James as much as
possible.
> Its just that we should approach it from a Java perspective, not a C on
> Unix one. The issues are different.
Not so much. We seem to all be in agreement except on the specific case of
how many JVMs per machine, and why. We can discuss that in more detail on
james-dev@, and take Community@ off the distribution. But we should keep
Pier on the list (and add Stefano?) for this discussion.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by Steve Brewin <sb...@synsys.com>.
Noel,
I'm sure that everyone is in favour of hardening James as much as possible.
Its just that we should approach it from a Java perspective, not a C on Unix
one. The issues are different.
> Frankly, I don't want to run anything at root that can avoid
> it. That is
> just good practice.
Sure, and you don't have to. Hey, we routinely run BEA WebLogic in
distributed cluster of 20 big Sun/Solaris servers under ids. with just the
required privileges rather than root and just the one JVM per machine, no
problem. There are much larger similar configurations around. There is a big
J2EE app. server base out there that has had to face these issues and
resolve them. They have, so we can.
> Consider Vincenzo's anti-spam matcher. Would you want that
> to run as root?
No, and there is no need to as explained above. Then I wouldn't want it to
run with the same privileges as James or a James component either. It would
be good to be able to define the launch environment for an executable, which
is where most of the vulnerabilities occur. To be fair, I believe that
Vincenzo has only looked at this from the Win32 perspective and doesn't have
the options available on a Unix platform.
> > There is no need to distribute James' components over
> multiple JVMs to
> > enhance security or robustness.
>
> I disagree as a technical matter.
Such as? I can tell you in advance that my response will be that if a
problem exists it should be fixed at source so that it works within a single
JVM and then scales. Otherwise we are temporarily circumventing the problem
and someday we will hit it again in a distributed solution and ultimately
will have to fix it much more expensively. Been there.
> I can give you plenty of good reasons for why you want the
> ability to run
> the mailet pipeline at privileges other than root:
I think we are agreed that we don't want to run at root and hopefully I have
shown that you don't need to. But the following is really useful in
explaining why we don't want to and what we might want to distribute for
performance reasons.
>
> - the chance of a JVM exploit.
> - potential exploits via native code in
> a JDBC driver.
> - the use of native code in matchers/mailets,
> e.g., the anti-virus matcher.
> ---------------------------------------------------
> - the use of third party matchers/mailets.
> - the use of user-defined scripting matchers/mailets
> - support for SOAP
> - one pipeline being extra busy or big
> performing lots of processing and/or
> handling large messages, should not
> deny service to other users.
>
I remain unconvinced that all of the issues are potential security risks,
but as I tried to say in an earlier posting, its a matter of trust. We
should certainly make James sufficiently configurable that it can be
integrated into trusted practises wherever feasible, whether that trust is
well placed or not. Conversely, James should be immune as possible from
vulnerabilities exposed by malconfiguration.
> Next, consider the dynamic reconfiguration issue, which
> thread I have no had
> time to respond to over the past couple of days....
Noel, I'll respond to this in the Dynamic Reconfiguration thread, but in
short we're in agreement, you are echoing my thoughts here.
-- Steve
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: WORA Considered Evil ;-)
Posted by Steve Brewin <sb...@synsys.com>.
Noel,
I'm sure that everyone is in favour of hardening James as much as possible.
Its just that we should approach it from a Java perspective, not a C on Unix
one. The issues are different.
> Frankly, I don't want to run anything at root that can avoid
> it. That is
> just good practice.
Sure, and you don't have to. Hey, we routinely run BEA WebLogic in
distributed cluster of 20 big Sun/Solaris servers under ids. with just the
required privileges rather than root and just the one JVM per machine, no
problem. There are much larger similar configurations around. There is a big
J2EE app. server base out there that has had to face these issues and
resolve them. They have, so we can.
> Consider Vincenzo's anti-spam matcher. Would you want that
> to run as root?
No, and there is no need to as explained above. Then I wouldn't want it to
run with the same privileges as James or a James component either. It would
be good to be able to define the launch environment for an executable, which
is where most of the vulnerabilities occur. To be fair, I believe that
Vincenzo has only looked at this from the Win32 perspective and doesn't have
the options available on a Unix platform.
> > There is no need to distribute James' components over
> multiple JVMs to
> > enhance security or robustness.
>
> I disagree as a technical matter.
Such as? I can tell you in advance that my response will be that if a
problem exists it should be fixed at source so that it works within a single
JVM and then scales. Otherwise we are temporarily circumventing the problem
and someday we will hit it again in a distributed solution and ultimately
will have to fix it much more expensively. Been there.
> I can give you plenty of good reasons for why you want the
> ability to run
> the mailet pipeline at privileges other than root:
I think we are agreed that we don't want to run at root and hopefully I have
shown that you don't need to. But the following is really useful in
explaining why we don't want to and what we might want to distribute for
performance reasons.
>
> - the chance of a JVM exploit.
> - potential exploits via native code in
> a JDBC driver.
> - the use of native code in matchers/mailets,
> e.g., the anti-virus matcher.
> ---------------------------------------------------
> - the use of third party matchers/mailets.
> - the use of user-defined scripting matchers/mailets
> - support for SOAP
> - one pipeline being extra busy or big
> performing lots of processing and/or
> handling large messages, should not
> deny service to other users.
>
I remain unconvinced that all of the issues are potential security risks,
but as I tried to say in an earlier posting, its a matter of trust. We
should certainly make James sufficiently configurable that it can be
integrated into trusted practises wherever feasible, whether that trust is
well placed or not. Conversely, James should be immune as possible from
vulnerabilities exposed by malconfiguration.
> Next, consider the dynamic reconfiguration issue, which
> thread I have no had
> time to respond to over the past couple of days....
Noel, I'll respond to this in the Dynamic Reconfiguration thread, but in
short we're in agreement, you are echoing my thoughts here.
-- Steve
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
Steve and Kenny,
There is a balance. Not all of Pier's issues may apply, but many of the
important ones do.
Frankly, I don't want to run anything at root that can avoid it. That is
just good practice.
Consider Vincenzo's anti-spam matcher. Would you want that to run as root?
I am not convinced that an exploit cannot be written that would cause the
commercial, native code, virus scanner to have a buffer overflow causing it
to execute code embedded in a carefully crafted message. Can you prove
otherwise? How big a bet would you be willing to place? Because that is
the core of decision theory: what are you willing to risk on your choice?
> There is no need to distribute James' components over multiple JVMs to
> enhance security or robustness.
I disagree as a technical matter. As a practical matter, it depends upon
one's configuration and comfort level. Clearly most users experience levels
of both security and robustness with James that meet their requirements.
> The paranoia comes from what's called a buffer overflow
> exploit.
> Java however doesn't suffer from this kind of attack
As a matter of fact, there was one involving the native compression code.
I agree that Java is safer than C/C++, but that does not diminish the point.
I can give you plenty of good reasons for why you want the ability to run
the mailet pipeline at privileges other than root:
- the chance of a JVM exploit.
- potential exploits via native code in
a JDBC driver.
- the use of native code in matchers/mailets,
e.g., the anti-virus matcher.
---------------------------------------------------
- the use of third party matchers/mailets.
- the use of user-defined scripting matchers/mailets
- support for SOAP
- one pipeline being extra busy or big
performing lots of processing and/or
handling large messages, should not
deny service to other users.
None of the items below the bar are related to native code exploits. A
single JVM is not secure when you start taking into account such things as
SOAP, user-defined scripts, etc. Would you run Tomcat as root, and allow
users to install whatever servlets they want at root privilege?
For most users, the risk involved may be effectively non-existent, but the
more flexible a system becomes, the more the system must be architected to
prevent exploits due to an oversight failure. For Tomcat, I make sure that
each virtual host's JVM(s) run(s) with the access right's of the host's
owner. If I have mail applications for multiple hosts, I also want to
ensure separation, security, and privacy.
Next, consider the dynamic reconfiguration issue, which thread I have no had
time to respond to over the past couple of days. The key issue there is
that to reconfigure the pipeline, I should not have to impact the other
services. What you might do for a graceful reconfig is to put the pipeline
into a mode where it will process all messages EXCEPT for those in the root
spool. That lets you flush out the current pipeline, while allowing the
SMTP server to continue spooling messages. Then I can gracefully restart
the pipeline processor without any impact on SMTP, POP3, NNTP or other
service.
We've already had the plan for a distributed pipeline cluster, so I don't
see any reason to be arguing the point. If you look at Pier's message, you
should see GOOD things, not bad. The architecture is basically exactly what
he wants.
> Using multiple VMs forces the use of some kind of remote interface.
We already have one: the spooler and message stores. Separation would occur
where natural divisions exist.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
Steve and Kenny,
There is a balance. Not all of Pier's issues may apply, but many of the
important ones do.
Frankly, I don't want to run anything at root that can avoid it. That is
just good practice.
Consider Vincenzo's anti-spam matcher. Would you want that to run as root?
I am not convinced that an exploit cannot be written that would cause the
commercial, native code, virus scanner to have a buffer overflow causing it
to execute code embedded in a carefully crafted message. Can you prove
otherwise? How big a bet would you be willing to place? Because that is
the core of decision theory: what are you willing to risk on your choice?
> There is no need to distribute James' components over multiple JVMs to
> enhance security or robustness.
I disagree as a technical matter. As a practical matter, it depends upon
one's configuration and comfort level. Clearly most users experience levels
of both security and robustness with James that meet their requirements.
> The paranoia comes from what's called a buffer overflow
> exploit.
> Java however doesn't suffer from this kind of attack
As a matter of fact, there was one involving the native compression code.
I agree that Java is safer than C/C++, but that does not diminish the point.
I can give you plenty of good reasons for why you want the ability to run
the mailet pipeline at privileges other than root:
- the chance of a JVM exploit.
- potential exploits via native code in
a JDBC driver.
- the use of native code in matchers/mailets,
e.g., the anti-virus matcher.
---------------------------------------------------
- the use of third party matchers/mailets.
- the use of user-defined scripting matchers/mailets
- support for SOAP
- one pipeline being extra busy or big
performing lots of processing and/or
handling large messages, should not
deny service to other users.
None of the items below the bar are related to native code exploits. A
single JVM is not secure when you start taking into account such things as
SOAP, user-defined scripts, etc. Would you run Tomcat as root, and allow
users to install whatever servlets they want at root privilege?
For most users, the risk involved may be effectively non-existent, but the
more flexible a system becomes, the more the system must be architected to
prevent exploits due to an oversight failure. For Tomcat, I make sure that
each virtual host's JVM(s) run(s) with the access right's of the host's
owner. If I have mail applications for multiple hosts, I also want to
ensure separation, security, and privacy.
Next, consider the dynamic reconfiguration issue, which thread I have no had
time to respond to over the past couple of days. The key issue there is
that to reconfigure the pipeline, I should not have to impact the other
services. What you might do for a graceful reconfig is to put the pipeline
into a mode where it will process all messages EXCEPT for those in the root
spool. That lets you flush out the current pipeline, while allowing the
SMTP server to continue spooling messages. Then I can gracefully restart
the pipeline processor without any impact on SMTP, POP3, NNTP or other
service.
We've already had the plan for a distributed pipeline cluster, so I don't
see any reason to be arguing the point. If you look at Pier's message, you
should see GOOD things, not bad. The architecture is basically exactly what
he wants.
> Using multiple VMs forces the use of some kind of remote interface.
We already have one: the spooler and message stores. Separation would occur
where natural divisions exist.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by Steve Brewin <sb...@synsys.com>.
Rather than picking off Piers points one by one I'ld say that the "rock
solid arguments *not* to use JAMES as a MTA" do not hold in a Java
environment and therefore do not apply to James.
Pier trusts the setup he describes because it solves the problems
encountered in a C on Unix world. These are not the same problems
encountered by James, or any other Java application running on a JVM with an
appropriate security policy, whatever the OS.
There is no need to distibute James' components over multiple JVMs to
enhance security or robustness.
Security isn't an issue in the same way as it is with C. With C I can
happily play around with any memory location in my address space but the JVM
ensures that objects are bound to and do not break their contracts. A
greater level of isolation is provided within a JVM than is provided by
running separate processes in Unix.
A thread going down within the JVM does not take the JVM down with it unless
an unhandled Error or Exception is thrown. If the Error/Exception handling
is correct, the JVM will only terminate when there is no point in
continuing, generally due to misconfiguration or resource depletion. These
are issues that would have the same consequence in a multiple JVM, same
machine environment.
Using multiple VMs forces the use of some kind of remote interface. The poor
performance of remote interfaces is well known and contributed much to the
poor performance of EJBs until local interfaces were introduced. This is not
a road down which to travel without serious cause.
In the early days of Java there was a good reason to run multiple JVMs.
Performance when garbage collecting large heaps was very poor. Having
several smaller heaps by way of multiple JVMs was a workaround. Current JVMs
are much much better, so its no longer a good trade off against the cost of
remote interfaces.
These days, the only reason for multiple JVMs is to distribute across
multiple machines for scalability and robustness (if a machine goes down or
runs out of resources, the app. continues to run on the others).
And finally, as a general observation. Modern JVMs make extremely good use
of the native threading libraries at their disposal. They can be configured
to exploit them as required, look at Sun's latest for instance. And if you
want JVM fast start (pre-loaded, ready to go JVMs) its on its way.
-- Steve
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: WORA Considered Evil ;-)
Posted by Steve Brewin <sb...@synsys.com>.
Rather than picking off Piers points one by one I'ld say that the "rock
solid arguments *not* to use JAMES as a MTA" do not hold in a Java
environment and therefore do not apply to James.
Pier trusts the setup he describes because it solves the problems
encountered in a C on Unix world. These are not the same problems
encountered by James, or any other Java application running on a JVM with an
appropriate security policy, whatever the OS.
There is no need to distibute James' components over multiple JVMs to
enhance security or robustness.
Security isn't an issue in the same way as it is with C. With C I can
happily play around with any memory location in my address space but the JVM
ensures that objects are bound to and do not break their contracts. A
greater level of isolation is provided within a JVM than is provided by
running separate processes in Unix.
A thread going down within the JVM does not take the JVM down with it unless
an unhandled Error or Exception is thrown. If the Error/Exception handling
is correct, the JVM will only terminate when there is no point in
continuing, generally due to misconfiguration or resource depletion. These
are issues that would have the same consequence in a multiple JVM, same
machine environment.
Using multiple VMs forces the use of some kind of remote interface. The poor
performance of remote interfaces is well known and contributed much to the
poor performance of EJBs until local interfaces were introduced. This is not
a road down which to travel without serious cause.
In the early days of Java there was a good reason to run multiple JVMs.
Performance when garbage collecting large heaps was very poor. Having
several smaller heaps by way of multiple JVMs was a workaround. Current JVMs
are much much better, so its no longer a good trade off against the cost of
remote interfaces.
These days, the only reason for multiple JVMs is to distribute across
multiple machines for scalability and robustness (if a machine goes down or
runs out of resources, the app. continues to run on the others).
And finally, as a general observation. Modern JVMs make extremely good use
of the native threading libraries at their disposal. They can be configured
to exploit them as required, look at Sun's latest for instance. And if you
want JVM fast start (pre-loaded, ready to go JVMs) its on its way.
-- Steve
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Pier Fumagalli <pi...@betaversion.org>.
On 26/6/03 23:37, "Noel J. Bergman" <no...@devtech.com> wrote:
>> Pier had pretty rock solid arguments *not* to use JAMES as a MTA
>> and all came from the sysadm paranoia
>
> One example would be that you run James as root in order to access the
> privileged ports. And if James runs as root for the ports, all of the code
> runs as root. There are OS specific workarounds, but it is a valid view.
Trivial argument... A couple of JNI calls, or a port redirector in front
would do the trick...
>> Unfortunately, I don't recall exactly what his arguments were, Pier, do
>> you have a minute to chime in? I think the JAMES people would love to
>> hear your criticism.
>
> Absolutely! :-) Although the discussion would probably be more appropriate
> on the James Developer mailing list than here.
Cced... But I ain't going to subscribe to YAML! :-)
> Actually ... hmmm ... now that you've got me thinking about it, it occurs to
> me that it should be possible even now to separate James into separate JVMs
> for the SMTP service, mailet pipeline, and POP3 service.
DO IT ! :-) NOW!
> [...]
> You know ... I think that would work just fine. Maybe a bit of a tweak in
> some spooler code.
And remember to be able to read from and write to something looking (and/or
behaving) like "/usr/(lib|sbin)/sendmail", which is the de-facto standard.
>> Java doesn't have concepts like native process forking capabilities
>
> Java has threads, instead. Not all operating systems support fork(). You
> can launch new processes, but they don't start as quickly as a fork. It
> would be nice if JVM startup were light(er)weight in the presence of an
> already running a JVM.
If an OS doesn't have fork() and/or threads (both of them), it's a piece of
crap, and I won't use it... OSes I don't use for this particular reason are
the old distributions of BSD (they saw the light), Linux, and Winzop...
Till not so much time ago I was left with Solaris and Darwin... :-(
>> no notion of OS security
>
> The Java 2 security model and JAAS are different, but effective. In terms
> of file system items, it seems to me that a JNDI service provider could
> provide full OS functionality by mapping file system specific properties to
> JNDI attributes.
Annnnnddddddd.... You need JNI to read those attributes from the
filesystem... GLUE, but polluted glue, evil glue, no WORA glue...
> So when you say:
>
>> I'm aware that Noel wants the apache mail infrastructure to run on james,
>> which is a goal I would *love* to see happening, but I fear that he's not
>> taking into account the sysadm paranoia that runs in the apache
>> infrastructure team.
>
> please understand that simply being the ASF mail server is only the
> destination. The value comes from what will have had to happen to get
> there. In other words, being the ASF mail server isn't the point. Being
> good enough to be the ASF mail server is the point.
Me and Stefano both want James running on Betaversion (frankly Master Brian
is managing Apache, and he is doing an excellent job)... But when you
convince one of the Master's disciples (me) to run it, I can put a good word
in (or at least try to).
There is also a pretty peculiar problem with Qmail, it doesn't have a
license, and therefore it cannot be forked... It is not open source, you can
see the sources and publish your own patches, but the chance of seeing that
patch incorporated in the next distribution is zero, as the last
distribution was made 3/4 years ago now... (reason why I still didn't change
it is that I compiled it last 2 years ago, and it ran fine since then).
But of course Postfix is IBM Public License (OSI approved), and James is
ours, so, if James could handle the mailing list processing (no SMTP in
and/or out, only processing the lists) I bet that together with Postfix,
Cyrus and EyeBrowse they'd make a good team...
Pier
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: WORA Considered Evil ;-)
Posted by Pier Fumagalli <pi...@betaversion.org>.
On 26/6/03 23:37, "Noel J. Bergman" <no...@devtech.com> wrote:
>> Pier had pretty rock solid arguments *not* to use JAMES as a MTA
>> and all came from the sysadm paranoia
>
> One example would be that you run James as root in order to access the
> privileged ports. And if James runs as root for the ports, all of the code
> runs as root. There are OS specific workarounds, but it is a valid view.
Trivial argument... A couple of JNI calls, or a port redirector in front
would do the trick...
>> Unfortunately, I don't recall exactly what his arguments were, Pier, do
>> you have a minute to chime in? I think the JAMES people would love to
>> hear your criticism.
>
> Absolutely! :-) Although the discussion would probably be more appropriate
> on the James Developer mailing list than here.
Cced... But I ain't going to subscribe to YAML! :-)
> Actually ... hmmm ... now that you've got me thinking about it, it occurs to
> me that it should be possible even now to separate James into separate JVMs
> for the SMTP service, mailet pipeline, and POP3 service.
DO IT ! :-) NOW!
> [...]
> You know ... I think that would work just fine. Maybe a bit of a tweak in
> some spooler code.
And remember to be able to read from and write to something looking (and/or
behaving) like "/usr/(lib|sbin)/sendmail", which is the de-facto standard.
>> Java doesn't have concepts like native process forking capabilities
>
> Java has threads, instead. Not all operating systems support fork(). You
> can launch new processes, but they don't start as quickly as a fork. It
> would be nice if JVM startup were light(er)weight in the presence of an
> already running a JVM.
If an OS doesn't have fork() and/or threads (both of them), it's a piece of
crap, and I won't use it... OSes I don't use for this particular reason are
the old distributions of BSD (they saw the light), Linux, and Winzop...
Till not so much time ago I was left with Solaris and Darwin... :-(
>> no notion of OS security
>
> The Java 2 security model and JAAS are different, but effective. In terms
> of file system items, it seems to me that a JNDI service provider could
> provide full OS functionality by mapping file system specific properties to
> JNDI attributes.
Annnnnddddddd.... You need JNI to read those attributes from the
filesystem... GLUE, but polluted glue, evil glue, no WORA glue...
> So when you say:
>
>> I'm aware that Noel wants the apache mail infrastructure to run on james,
>> which is a goal I would *love* to see happening, but I fear that he's not
>> taking into account the sysadm paranoia that runs in the apache
>> infrastructure team.
>
> please understand that simply being the ASF mail server is only the
> destination. The value comes from what will have had to happen to get
> there. In other words, being the ASF mail server isn't the point. Being
> good enough to be the ASF mail server is the point.
Me and Stefano both want James running on Betaversion (frankly Master Brian
is managing Apache, and he is doing an excellent job)... But when you
convince one of the Master's disciples (me) to run it, I can put a good word
in (or at least try to).
There is also a pretty peculiar problem with Qmail, it doesn't have a
license, and therefore it cannot be forked... It is not open source, you can
see the sources and publish your own patches, but the chance of seeing that
patch incorporated in the next distribution is zero, as the last
distribution was made 3/4 years ago now... (reason why I still didn't change
it is that I compiled it last 2 years ago, and it ran fine since then).
But of course Postfix is IBM Public License (OSI approved), and James is
ours, so, if James could handle the mailing list processing (no SMTP in
and/or out, only processing the lists) I bet that together with Postfix,
Cyrus and EyeBrowse they'd make a good team...
Pier
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
[Reply in multiple pieces based on sub-topic]
> A few months ago, I had a very interesting conversation with Pier on
JAMES.
Thanks for the background. I'd heard some of it from Serge over time. And
the servlet topic gets brought up from time to time by people who see the
obvious similarities, but not the key difference.
> Pier had pretty rock solid arguments *not* to use JAMES as a MTA
> and all came from the sysadm paranoia
One example would be that you run James as root in order to access the
privileged ports. And if James runs as root for the ports, all of the code
runs as root. There are OS specific workarounds, but it is a valid view.
> Unfortunately, I don't recall exactly what his arguments were, Pier, do
> you have a minute to chime in? I think the JAMES people would love to
> hear your criticism.
Absolutely! :-) Although the discussion would probably be more appropriate
on the James Developer mailing list than here.
> [qmail's] separation-of-concerns-driven design is incredibly solid
> and very scalable, something that can be matched (or even improved,
> I believe) in a multi-threaded environment, but this gives a single
> point of failure to the system that is not acceptable from a paranoid
> sysadm point of view.
James already has a pipeline architecture. The issue you are raising is
that the pipeline is in a single process. One of the things we've talked
about is allowing multiple distributed processes coupled through stores.
There is a plan; it just takes time and resources to get there.
Actually ... hmmm ... now that you've got me thinking about it, it occurs to
me that it should be possible even now to separate James into separate JVMs
for the SMTP service, mailet pipeline, and POP3 service. Once the SMTP
service puts a message in the pipeline, its done with it, so that's OK.
Once the mailet pipeline puts a message in the pop3 store, its done with it,
so that's OK. This approach would not have a multi-process mailet pipeline
(which requires the new spooler), but it would allow the majority of the
code to run in normal user mode, and would allow different parts to be
restarted if/when desired.
You know ... I think that would work just fine. Maybe a bit of a tweak in
some spooler code.
> Java doesn't have concepts like native process forking capabilities
Java has threads, instead. Not all operating systems support fork(). You
can launch new processes, but they don't start as quickly as a fork. It
would be nice if JVM startup were light(er)weight in the presence of an
already running a JVM.
> no notion of OS security
The Java 2 security model and JAAS are different, but effective. In terms
of file system items, it seems to me that a JNDI service provider could
provide full OS functionality by mapping file system specific properties to
JNDI attributes.
> As you, I would love to see JAMES be used as the ASF email
> infrastructure because it would allow fancy interoperability,
> expecially between the email realm and the web realm, which are,
> historically, hard to mix in an elegant and coherent way.
> At the same time, I strongly believe that without a healthy amount of
> native code, it is impossible to match the sysadm needs and fears.
As I've said, one of the reasons for picking the ASF mail server as a goal
is that provides a concrete set of important, real-world, requirements that
should be met.
So when you say:
> I'm aware that Noel wants the apache mail infrastructure to run on james,
> which is a goal I would *love* to see happening, but I fear that he's not
> taking into account the sysadm paranoia that runs in the apache
> infrastructure team.
please understand that simply being the ASF mail server is only the
destination. The value comes from what will have had to happen to get
there. In other words, being the ASF mail server isn't the point. Being
good enough to be the ASF mail server is the point.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> I don't see Java as "glue" because it portrays integration with
> non-Java as anathema.
It favors portability, which means that the abstractions have to be
portable. No reason why you can't have glue classes using JNI to call
things, but I would expect whatever is part of a standard distribution to be
portable. You mentioned zlib and bzip2, amongst others.
> I just discovered that somebody wrote a bzip2 implementation
> in Java rather than use an existing codebase.
javax.util.zip.Inflater and javax.util.zip.Deflater are native code (zlib,
IIRC). I don't see why the same approach couldn't have been done with
bzip2; the library code is portable. Perhaps they wanted to reduce the
support burden of having to provide binaries for each platform. I wonder if
they benchmarked the performance difference.
With respect to DBM: http://aurora.rg.iupui.edu/~schadow/dbm-java/.
> even simple things like querying the system for the name
> of a user
String user = System.getProperty("user.name");
> and the groups the user belongs to.
public class grouptest {
static public void main(String[] args) {
System.out.println("User " + System.getProperty("user.name") +
" belongs to " +
System.getProperty("user.groups"));
}
}
$ /usr/local/java/bin/java -cp . "-Duser.groups=`groups`" grouptest
$ sudo -u <user> /usr/local/java/bin/java -cp . "-Duser.groups=`groups`"
grouptest
Dynamic queries and updates could be exposed through a JNDI service
provider, which is how I believe that it should be exposed (an alternative
would be to just run the *nix commands via Runtime.exec). People just don't
give JNDI the weight that it should have. Most think of it as an alien
concept used to connect to corporate directories. I'm doing a presentation
on JNDI in November discussing this subject.
I'm not saying that Java is perfect; just pointing out that there are
solutions, and that the "WORA" emphasis on portability promotes different
ways of approaching a solution.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Greg Stein <gs...@lyra.org>.
On Fri, Jun 27, 2003 at 08:48:18AM +0200, David N. Welton wrote:
>...
> It's also really easy to use as a command language, controlling
> various bits of other, larger, C systems. You can embed, it, extend
> it, push it this way and that - even take the source and hack on it,
> because the implementation is free software.
>
> I think I like that world view more, even if sometimes the results
> aren't so elegant. I think Lisp has a lot more going for it than
> Java, in some ways, but they really seem to suffer from the "don't
> pollute it" syndrome, down to the fact that a lot of them still harbor
> dreams of a LispOS.
I've got the same feelings about Python. I don't see Java as "glue" because
it portrays integration with non-Java as anathema. The amount of integration
(aka "glue") is *VERY* minimal. Look at Perl or Python (I'm not familiar
with Tcl, but will assume it has the same): integration with Expat, *DBM,
GD, curses, PCRE/regex-stuff, zlib, etc. Heck, even simple things like
querying the system for the name of a user and the groups the user belongs
to. All just part of Perl/Python/etc. But none of that is available to Java.
It's all reimplemented. Heck, I just discovered that somebody wrote a bzip2
implementation in Java rather than use an existing codebase.
Glue? Java? Nope. Java avoids integration with system facilities, so there
isn't a way to use it to glue together pieces. I'll use Python first, then
Perl if I want glue. Those two have always satisfied. I really can't see how
to use Java to glue, with its lack of integration and especially with its
startup time and memory footprint.
Bah. This has gone too far into language wars :-)
The one thing that I would state: when you're glueing stuff together... that
implies you want fast, quick, and dirty. It is usually a one-off. Thus, most
rules about programming go out the window. "Just get it done." And to do
that, you use the most comfortable language. If Java works for people, then
hoo-yah. More power to them. That's the way it should be. Use what works.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by "David N. Welton" <da...@dedasys.com>.
Pier Fumagalli <pi...@betaversion.org> writes:
As long as we're talking about language things, I'm going to toot the
Tcl horn some:-)
> I've never seen anyone saying that the "code-reuse" concept applies
> only to Java code, it is nice to open your eyes and see that Java
> can be the glue for PERL, JavaScript, Java, and C all together. At
> least, that's my vision, otherwise I wouldn't have learned the Java
> Native Interface by heart over the past 6 years...
The Tcl language has been a wonderful glue for the past... oh, more
than 10 years. In fact, it was designed for that. Even if you're not
crazy about the language itself (it won't win any awards from computer
scientists... err, well, besides the ACM:-), have a look at the C API.
It's documented beautifully, with man pages for each function. The
source code is, in general, worth a look. I think it's written very
well, and is a pleasure to read.
It is, of course, WORA, with regards to the main language *and* the Tk
graphical toolkit, but doesn't shy away from its origins as a glue
language. There are all kinds of connectors to other packages - Tclx
for posix calls, tclblend for a JNI interface (even Tcl implemented in
Java - Jacl), XML, SOAP, databases, and so on. And if you want, you
can use its sandbox to run code in a place where it won't do any harm.
Oh yeah, it has expect, too, which is kind of ugly, but sometimes glue
isn't pretty.
It's also really easy to use as a command language, controlling
various bits of other, larger, C systems. You can embed, it, extend
it, push it this way and that - even take the source and hack on it,
because the implementation is free software.
I think I like that world view more, even if sometimes the results
aren't so elegant. I think Lisp has a lot more going for it than
Java, in some ways, but they really seem to suffer from the "don't
pollute it" syndrome, down to the fact that a lot of them still harbor
dreams of a LispOS.
My two cents,
--
David N. Welton
Consulting: http://www.dedasys.com/
Personal: http://www.dedasys.com/davidw/
Free Software: http://www.dedasys.com/freesoftware/
Apache Tcl: http://tcl.apache.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Piping to external applications
Posted by "Noel J. Bergman" <no...@devtech.com>.
[From community@: RE: WORA Considered Evil ;-)]
> Java is the glue, let's glue it to what we have and trust... I believe
that
> 3 years ago, I asked him to do a mailet that could be an interface to
> "/usr/(lib|sbin)/sendmail", using Runtime.createProcess... After 2 years,
< that mailet didn't come up, and well, I decided to use EZMLM.
That is "before my time", but I will point out that we do have users who
have implemented anti-virus scanning in a similar manner. The message is
written out to disk, and Runtime.exec() is used to launch the native
application.
ref: http://portale.praxis.it/pub/james/
I don't see why the same basic code couldn't be used for:
<mailet match="..." class="PipeToExternal">
<command>/<path>/<executable> [replaceable parameter list]</command>
[<passthrough>false</passthrough>]
</mailet>
e.g.,
<mailet match="..." class="PipeToExternal">
<command>/usr/lib/sendmail -B7bit -f${sender} -- ${recipients}</command>
</mailet>
This would not be a difficult, or even non-portable, construction. One
question is what, if anything, to do with the output from the command. What
did you have in mind?
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: WORA Considered Evil ;-)
Posted by Stefano Mazzocchi <st...@apache.org>.
on 6/27/03 1:23 PM Noel J. Bergman wrote:
> There are good reasons for allowing the mail administrator to
> choose (easy of use and reduced memory consumption vs robustness and
> flexibility).
Agreed.
--
Stefano.
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> > I admit that doing that is not my highest priority right now. We've
> > got a lot of nice new contributions that need to be merged.
> Noel, I'm not suggesting that you do do it, certainly not that
> you do it soon either, chill out man!
LOL Don't worry. It hadn't even occured to me to take it that way, Danny.
:-) I was just setting expectations in case someone wanted to look for it
soon.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> > I admit that doing that is not my highest priority right now. We've
> > got a lot of nice new contributions that need to be merged.
> Noel, I'm not suggesting that you do do it, certainly not that
> you do it soon either, chill out man!
LOL Don't worry. It hadn't even occured to me to take it that way, Danny.
:-) I was just setting expectations in case someone wanted to look for it
soon.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by Danny Angus <da...@apache.org>.
Noel wrote:
> I admit that doing that is not my highest priority right now. We've got a
> lot of nice new contributions that need to be merged.
Noel, I'm not suggesting that you do do it, certainly not that you do it soon either, chill out man!
d.
RE: WORA Considered Evil ;-)
Posted by Danny Angus <da...@apache.org>.
Noel wrote:
> I admit that doing that is not my highest priority right now. We've got a
> lot of nice new contributions that need to be merged.
Noel, I'm not suggesting that you do do it, certainly not that you do it soon either, chill out man!
d.
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> > The internals of James models the desired architecture. The issue
> > concerning you is that the current implementation does it in a single
> > process. That is at least partially addressable with no change to the
> > architecture, minimal (if any) change to the implementation, and some
> > changes to configuration files.
> I don't see why you couldn't run multiple instances of James each only
> configured to provide one service (SMTP, SPOOL, POP etc) as it stands
> using shared repositories.
Exactly.
> I think it is probably just a configuration excercise (nightmare?).
Pretty much. :-) I think might best be a tweak to the spooler, since it
would no longer be getting a notify() from the SMTP handler. And both James
(the class) and dnsserver would be shared, since they provide services to
all.
> The problem is probably as simple as working out how to define more than
> one $PHOENIX_HOME for start/stop scripts
I admit that doing that is not my highest priority right now. We've got a
lot of nice new contributions that need to be merged. But Stefano and Pier
have raised excellent points, which should be addressed. If anything, they
validate James' architecture.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> > The internals of James models the desired architecture. The issue
> > concerning you is that the current implementation does it in a single
> > process. That is at least partially addressable with no change to the
> > architecture, minimal (if any) change to the implementation, and some
> > changes to configuration files.
> I don't see why you couldn't run multiple instances of James each only
> configured to provide one service (SMTP, SPOOL, POP etc) as it stands
> using shared repositories.
Exactly.
> I think it is probably just a configuration excercise (nightmare?).
Pretty much. :-) I think might best be a tweak to the spooler, since it
would no longer be getting a notify() from the SMTP handler. And both James
(the class) and dnsserver would be shared, since they provide services to
all.
> The problem is probably as simple as working out how to define more than
> one $PHOENIX_HOME for start/stop scripts
I admit that doing that is not my highest priority right now. We've got a
lot of nice new contributions that need to be merged. But Stefano and Pier
have raised excellent points, which should be addressed. If anything, they
validate James' architecture.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by Danny Angus <da...@apache.org>.
> The internals of James models the desired architecture. The issue
> concerning you is that the current implementation does it in a single
> process. That is at least partially addressable with no change to the
> architecture, minimal (if any) change to the implementation, and some
> changes to configuration files.
I don't see why you couldn't run multiple instances of James each only configured to provide one service (SMTP, SPOOL, POP etc) as it stands using shared repositories. I think it is probably just a configuration excercise (nightmare?).
The problem is probably as simple as working out how to define more than one $PHOENIX_HOME for start/stop scripts
d.
RE: WORA Considered Evil ;-)
Posted by Danny Angus <da...@apache.org>.
> The internals of James models the desired architecture. The issue
> concerning you is that the current implementation does it in a single
> process. That is at least partially addressable with no change to the
> architecture, minimal (if any) change to the implementation, and some
> changes to configuration files.
I don't see why you couldn't run multiple instances of James each only configured to provide one service (SMTP, SPOOL, POP etc) as it stands using shared repositories. I think it is probably just a configuration excercise (nightmare?).
The problem is probably as simple as working out how to define more than one $PHOENIX_HOME for start/stop scripts
d.
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> I was raising concerns that *real* sysadms have. I don't know any sysadm
> who, free to choose his server operating system, would choose windosh.
I personally don't consider Windows to be robust and secure enough as a
server operating system. But that is far from a universal view, and I don't
feel compelled to enforce my personal opinion by abandoning over 40% of my
community. Is httpd deciding to drop support for Windows (rhetorical)?
> If you want James to be good-enough for the ASF (even if it doesn't get
> into production), forget your windows user base and target the real
> heavy duty servers.
I don't believe it necessary to architect or implement something that
excludes Windows in order to deliver a suitable product.
The internals of James models the desired architecture. The issue
concerning you is that the current implementation does it in a single
process. That is at least partially addressable with no change to the
architecture, minimal (if any) change to the implementation, and some
changes to configuration files. Providing that configuration as an
alternative to a single JVM is certainly something we can take a close look
at offering. There are good reasons for allowing the mail administrator to
choose (easy of use and reduced memory consumption vs robustness and
flexibility).
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> I was raising concerns that *real* sysadms have. I don't know any sysadm
> who, free to choose his server operating system, would choose windosh.
I personally don't consider Windows to be robust and secure enough as a
server operating system. But that is far from a universal view, and I don't
feel compelled to enforce my personal opinion by abandoning over 40% of my
community. Is httpd deciding to drop support for Windows (rhetorical)?
> If you want James to be good-enough for the ASF (even if it doesn't get
> into production), forget your windows user base and target the real
> heavy duty servers.
I don't believe it necessary to architect or implement something that
excludes Windows in order to deliver a suitable product.
The internals of James models the desired architecture. The issue
concerning you is that the current implementation does it in a single
process. That is at least partially addressable with no change to the
architecture, minimal (if any) change to the implementation, and some
changes to configuration files. Providing that configuration as an
alternative to a single JVM is certainly something we can take a close look
at offering. There are good reasons for allowing the mail administrator to
choose (easy of use and reduced memory consumption vs robustness and
flexibility).
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: WORA Considered Evil ;-)
Posted by Stefano Mazzocchi <st...@apache.org>.
on 6/26/03 9:01 PM Noel J. Bergman wrote:
>>Well, all "decent" OSes... You won't find "fork" in stupid WindoSH...
>
> "According to market researcher OneStat.com, Windows now controls 97.46
> percent of the global desktop operating system market, compared to just 1.43
> percent for Apple Macintosh and 0.26 percent for Linux."
Bah, we are talking about different market here. I was raising concerns
that *real* sysadms have. I don't know any sysadm who, free to choose
his server operating system, would choose windosh. If only for the
frequency rate of those security holes and having to wait for them to
patch it.
If you want James to be good-enough for the ASF (even if it doesn't get
into production), forget your windows user base and target the real
heavy duty servers. They will also help you create a community of
*serious* email users, which probably have 0.1% of the market, but
handle 90% of the email of the planet.
Don't forget that Microsoft owns HotMail but it still runs on FreeBSD
machines. [I even heard that they recompiled httpd to show an ISS as the
headers but people doing TCP/IP stack analysis showed they were not
using the windows TCP/IP stack, but FreeBSD's, but maybe it's a rumor, I
can't prove this]
--
Stefano.
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> Well, all "decent" OSes... You won't find "fork" in stupid WindoSH...
"According to market researcher OneStat.com, Windows now controls 97.46
percent of the global desktop operating system market, compared to just 1.43
percent for Apple Macintosh and 0.26 percent for Linux."
Do you have statistics on the percentage of Windows downloads for httpd?
For the Java apps, the best guess would be .zip vs .tar.gz downloads. When
we did a survey of James users, the results were:
41.5% Windows
51.2% Linux
7.3% Sun Solaris
0.0% Others
So even in our community, there is a large population of Windows users.
Personally, I am typing this on a Windows system with SSH open to 4 Linux
boxes and a FreeBSD box. My next Thinkpad, perhaps I'll be able to go with
linux for the primary OS, and use a VMWare guest for Office.
> So, the silly java architects want java to be your OS and in order to
> have you buy this shit, they tell you that if you don't do this, you
> won't have WORA.
> Java is a great "glue" and inside its vision, there is a deep concept
> of "reuse"...
And I see nothing wrong with using native code, so long as you abstract the
interface to provide portability. Two examples:
- JNDI Service Provider that full access to all file system
features by mapping attributes and file system properties.
Sure, there is native code involved. But the client does
not know that, and since it has to play by the JNDI rules,
it should be happy with whatever it finds on a particular
platform.
This is how I'd have handled the Mac resource fork. I do
not know what Apple did.
- Tuple-space, or object-space, server. Sharing could be
via shared memory, database, RMI, or who cares what, so
long as the appplication doesn't know (or care).
> And I'm afraid that WORA is getting in the way and it's preventing us
> from taking java to the next level, where it enters its medieval
> religious crusade-like attitude and simply becomes a modern, peaceful
> and cooperating technology.
WORA to me means that WHEN you use native code, you have to think about how
to abstract and encapsulate, so that application code remains portable. The
JNDI example is one that comes to mind because I can take Squeak and run it
on all sorts of devices, including my PDA, but access to advanced file
system features and some other things have not been portable. The model was
tied to the platform.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Pier Fumagalli <pi...@betaversion.org>.
And now for the fun part...
On 26/6/03 17:28, "Stefano Mazzocchi" <st...@apache.org> wrote:
> [...]
> Java doesn't have concepts like native process forking capabilities, no
> notion of OS security, nothing that cannot be found in all OS is present
> and, if present in different ways, it's virtualized.
Well, all "decent" OSes... You won't find "fork" in stupid WindoSH...
> So, the silly java architects want java to be your OS and in order to
> have you buy this shit, they tell you that if you don't do this, you
> won't have WORA.
>
> And people buy it! The power of marketing.
I got rid of the marketing-centric vision imposed by Sun on Java and
suddently I realized that I started running some PERL scripts...
Ok, they might not be that nice, but in 90% of the cases, they _do_ work...
> This attitude is hurting the ability to java to be a *solution* for your
> problems, whatever they are, not simply a rotation of you problem realm.
I learned that Java is a great "glue" and inside its vision (buried beneath
a pile of crap left there by some marketing team), there is a deep concept
of "reuse"...
I've never seen anyone saying that the "code-reuse" concept applies only to
Java code, it is nice to open your eyes and see that Java can be the glue
for PERL, JavaScript, Java, and C all together. At least, that's my vision,
otherwise I wouldn't have learned the Java Native Interface by heart over
the past 6 years...
> As you, I would love to see JAMES be used as the ASF email
> infrastructure because it would allow fancy interoperability, expecially
> between the email realm and the web realm, which are, historically, hard
> to mix in an elegant and coherent way.
>
> At the same time, I strongly believe that without a healthy amount of
> native code, it is impossible to match the sysadm needs and fears.
Java is the glue, let's glue it to what we have and trust... I believe that
3 years ago, I asked him to do a mailet that could be an interface to
"/usr/(lib|sbin)/sendmail", using Runtime.createProcess... After 2 years,
that mailet didn't come up, and well, I decided to use EZMLM...
> And I'm afraid that WORA is getting in the way and it's preventing us
> from taking java to the next level, where it enters its medieval
> religious crusade-like attitude and simply becomes a modern, peaceful
> and cooperating technology.
Now you're talking like a good UNIX user (I see that MacOS/X is changing
your perception of the world). It's all about being open, and accepting what
you're given, and where others come from.
Look at Cocoon: the "Java" glue between C (Apache), Java (the unnamed
servlet container), JavaScript (the flow layer), XML (the data model) and
databases.
THAT is what Java is all about, f*** WORA...
> Apache made java known and possible on the server side.
>
> Now Apache has to liberate java from its WORA mental jail in order to
> move it to another level.
>
> And in order to do this, we must commit a few sins, one of which could
> be compiling our existing code for .NET CLI and another creating APR
> Java API and stop using the java standard libraries.
But I might ask, is it a "sin" to be open and accept something else? Some
PERL code, some C#/CLI or some C++?
> Java is a lot more than what Sun wants us to believe.
>
> Take the red pill and exit your virtualized common denominator matrix
> that the big ball of fire in the sky calls "platform", and see with your
> own eyes what java is really about.
I was taking a lot of pills few years ago... The red one must have been in
there as well, apparently! :-) :-)
Pier
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
[Reply in multiple pieces based on sub-topic]
> A few months ago, I had a very interesting conversation with Pier on
JAMES.
Thanks for the background. I'd heard some of it from Serge over time. And
the servlet topic gets brought up from time to time by people who see the
obvious similarities, but not the key difference.
> Pier had pretty rock solid arguments *not* to use JAMES as a MTA
> and all came from the sysadm paranoia
One example would be that you run James as root in order to access the
privileged ports. And if James runs as root for the ports, all of the code
runs as root. There are OS specific workarounds, but it is a valid view.
> Unfortunately, I don't recall exactly what his arguments were, Pier, do
> you have a minute to chime in? I think the JAMES people would love to
> hear your criticism.
Absolutely! :-) Although the discussion would probably be more appropriate
on the James Developer mailing list than here.
> [qmail's] separation-of-concerns-driven design is incredibly solid
> and very scalable, something that can be matched (or even improved,
> I believe) in a multi-threaded environment, but this gives a single
> point of failure to the system that is not acceptable from a paranoid
> sysadm point of view.
James already has a pipeline architecture. The issue you are raising is
that the pipeline is in a single process. One of the things we've talked
about is allowing multiple distributed processes coupled through stores.
There is a plan; it just takes time and resources to get there.
Actually ... hmmm ... now that you've got me thinking about it, it occurs to
me that it should be possible even now to separate James into separate JVMs
for the SMTP service, mailet pipeline, and POP3 service. Once the SMTP
service puts a message in the pipeline, its done with it, so that's OK.
Once the mailet pipeline puts a message in the pop3 store, its done with it,
so that's OK. This approach would not have a multi-process mailet pipeline
(which requires the new spooler), but it would allow the majority of the
code to run in normal user mode, and would allow different parts to be
restarted if/when desired.
You know ... I think that would work just fine. Maybe a bit of a tweak in
some spooler code.
> Java doesn't have concepts like native process forking capabilities
Java has threads, instead. Not all operating systems support fork(). You
can launch new processes, but they don't start as quickly as a fork. It
would be nice if JVM startup were light(er)weight in the presence of an
already running a JVM.
> no notion of OS security
The Java 2 security model and JAAS are different, but effective. In terms
of file system items, it seems to me that a JNDI service provider could
provide full OS functionality by mapping file system specific properties to
JNDI attributes.
> As you, I would love to see JAMES be used as the ASF email
> infrastructure because it would allow fancy interoperability,
> expecially between the email realm and the web realm, which are,
> historically, hard to mix in an elegant and coherent way.
> At the same time, I strongly believe that without a healthy amount of
> native code, it is impossible to match the sysadm needs and fears.
As I've said, one of the reasons for picking the ASF mail server as a goal
is that provides a concrete set of important, real-world, requirements that
should be met.
So when you say:
> I'm aware that Noel wants the apache mail infrastructure to run on james,
> which is a goal I would *love* to see happening, but I fear that he's not
> taking into account the sysadm paranoia that runs in the apache
> infrastructure team.
please understand that simply being the ASF mail server is only the
destination. The value comes from what will have had to happen to get
there. In other words, being the ASF mail server isn't the point. Being
good enough to be the ASF mail server is the point.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: WORA Considered Evil ;-)
Posted by Stefano Mazzocchi <st...@apache.org>.
on 6/26/03 4:47 PM Noel J. Bergman wrote:
> [Reply in multiple pieces based on sub-topic]
>
>
>>>A problem with multiple JVM instances is the lack of sharing between
>>>multiple instances.
>
>
>>on some operating systems, different JVMs share as much as 80% of
>>their memory.
>
>
>>>I would like to see the JVM/JIT generate and share common class
>>>code on platforms that support it.
>
>
>>This is already there.
>
>
> On which platforms? And are we talking about the same thing? Are you
> saying that if I do
>
> export CATALINA_BASE=/site1; startup.sh
> export CATALINA_BASE=/site2; startup.sh
> export CATALINA_BASE=/site3; startup.sh
> ...
> export CATALINA_BASE=/siteN; startup.sh
> service james start
>
> that the N separate sets of Tomcat processes and James are sharing memory
> images for all common class code?
I don't remember where or when, but I remember reading about some JVM (I
would suspect JVM for Solaris on Sparc, from a marketing perspective,
but Sun hardware marketing never talks to the Sun software marketing so
you never know how dumb they can end up being) that would share memory
if many JVM were running on the same machine.
Now, I really can't tell you *what* kind of memory they share. I think
that sharing *all* common classes is a potentially high security
problem, so probably they just share native code, jvm code (not the
local stack, of course) rt.jar and endorsed libraries. Which is probably
20Mb of stuff. I think it's totally reasonable to think that you can
have james and tomcat running on the same machine where both JVM use
30Mb of RAM (each), but 20Mb are shared so the real usage is
20Mb + 10Mb (for tomcat) + 10Mb (for James) = 40Mb
which is less than what your "top" says 30Mb + 30Mb = 60Mb.
of course, the more processes you run, the better because at that point,
the 20Mb overhead of the JVM is diluted across all running processes and
become, more or less, equivalent to an OS overhead.
>From memory analysis of my very laptop, I suspect that the Apple JVM
that ships with MacOSX does exactly this. In fact, you can have many
java stuff running on MacOSX without noticing (unless it's eclipse, but
that's another story since it's not even close to be optimized for macosx)
I heard that Oracle had a superscalar multi-process JVM (running inside
their database! to allow stored procedures in java) but don't know if
ever went in production.
I would not be surprised to see even IBM JVM having some fancy
inter-process shared memory facility as well.
> Do you know when/where that was
> introduced?
If not with 1.2 for sure with 1.3 but I really don't remember more (or
maybe I thought about it so much that I convinced myself it's already
working this way ;-)
--
Stefano.
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> The separate processes would share memory including a part of the
> actual JVM code as well as much of the Java library classes if (and I'm
> pretty sure only if) you are running on OS X. Apple has provided the
> technology to do this back to Sun and it will likely appear in a future
> version of Java.
I've raised that issue with folks from IBM and SUN about that for Software
Summit's running. Glad to hear that SOMEONE (or someones) have it working
finally. Very much looking forward to it when it is generally available.
:-)
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Adrian Sutton <ad...@intencha.com>.
> On which platforms? And are we talking about the same thing? Are you
> saying that if I do
>
> export CATALINA_BASE=/site1; startup.sh
> export CATALINA_BASE=/site2; startup.sh
> export CATALINA_BASE=/site3; startup.sh
> ...
> export CATALINA_BASE=/siteN; startup.sh
> service james start
>
> that the N separate sets of Tomcat processes and James are sharing
> memory
> images for all common class code? Do you know when/where that was
> introduced?
The separate processes would share memory including a part of the
actual JVM code as well as much of the Java library classes if (and I'm
pretty sure only if) you are running on OS X. Apple has provided the
technology to do this back to Sun and it will likely appear in a future
version of Java. At one stage Apple was suggesting it would likely
appear in Java 1.5 but it seems to have slipped from there.
> --- Noel
Adrian Sutton.
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
[Reply in multiple pieces based on sub-topic]
> > A problem with multiple JVM instances is the lack of sharing between
> > multiple instances.
> on some operating systems, different JVMs share as much as 80% of
> their memory.
> > I would like to see the JVM/JIT generate and share common class
> > code on platforms that support it.
> This is already there.
On which platforms? And are we talking about the same thing? Are you
saying that if I do
export CATALINA_BASE=/site1; startup.sh
export CATALINA_BASE=/site2; startup.sh
export CATALINA_BASE=/site3; startup.sh
...
export CATALINA_BASE=/siteN; startup.sh
service james start
that the N separate sets of Tomcat processes and James are sharing memory
images for all common class code? Do you know when/where that was
introduced?
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by "Richard O. Hammer" <RO...@EarthLink.net>.
Pier Fumagalli wrote:
> All those components must run ... (for security) under different user
> privileges.
Pier mentioned this point repeatedly, asserting that security can be
gained by running the various pieces of the MTA under different users'
privileges. Since I also lack sysadmin experience, I wonder if
someone could tell me the motivation for this precaution.
Historically, what went wrong that caused sysadmins to prefer running
separate pieces of an MTA under separate users' privileges?
I wonder if that kind of thing, whatever it was that went wrong, could
happen with Java and James. Java has a lot of security built in which
is lacking in C and other languages.
I wonder if a lot of the traditional sysadmin's paranoia comes from
growing up with Sendmail. As I understand the history, Sendmail had
its architectural foundations laid before anyone thought much about
security. As such, it helped raise a generation of paranoid sysadmins.
But when you understand a problem it usually suffices to solve the
problem once, just exactly once. After you have stepped on a bug, it
does not always help to step on it again a second, third, ... tenth time.
But of course paranoia evolved into the human psyche for some good
reason. Until you have mastered a problem paranoia often pays: do
anything, do everything.
Rich Hammer
Hillsborough, N.C.
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: WORA Considered Evil ;-)
Posted by Kenny Smith <ja...@journalscape.com>.
> Pier mentioned this point repeatedly, asserting that security can be
> gained by running the various pieces of the MTA under different users'
> privileges. Since I also lack sysadmin experience, I wonder if someone
> could tell me the motivation for this precaution. Historically, what
> went wrong that caused sysadmins to prefer running separate pieces of an
> MTA under separate users' privileges?
You are correct, it is a holdover from dealing with so many applications
written in C. The paranoia comes from what's called a buffer overflow
exploit. In C, if you write special data into memory that goes beyond
the end of an allocated memory chunk, then the attacker can gain access
to a shell prompt running as the owner of the process. This means, if
you were running it as root, the attacker now has root access. However,
if you segment the whole setup into multiple pieces with multiple users
and one of the pieces suffers from a buffer overflow exploit, then the
attacker only gains access to a very small piece of the system. It's all
about minimizing the potential damage should a programming bug be found.
Java however doesn't suffer from this kind of attack because the JVM
does bounds checking on your memory, so you can't overflow a buffer.
Kenny Smith
JournalScape.com
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: WORA Considered Evil ;-)
Posted by Pier Fumagalli <pi...@betaversion.org>.
NOTE: This is a technical digression on the ethics of MTAs, as Brian B.
inculcated in my brain over the past 4 years, or better, as I developed his
teachings. (Brian is master of Qmail, and Qmail is master of MTAs, being
master a transitive property). I am just a slow learner...
So, if you don't want to get into technical details about email, well, hit
DELETE now...
On 26/6/03 17:28, "Stefano Mazzocchi" <st...@apache.org> wrote:
> [...]
> So, one night, when I was visiting them in London, Pier and I sit down
> and talked about how feasible/useful/dangerous was to update our email
> infrastructure to JAMES. [despite the little coding I've done on it, I'm
> still emotionally attached to the idea of having an entire email
> infrastructure based on the beauty of java modularity and pluggability]
>
> It turned out that Pier had pretty rock solid arguments *not* to use
> JAMES as a MTA and all came from the sysadm paranoia that he grew
> accustomed to (and which I totally lack, given my very basic sysadm
> skills and experience).
>
> Unfortunately, I don't recall exactly what his arguments were, Pier, do
> you have a minute to chime in? I think the JAMES people would love to
> hear your criticism.
There are a quite consistent number of advantages in running a native MTA
compared to a Java-only solution on UNIX systems, all derived from one
single winning point: multi-processing.
Let's try to identify the main components of an MTA:
The most important piece is the mail queue: a queue is a transient storage
where messages are held temporarily, during the message processing stage.
There might be different queues per MTA (incoming, outgoing, in-process),
but one point is fundamental: the queue needs to be fast, reliable and less
messages are in any queue, better it is, at all time.
Other vital part is the "injector", aka something that reads a message from
somewhere (file, network, another queue), and stores it in a queue.
Third part is the despooler, taking a message from the queue and delivering
it somehow (to a file, a pipe, through the network, or to another queue).
Fourth and final component is the "processor", which is no more, no less,
than the union of a injector and a despooler, but only operating on queues
(therefore, a processor reads a message from the queue, does something with
it, and puts it back into the queue).
Diagram:
+----------+ +-------+ +-----------+
INPUT----->| injector |----->| |----->| despooler |----->OUTPUT
+----------+ | queue | +-----------+
+-->| |--+
| +-------+ |
| V
+----------------+
| processor |
+----------------+
Complicate it as much as you want, but this is the basic...
Add to this diagram another component, pervasive throughout the entire
drawing, a "controller", or something that makes all those separated
components talk together.
All those components must run asynchronously, independently, completely
separated from one another and (for security) under different user
privileges. NOTHING (apart from the master controlling daemon, doing
nothing) runs as root.
The lifecycle of the MTA, then, is the following:
1) the controller starts up (root) and binds to all required listening ports
2a) once a connection from the input is established (to the controller), the
controller forks, downgrades to the "queue" user and executes the
"queue" process
2a) the controller forks again, downgrades to "injector", executes the
"injector" process.
2b) the controller connects (usually via pipes, but could also work on local
network) the newly created "injector" output with the input of the
"queue" created in step 2a.
2c) the message is read from the original INPUT as the "injector" user, and
"piped" to the queue by the other process as the "queue" user. No I/O
happens as ROOT (call it defensive programming, Brian B., late 1999).
3) once the message is in the queue, if required the controller connects
"queue" with "processor" and again with "queue" in a similar way as
described in step 2. This happens as many times as it is required (a
message can be re-injected, altered, god knows, but again, nothing
works as "root" and everything is isolated from anything else).
4) once the message doesn't require further processing, again as in step
2, the controller connects "queue" with "despooler" and sends the
message.
So, overall, every single part is completely isolated from any other,
nothing runs as a privileged user, no process has power to interact or
disrupt the operation of another, apart from the controller that all it does
is "create pipes, fork, downgrade, and execute".
Notably, each interaction is transactional (so, for example, unless the
"queue" process is terminated successfully, the SMTP injector won't report
to the other end that the message has been accepted, and so on)... No
messages are lost (in theory).
You see how multi processing can hugely help in terms of reliability and
security, but there are several other advantages: every process is TINY (on
qmail in the order of 1 megabyte... It's fast to create, it's fast to
destroy, and it runs in its little sandbox, if it dies (out of memory?) all
other running processes are untouched...
That is _BY_FAR_ the best architecture ever, it might be not the fastest
one, but for sure it is the most secure and reliable.
Plus you get the advantage of running other processes most of the times. For
example, anti-virus engines, anti-spam engines, or even MUAs (mail user
agents, like IMAP/POP3 server) are all little tiny things, they come
packaged as simple binaries, and can be executed completely independently
and separated completely from the whole mail injection-process-despooling
thing.
Now, take our (betaversion) example:
> Now, Pier, Fede and I share our email infrastructure on betaversion.org.
> It's a pretty complex (and very powerful) setup made with
> qmail+cyrus+bogofilter+sieve+Horde/IMP
> [...]
Qmail works as described above (N processes running as N users doing the
different bits and bobs), bogofilter is running as a "processor" completely
separated from the Qmail processors (the ones doing alias rewriting and
stuff), cyrus runs as an injector (and in its own is separated into several
different processes as well), sieve/horde and similia run under Apache...
What I get from all this "separation" and independence? Example:
Qmail fails (or I have to take it down for some odd reason?), Stefano can
read his email from Horde via Apache on his IMAP store running on Cyrus.
Cyrus crashes? Well the message my mom sent me at the same time is queued by
Qmail and will be delivered when Cyrus comes back up.
I started disliking Qmail? Simple, I install postfix and don't touch
anything else in my entire configuration...
It's a "concerto" as Stefano pointed out correctly, of interconnected but
completely independent and self-reliant pieces of software, and it works...
Now, when I see James, I see a nice mail server, yeah, cool, but it has
everything inside it... SMTP server, QUEUE, mailing list processor, MUA,
SMTP client, web server EVERYTHING running in one big huge process, all with
the same privileges from the OS point of view, and know what, if my SMTP
engine causes a JVM internal error, my IMAP, my webmail, my mailing lists,
and my outgoing SMTP queue are stalled as well...
NOT nice, actually, it looks so much alike to Lotus Notes running on a
Windoze Server... Bulky, monolithic, hardly scalable, or interoperable with
other software...
Pier
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Joerg Pietschmann <pi...@apache.org>.
Stefano Mazzocchi wrote:
> 1) we have no access to the JVM code, we can't make it faster, even if
> we wanted and knew how (java would be *so* much faster if we could
> reimplement part of the standard library natively! expecially Strings!)
But you can reimplement parts of the standard library natively. Especially
strings. Whether this would make Java programs faster is anyones guess
though, because this would deprive HotSpot of possibilities for inlining.
J.Pietschmann
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Pier Fumagalli <pi...@betaversion.org>.
On 28/6/03 0:04, "Costin Manolache" <co...@apache.org> wrote:
> And I dislike the fact that JCP leads to a world of continuous change
> and bloating - instead of stability and depth ( what you call "stagnation"
> :-).
Funny... You should write software in Cobol, then! :-) :-) :-) :-)
Change, but only when inspired by a true innovative mind, is the very spice
that moves this world forward...
Otherwise we would be still building our tools with the more stable and deep
materials, like stones, for instance! :-)
Pier - welcome to the iron age :-)
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Costin Manolache <co...@apache.org>.
On Fri, 27 Jun 2003, Stefano Mazzocchi wrote:
> Nono, you didn't get my point: I think it's possible to get *existing*
> java classes and re-compile them into CLI.
Why would you do that ? AFAIK the CLI is more complex ( i.e. more bugs,
harder to port, harder to optimize ) than the JVM. And the security model
in java has been tested much more.
At least for me one of the reasons to use java was the sandbox and the
guarantees of the security model - which are one of the reason the
integration with other languages and JNI is so difficult.
If you don't care about the security - then it's easier and faster to just
compile to native using GCJ.
> Why bothering?
>
> 1) we have no access to the JVM code, we can't make it faster, even if
> we wanted and knew how (java would be *so* much faster if we could
> reimplement part of the standard library natively! expecially Strings!)
The GCJ library is available, nothing stops you to make it faster.
It shares the optimizer with the C++ compiler - so your work will
have double value :-)
And the class library is shared by almost all free VMs ( the classpath
project ).
> 2) the JVM bytecode was created for settop boxes. one register and one
> stack. most of the operations are performed in getting data in and out
> of that single register. Maybe it was cool 10 years ago when targetting
> *7, but today it's pure bullshit. CLI is designed as a modern CPU and
> it's much easier to compile into native code.
That's featurism talking :-) Or we are going to go into RISC vs CISC
religious debate.
If you believe the RISC argument - the JVM is easier to optimize, more
stable and may have fewer security problems ( the last one is more
important for me at least ). Usually more features and bloat in the
instruction set results doesn't help. I know today CISC are as powerfull
or more - but that doesn't mean that "simpler" is completely outdated.
> If I had time and energy, I would submit a JSR to modernize the
> bytecode. But then politics would get in, corporations would slow things
> down, marketing would impose visions and all that shit.
"Modernizing" ( bloating ) the bytecode - as well as the core APIs -
would be very bad IMO. So for one the politics would do the right thing
:-)
> I don't have it, so I think I'll just route around the big obstacle to
> the evolution of java which is called JCP.
>
> And they go around saying that JCP means innovation.
>
> I guess that in a world of perennial stagnation, anything that changes
> is named "innovation".
Funny - I dislike JCP for oposite reasons, I think it's hurting java by
adding lots of bloat and by trying to "innovate". I don't think
"inovating" and "standardization" fit very well togheter...
And I dislike the fact that JCP leads to a world of continuous change
and bloating - instead of stability and depth ( what you call "stagnation"
:-).
Costin
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Santiago Gala <sg...@hisitech.com>.
Stefano Mazzocchi escribió:
> on 6/27/03 6:43 AM Danny Angus wrote:
>
>
>>Stephano wrote:
>>
>>
>>
>>>And in order to do this, we must commit a few sins, one of which could
>>>be compiling our existing code for .NET CLI
>>
>>
>>Funny you should mention that... because I'm porting the Mailet API to .NET.
>>
>>The problem isn't with the API, but with the dependance on javax.mail in particular MimeMessage et al. For which an OS (or at least free) .NET version doesn't exist.
>
>
> Nono, you didn't get my point: I think it's possible to get *existing*
> java classes and re-compile them into CLI.
>
If I understood it correctly, running java.* or javax.* under a non Sun
VM is a license violation.
I discovered it while I was trying to run swing applications under
kaffe. I had to use javax.swing.* classes and it worked (partly, some
native patches were required in kaffe). I was evaluating the possibility
of a VNC-based AWT for the classic headless-java problem under Linux.
We could use classpath (or however they call it now) for this.
> Since java and C# are converging into two different syntaxes of the
> exact same functionality, and we have millions of lines of java code, I
> think that working at the bytecode level will give us a much better (and
> easier) portability choice.
>
> And, if that could do it for CLI or Parrot at the same time, it would be
> even better, more choice equals more freedom.
>
> Why bothering?
>
> 1) we have no access to the JVM code, we can't make it faster, even if
> we wanted and knew how (java would be *so* much faster if we could
> reimplement part of the standard library natively! expecially Strings!)
>
The hotspot is supposed to do this, isn't it? (I would treat some
standard classes as *very* special if I was in the hotspot team).
> 2) the JVM bytecode was created for settop boxes. one register and one
> stack. most of the operations are performed in getting data in and out
> of that single register. Maybe it was cool 10 years ago when targetting
> *7, but today it's pure bullshit. CLI is designed as a modern CPU and
> it's much easier to compile into native code.
>
> 3) parrot has native continuations. I expect Java to have continuations
> in 1.6 or sooner if .NET adds them. But the VM functionalities are
> converging.
>
> If I had time and energy, I would submit a JSR to modernize the
> bytecode. But then politics would get in, corporations would slow things
> down, marketing would impose visions and all that shit.
>
It can be done (I'm not saying "I'd" or "You could" just in case someone
thinks I'm volunteering ;-) ) by "embrace and extend", aka "Just do
it!". I mean, completely outside of the JCP, as a "de facto" standard.
If they are slow, we can invite them to join us in the future. This
would only work if it effectively outpaced java evolution while being a
solid alternative.
It could not be named java (jamaica is a nice island too ;-) ), also
licensing could bring problems, see above. But it could be a way to
expose that the "Emperor new clothes" are really thin to the public.
> I don't have it, so I think I'll just route around the big obstacle to
> the evolution of java which is called JCP.
>
> And they go around saying that JCP means innovation.
>
> I guess that in a world of perennial stagnation, anything that changes
> is named "innovation".
>
> Bah, there is nobody blinder that those who don't want to see.
>
--
Santiago Gala
High Sierra Technology, S.L. (http://hisitech.com)
http://memojo.com?page=SantiagoGalaBlog
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> Aren't the same people right now are saying that Java DOT-NET means
> community-based software development???
It is a start. java.net is not the ASF (believe me, I have had that
discussion), but give them credit for what they are doing, and encourage
them to go further.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Pier Fumagalli <pi...@betaversion.org>.
On 27/6/03 17:37, "Stefano Mazzocchi" <st...@apache.org> wrote:
> And they go around saying that JCP means innovation.
Aren't the same people right now are saying that Java DOT-NET means
community-based software development???
> I guess that in a world of perennial stagnation, anything that changes
> is named "innovation".
But isn't that change locking back into the same stagnation?
At the end, what is sold to you as innovation, is nothing but a small change
that simply ties you more and more into a more profitable marketing
solution...
Pier (disgruntled)
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> >Sun is moving to do a lot more with Open Source licenses. Let's hope
that
> >discussions about migrating that code to .NET doesn't discourage them.
> Exactly how many years have they ignored such issues?
I'm not a Sun apologist. Just reporting. See
http://www.java.net/choose_license.html, and related projects from Sun.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
At 04:05 PM 6/27/2003, Noel J. Bergman wrote:
>> the problem I have encoutered 1st hand is that there is are *LOT* of java
>> libraries that we all use everyday which are licenced to us by Sun
>> (and possibly others) under proprietery licences of one kind or another
>
>Sun is moving to do a lot more with Open Source licenses. Let's hope that
>discussions about migrating that code to .NET doesn't discourage them.
Exactly how many years have they ignored such issues?
.NET discussions haven't discouraged them, they seem to be incited to
finally react to the real world, wherein java microcode engines could be
seriously displaced by mono-style engines.
Bill
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> the problem I have encoutered 1st hand is that there is are *LOT* of java
> libraries that we all use everyday which are licenced to us by Sun
> (and possibly others) under proprietery licences of one kind or another
Sun is moving to do a lot more with Open Source licenses. Let's hope that
discussions about migrating that code to .NET doesn't discourage them.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by Danny Angus <da...@apache.org>.
Stefhano wrote:
> Nono, you didn't get my point: I think it's possible to get *existing*
> java classes and re-compile them into CLI.
>
> Since java and C# are converging into two different syntaxes of the
> exact same functionality, and we have millions of lines of java code, I
> think that working at the bytecode level will give us a much better (and
> easier) portability choice.
You're right, I kind of missed the point, I get the bytecode argument though.
And yes, the language similarity between c# and java is staggering. And no, I don't want to get into big discussions about delegates (aka function pointers) and structs here.
The point I was making (badly) is that the problem isn't translating java to c#, or even presumaby compiling java code (Visual J .NET) to .NET bytecode, the problem I have encoutered 1st hand is that there is are *LOT* of java libraries that we all use everyday which are licenced to us by Sun (and possibly others) under proprietery licences of one kind or another with varying degrees of freedom.
For this idea to take off there would have to be either a lot of things re-written from scratch without dependance on proprietery code, or some new VM which would allow java and .NET bytecode to be used interchangeably, and I'm guessing here but I bet that Sun wouldn't like to see anyone try this with any parts of J2EE.
I suspect the that issues which might sink this are ones of scale and politics not technology.
d.
Re: WORA Considered Evil ;-)
Posted by Stefano Mazzocchi <st...@apache.org>.
on 6/27/03 6:43 AM Danny Angus wrote:
> Stephano wrote:
>
>
>>And in order to do this, we must commit a few sins, one of which could
>>be compiling our existing code for .NET CLI
>
>
> Funny you should mention that... because I'm porting the Mailet API to .NET.
>
> The problem isn't with the API, but with the dependance on javax.mail in particular MimeMessage et al. For which an OS (or at least free) .NET version doesn't exist.
Nono, you didn't get my point: I think it's possible to get *existing*
java classes and re-compile them into CLI.
Since java and C# are converging into two different syntaxes of the
exact same functionality, and we have millions of lines of java code, I
think that working at the bytecode level will give us a much better (and
easier) portability choice.
And, if that could do it for CLI or Parrot at the same time, it would be
even better, more choice equals more freedom.
Why bothering?
1) we have no access to the JVM code, we can't make it faster, even if
we wanted and knew how (java would be *so* much faster if we could
reimplement part of the standard library natively! expecially Strings!)
2) the JVM bytecode was created for settop boxes. one register and one
stack. most of the operations are performed in getting data in and out
of that single register. Maybe it was cool 10 years ago when targetting
*7, but today it's pure bullshit. CLI is designed as a modern CPU and
it's much easier to compile into native code.
3) parrot has native continuations. I expect Java to have continuations
in 1.6 or sooner if .NET adds them. But the VM functionalities are
converging.
If I had time and energy, I would submit a JSR to modernize the
bytecode. But then politics would get in, corporations would slow things
down, marketing would impose visions and all that shit.
I don't have it, so I think I'll just route around the big obstacle to
the evolution of java which is called JCP.
And they go around saying that JCP means innovation.
I guess that in a world of perennial stagnation, anything that changes
is named "innovation".
Bah, there is nobody blinder that those who don't want to see.
--
Stefano.
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: WORA Considered Evil ;-)
Posted by Danny Angus <da...@apache.org>.
Stephano wrote:
>And in order to do this, we must commit a few sins, one of which could
> be compiling our existing code for .NET CLI
Funny you should mention that... because I'm porting the Mailet API to .NET.
The problem isn't with the API, but with the dependance on javax.mail in particular MimeMessage et al. For which an OS (or at least free) .NET version doesn't exist.
d.
Re: WORA Considered Evil ;-)
Posted by Pier Fumagalli <pi...@betaversion.org>.
NOTE: This is a technical digression on the ethics of MTAs, as Brian B.
inculcated in my brain over the past 4 years, or better, as I developed his
teachings. (Brian is master of Qmail, and Qmail is master of MTAs, being
master a transitive property). I am just a slow learner...
So, if you don't want to get into technical details about email, well, hit
DELETE now...
On 26/6/03 17:28, "Stefano Mazzocchi" <st...@apache.org> wrote:
> [...]
> So, one night, when I was visiting them in London, Pier and I sit down
> and talked about how feasible/useful/dangerous was to update our email
> infrastructure to JAMES. [despite the little coding I've done on it, I'm
> still emotionally attached to the idea of having an entire email
> infrastructure based on the beauty of java modularity and pluggability]
>
> It turned out that Pier had pretty rock solid arguments *not* to use
> JAMES as a MTA and all came from the sysadm paranoia that he grew
> accustomed to (and which I totally lack, given my very basic sysadm
> skills and experience).
>
> Unfortunately, I don't recall exactly what his arguments were, Pier, do
> you have a minute to chime in? I think the JAMES people would love to
> hear your criticism.
There are a quite consistent number of advantages in running a native MTA
compared to a Java-only solution on UNIX systems, all derived from one
single winning point: multi-processing.
Let's try to identify the main components of an MTA:
The most important piece is the mail queue: a queue is a transient storage
where messages are held temporarily, during the message processing stage.
There might be different queues per MTA (incoming, outgoing, in-process),
but one point is fundamental: the queue needs to be fast, reliable and less
messages are in any queue, better it is, at all time.
Other vital part is the "injector", aka something that reads a message from
somewhere (file, network, another queue), and stores it in a queue.
Third part is the despooler, taking a message from the queue and delivering
it somehow (to a file, a pipe, through the network, or to another queue).
Fourth and final component is the "processor", which is no more, no less,
than the union of a injector and a despooler, but only operating on queues
(therefore, a processor reads a message from the queue, does something with
it, and puts it back into the queue).
Diagram:
+----------+ +-------+ +-----------+
INPUT----->| injector |----->| |----->| despooler |----->OUTPUT
+----------+ | queue | +-----------+
+-->| |--+
| +-------+ |
| V
+----------------+
| processor |
+----------------+
Complicate it as much as you want, but this is the basic...
Add to this diagram another component, pervasive throughout the entire
drawing, a "controller", or something that makes all those separated
components talk together.
All those components must run asynchronously, independently, completely
separated from one another and (for security) under different user
privileges. NOTHING (apart from the master controlling daemon, doing
nothing) runs as root.
The lifecycle of the MTA, then, is the following:
1) the controller starts up (root) and binds to all required listening ports
2a) once a connection from the input is established (to the controller), the
controller forks, downgrades to the "queue" user and executes the
"queue" process
2a) the controller forks again, downgrades to "injector", executes the
"injector" process.
2b) the controller connects (usually via pipes, but could also work on local
network) the newly created "injector" output with the input of the
"queue" created in step 2a.
2c) the message is read from the original INPUT as the "injector" user, and
"piped" to the queue by the other process as the "queue" user. No I/O
happens as ROOT (call it defensive programming, Brian B., late 1999).
3) once the message is in the queue, if required the controller connects
"queue" with "processor" and again with "queue" in a similar way as
described in step 2. This happens as many times as it is required (a
message can be re-injected, altered, god knows, but again, nothing
works as "root" and everything is isolated from anything else).
4) once the message doesn't require further processing, again as in step
2, the controller connects "queue" with "despooler" and sends the
message.
So, overall, every single part is completely isolated from any other,
nothing runs as a privileged user, no process has power to interact or
disrupt the operation of another, apart from the controller that all it does
is "create pipes, fork, downgrade, and execute".
Notably, each interaction is transactional (so, for example, unless the
"queue" process is terminated successfully, the SMTP injector won't report
to the other end that the message has been accepted, and so on)... No
messages are lost (in theory).
You see how multi processing can hugely help in terms of reliability and
security, but there are several other advantages: every process is TINY (on
qmail in the order of 1 megabyte... It's fast to create, it's fast to
destroy, and it runs in its little sandbox, if it dies (out of memory?) all
other running processes are untouched...
That is _BY_FAR_ the best architecture ever, it might be not the fastest
one, but for sure it is the most secure and reliable.
Plus you get the advantage of running other processes most of the times. For
example, anti-virus engines, anti-spam engines, or even MUAs (mail user
agents, like IMAP/POP3 server) are all little tiny things, they come
packaged as simple binaries, and can be executed completely independently
and separated completely from the whole mail injection-process-despooling
thing.
Now, take our (betaversion) example:
> Now, Pier, Fede and I share our email infrastructure on betaversion.org.
> It's a pretty complex (and very powerful) setup made with
> qmail+cyrus+bogofilter+sieve+Horde/IMP
> [...]
Qmail works as described above (N processes running as N users doing the
different bits and bobs), bogofilter is running as a "processor" completely
separated from the Qmail processors (the ones doing alias rewriting and
stuff), cyrus runs as an injector (and in its own is separated into several
different processes as well), sieve/horde and similia run under Apache...
What I get from all this "separation" and independence? Example:
Qmail fails (or I have to take it down for some odd reason?), Stefano can
read his email from Horde via Apache on his IMAP store running on Cyrus.
Cyrus crashes? Well the message my mom sent me at the same time is queued by
Qmail and will be delivered when Cyrus comes back up.
I started disliking Qmail? Simple, I install postfix and don't touch
anything else in my entire configuration...
It's a "concerto" as Stefano pointed out correctly, of interconnected but
completely independent and self-reliant pieces of software, and it works...
Now, when I see James, I see a nice mail server, yeah, cool, but it has
everything inside it... SMTP server, QUEUE, mailing list processor, MUA,
SMTP client, web server EVERYTHING running in one big huge process, all with
the same privileges from the OS point of view, and know what, if my SMTP
engine causes a JVM internal error, my IMAP, my webmail, my mailing lists,
and my outgoing SMTP queue are stalled as well...
NOT nice, actually, it looks so much alike to Lotus Notes running on a
Windoze Server... Bulky, monolithic, hardly scalable, or interoperable with
other software...
Pier
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: WORA Considered Evil ;-)
Posted by "Noel J. Bergman" <no...@devtech.com>.
[Reply in multiple, shorter, pieces based on sub-topic]
> > However, having something like httpd front-end lots of backend JVMs on
> > multiple machines is nice.
> Hey, I know that. I was one of the designers of mod_jserv, you know? ;-)
Let me think ... mod_jserv ... would that be the thing I started using after
Jeeves, and which I still have about a dozen hosts running (although not for
long)?
Yes, I knew that. :-) I was surprised that you didn't mention it in your
otherwise well-explained lament about the drawbacks of a single JVM.
> > And what prevents mod_jk2 from using a launcher
> > to control and monitor JVM processes?
> Nothing. that's exactly how mod_jserv worked.
When not in manual mode (ApJServManual on), mod_jserv launched the JVM, but
unless I missed a switch, there was no way to control per-JVM user/group.
Also, I was talking about a separate launcher service that could run on
other nodes in the network. Sort of like an inetd (don't take that too
literally!) tailored to mod_jk needs.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Stefano Mazzocchi <st...@apache.org>.
on 6/26/03 1:03 AM Noel J. Bergman wrote:
> Do you really know only such cloistered Java programmers despite all of the
> ones you meet?
Yes. Very few have the guts to exit the celebrated monocultural java
vision (and I was one of them since not so long ago)
>>basically they are totally isolated, which leads to concepts such as
>
> server
>
>>microkernel architectures (avalon Phoenix) which look cool from a purely
>>architectural perspective, but are totally weak from a security and
>>stability perspective, because they use one JVM for the entire thing, a
>>very weak setup.
>
>
> However, having something like httpd front-end lots of backend JVMs on
> multiple machines is nice.
Hey, I know that. I was one of the designers of mod_jserv, you know? ;-)
> And what prevents mod_jk2 from using a launcher
> to control and monitor JVM processes?
Nothing. that's exactly how mod_jserv worked.
> It would be nice to be able to do
> something like:
>
> <virtualhost ...>
> ServerName RingBearer
> JkUserID frodo FoTR
> Jk...
> ...
> </virtualhost>
>
> and have Coyote communicate to a "jkLauncher" a connection request for a JVM
> running as that user (configuration of the JVM not being the launcher's
> responsibility). It would certainly make my configuration easier, and more
> robust. Currently, I do something similar by hand.
>
> A problem with multiple JVM instances is the lack of sharing between
> multiple instances.
??? on some operating systems, different JVMs share as much as 80% of
their memory.
> f you run multiple instances of tomcat, one per
> virtual host for example, your memory overhead is considerable since none of
> the class code is shared. I would like to see the JVM/JIT generate and
> share common class code on platforms that support it.
This is already there.
> Sun appears to prefer intra-process threading or inter-process communication
> over network interfaces. You might find:
>
> http://www.iee.org/Publish/Journals/ProfJourn/Proc/SEN/vinter.pdf
> http://www.cs.berkeley.edu/~szewczyk/cs267/Final.html
> http://matsu-www.is.titech.ac.jp/~sohda/research/javag2001.pdf
>
> interesting. I didn't say practical. :-) Although JPS looks interesting.
>
> True, Java is not a systems programming language. But without WORA, I do
> not believe that Java would have the success that it has on the server.
A few months ago, I had a very interesting conversation with Pier on JAMES.
<background>JAMES (james.apache.org) is a mail server written in Java
that contains pluggable programmatic modules called "mailet" to extend
its functionality (much like servlets for an HTTP server)
Pier and I designed the concept of Mailet on the plane going to
ApacheCON '98, then we met with James Duncan Davidson at Sun to discuss
the issues we had with the Servlet API at that time (James was not yet
an apache fellow, but he was the spec lead for the Servlet API). There
we proposed to add "mail" capabilities to the servlet API.
After a long discussion, James turned the proposal down, basically
because the Servlet API *looked* like it's abstract enough to take mail
operation in consideration, but it's not: it's designed with the concept
that the response goes back to the requestor. Which is not the case for
email.
Also, there were major marketing problems in moving a clearly
web-centric architecture in a realm (email servers) where Sun (nor the
other vendors) had no interest in going.
So, we created the Mailet API and started JAMES, later we had Federico
involved that did most of the coding.</background>
Now, Pier, Fede and I share our email infrastructure on betaversion.org.
It's a pretty complex (and very powerful) setup made with
qmail+cyrus+bogofilter+sieve+Horde/IMP glued with some croned-invoqued
python code that updates every 5 minutes the bogofilter database
extracting the spam/ham from the cyrus folders. I also added some PHP to
IMP to allow me to update my sieve file directly from a web interface.
The system is solid, very powerful, but hey, from the eyes of a software
architect, looks messy as hell.
So, one night, when I was visiting them in London, Pier and I sit down
and talked about how feasible/useful/dangerous was to update our email
infrastructure to JAMES. [despite the little coding I've done on it, I'm
still emotionally attached to the idea of having an entire email
infrastructure based on the beauty of java modularity and pluggability]
It turned out that Pier had pretty rock solid arguments *not* to use
JAMES as a MTA and all came from the sysadm paranoia that he grew
accustomed to (and which I totally lack, given my very basic sysadm
skills and experience).
Unfortunately, I don't recall exactly what his arguments were, Pier, do
you have a minute to chime in? I think the JAMES people would love to
hear your criticism.
Anyway, i remember that all of his points came from the fact that by
using the JVM you are basically giving away all multi-processing
features of the operating system and since UNIX is basically designed
around the concept of processes, java strongly limits your ability to
'well behave' in a modern operating system environment.
this is something I remember Brian was strongly concerned about as well:
qmail is *strongly* multi-processed and this is something that qmail
users normally love. qmail is an orchestra of processes, each with
different capabilities and security protections, that work toward a the
common goal of delivering email. This separation-of-concerns-driven
design is incredibly solid and very scalable (too bad qmail is not
opensource and it's a one-man show), something that can be matched (or
even improved, I believe) in a multi-threaded environment, but this
gives a single point of failure to the system that is not acceptable
from a paranoid sysadm point of view.
So, my point is: java is great and solves a lot of problems, but it's
'common denominator abstraction' design paradigm is, IMO, hurting
severely the ability to adapt to different environments.
SWT vs. Swing shows exactly what I mean: SWT is native, has to be
implemented completely in every OS, but it *ROCKS* compared to Swing,
which is a common denominator abstraction, but it's *SLOW* as hell,
because such virtualization stops it from being hardware accelerated
(so, basically, Swing is 10 years behind from a graphic architecture
perspective, except on MacOSX, where Swing is natively implemented, thus
accelerated)
Java doesn't have concepts like native process forking capabilities, no
notion of OS security, nothing that cannot be found in all OS is present
and, if present in different ways, it's virtualized.
So, the silly java architects want java to be your OS and in order to
have you buy this shit, they tell you that if you don't do this, you
won't have WORA.
And people buy it! The power of marketing.
This attitude is hurting the ability to java to be a *solution* for your
problems, whatever they are, not simply a rotation of you problem realm.
As you, I would love to see JAMES be used as the ASF email
infrastructure because it would allow fancy interoperability, expecially
between the email realm and the web realm, which are, historically, hard
to mix in an elegant and coherent way.
At the same time, I strongly believe that without a healthy amount of
native code, it is impossible to match the sysadm needs and fears.
And I'm afraid that WORA is getting in the way and it's preventing us
from taking java to the next level, where it enters its medieval
religious crusade-like attitude and simply becomes a modern, peaceful
and cooperating technology.
Apache made java known and possible on the server side.
Now Apache has to liberate java from its WORA mental jail in order to
move it to another level.
And in order to do this, we must commit a few sins, one of which could
be compiling our existing code for .NET CLI and another creating APR
Java API and stop using the java standard libraries.
Java is a lot more than what Sun wants us to believe.
Take the red pill and exit your virtualized common denominator matrix
that the big ball of fire in the sky calls "platform", and see with your
own eyes what java is really about.
--
Stefano.
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Glen Stampoultzis <gs...@iinet.net.au>.
At 01:21 AM 27/06/2003, you wrote:
>BTW, Glen, HTTPd is written in C and it's *by far* more portable than
>any java program out there.
I'll buy that.
>At the same time, the amount of work done to allow this portability is
>impressive, while, compiling for a JVM, gets you instant portability and
>almost no cost.
This was basically what I was referring to in my original post. Ease of
portability not that Java was more portable.
Glen Stampoultzis
gstamp@iinet.net.au
http://members.iinet.net.au/~gstamp/glen/
Re: WORA Considered Evil ;-)
Posted by Stefano Mazzocchi <st...@apache.org>.
on 6/26/03 8:03 AM David N. Welton wrote:
> Glen Stampoultzis <gs...@iinet.net.au> writes:
>
>
>>Yes. As dogmatic as Sun has been about "pure" Java it's still a
>>success factor in the adoption of Java. There's still no other
>>platform out there that makes it as easy as Java to write for
>>multiple platforms.
>
>
> Errr... really?
Glen is talking about the JVM, I suspect, more than the java language.
This is in line with things like Jython and PHP5 being all compiled to
java bytecode.
It is even more so in .NET where the CLI virtual machine has been
designed to be similar to a modern high-end processor, not as a
microwave oven microcontroller one (as, unfortunately, it is the case
with java, where the JVM architecture sucks ass).
And exactly for this reason, if Mono is successful and doesn't get
locked by Microsoft legal battles and Sun doesn't open up the JVM, we'll
simply write a compiler from JVM to CLI and simply mix the best of both
worlds, escaping, at least, vendor lockin from the JVM part.... and
hopefully removing some of that "holy WORA" syndrome that is really
stopping people from thinking about the fact that diversity is good for
you and doesn't necessarely prevent portability.
BTW, Glen, HTTPd is written in C and it's *by far* more portable than
any java program out there.
At the same time, the amount of work done to allow this portability is
impressive, while, compiling for a JVM, gets you instant portability and
almost no cost.
--
Stefano.
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by "David N. Welton" <da...@dedasys.com>.
Glen Stampoultzis <gs...@iinet.net.au> writes:
> Yes. As dogmatic as Sun has been about "pure" Java it's still a
> success factor in the adoption of Java.� There's still no other
> platform out there that makes it as easy as Java to write for
> multiple platforms.�
Errr... really?
--
David N. Welton
Consulting: http://www.dedasys.com/
Personal: http://www.dedasys.com/davidw/
Free Software: http://www.dedasys.com/freesoftware/
Apache Tcl: http://tcl.apache.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: Java + Scripting languages
Posted by Sanjiva Weerawarana <sa...@opensource.lk>.
I would also like to understand how this relates to BSF
http://jakarta.apache.org/bsf
BSF also defines such an API ...
Sanjiva.
----- Original Message -----
From: "David N. Welton" <da...@dedasys.com>
To: <co...@apache.org>
Sent: Thursday, June 26, 2003 6:55 PM
Subject: Java + Scripting languages
>
> Hi guys, I saw this:
>
> http://www.jcp.org/en/jsr/detail?id=223
>
> The specification may include a Java API that can be used,
> possibly through JNI, by an scripting language engine to
> access the desired Java objects.
>
> Can anyone give us a more concrete description of what this is really
> about?
>
> It looks interesting, because... hey, who wouldn't want to associate
> with a million dollar marketing machine:-)
>
> --
> David N. Welton
> Consulting: http://www.dedasys.com/
> Personal: http://www.dedasys.com/davidw/
> Free Software: http://www.dedasys.com/freesoftware/
> Apache Tcl: http://tcl.apache.org/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: community-unsubscribe@apache.org
> For additional commands, e-mail: community-help@apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: Java + Scripting languages
Posted by Stefano Mazzocchi <st...@apache.org>.
on 6/27/03 5:58 AM Sam Ruby wrote:
> Sometimes it sucks to be four years ahead of your time.
Amen.
--
Stefano.
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: Java + Scripting languages
Posted by Santiago Gala <sg...@hisitech.com>.
Sam Ruby escribió:
>
> Sometimes it sucks to be four years ahead of your time.
>
Read Marc Portier's blog entry today, (
http://radio.weblogs.com/0116284/ ), specifically the sentence:
"Mmmm. Bumpy lives for those that see, but fail to make others see..."
BTW, it is about Galileo
> - Sam Ruby
>
>
--
Santiago Gala
High Sierra Technology, S.L. (http://hisitech.com)
http://memojo.com?page=SantiagoGalaBlog
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: Java + Scripting languages
Posted by Sam Ruby <ru...@apache.org>.
Craig R. McClanahan wrote:
>
>>http://www.jcp.org/en/jsr/detail?id=223
>>
>> The specification may include a Java API that can be used,
>> possibly through JNI, by an scripting language engine to
>> access the desired Java objects.
>>
>>Can anyone give us a more concrete description of what this is really
>>about?
>
> The basis for this is exactly what that sentence states -- scripting
> language users have said they would like to be able to access business
> logic and data objects inside a servlet-based application from their
> scripts, in a portable manner. The point of the JSR is to make that sort
> of thing possible.
>
> As Stefano points out, Sam did indeed create some code to do this. Just
> two little problem though, it's non-thread safe (uses instance variables
> for per-request state), and it's not scalable. And, it only deals with
> PHP, but there's lots of other scripting languages (and scripting language
> users) in the world that could benefit from the same ability.
What I did with PHP was only a proof of concept. My focus wasn't only
on PHP, but on a wide number of languages.
I met personally with Eduardo on more than one occasion trying to
generate interest in the subject. At the time, it was clear that his
focus on on the 'one true programming language'.
Sometimes it sucks to be four years ahead of your time.
- Sam Ruby
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: Java + Scripting languages
Posted by "Victor J. Orlikowski" <vi...@alumni.duke.edu>.
On Thu, Jun 26, 2003 at 05:22:38PM -0700, Craig R. McClanahan wrote:
> Accessing Java objects defined in the system class loader doesn't require
> anything new -- JNI provides all the necessary hooks. But, to interact
> with web app resources, you need to do things like load classes from the
> webapp's class loader, and gain access to the ServletContext instance,
> and perhaps even do nice things like utilize the servlet container's
> session mechanism for scripting languages that don't have such a notion.
> Such things can be designed and built for a particular server today, but
> there's no standard approach; hence the JSR.
>
Hmmm.
It's going to be interesting to see the confluence of 223 and 045
(debugging support for other languages); I think the integration
of languages via JNI is going to present a near-brick wall when it
comes to sane debugging (that said, 045 assumes that compilation
to bytecode is possible, which may allow it to preclude some of
the languages brought in under 223).
But then, what do I know? :)
I will say that BSF does provide the object registry for languages
that are implemented in Java; it's up to the individual BSFEngine
implementation to mediate access to those objects. Having a
BSFEngine that "does the right thing" via JNI is not hard to
envision (in fact, I think the ActiveScript engine did something
similar for WSH).
Victor
--
Victor J. Orlikowski | The Wall is Down, But the Threat Remains!
=================================================================
orlikowski@apache.org | victor.j.orlikowski@alumni.duke.edu
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: Java + Scripting languages
Posted by "David N. Welton" <da...@dedasys.com>.
"Craig R. McClanahan" <cr...@apache.org> writes:
> On Thu, 26 Jun 2003, David N. Welton wrote:
> > Hi guys, I saw this:
> > http://www.jcp.org/en/jsr/detail?id=223
> > The specification may include a Java API that can be used,
> > possibly through JNI, by an scripting language engine to
> > access the desired Java objects.
> > Can anyone give us a more concrete description of what this is
> > really about?
> The basis for this is exactly what that sentence states -- scripting
> language users have said they would like to be able to access
> business logic and data objects inside a servlet-based application
> from their scripts, in a portable manner. The point of the JSR is
> to make that sort of thing possible.
Part of what made me ask was the 'may' and 'possibly'. They don't
sound very convinced.
> Accessing Java objects defined in the system class loader doesn't
> require anything new -- JNI provides all the necessary hooks. But,
> to interact with web app resources, you need to do things like load
> classes from the webapp's class loader, and gain access to the
> ServletContext instance, and perhaps even do nice things like
> utilize the servlet container's session mechanism for scripting
> languages that don't have such a notion. Such things can be
> designed and built for a particular server today, but there's no
> standard approach; hence the JSR.
Neat, it could be interesting indeed - thankyou for the clear
explanation Craig.
--
David N. Welton
Consulting: http://www.dedasys.com/
Personal: http://www.dedasys.com/davidw/
Free Software: http://www.dedasys.com/freesoftware/
Apache Tcl: http://tcl.apache.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: Java + Scripting languages
Posted by "Craig R. McClanahan" <cr...@apache.org>.
On Thu, 26 Jun 2003, David N. Welton wrote:
> Date: 26 Jun 2003 14:55:02 +0200
> From: David N. Welton <da...@dedasys.com>
> Reply-To: community@apache.org
> To: community@apache.org
> Subject: Java + Scripting languages
>
>
> Hi guys, I saw this:
>
> http://www.jcp.org/en/jsr/detail?id=223
>
> The specification may include a Java API that can be used,
> possibly through JNI, by an scripting language engine to
> access the desired Java objects.
>
> Can anyone give us a more concrete description of what this is really
> about?
>
The basis for this is exactly what that sentence states -- scripting
language users have said they would like to be able to access business
logic and data objects inside a servlet-based application from their
scripts, in a portable manner. The point of the JSR is to make that sort
of thing possible.
As Stefano points out, Sam did indeed create some code to do this. Just
two little problem though, it's non-thread safe (uses instance variables
for per-request state), and it's not scalable. And, it only deals with
PHP, but there's lots of other scripting languages (and scripting language
users) in the world that could benefit from the same ability.
Doing this kind of integration for a scripting language written in Java
(or indirectly wrapped by something like BSF) is pretty easy, because the
Java code can directly talk to servlet APIs. It's not quite that easy for
a scripting language implemented in a non-Java language (like PHP), where
you have to create some additional mechanisms to access web application
resources from a scripting page.
Accessing Java objects defined in the system class loader doesn't require
anything new -- JNI provides all the necessary hooks. But, to interact
with web app resources, you need to do things like load classes from the
webapp's class loader, and gain access to the ServletContext instance,
and perhaps even do nice things like utilize the servlet container's
session mechanism for scripting languages that don't have such a notion.
Such things can be designed and built for a particular server today, but
there's no standard approach; hence the JSR.
> It looks interesting, because... hey, who wouldn't want to associate
> with a million dollar marketing machine:-)
It's actually going to be a pretty good win-win ... scripting language
users gain access to business logic and services already written in Java,
and the Java community has the opportunity to grow by virtue of being
useful to people who don't currently use it.
>
> --
> David N. Welton
Craig McClanahan
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: Java + Scripting languages
Posted by "Noel J. Bergman" <no...@devtech.com>.
>>>http://www.jcp.org/en/jsr/detail?id=223
>>I am a priori discouraged because I don't see IBM represented. To have
this
>>JSR without participation from the BSF folks seems wrong. I don't know if
>>this is payback for Eclipse and the WS-I, an oversight, or what.
>Lack of warm bodies with the right skills who are not already booked
>110%. I am twisting Chuck Murcko and Victor Orlikowski's arm to
>represent IBM.
Color me much relieved! I've known about JSR 223 for a couple of weeks now
from another e-mail, was initially pleased to see that there was one, and
absolutely shocked (and dismayed) not to see BSF represented.
Should Chuck and Victor represent IBM or should they represent the ASF, now
that BSF has been transfered? You, Chuck and Victor are all ASF Members.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: Java + Scripting languages
Posted by Bill Stoddard <bi...@wstoddard.com>.
Noel J. Bergman wrote:
>>>http://www.jcp.org/en/jsr/detail?id=223
>>>
>>>
>
>
>
>>At the end, I will not be surprised if this turns out to be yet another
>>JSP-like political compromise between vendors, with no technological
>>value associated to it.
>>
>>
>
>I am a priori discouraged because I don't see IBM represented. To have this
>JSR without participation from the BSF folks seems wrong. I don't know if
>this is payback for Eclipse and the WS-I, an oversight, or what.
>
> --- Noel
>
Lack of warm bodies with the right skills who are not already booked
110%. I am twisting Chuck Murcko and Victor Orlikowski's arm to
represent IBM.
Bill
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: Java + Scripting languages
Posted by "Victor J. Orlikowski" <vi...@alumni.duke.edu>.
On Thu, Jun 26, 2003 at 02:13:37PM -0400, Noel J. Bergman wrote:
> > > I am a priori discouraged because I don't see IBM represented. To have
> this
> > > JSR without participation from the BSF folks seems wrong. I don't know
> if
> > > this is payback for Eclipse and the WS-I, an oversight, or what.
>
> > BSF is Apache now, not so ?
>
> Politics and memories. Hopefully this isn't what is happening. It is just
> an obvious conclusion in lieu of facts.
>
Eh. We were only recently made aware of 223, and either I or Chuck
plan on getting involved.
Victor
--
Victor J. Orlikowski | The Wall is Down, But the Threat Remains!
=================================================================
orlikowski@apache.org | victor.j.orlikowski@alumni.duke.edu
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: Java + Scripting languages
Posted by "Noel J. Bergman" <no...@devtech.com>.
> > I am a priori discouraged because I don't see IBM represented. To have
this
> > JSR without participation from the BSF folks seems wrong. I don't know
if
> > this is payback for Eclipse and the WS-I, an oversight, or what.
> BSF is Apache now, not so ?
Politics and memories. Hopefully this isn't what is happening. It is just
an obvious conclusion in lieu of facts.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: Java + Scripting languages
Posted by Paul Hammant <pa...@yahoo.com>.
> I am a priori discouraged because I don't see IBM represented. To have this
> JSR without participation from the BSF folks seems wrong. I don't know if
> this is payback for Eclipse and the WS-I, an oversight, or what.
BSF is Apache now, not so ?
- Paul
________________________________________________________________________
Want to chat instantly with your online friends? Get the FREE Yahoo!
Messenger http://uk.messenger.yahoo.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: Java + Scripting languages
Posted by "Noel J. Bergman" <no...@devtech.com>.
> > http://www.jcp.org/en/jsr/detail?id=223
> At the end, I will not be surprised if this turns out to be yet another
> JSP-like political compromise between vendors, with no technological
> value associated to it.
I am a priori discouraged because I don't see IBM represented. To have this
JSR without participation from the BSF folks seems wrong. I don't know if
this is payback for Eclipse and the WS-I, an oversight, or what.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: Java + Scripting languages
Posted by Stefano Mazzocchi <st...@apache.org>.
on 6/26/03 7:55 AM David N. Welton wrote:
> Hi guys, I saw this:
>
> http://www.jcp.org/en/jsr/detail?id=223
>
> The specification may include a Java API that can be used,
> possibly through JNI, by an scripting language engine to
> access the desired Java objects.
>
> Can anyone give us a more concrete description of what this is really
> about?
>From where I stand, it looks like the JSP folks changed gears in their
marketing engine. Now they want to make it easy to move millions of PHP
folks into the J2EE.
> It looks interesting, because... hey, who wouldn't want to associate
> with a million dollar marketing machine:-)
Yeah, that's exactly how they are going fishing for new market
opportunities, so you wait for them instead of doing your own stuff and
you get locked into their politics.
I heard rumors that PHP5 might be (partially) compiled into bytecode,
with JNI hooks to existing php libraries, or, on the other hand, use JNI
hooks to access the servlet API exposed objects.
btw, our good old Sam Ruby showed it's already possible both hooking
from java to php or from php to java. Code is already into PHP and there
was a time where you could write an XSP page for Cocoon using PHP as a
scripting language (but nobody cared and the code died)
At the end, I will not be surprised if this turns out to be yet another
JSP-like political compromise between vendors, with no technological
value associated to it.
[note: I was part of the JSP JSR and asked to be removed exactly because
I couldn't stand their political-driven design attitude.]
Sorry to sound cynical or rain on the party, but I lost hope in the JCP
(or all committee-driven design, for that matter) a long time ago and
it's simply getting worse.
My point is: you can hook to java *right* now, if you care (look at
Sam's code in PHP if you want to see how). JNI is there and it's all you
need.
It is true that a *common* native abstraction for hooking into
scriptable java objects would make it much easier to hook different
scripting languages to the java platform (today, you have to do
everything by hand for every language you hook), but then again, it's
nothing that is not already possible.
Hope this helps.
--
Stefano.
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Java + Scripting languages
Posted by "David N. Welton" <da...@dedasys.com>.
Hi guys, I saw this:
http://www.jcp.org/en/jsr/detail?id=223
The specification may include a Java API that can be used,
possibly through JNI, by an scripting language engine to
access the desired Java objects.
Can anyone give us a more concrete description of what this is really
about?
It looks interesting, because... hey, who wouldn't want to associate
with a million dollar marketing machine:-)
--
David N. Welton
Consulting: http://www.dedasys.com/
Personal: http://www.dedasys.com/davidw/
Free Software: http://www.dedasys.com/freesoftware/
Apache Tcl: http://tcl.apache.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: WORA Considered Evil ;-)
Posted by Glen Stampoultzis <gs...@iinet.net.au>.
At 04:03 PM 26/06/2003, you wrote:
>True, Java is not a systems programming language. But without WORA, I do
>not believe that Java would have the success that it has on the server.
Yes. As dogmatic as Sun has been about "pure" Java it's still a success
factor in the adoption of Java. There's still no other platform out there
that makes it as easy as Java to write for multiple platforms. A little
JNI in the right places certainly isn't evil though.
Regards,
Glen Stampoultzis
gstamp@iinet.net.au
http://members.iinet.net.au/~gstamp/glen/