You are viewing a plain text version of this content. The canonical link for it is here.

Posted to cvs@cocoon.apache.org by pi...@locus.apache.org on 2000/09/14 13:08:11 UTC

cvs commit: xml-cocoon/src/org/apache/cocoon/util IOUtils.java

pier        00/09/14 04:08:10

  Modified:    src/org/apache/cocoon/util Tag: xml-cocoon2 IOUtils.java
  Log:
  Please don't use System resources, as we're TOTALLY unaware of the classloader used.
  (Fix 2 towards class loading issues)
  
  Revision  Changes    Path
  No                   revision
  
  
  No                   revision
  
  
  1.1.2.5   +2 -2      xml-cocoon/src/org/apache/cocoon/util/Attic/IOUtils.java
  
  Index: IOUtils.java
  ===================================================================
  RCS file: /home/cvs/xml-cocoon/src/org/apache/cocoon/util/Attic/IOUtils.java,v
  retrieving revision 1.1.2.4
  retrieving revision 1.1.2.5
  diff -u -r1.1.2.4 -r1.1.2.5
  --- IOUtils.java	2000/09/06 23:22:26	1.1.2.4
  +++ IOUtils.java	2000/09/14 11:08:08	1.1.2.5
  @@ -25,7 +25,7 @@
    * utility methods
    *
    * @author <a href="mailto:ricardo@apache.org">Ricardo Rocha</a>
  - * @version CVS $Revision: 1.1.2.4 $ $Date: 2000/09/06 23:22:26 $
  + * @version CVS $Revision: 1.1.2.5 $ $Date: 2000/09/14 11:08:08 $
    */
   public class IOUtils {
     /**
  @@ -49,7 +49,7 @@
   	getFullFilename(file).replace(File.separatorChar, '/')
         );
       } else if (location.startsWith("resource://")) {
  -      return ClassUtils.getClassLoader().getSystemResource(
  +      return ClassUtils.getClassLoader().getResource(
           location.substring("resource://".length())
         );
       } else {

Re: C2: Uploader needed

Posted by Nicola Ken Barozzi <ni...@supereva.it>.

From: "Paul Russell" <pa...@luminas.co.uk>

> On Thu, Sep 14, 2000 at 02:51:33PM +0200, Matthew Langham wrote:
> > At the moment we are considering an "UploadTransformer" which would grab the
> > uploaded file from the HTTP stream and write that out to wherever..
> 
> Can't help thinking this should be part of the FP taglib...
> others?

Think so too, the principle is the same.

nicola_ken

Re: C2: Uploader needed

Posted by Stefano Mazzocchi <st...@apache.org>.

Jeremy Quinn wrote:
> 
> Thanks Neeme,
> 
> I think I found what you meant.
> 
> X-Mozilla-Status: 0009of the source code from Turbine in FP, what would be
> the "polite" way of doing it?
> 
> We should'nt expect people to download Turbine, just for this functionality?
> 
> I'd probably want to incorporate the modified Turbine source into FP.
> 
> Is this OK?

Yes, we already stole... ehmmm, "borrowed" the pool from turbine...
another piece shouldn't hurt :)

Don't take the whole turbine jar as a whole: it might create the
illusion you can mix Turbine and Cocoon, but this is clearly not the
case from this side of the fence.

-- 
Stefano Mazzocchi      One must still have chaos in oneself to be
                          able to give birth to a dancing star.
<st...@apache.org>                             Friedrich Nietzsche
--------------------------------------------------------------------
 Missed us in Orlando? Make it up with ApacheCON Europe in London!
------------------------- http://ApacheCon.Com ---------------------

Re: C2: Uploader needed

Posted by Stefano Mazzocchi <st...@apache.org>.

Jeremy Quinn wrote:
> 
> Thanks Neeme,
> 
> I think I found what you meant.
> 
> If I were to use some of the source code from Turbine in FP, what would be
> the "polite" way of doing it?
> 
> We should'nt expect people to download Turbine, just for this functionality?
> 
> I'd probably want to incorporate the modified Turbine source into FP.
> 
> Is this OK?

Yes, we already stole... ehmmm, "borrowed" the pool from turbine...
another piece shouldn't hurt :)

Don't take the whole turbine jar as a whole: it might create the
illusion you can mix Turbine and Cocoon, but this is clearly not the
case from this side of the fence.

-- 
Stefano Mazzocchi      One must still have chaos in oneself to be
                          able to give birth to a dancing star.
<st...@apache.org>                             Friedrich Nietzsche
--------------------------------------------------------------------
 Missed us in Orlando? Make it up with ApacheCON Europe in London!
------------------------- http://ApacheCon.Com ---------------------

Re: C2: Uploader needed

Posted by Stefano Mazzocchi <st...@apache.org>.

Neeme Praks wrote:
> 
> I think it would be "polite enough" if you would give the due credit to the
> authors of the code and let them know also that you included their code in
> FP...

Oh, I think that "not give due credit" was not even in consideration and
yes, letting them know doesn't hurt either.

-- 
Stefano Mazzocchi      One must still have chaos in oneself to be
                          able to give birth to a dancing star.
<st...@apache.org>                             Friedrich Nietzsche
--------------------------------------------------------------------
 Missed us in Orlando? Make it up with ApacheCON Europe in London!
------------------------- http://ApacheCon.Com ---------------------

RE: C2: Uploader needed

Posted by Jeremy Quinn <je...@media.demon.co.uk>.

At 16:01 +0200 17/09/00, Neeme Praks wrote:
>
>I think it would be "polite enough" if you would give the due credit to the
>authors of the code and let them know also that you included their code in
>FP...
>
>Neeme

cheers

Jeremy
-- 
   ___________________________________________________________________

   Jeremy Quinn                                           Karma Divers
                                                       webSpace Design
                                            HyperMedia Research Centre

   <ma...@mac.com>     		 <http://www.media.demon.co.uk>
    <phone:+44.[0].20.7737.6831>        <pa...@sms.genie.co.uk>

RE: C2: Uploader needed

Posted by Neeme Praks <ne...@one.lv>.

I think it would be "polite enough" if you would give the due credit to the
authors of the code and let them know also that you included their code in
FP...

Neeme

> -----Original Message-----
> From: Jeremy Quinn [mailto:jeremy@media.demon.co.uk]
> Sent: Sunday, September 17, 2000 1:25 PM
> To: cocoon-dev@xml.apache.org
> Subject: RE: C2: Uploader needed
>
>
>
> Thanks Neeme,
>
> I think I found what you meant.
>
> If I were to use some of the source code from Turbine in FP,
> what would be
> the "polite" way of doing it?
>
> We should'nt expect people to download Turbine, just for this
> functionality?
>
> I'd probably want to incorporate the modified Turbine source into FP.
>
> Is this OK?
>
> thanks
>
> regards Jeremy
>
>
>
> At 18:32 +0200 16/09/00, Neeme Praks wrote:
> >check out Turbine, it has three different implementations of this
> >functionality...
> >
> >> -----Original Message-----
> >> From: Jeremy Quinn [mailto:jeremy@media.demon.co.uk]
> >> Sent: Friday, September 15, 2000 10:43 AM
> >> To: cocoon-dev@xml.apache.org
> >> Subject: Re: C2: Uploader needed
> >>
> >> Good idea, I thought of this before, but was put off by my
> >> understanding
> >> that code for decoding uploaded files in the HTTP Request was
> >> not available
> >> in the open source.
>
>
> --
>    ___________________________________________________________________
>
>    Jeremy Quinn                                           Karma Divers
>                                                        webSpace Design
>                                             HyperMedia Research Centre
>
>    <ma...@mac.com>
<http://www.media.demon.co.uk>
    <phone:+44.[0].20.7737.6831>        <pa...@sms.genie.co.uk>

RE: C2: Uploader needed

Posted by Jeremy Quinn <je...@media.demon.co.uk>.

Thanks Neeme,

I think I found what you meant.

If I were to use some of the source code from Turbine in FP, what would be
the "polite" way of doing it?

We should'nt expect people to download Turbine, just for this functionality?

I'd probably want to incorporate the modified Turbine source into FP.

Is this OK?

thanks

regards Jeremy



At 18:32 +0200 16/09/00, Neeme Praks wrote:
>check out Turbine, it has three different implementations of this
>functionality...
>
>> -----Original Message-----
>> From: Jeremy Quinn [mailto:jeremy@media.demon.co.uk]
>> Sent: Friday, September 15, 2000 10:43 AM
>> To: cocoon-dev@xml.apache.org
>> Subject: Re: C2: Uploader needed
>>
>> Good idea, I thought of this before, but was put off by my
>> understanding
>> that code for decoding uploaded files in the HTTP Request was
>> not available
>> in the open source.


-- 
   ___________________________________________________________________

   Jeremy Quinn                                           Karma Divers
                                                       webSpace Design
                                            HyperMedia Research Centre

   <ma...@mac.com>     		 <http://www.media.demon.co.uk>
    <phone:+44.[0].20.7737.6831>        <pa...@sms.genie.co.uk>

RE: C2: Uploader needed

Posted by Neeme Praks <ne...@one.lv>.

check out Turbine, it has three different implementations of this
functionality...

> -----Original Message-----
> From: Jeremy Quinn [mailto:jeremy@media.demon.co.uk]
> Sent: Friday, September 15, 2000 10:43 AM
> To: cocoon-dev@xml.apache.org
> Subject: Re: C2: Uploader needed
>
>
> At 13:55 +0100 14/09/00, Paul Russell wrote:
> >On Thu, Sep 14, 2000 at 02:51:33PM +0200, Matthew Langham wrote:
> >> At the moment we are considering an "UploadTransformer"
> which would grab the
> >> uploaded file from the HTTP stream and write that out to wherever..
> >
> >Can't help thinking this should be part of the FP taglib...
> >others?
>
> Good idea, I thought of this before, but was put off by my
> understanding
> that code for decoding uploaded files in the HTTP Request was
> not available
> in the open source.
>
> Please correct me on this, point me to some code, and I'll include it.
>
> regards Jeremy
> --
>    ___________________________________________________________________
>
>    Jeremy Quinn                                           Karma Divers
>                                                        webSpace Design
>                                             HyperMedia Research Centre
>
>    <ma...@mac.com>
<http://www.media.demon.co.uk>
    <phone:+44.[0].20.7737.6831>        <pa...@sms.genie.co.uk>

Re: C2: Uploader needed

Posted by Jeremy Quinn <je...@media.demon.co.uk>.

At 13:55 +0100 14/09/00, Paul Russell wrote:
>On Thu, Sep 14, 2000 at 02:51:33PM +0200, Matthew Langham wrote:
>> At the moment we are considering an "UploadTransformer" which would grab the
>> uploaded file from the HTTP stream and write that out to wherever..
>
>Can't help thinking this should be part of the FP taglib...
>others?

Good idea, I thought of this before, but was put off by my understanding
that code for decoding uploaded files in the HTTP Request was not available
in the open source.

Please correct me on this, point me to some code, and I'll include it.

regards Jeremy
-- 
   ___________________________________________________________________

   Jeremy Quinn                                           Karma Divers
                                                       webSpace Design
                                            HyperMedia Research Centre

   <ma...@mac.com>     		 <http://www.media.demon.co.uk>
    <phone:+44.[0].20.7737.6831>        <pa...@sms.genie.co.uk>

RE: C2: Uploader needed

Posted by Per Kreipke <pe...@onclave.com>.

Sorry if this is a stupid question, but I can't find the FP taglib anywhere
in my C1 installation. Where can I find it?

Per.

> -----Original Message-----
> From: Paul Russell [mailto:paul@luminas.co.uk]
> Sent: Thursday, September 14, 2000 8:55 AM
> To: cocoon-dev@xml.apache.org
> Subject: Re: C2: Uploader needed
>
>
> On Thu, Sep 14, 2000 at 02:51:33PM +0200, Matthew Langham wrote:
> > At the moment we are considering an "UploadTransformer" which
> would grab the
> > uploaded file from the HTTP stream and write that out to wherever..
>
> Can't help thinking this should be part of the FP taglib...
> others?
>
>
> Paul
>
> --
> Paul Russell                               <pa...@luminas.co.uk>
> Technical Director,                   http://www.luminas.co.uk
> Luminas Ltd.
>

Re: C2: Uploader needed

Posted by Paul Russell <pa...@luminas.co.uk>.

On Thu, Sep 14, 2000 at 02:51:33PM +0200, Matthew Langham wrote:
> At the moment we are considering an "UploadTransformer" which would grab the
> uploaded file from the HTTP stream and write that out to wherever..

Can't help thinking this should be part of the FP taglib...
others?


Paul

-- 
Paul Russell                               <pa...@luminas.co.uk>
Technical Director,                   http://www.luminas.co.uk
Luminas Ltd.

RE: C2: Uploader needed

Posted by Per Kreipke <pe...@onclave.com>.

I was asking about something similar on the '[C2] Access Control Needed'
thread. Similar to upload of files would be the ability to input XML
document into the system from outside via POSTs. Specifically, I'm thinking
of having Cocoon handle XML-RPC requests.

XML-RPC uses a "text/xml" POST HTTP request to send a XML payload
(containing function name and params) to the server which then executes the
referenced function and ships back another XML payload containing the return
code/contents.

An example use of this is to add messages to discussion groups or posting
images, etc.

I asked in the other thread whether the sitemap would allow XML transforms
in the input direction. I didn't get a clear answer [or failed to understand
it :) ].

Would this be a similar transformer/whatever?

Per.

> -----Original Message-----
> From: Matthew Langham [mailto:mlangham@sundn.de]
> Sent: Thursday, September 14, 2000 8:52 AM
> To: cocoon-dev@xml.apache.org
> Subject: C2: Uploader needed
>
>
> We need to integrate the possibility of uploading files into Cocoon and I
> have been thinking about how best to do this - but then I thought hey -
> maybe someone has already done this.
>
> At the moment we are considering an "UploadTransformer" which
> would grab the
> uploaded file from the HTTP stream and write that out to wherever..
>
> Comments or "done that"s?
>
> Matthew Langham
>
> --
> Open Source Group               sunShine - Lighting up e:Business
> =================================================================
> Matthew Langham, S&N AG, Klingenderstrasse 5, D-33100 Paderborn
> Tel: +49-5251-1581-30   [mlangham@sundn.de - http://www.sundn.de]
> =================================================================
>
>
>

C2: Uploader needed

Posted by Matthew Langham <ml...@sundn.de>.

We need to integrate the possibility of uploading files into Cocoon and I
have been thinking about how best to do this - but then I thought hey -
maybe someone has already done this.

At the moment we are considering an "UploadTransformer" which would grab the
uploaded file from the HTTP stream and write that out to wherever..

Comments or "done that"s?

Matthew Langham

--
Open Source Group               sunShine - Lighting up e:Business
=================================================================
Matthew Langham, S&N AG, Klingenderstrasse 5, D-33100 Paderborn
Tel: +49-5251-1581-30   [mlangham@sundn.de - http://www.sundn.de]
=================================================================

Re: WEB-INF classloading and on the fly compilation

Posted by Paul Russell <pa...@luminas.co.uk>.

On Thu, Sep 14, 2000 at 04:42:48AM -0700, Pier P. Fumagalli wrote:
> Now, either am I a complete MORON or JavaC resolves everything using the
> System classloader, completely ignoring whatever classloader loaded
> itself? NOTICE: If this is true, we have a bunch of problems basically
> everywhere we want to do some on-the-fly compilation (tomcat people
> read: JSP!).

I'm pretty sure your're right. I spent a while battling with
this a few months ago. Basically, as far as I'm concerned, the
solution is to have access to the compiler at a much more
fundamental level. I'd like to be able to hand the thing parse
trees, ideally, and we *certainly* need to be able to use our
own classloaders.

A while back I was looking at building an application server
with a distributed VFS looking after all objects required in
the system (including classes, XSP, EJB components, the lot).
I stuck it on hold because it seems to be very difficult to
persuade Javac to load classes from a non-system classloader.

Hrmph.

Paul
-- 
Paul Russell                               <pa...@luminas.co.uk>
Technical Director,                   http://www.luminas.co.uk
Luminas Ltd.

Re: WEB-INF classloading and on the fly compilation

Posted by Stefano Mazzocchi <st...@apache.org>.

Ross Burton wrote:
> 
> I presume this problem is the same problem which means Jikes as a XSP
> compiler is... tricky.  Shame.

For now, yes, unfortunately :(

-- 
Stefano Mazzocchi      One must still have chaos in oneself to be
                          able to give birth to a dancing star.
<st...@apache.org>                             Friedrich Nietzsche
--------------------------------------------------------------------
 Missed us in Orlando? Make it up with ApacheCON Europe in London!
------------------------- http://ApacheCon.Com ---------------------

Re: WEB-INF classloading and on the fly compilation

Posted by "Pier P. Fumagalli" <pi...@apache.org>.

Ross Burton wrote:
> 
> I presume this problem is the same problem which means Jikes as a XSP
> compiler is... tricky.  Shame.

There is an attribute passed to all web apps that can be used to
determine the actual classpath... It's a tomcat-only feature, though and
it's what Jasper uses for compiling JSPs... We could use the same...
I'm to tired to patch it tonight...

	Pier

Re: WEB-INF classloading and on the fly compilation

Posted by Ross Burton <ro...@mail.com>.

I presume this problem is the same problem which means Jikes as a XSP
compiler is... tricky.  Shame.

Ross Burton

TC 3.2 Beta 4 BAT files

Posted by Hans Bergsten <ha...@gefionsoftware.com>.

Hi guys,

I've just tested TC 3.2 Beta 4 on Windows 98 and NT 4.0, and everything seems
to work fine. Except that all BAT files are still in Unix format.

I have asked about this before, but I didn't get any response so I try again.
Will this be handled by the real build process (as opposed to the beta build
process), or must someone (could be me) check in Windows versions of all BAT
files to get it corrected? I would prefer if it can be handled by the build
process, because otherwise it will break again as soon as someone edits these
file in Unix and checks them in.

Hans
-- 
Hans Bergsten		hans@gefionsoftware.com
Gefion Software		http://www.gefionsoftware.com

Re: WEB-INF classloading and on the fly compilation

Posted by "Pier P. Fumagalli" <pi...@eng.sun.com>.

Nick Bauman wrote:
> 
> On page 24 section 4.6 "a Container Provider ... must ensure that all
> servlets, and classes that they may use, are loaded in the scope of a
> single class loader ..."

That's exactly what happens... The scope of the WEB-APP classloader is
itself or its parental chain up to the system class loader...
It's a different problem...

	Pier

Re: WEB-INF classloading and on the fly compilation

Posted by "Craig R. McClanahan" <Cr...@eng.sun.com>.

Nick Bauman wrote:

> On Thu, 14 Sep 2000, Nick Bauman wrote:
>
> > On Thu, 14 Sep 2000 cmanolache@yahoo.com wrote:
>
> > > > On page 24 section 4.6 "a Container Provider ... must ensure that all
> > > > servlets, and classes that they may use, are loaded in the scope of a
> > > > single class loader ..."
> > >
> > > Well, I'm confused - this is how tomcat works. Each context has it's class
> > > loader and all servlets and classes are loaded by that class loader. (
> > > starting with tomcat 3.3 it'll be the URLClassLoader - the
> > > "most" standard/compatible/secure loader we could use).
> > >
> > > Costin
> > >
> >
> > Is it a Singleton? The spec states it should be.
> >
>
> At least that's how it could be interpreted. Maybe what is meant is one
> classloader per context. Any experts on interpretation out there?
>

In 2.2, the scope of this particular statement in the spec is the part of the
sentence that was replaced by ... in the original citation:

    Therefore, when a Container Provider implements a class loading
    scheme for ease of development [i.e. supports automatic reloading
    when classes are changed], they must ensure that all servlets, and
    classes that they use, are loaded in the scope of a single class
    loader guaranteeing that the application will behave as expected by
    the Developer.

In essence, this restricts the container to AT MOST one class loader per web-app
(some early containers used a classloader per servlet to implement autoreload), to
avoid class casting problems.  There are other aspects of the spec that require AT
LEAST one class loader (for instance, the fact that a webapp can see it's own
WEB-INF/classes and WEB-INF/lib/*.jar but not any from other webapps), so it's
pretty clear that there is a one-to-one relationship between webapps and
classloaders.  That's the way it's implemented in Tomcat 3.x and 4.0 as well.

In 2.3, there is specific reference to the "Web Application Classloader" (Section
9.6.2) plus some additional requirements on the order in which the various
repositories are searched and respecting declared version dependencies between JAR
files.

Craig

====================
See you at ApacheCon Europe <http://www.apachecon.com>!
Session VS01 (23-Oct 13h00-17h00):  Sun Technical Briefing
Session T06  (24-Oct 14h00-15h00):  Migrating Apache JServ
                                    Applications to Tomcat

Re: WEB-INF classloading and on the fly compilation

Posted by Nick Bauman <ni...@cortexity.com>.

On Thu, 14 Sep 2000, Nick Bauman wrote:

> On Thu, 14 Sep 2000 cmanolache@yahoo.com wrote:

> > > On page 24 section 4.6 "a Container Provider ... must ensure that all
> > > servlets, and classes that they may use, are loaded in the scope of a
> > > single class loader ..."
> > 
> > Well, I'm confused - this is how tomcat works. Each context has it's class
> > loader and all servlets and classes are loaded by that class loader. (
> > starting with tomcat 3.3 it'll be the URLClassLoader - the
> > "most" standard/compatible/secure loader we could use). 
> > 
> > Costin
> > 
> 
> Is it a Singleton? The spec states it should be.
> 

At least that's how it could be interpreted. Maybe what is meant is one
classloader per context. Any experts on interpretation out there?

Re: WEB-INF classloading and on the fly compilation

Posted by Nick Bauman <ni...@cortexity.com>.

On Thu, 14 Sep 2000 cmanolache@yahoo.com wrote:

> > > > > Now, either am I a complete MORON or JavaC resolves everything using the
> > > > > System classloader, completely ignoring whatever classloader loaded
> > > > > itself? NOTICE: If this is true, we have a bunch of problems basically
> > > > > everywhere we want to do some on-the-fly compilation (tomcat people
> > > > > read: JSP!).
> > > > >
> > > > 
> > > > This shouldn't be. The 2.2 spec says so on page 24.
> > > 
> > > ??? Where on page 24? On my copy it talks about resources and virtual
> > > hosts....
> > > 
> > On page 24 section 4.6 "a Container Provider ... must ensure that all
> > servlets, and classes that they may use, are loaded in the scope of a
> > single class loader ..."
> 
> Well, I'm confused - this is how tomcat works. Each context has it's class
> loader and all servlets and classes are loaded by that class loader. (
> starting with tomcat 3.3 it'll be the URLClassLoader - the
> "most" standard/compatible/secure loader we could use). 
> 
> Costin
> 

Is it a Singleton? The spec states it should be.

Re: WEB-INF classloading and on the fly compilation

Posted by cm...@yahoo.com.

> > > > Now, either am I a complete MORON or JavaC resolves everything using the
> > > > System classloader, completely ignoring whatever classloader loaded
> > > > itself? NOTICE: If this is true, we have a bunch of problems basically
> > > > everywhere we want to do some on-the-fly compilation (tomcat people
> > > > read: JSP!).
> > > >
> > > 
> > > This shouldn't be. The 2.2 spec says so on page 24.
> > 
> > ??? Where on page 24? On my copy it talks about resources and virtual
> > hosts....
> > 
> On page 24 section 4.6 "a Container Provider ... must ensure that all
> servlets, and classes that they may use, are loaded in the scope of a
> single class loader ..."

Well, I'm confused - this is how tomcat works. Each context has it's class
loader and all servlets and classes are loaded by that class loader. (
starting with tomcat 3.3 it'll be the URLClassLoader - the
"most" standard/compatible/secure loader we could use). 

Costin

Re: WEB-INF classloading and on the fly compilation

Posted by Nick Bauman <ni...@cortexity.com>.

On Thu, 14 Sep 2000, Pier P. Fumagalli wrote:

> Nick Bauman wrote:
> > 
> > On Thu, 14 Sep 2000, Pier P. Fumagalli wrote:
> > 
> > > Now, either am I a complete MORON or JavaC resolves everything using the
> > > System classloader, completely ignoring whatever classloader loaded
> > > itself? NOTICE: If this is true, we have a bunch of problems basically
> > > everywhere we want to do some on-the-fly compilation (tomcat people
> > > read: JSP!).
> > >
> > 
> > This shouldn't be. The 2.2 spec says so on page 24.
> 
> ??? Where on page 24? On my copy it talks about resources and virtual
> hosts....
> 
> 	Pier

On page 24 section 4.6 "a Container Provider ... must ensure that all
servlets, and classes that they may use, are loaded in the scope of a
single class loader ..."
 

-- 
Nicolaus Bauman
Software Engineer
Simplexity Systems
I'll give you $5 if you follow this link:
https://secure.paypal.x.com/refer/pal=nixnixnix%40yahoo.com

Re: WEB-INF classloading and on the fly compilation

Posted by "Pier P. Fumagalli" <pi...@apache.org>.

Nick Bauman wrote:
> 
> On Thu, 14 Sep 2000, Pier P. Fumagalli wrote:
> 
> > Now, either am I a complete MORON or JavaC resolves everything using the
> > System classloader, completely ignoring whatever classloader loaded
> > itself? NOTICE: If this is true, we have a bunch of problems basically
> > everywhere we want to do some on-the-fly compilation (tomcat people
> > read: JSP!).
> >
> 
> This shouldn't be. The 2.2 spec says so on page 24.

??? Where on page 24? On my copy it talks about resources and virtual
hosts....

	Pier

Re: WEB-INF classloading and on the fly compilation

Posted by Nick Bauman <ni...@cortexity.com>.

On Thu, 14 Sep 2000, Pier P. Fumagalli wrote:

> Now, either am I a complete MORON or JavaC resolves everything using the
> System classloader, completely ignoring whatever classloader loaded
> itself? NOTICE: If this is true, we have a bunch of problems basically
> everywhere we want to do some on-the-fly compilation (tomcat people
> read: JSP!).
> 

This shouldn't be. The 2.2 spec says so on page 24. 

-Nick

Re: WEB-INF classloading and on the fly compilation

Posted by "Pier P. Fumagalli" <pi...@eng.sun.com>.

Vadim Gritsenko wrote:
> 
> [...]
> So, in some sence, you are completely right: it fully ignores any
> ClassLoaders.

Wohooo... That's what I thought, and I'm definitely not happy :(

	Pier

Re: WEB-INF classloading and on the fly compilation

Posted by Stefano Mazzocchi <st...@apache.org>.

Vadim Gritsenko wrote:
> 
> >
> > Now, either am I a complete MORON or JavaC resolves everything using the
> > System classloader, completely ignoring whatever classloader loaded
> > itself? NOTICE: If this is true, we have a bunch of problems basically
> > everywhere we want to do some on-the-fly compilation (tomcat people
> > read: JSP!).
> >
> 
> I took a look into Javac and found that there is no any ClassLoader's used
> or
> Class.forName. It just loads binary class data and processes it (class
> sun.tools.java.BinaryClass, static method load(Environment env,
> DataInputStream in, int mask) throws IOException).
> 
> To find class file, it uses (in this order):
> 
> 1. argument -sysclasspath (or -bootclasspath), or, if it not present, system
> property
> sun.boot.class.path. If property is absent (old java), then it uses property
> java.class.path.
> 
> 2. All .jar's from directories specified with argument -extdirs, or, if not
> present,
> system property java.ext.dirs.
> 
> 3. argument -classpath
> 
> All this take place in class sun.tools.javac.BatchEnvironment, protected
> static method classPaths(...).
> 
> So, in some sence, you are completely right: it fully ignores any
> ClassLoaders.
> 
> Sincerely,
> Vadim

We are in deep shit, then. I'm serious.

All right... need to do something about it.

Stay tuned for more soon.

-- 
Stefano Mazzocchi      One must still have chaos in oneself to be
                          able to give birth to a dancing star.
<st...@apache.org>                             Friedrich Nietzsche
--------------------------------------------------------------------
 Missed us in Orlando? Make it up with ApacheCON Europe in London!
------------------------- http://ApacheCon.Com ---------------------

RE: WEB-INF classloading and on the fly compilation

Posted by Vadim Gritsenko <vg...@hns.com>.

>
> Now, either am I a complete MORON or JavaC resolves everything using the
> System classloader, completely ignoring whatever classloader loaded
> itself? NOTICE: If this is true, we have a bunch of problems basically
> everywhere we want to do some on-the-fly compilation (tomcat people
> read: JSP!).
>

I took a look into Javac and found that there is no any ClassLoader's used
or
Class.forName. It just loads binary class data and processes it (class
sun.tools.java.BinaryClass, static method load(Environment env,
DataInputStream in, int mask) throws IOException).

To find class file, it uses (in this order):

1. argument -sysclasspath (or -bootclasspath), or, if it not present, system
property
sun.boot.class.path. If property is absent (old java), then it uses property
java.class.path.

2. All .jar's from directories specified with argument -extdirs, or, if not
present,
system property java.ext.dirs.

3. argument -classpath

All this take place in class sun.tools.javac.BatchEnvironment, protected
static method classPaths(...).


So, in some sence, you are completely right: it fully ignores any
ClassLoaders.


Sincerely,
Vadim

Re: WEB-INF classloading and on the fly compilation

Posted by "Pier P. Fumagalli" <pi...@eng.sun.com>.

Chon-Chon Tang wrote:
> 
> Maybe I'm missing something,

Yes, you are :) See my first message...

> but there's absolutely no need for the
> compiler to see the 'final' class that will be loaded at run-time
> (possibly through any strange implementation of a classloader).
> If you wish to do on-the-fly compilation (presumably to take
> advantage of execution traces?), it just needs to preserve binary
> compatibility with the class that the using class was compiled
> with.

If I want to compile that has an "import ....;" statement of something
placed in WEB-INF/lib ,well, that import will not be resolved and you
will not be able to compile...

	Pier

RE: WEB-INF classloading and on the fly compilation

Posted by Chon-Chon Tang <zt...@mit.edu>.

> That's a problem related to the compiler, relying on files instead on
> classes loaded by the classloaders... I know that at Olliance they're
> thinking about distributing WEB-APPs on a set of different servlet
> engines (a some sort of load balancing but with more guts :). This will
> surely add lots of hacks in class-loading mechanisms, since WEB-APPs
> WARs will be loaded from the network and so on...
> If the compiler relies on files on the disk, then, we're screwed,
> because we cannot possibly have a portable servlet using on-the-fly
> compilation...
> 

Maybe I'm missing something, but there's absolutely no need for the
compiler to see the 'final' class that will be loaded at run-time
(possibly through any strange implementation of a classloader).
If you wish to do on-the-fly compilation (presumably to take 
advantage of execution traces?), it just needs to preserve binary
compatibility with the class that the using class was compiled
with.

I don't see how that will be a significant problem.  Presumably the
using code is using the class through some well-defined 'interface'
(not necessarily Java interface) that will remain static, regardless
of where the underlying class implementation comes from.  If you 
must, define a stub class that defines this interface properly
and that is used during compilation.  When the class is loaded
at run-time through any mechanism you choose, it will be correctly
used as long as it preserves the various rules for binary 
compatibility.

Re: WEB-INF classloading and on the fly compilation

Posted by cm...@yahoo.com.

> > A compiler is a very complex tool, with a very narrow goal - to produce
> > bytecode from a java source. It doesn't try to be the coolest
> > distributable-location-independent-load-from-network app.
> > 
> > I would be more than happy if the compiler would do a good job at
> > generating a reasonable good code at a reasonable speed.
> 
> It's such an easy hack... Instead of opening files, once you have the
> class name, you call getResource();

Is it? What about modification time, random access ( URL doesn't have it)?
What about speed ( adding a layer that reduces the available API does that
most of the time ) ?

Costin

Re: WEB-INF classloading and on the fly compilation

Posted by "Pier P. Fumagalli" <pi...@eng.sun.com>.

cmanolache@yahoo.com wrote:
> 
> > That's a problem related to the compiler, relying on files instead on
> > classes loaded by the classloaders... I know that at Olliance they're
> > thinking about distributing WEB-APPs on a set of different servlet
> > engines (a some sort of load balancing but with more guts :). This will
> > surely add lots of hacks in class-loading mechanisms, since WEB-APPs
> > WARs will be loaded from the network and so on...
> > If the compiler relies on files on the disk, then, we're screwed,
> > because we cannot possibly have a portable servlet using on-the-fly
> > compilation...
> 
> Deployment != Runtime.
> 
> A compiler is a very complex tool, with a very narrow goal - to produce
> bytecode from a java source. It doesn't try to be the coolest
> distributable-location-independent-load-from-network app.
> 
> I would be more than happy if the compiler would do a good job at
> generating a reasonable good code at a reasonable speed.

It's such an easy hack... Instead of opening files, once you have the
class name, you call getResource();

	Pier

RE: WEB-INF classloading and on the fly compilation

Posted by Nick Bauman <ni...@cortexity.com>.

Both of you: Paulo and Costin, have concerns that I think are valid. 

I run a small ASP outside of work. This ASP has many concerns typical of
an ISP, which makes me interested in Costin's concerns. But at work,
moving "at the speed of business", I can see reasons why I would
want the ability to compile Java from something other than a
filesystem. (Although I can't think of a specific business requirement off
the top of my head.)

On Sat, 16 Sep 2000 cmanolache@yahoo.com wrote:

> > > My goal is to implement a server where webapps can be deployed in real
> > > sandboxes, like applets in browsers. If Cocoon and other web-development
> > > environments will go this way ( putting too much complexity
> > > inside the web app and expecting it to work ) than the whole security will
> > > be useless - you'll have a sandbox but nothing will be able to
> > > run inside.
> > 
> > *Your* goal.
> 
> I certainly hope there are other people interested in that. 
> 
> 
> > For others - like me - sandboxing is not that important. Sandboxing might be
> > very important for an ISP. However, these days, more and more "private" web
> > servers exist.
> 
> More and more insecure web servers exist too. 
> 
> 
> > In my company, we control all the code that is placed in the Web Server. 
> 
> Well, you may think so, and I hope you are right. I would be happy to
> accept all your flames and "narrow-minded" if I can make you have just a
> little doubt about this statement. 
> 
> Do you control your own code ( from a security point of view ) ? Do you
> control the code ( including jsp pages, etc) written by all programmers
> that work for your organization? And all the code that is included in
> various APIs and packges you use ?
> 
> > On
> > the other hand, the applications that we have to develop are becoming more
> > and more complex.
> 
> And you still think you are controlling it? 
> 
>  
> > Then, versatility and fast problem solving becomes much more important than
> > sandboxing.
> 
> I hope you're not running an e-commerce site :-)
> 
> 
> > Prioritizing sandboxing is very nice for a campus server used by students:
> > many users are trying to hack the thing and they do not have great
> > complexity demands.
> > 
> > Are you giving any priority to commercial users needs?
> 
> I certainly hope so - I spent some time running servers that many users
> tryed to hack, and I have an ideea about how many commercial sites are
> run. And indeed, there is a huge difference. 
> 
> 
> Do you believe that sandbox is required more for "campus" servers ( where
> a traditional experience exists with hacking and admins are well aware of
> the risks, and more important - the possible damage is minor) 
> 
> And on commercial sites ( where most effort goes into creating bloated
> sites and delivering on short schedules, security policies don't exist or
> are ignored and a break in means lots of $$$ losts ) it's less important ?
> 
> What's your security policy ? How much experience do you have on detecting
> break-ins ? How much % time do you spend reviewing the code ( that is more
> and more complex ) for security ?
> ( do you use XSL ? Are you aware that in a stylesheet you can have embeded
> scripts - and you can call exec("rm -rf ~") from inside ? Or 
> "exec( mail hacker < creditcard_numbers )" ? ) Same for jsps btw.
>   
> 
> > > I am also interested in performance - and I don't like the idea of running
> > > 3-4 compiler instances at the same time ( or even 1 ! ) on a production
> > > site. In most cases this can be avoided by compiling at deployment time.
> > 
> > "In most cases" is the key expression.
> > 
> > Well, in most cases you can build a site with static HTML... so, let's kill
> > both Servlet and JSP technologies too, right?
> 
> Using them in a secure way isn't the same as killing them. In fact
> servlets and Jsps are the only technologies today that may provide this. 
> ( but that's changing, with less and less interest in security and
> performance and more and more interest in feature bloat ) 
>   
> 
> > Is Tomcat supposed to be ONLY a production server? Without being good for
> > development???
> 
> I certainly hope tomcat will not be the only production server interested
> in security and performance. 
> 
> And so far we put a lot of effort into making it good for development. 
> 
> 
> > Besides, I do not have a problem with most "compile on first demand"
> > performance costs. Many other people do not have problems with that too.
> >  - JSP use "compile on first demand";
> 
> and it does that without including a compiler in the webapp. And it have (
> thanks to Danno ) a simpe tool to allow you to precompile the jsps ( and
> if you do a simple benchmark you'll notice the difference - and not only
> on the first request). 
> 
> That means it is possible to satisfy both sides.  
> 
> > Do you also disaprove Hot Spot?
> 
> I will quote Stefano - sometimes it's better to delete the mail before
> sending it and start again. I did that with this mail - 2 times :-)
> 
> > Well, I wonder if it is not the other way around:
> >   I understand your concerns (even if they are not mine too), but do you
> >   understand mine?
> 
> If I'm not too narrow-minded for that :-).
> 
> Costin
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
>

RE: WEB-INF classloading and on the fly compilation

Posted by Paulo Gaspar <pa...@krankikom.de>.

That was my (mis?)understanding too.
=8oP

Anyway, all is clear and fine now.
=:o)

Have fun,
Paulo

> -----Original Message-----
> From: Pier P. Fumagalli [mailto:pier.fumagalli@eng.sun.com]
> Sent: Tuesday, September 19, 2000 04:00
> To: tomcat-dev@jakarta.apache.org
> Subject: Re: WEB-INF classloading and on the fly compilation
> 
> 
> cmanolache@yahoo.com wrote:
> > 
> > [...]
> > > However, having access to dynamic compilation is a functional issue
> > > wich is important to me.
> > 
> > Nobody disagreed with dynamic compilation ( and if you got this 
> impression
> > from my mail than it's a sign I'm very bad at writting ). I just argued
> > about how to implement this - i.e. include everything in a WAR 
> or make it
> > a server-provided service.
> 
> As I perceived it the first time, you seemed more oriented towards
> "preventing" dynamic compilation from within WEB-APPs... But maybe I
> didn't read your email correctly...
> 
> Once you explained to me what you wanted to do today, I perfectly agree
> that SANDBOXING needs to be kept in mind also when designing such a
> WEB-APP, and don't worry, Cocoon will be a sandboxable WEB-APP...
> 
> 	Pier
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
>

Re: WEB-INF classloading and on the fly compilation

Posted by "Pier P. Fumagalli" <pi...@eng.sun.com>.

cmanolache@yahoo.com wrote:
> 
> [...]
> > However, having access to dynamic compilation is a functional issue
> > wich is important to me.
> 
> Nobody disagreed with dynamic compilation ( and if you got this impression
> from my mail than it's a sign I'm very bad at writting ). I just argued
> about how to implement this - i.e. include everything in a WAR or make it
> a server-provided service.

As I perceived it the first time, you seemed more oriented towards
"preventing" dynamic compilation from within WEB-APPs... But maybe I
didn't read your email correctly...

Once you explained to me what you wanted to do today, I perfectly agree
that SANDBOXING needs to be kept in mind also when designing such a
WEB-APP, and don't worry, Cocoon will be a sandboxable WEB-APP...

	Pier

RE: WEB-INF classloading and on the fly compilation

Posted by Paulo Gaspar <pa...@krankikom.de>.

> -----Original Message-----
> From: cmanolache@yahoo.com [mailto:cmanolache@yahoo.com]
> Sent: Monday, September 18, 2000 17:31
>
> Most of the time I don't even read the name of the sender :-). And for
> sure I had no intention to question your job or background, and I've got
> much worse flames to get it personal.

Well... I have a flaming style... but it was not to be a flame - just a
discussion over functionality.
=;o)


> I am very scared by the idea that many people consider security as a
> "university" problem - commercial sites shouldn't worry about this, only
> school admins. And your mail is the first I've seen in a long time that
> states this explicitly.

NO, no, no. Do not put words on my mouth.

Since I am using my job's e-mail, I must clear up this one.

Quoting myself:
  In an University or in a ISP one does not have control over the code that
  is placed in the server. In a company, such mechanisms(*) can and should
  be implemented.

(*) "mechanisms" to control what code goes into the server.

So, we have the same security concerns but we can use other means of
controlling "what code goes into the server" - while ISPs and universities
do NOT have so much control over "what source code goes on the server".

I was talking ONLY on sandboxing, and I was saying that SANDBOXING has less
priority for us WHEN we control "what code goes into the server" in other
ways.

This means that we CAN control that specific security problem ("what code
goes into the server") trough other means. It does NOT mean that we care
less about that.


Security is clearly MORE important for a commercial site than for a
university.
(Kind of obvious if you think on the consequences for each case.)


> ... I think it's important for this list to be
> aware and realize the importance and _imense_ complexity of security, and
> for people on this list to start understanding that in certain situations
> ( and that includes most commercial sites ) it is far more improtant to
> insure security than some damn features ( or even easy-of-programming !).
> Who cares if you wrote a site in only 3 days and it has all the features
> in the world if after a week someone is able to get all the credit cards
> used by your customers ?

100% agreed. And also 100% agreed on the need for a security policy.

Most of our sites do not have e-commerce and even for those we have a load
of security work going on.


The question is not just about the credit card numbers - down time and data
corruption are bad enough.

For those possible readers that are not aware of this need:
 - Our monitoring reveals that our in-house servers are attacked every
single
   day;
 - This includes mail servers;
 - It even includes people working at home with a normal dial-up connection
   (our webmaster installed monitoring at home);
 - Just for me, the anti-virus system installed at our mail server detected
   4 or 5 infected attachments during the last week (all from Apache
   related mailing lists - be aware of that too).

However, our sites with most traffic are installed in an external ISP and
we do not store any credit card numbers.

Imagine the attacks to e-commerce sites.


> > However, having access to dynamic compilation is a functional issue
> > wich is important to me.
>
> Nobody disagreed with dynamic compilation ( and if you got this impression
> from my mail than it's a sign I'm very bad at writing ). I just argued
> about how to implement this - i.e. include everything in a WAR or make it
> a server-provided service.
>
> Costin


Yes, I understand that now. Sorry if I overreacted to that.

I must not be better at writing since I often sound like flaming when I
just want to defend a POV. (Yes... it is not the first time...)
=;o)


Have fun,
Paulo

Re: WEB-INF classloading and on the fly compilation

Posted by "Pier P. Fumagalli" <pi...@eng.sun.com>.

"Pier P. Fumagalli" wrote:
> 
> cmanolache@yahoo.com wrote:
> >
> > Nick Bauman wrote:
> >
> > > On Mon, 18 Sep 2000, yhs@mimic.onesourcecorp.com wrote:
> > >
> > > > compilation will cause r00ting of the machine instantly. tomcat runs as
> > > > root on many systems anyway...a "rm -rf /" would not be nice on a
> > >
> > > Why in the hell run Tomcat as root? Totally unecessary. There's the big
> > > mistake right there.
> >
> > Maybe because we don't have a small native library to change the UID.
> > Volunteers :-) ?
> >
> > ( in case you run tomcat standalone on port 80 you need to start as
> > root to get the port, but java can't change the id.
> > I believe a production site should run apache+tomcat, but of course
> > some people have a different opinion )
> 
> It's coming kids... It's coming (I just have to write a small
> configuration parser...) I promise it will be in the CVS and in binary
> to download by the end of this week... PROMISE!

I AM AN IDIOT (Yes, I publically admit it)...

Ok, in my code, I change userID before starting up the VM, but that
prevents (of course) the creation of a ServerSocket from within the JVM
(because when the socket is created, the USER ID is already set to be !=
root).

So, the USER ID must be switched after the socket is created, but that
requires some tweaking in the container code...

I need to figure out a different solution for working around the
problem, I want the container to be usable even if no native libraries
are included...

	Pier

Re: WEB-INF classloading and on the fly compilation

Posted by "Pier P. Fumagalli" <pi...@eng.sun.com>.

cmanolache@yahoo.com wrote:
> 
> Nick Bauman wrote:
> 
> > On Mon, 18 Sep 2000, yhs@mimic.onesourcecorp.com wrote:
> >
> > > compilation will cause r00ting of the machine instantly. tomcat runs as
> > > root on many systems anyway...a "rm -rf /" would not be nice on a
> >
> > Why in the hell run Tomcat as root? Totally unecessary. There's the big
> > mistake right there.
> 
> Maybe because we don't have a small native library to change the UID.
> Volunteers :-) ?
> 
> ( in case you run tomcat standalone on port 80 you need to start as
> root to get the port, but java can't change the id.
> I believe a production site should run apache+tomcat, but of course
> some people have a different opinion )

It's coming kids... It's coming (I just have to write a small
configuration parser...) I promise it will be in the CVS and in binary
to download by the end of this week... PROMISE!

	Pier

Re: WEB-INF classloading and on the fly compilation

Posted by cm...@yahoo.com.

Nick Bauman wrote:

> On Mon, 18 Sep 2000, yhs@mimic.onesourcecorp.com wrote:
>
> > compilation will cause r00ting of the machine instantly. tomcat runs as
> > root on many systems anyway...a "rm -rf /" would not be nice on a
>
> Why in the hell run Tomcat as root? Totally unecessary. There's the big
> mistake right there.

Maybe because we don't have a small native library to change the UID.
Volunteers :-) ?

( in case you run tomcat standalone on port 80 you need to start as
root to get the port, but java can't change the id.
I believe a production site should run apache+tomcat, but of course
some people have a different opinion )

Costin

Re: WEB-INF classloading and on the fly compilation

Posted by Nick Bauman <ni...@cortexity.com>.

I use Linux and OpenBSD. I prefer Linux for development, OpenBSD for
production and / or well-defined uses. While Linux has great support and
cutting edge tools, OpenBSD is very proactive about security and
seems to have a more flexible, cleaner and forward-looking TCP/IP
implementation. The two together are great in their respective roles.

portfwd is a tiny C++ program that is easy to verify and configure. Here
is a sample config for portfwd:

--------8<----CUT----8<--------

/*
  portfwd.cfg
*/

tcp /* TCP connections */
{ 
        /*
         * Connections on port 12000 from subnet localhost/24 and
         * ports in the range 0-5000 are forwarded to 127.0.0.1:23.
         * Anything else goes to 127.0.0.1:80.
         */
        12000 { 
                localhost/24:+5000  => 127.0.0.1:23;
                /* anything else */ => 127.0.0.1:80
        };

        /*
         * On port 11000, connections from address localhost, any port,
         * are forwarded to 127.0.0.1:23.
         */
        11000 { localhost => 127.0.0.1:23 }
}

udp /* UDP packets */
{
        /*
         * All packets on port 10000 are forwarded to 127.0.0.1:11000.
         */
        10000 { => 127.0.0.1:11000 }
}

--------8<----CUT----8<--------

portfwd can be found at:

http://sourceforge.net/project/showfiles.php?group_id=771

For greater security, run a logging firewall in front of your Tomcat
webserver. The firewall can be configured to port-forward a privledged
port (80) to an unprivledged port on another machine behind your firewall
where your unprivledged user running Tomcat is listening. Consult your
firewall documentation on how to do this.

On Mon, 18 Sep 2000 cmanolache@yahoo.com wrote:

> Nick Bauman wrote:
> 
> > yhs,
> >
> > No kiddin. Use apache or portfwd. I use a firewall device in front of
> > Tomcat which logs and port-forwards requests to the unpriv. port.
> >
> 
> Can you write some documentation about that ? ( how to set up portfwd,
> etc).  What platforms do you use ?
> 
> Costin
> 
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
> 

-- 
Nicolaus Bauman
Software Engineer
Simplexity Systems
I'll give you $5 if you follow this link:
https://secure.paypal.x.com/refer/pal=nixnixnix%40yahoo.com

Re: WEB-INF classloading and on the fly compilation

Posted by cm...@yahoo.com.

Nick Bauman wrote:

> yhs,
>
> No kiddin. Use apache or portfwd. I use a firewall device in front of
> Tomcat which logs and port-forwards requests to the unpriv. port.
>

Can you write some documentation about that ? ( how to set up portfwd,
etc).  What platforms do you use ?

Costin

RE: WEB-INF classloading and on the fly compilation

Posted by Nick Bauman <ni...@cortexity.com>.

On Mon, 18 Sep 2000, yhs@mimic.onesourcecorp.com wrote:

> > yhs,
> > 
> > No kiddin. Use apache or portfwd. I use a firewall device in front of
> > Tomcat which logs and port-forwards requests to the unpriv. port.
> > 
> 
> not practical for all platforms. tomcat can be (and is) used on various
> devices which run java but do not have/cannot implement due to space
> restrictions etc a firewall ruleset. unless you write the firewall device
> in java (and of course handle the load balancing parts as well) in which
> case i'd love to see your code. :)
> Of course, it would be easier to release a tomcat without huge security
> holes put in due to weird dynamic compilation features which could be
> run as r00t without problems. :) 
> -Ys-
> yhs@mimic.onesourcecorp.com

Right, sure. I'm saying there are ways around the setUID 
requirement. YMMV. But "embedding" Tomcat it in a "device" and letting it
run as "root" on that deivce seems kinda like being given all the tools to
solve a problem and only using the shortest path and the fewest tools to
do it. IOW, if it's a device and it's running a unix or unix-like OS, why
can't it run a tiny port forwarder too? 

Just a thought.

RE: WEB-INF classloading and on the fly compilation

Posted by "yhs@mimic.onesourcecorp.com" <yh...@mimic.onesourcecorp.com>.


On Mon, 18 Sep 2000, Nick Bauman wrote:

> yhs,
> 
> No kiddin. Use apache or portfwd. I use a firewall device in front of
> Tomcat which logs and port-forwards requests to the unpriv. port.
> 

not practical for all platforms. tomcat can be (and is) used on various
devices which run java but do not have/cannot implement due to space
restrictions etc a firewall ruleset. unless you write the firewall device
in java (and of course handle the load balancing parts as well) in which
case i'd love to see your code. :)
Of course, it would be easier to release a tomcat without huge security
holes put in due to weird dynamic compilation features which could be
run as r00t without problems. :) 
-Ys-
yhs@mimic.onesourcecorp.com

RE: WEB-INF classloading and on the fly compilation

Posted by Nick Bauman <ni...@cortexity.com>.

yhs,

No kiddin. Use apache or portfwd. I use a firewall device in front of
Tomcat which logs and port-forwards requests to the unpriv. port.

On Mon, 18 Sep 2000, yhs@mimic.onesourcecorp.com wrote:

> 
> 
> On Mon, 18 Sep 2000, Nick Bauman wrote:
> 
> > On Mon, 18 Sep 2000, yhs@mimic.onesourcecorp.com wrote:
> > 
> > > compilation will cause r00ting of the machine instantly. tomcat runs as
> > > root on many systems anyway...a "rm -rf /" would not be nice on a
> > 
> > Why in the hell run Tomcat as root? Totally unecessary. There's the big
> > mistake right there.
> > 
> > -Nick
> > 
> 
> ports < 1023 are privilidged. you need r00t for 80.
> -Ys-
> yhs@mimic.onesourcecorp.com
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
> 

-- 
Nicolaus Bauman
Software Engineer
Simplexity Systems
I'll give you $5 if you follow this link:
https://secure.paypal.x.com/refer/pal=nixnixnix%40yahoo.com

RE: WEB-INF classloading and on the fly compilation

Posted by "yhs@mimic.onesourcecorp.com" <yh...@mimic.onesourcecorp.com>.


On Mon, 18 Sep 2000, Nick Bauman wrote:

> On Mon, 18 Sep 2000, yhs@mimic.onesourcecorp.com wrote:
> 
> > compilation will cause r00ting of the machine instantly. tomcat runs as
> > root on many systems anyway...a "rm -rf /" would not be nice on a
> 
> Why in the hell run Tomcat as root? Totally unecessary. There's the big
> mistake right there.
> 
> -Nick
> 

ports < 1023 are privilidged. you need r00t for 80.
-Ys-
yhs@mimic.onesourcecorp.com

RE: WEB-INF classloading and on the fly compilation

Posted by Nick Bauman <ni...@cortexity.com>.

On Mon, 18 Sep 2000, yhs@mimic.onesourcecorp.com wrote:

> compilation will cause r00ting of the machine instantly. tomcat runs as
> root on many systems anyway...a "rm -rf /" would not be nice on a

Why in the hell run Tomcat as root? Totally unecessary. There's the big
mistake right there.

-Nick

RE: WEB-INF classloading and on the fly compilation

Posted by cm...@yahoo.com.

> 
> hell...YES. considering the problems/complaints caused by the breakage of
> tomcats session management (the not too random random number bug) it
> should be clear that everyone does have a security policy..or at least a
> notion of it. a webserver is the service thats most likely to be running
> on any machine - if the webserver gets r00ted thats the end of the box and
> your business. and tomcat DOES run as a webserver. personally if i thought
> tomcat was insecure i'd dump it for iPlanet/javawebserver in a
> flash.

The session management problem was minor compared with the admin problems
( admin was enabled by default with no password - as tomcat was intended
for development. )

> > Nobody disagreed with dynamic compilation ( and if you got this impression
> > from my mail than it's a sign I'm very bad at writting ). I just argued
> > about how to implement this - i.e. include everything in a WAR or make it
> > a server-provided service. 
> 
> IMHO dynamic compilation should be off by default if implemented..i think
> its a bad idea anyway. thats usually how stuff gets r00ted. i can easily
> write a servlet with a System call, upload it somehow and your dynamic
> compilation will cause r00ting of the machine instantly. tomcat runs as
> root on many systems anyway...a "rm -rf /" would not be nice on a
> production box. dump it in a WAR and put a big OFF switch with a warning
> notice or at least make sure we can get rid of it easily if need be.

Dynamic compilation is not so wrong if the Policy is enabled and is
restrictive enough. As long as the webapp has a java.policy with the same
rights as applets - nothing bad should happen.

But if we start to make holes - add permission to create arbitrary  class
loaders, all extra permissions required by complex programs like javac and
whatever else may end up in the WAR - then nobody can make any guarantee.

At least the applet-restricted sandbox was tested ( and even there, many
holes found and fixed !). 

Again - it's about how do you implement dynamic compilation and all the
sophisticated template systems ( jsp included ). (so far jasper works fine
with the sandbox )

Costin

RE: WEB-INF classloading and on the fly compilation

Posted by "yhs@mimic.onesourcecorp.com" <yh...@mimic.onesourcecorp.com>.


On Mon, 18 Sep 2000 cmanolache@yahoo.com wrote:

><SNIP>
> The question " do you have a security policy " was addressed to everyone
> on this list - and I don't want to offend anyone. If you feel ofended ( or
> if the answer is no ) then it's your problem :-)
> ( by security policy I mean a general - do you think about that and have a
> plan in case something get wrong and an understanding of what will you
> loose if something happens ? )

hell...YES. considering the problems/complaints caused by the breakage of
tomcats session management (the not too random random number bug) it
should be clear that everyone does have a security policy..or at least a
notion of it. a webserver is the service thats most likely to be running
on any machine - if the webserver gets r00ted thats the end of the box and
your business. and tomcat DOES run as a webserver. personally if i thought
tomcat was insecure i'd dump it for iPlanet/javawebserver in a
flash.

> > However, having access to dynamic compilation is a functional issue 
> > wich is important to me.
> 
> Nobody disagreed with dynamic compilation ( and if you got this impression
> from my mail than it's a sign I'm very bad at writting ). I just argued
> about how to implement this - i.e. include everything in a WAR or make it
> a server-provided service. 
> 
> Costin
> 

IMHO dynamic compilation should be off by default if implemented..i think
its a bad idea anyway. thats usually how stuff gets r00ted. i can easily
write a servlet with a System call, upload it somehow and your dynamic
compilation will cause r00ting of the machine instantly. tomcat runs as
root on many systems anyway...a "rm -rf /" would not be nice on a
production box. dump it in a WAR and put a big OFF switch with a warning
notice or at least make sure we can get rid of it easily if need be.
-Ys-
yhs@mimic.onesourcecorp.com

RE: WEB-INF classloading and on the fly compilation

Posted by cm...@yahoo.com.

> When you feel ofended by my remarks and then make insinuations about 
> the way I perform my job, without knowing anything about it or about my 
> background, you are getting much more personal than I ever did.

Most of the time I don't even read the name of the sender :-). And for
sure I had no intention to question your job or background, and I've got
much worse flames to get it personal. 

I am very scared by the idea that many people consider security as a
"university" problem - commercial sites shouldn't worry about this, only
school admins. And your mail is the first I've seen in a long time that
states this explicitely.

Sorry if I overreacted, but I think it's important for this list to be
aware and realize the importance and _imense_ complexity of security, and
for people on this list to start understanding that in certain situations
( and that includes most commercial sites ) it is far more improtant to
insure security than some damn features ( or even easy-of-programming !).
Who cares if you wrote a site in only 3 days and it has all the features
in the world if after a week someone is able to get all the credit cards
used by your customers ? 

The question " do you have a security policy " was addressed to everyone
on this list - and I don't want to offend anyone. If you feel ofended ( or
if the answer is no ) then it's your problem :-)
( by security policy I mean a general - do you think about that and have a
plan in case something get wrong and an understanding of what will you
loose if something happens ? )

 
> However, having access to dynamic compilation is a functional issue 
> wich is important to me.

Nobody disagreed with dynamic compilation ( and if you got this impression
from my mail than it's a sign I'm very bad at writting ). I just argued
about how to implement this - i.e. include everything in a WAR or make it
a server-provided service. 

Costin

RE: WEB-INF classloading and on the fly compilation

Posted by Paulo Gaspar <pa...@krankikom.de>.

Costin, 


My writing is not diplomatic but I want no fights either.

When I make a provocative question like:
  "Do you also disaprove Hot Spot?"
what I mean is to highlight something that I think to be wrong with an 
idea. I do it trough exageration just to make what I think to be the 
negative aspects more obvious.

I am atacking and idea, not a person.

When you feel ofended by my remarks and then make insinuations about 
the way I perform my job, without knowing anything about it or about my 
background, you are getting much more personal than I ever did.

This is the only thing that made this discussion so hard.


If you disagree with me, attack my ideas back with logical arguments, 
and highlight their week points by making them look ridicolous if that
helps to make issues clear. No problem. That is what I do.

Just focus on logical argumentation without taking it or making it 
personal.


Your P.O.V. has to be valid. With your experience, there is no doubt 
about that.

In the other hand, none of us is right all the time.


BTW, I think that in terms of functionality the differences of opinion
look to be even less than those 60-70%. It is on the way to implement
it that I see most discussions going on.


However, having access to dynamic compilation is a functional issue 
wich is important to me.

Again, your general principles on the subject are right. The problem 
are all the exceptional situations - and I have some. If you just 
leave a door open, or a door that can be open, everyone is happy.


Have fun,
Paulo Gaspar


> -----Original Message-----
> From: cmanolache@yahoo.com [mailto:cmanolache@yahoo.com]
> Sent: Sunday, September 17, 2000 23:43
> To: tomcat-dev@jakarta.apache.org; pier@betaversion.org
> Subject: Re: WEB-INF classloading and on the fly compilation
> 
> 
> > I don't really care about what Costin says. He's a good developer, a
> > good friend, and an excellent co-worker, but I believe we both realize
> > that we have different interests and different priorities on circa 90%
> > of the stuff :) :) :)
> 
> Thanks, Pier :-)
> 
> After all that's how open source works - we all try to improve a product
> based on our interest. There is a voting system - so if something I do
> brakes your code you can -1 it and it'll be rolled back. 
> 
> I try to read most ( ok, 1/2 ) of the mailing lists and get informations,
> and I try to send back informations. Each of us have a certain background
> and experience that shapes our opinions, and IMHO the filter for
> technical opinions must be at the receiver ( and the filter for
> non-technical opinions at the sender !). 
> 
> 
> Costin
> ( well, I hope the differences are not at 90%, but closer to 60-70% :-)
> 
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tomcat-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tomcat-dev-help@jakarta.apache.org
>

Re: WEB-INF classloading and on the fly compilation

Posted by cm...@yahoo.com.

> I don't really care about what Costin says. He's a good developer, a
> good friend, and an excellent co-worker, but I believe we both realize
> that we have different interests and different priorities on circa 90%
> of the stuff :) :) :)

Thanks, Pier :-)

After all that's how open source works - we all try to improve a product
based on our interest. There is a voting system - so if something I do
brakes your code you can -1 it and it'll be rolled back. 

I try to read most ( ok, 1/2 ) of the mailing lists and get informations,
and I try to send back informations. Each of us have a certain background
and experience that shapes our opinions, and IMHO the filter for
technical opinions must be at the receiver ( and the filter for
non-technical opinions at the sender !). 


Costin
( well, I hope the differences are not at 90%, but closer to 60-70% :-)

Re: WEB-INF classloading and on the fly compilation

Posted by Pierpaolo Fumagalli <pi...@apache.org>.

Paulo Gaspar wrote:
> 
> From: cmanolache@yahoo.com [mailto:cmanolache@yahoo.com]
> >
> > Sorry for using your bandwith with this thread - I ( personally ) believe
> > that complex services like code generation, compilation, template systems,
> > etc are better if implemented as standalone components that can be
> > integrated in the server instead of puting them in a webapp.
> > 
> > [...]
> > 
> > I do believe that pre-compilation is a better solution for a production
> > site and jspc is a prove.  That doesn't mean it should be used for
> > develoment  - just that it's something that systems that
> > use dynamic compilation should think about. ( afaik j2ee is also
> > generating its stubs at deployment time, and probably there are other
> > examples - that means it is possible to do that even in complex solutions)
> 
> With that I can agree.
> 
> [...]
> 
> What I found problematic was to have the doors completelly closed to
> "on demand" solutions.

I don't really care about what Costin says. He's a good developer, a
good friend, and an excellent co-worker, but I believe we both realize
that we have different interests and different priorities on circa 90%
of the stuff :) :) :)

I need on-the-fly compilation, and goddamit, it's an open-source project
I'm going to do something for it :) :) :)

And if some of us don't want to use it for their thowsands of reasons
that I might understand, or not, well.... I could care less :) :)

	Pier

RE: WEB-INF classloading and on the fly compilation

Posted by Paulo Gaspar <pa...@krankikom.de>.

With that I can agree.

What I found problematic was to have the doors completelly closed to
"on demand" solutions.

There are just too many exceptions in software development for absolute
rules to be acceptable.


I just find a pitty that you end a perfectly reasonable posting with a
  "You may agree or not - I'll not continue this thread".

You (not me) tourned this thread into a personal thing.

This is just me, acting as a user of Tomcat and trying to influence its
evolution in such a way that doors are not closed to things I do or plan
to do. (And yes, I can also go somewhere else.)

This kind of talk shouldn't be this hard. This way, I do not know if my
very poor diplomatic skils are up to the task.


Have fun,
Paulo


> -----Original Message-----
> From: cmanolache@yahoo.com [mailto:cmanolache@yahoo.com]
> Sent: Sunday, September 17, 2000 08:39
>
> Hi,
>
> Sorry for using your bandwith with this thread - I ( personally ) believe
> that complex services like code generation, compilation, template systems,
> etc are better if implemented as standalone components that can be
> integrated in the server instead of puting them in a webapp.
>
> The interface between jasper and tomcat and all the bugs and tricks we had
> to do are probably good examples.
>
> This has nothing to do with how compilers read their input, and it doesn't
> mean dynamic code generation is not good - it's in fact an excelent
> solution. It's just about how to implement dynamic code generation and
> compilation, and how to interface this with the rest of the system.
>
> I do believe that pre-compilation is a better solution for a production
> site and jspc is a prove.  That doesn't mean it should be used for
> develoment  - just that it's something that systems that
> use dynamic compilation should think about. ( afaik j2ee is also
> generating its stubs at deployment time, and probably there are other
> examples - that means it is possible to do that even in complex solutions)
>
>
>
> You may agree or not - I'll not continue this thread.
>
> Costin

RE: WEB-INF classloading and on the fly compilation

Posted by cm...@yahoo.com.

Hi,

Sorry for using your bandwith with this thread - I ( personally ) believe
that complex services like code generation, compilation, template systems,
etc are better if implemented as standalone components that can be
integrated in the server instead of puting them in a webapp. 

The interface between jasper and tomcat and all the bugs and tricks we had
to do are probably good examples. 

This has nothing to do with how compilers read their input, and it doesn't
mean dynamic code generation is not good - it's in fact an excelent
solution. It's just about how to implement dynamic code generation and
compilation, and how to interface this with the rest of the system.

I do believe that pre-compilation is a better solution for a production
site and jspc is a prove.  That doesn't mean it should be used for
develoment  - just that it's something that systems that
use dynamic compilation should think about. ( afaik j2ee is also
generating its stubs at deployment time, and probably there are other
examples - that means it is possible to do that even in complex solutions)



You may agree or not - I'll not continue this thread.  

Costin

RE: WEB-INF classloading and on the fly compilation

Posted by Paulo Gaspar <pa...@krankikom.de>.

> -----Original Message-----
> From: cmanolache@yahoo.com [mailto:cmanolache@yahoo.com]
> Sent: Saturday, September 16, 2000 21:48
>
> > In my company, we control all the code that is placed in the
> Web Server.
>
> Well, you may think so, and I hope you are right. I would be happy to
> accept all your flames and "narrow-minded" if I can make you have just a
> little doubt about this statement.

Mentioning "narrow-minded" so often again seems exagerated. My sentence
was:
  "Sorry, but that really sounds narrow minded."

It is a clear reference to ONE position you took. I did NOT say that "you
are narrow minded". I said "that... sounds".

Is different, isn't it?

Discussions get healtier and more constructive if you do not take
criticizing as personal, especially when it isn't.


> And you still think you are controlling it?

Yes. By using anything that provides a simpler solution, keeping
it all as small as possible.

> I hope you're not running an e-commerce site :-)

Is my work under examination here? At the Tomcat mailling list?


> I certainly hope so - I spent some time running servers that many users
> tryed to hack, and I have an ideea about how many commercial sites are
> run. And indeed, there is a huge difference.

"Many" is not even sinonym to "majority" and much less to "all".


> Do you believe that sandbox is required more for "campus" servers ( where
> a traditional experience exists with hacking and admins are well aware of
> the risks, and more important - the possible damage is minor)
>
> And on commercial sites ( where most effort goes into creating bloated
> sites and delivering on short schedules, security policies don't exist or
> are ignored and a break in means lots of $$$ losts ) it's less important ?

No, it is not less important. But the admins are not necessarily less aware
of the risks and there are other mechanisms to control THAT security we are
talking about - let's remember we are talking about sandboxing.

In an University or in a ISP one does not have control over the code that is
placed in the server. In a company, such mechanisms can and should be
implemented.


> What's your security policy ? How much experience do you have on detecting
> break-ins ? How much % time do you spend reviewing the code ( that is more
> and more complex ) for security ?
> ( do you use XSL ? Are you aware that in a stylesheet you can have embeded
> scripts - and you can call exec("rm -rf ~") from inside ? Or
> "exec( mail hacker < creditcard_numbers )" ? ) Same for jsps btw.

Again, my work is not the point here.

Anyway:
 - We are talking about sandboxing. Controlling break-ins is more on the
   admin field and is not done trough sandboxing - which only limits some
   effects of a break-in;
 - I am always reviewing code. I just do not consider it dead at any time;
 - I am quite aware of those (and other) scripting/shell issues.

So what? What does it have to do with sandboxing and it's limitations on the
use of a compiler?


> Using them in a secure way isn't the same as killing them.

"Using them in a secure way" sounds more constructive than just saying "do
not do it" to the dynamic compilation issue.

Again, I do NOT mean that you have to do it.

I do mean that, IMO, one should remain a bit more open minded to that
possibility than you do.

This last sentence sumarizes the point I want to make.


> In fact
> servlets and Jsps are the only technologies today that may provide this.
> ( but that's changing, with less and less interest in security and
> performance and more and more interest in feature bloat )

I do not see less interest in security in the people I work with - I see
growing interest.

A bit of the same applies to peformance, although faster CPU's and the
evolution of some tools (as Hotspot in Java) tend to make things easier.

(BTW - I am not using the above mentioned XSLT anymore just because there
are faster and simpler to use template mechanisms.)


> > Besides, I do not have a problem with most "compile on first demand"
> > performance costs. Many other people do not have problems with that too.
> >  - JSP use "compile on first demand";
>
> and it does that without including a compiler in the webapp. And it have (
> thanks to Danno ) a simpe tool to allow you to precompile the jsps ( and
> if you do a simple benchmark you'll notice the difference - and not only
> on the first request).

I do not use JSPs. But using a tool is not the most pratical way to do it in
development time - especially when part of the work is made by a designer
using a visual tool (e.g.: Ultradev supports JSPs).

I hope you do not reply just by saying that this is not a best practice.
Tools that are too strict by only allowing what its authors consider to be
best practices just tend to be less used.


> > Do you also disaprove Hot Spot?
>
> I will quote Stefano - sometimes it's better to delete the mail before
> sending it and start again. I did that with this mail - 2 times :-)

I wonder why you had to do that.

My question makes sense: Hot Spot is a form of compilation that happens in
a running server and that might have negative performance impact in the
initial execution time (although things really speed up afterwards).

Other forms of "compile on first/initial demand" can have similar
advantages to Hot Spot.

You used the non-sense analogy of compiling Apache sources at run time to
demonstrate you point of view and I used Hot Spot to demo mine.


> > Well, I wonder if it is not the other way around:
> >   I understand your concerns (even if they are not mine too), but do you
> >   understand mine?
>
> If I'm not too narrow-minded for that :-).
>
> Costin

I hope not. I hope that you are at least trying.

Again, you are taking it as personal. But I already wrote about that above.


Have fun,
Paulo Gaspar

RE: WEB-INF classloading and on the fly compilation

Posted by cm...@yahoo.com.

> > My goal is to implement a server where webapps can be deployed in real
> > sandboxes, like applets in browsers. If Cocoon and other web-development
> > environments will go this way ( putting too much complexity
> > inside the web app and expecting it to work ) than the whole security will
> > be useless - you'll have a sandbox but nothing will be able to
> > run inside.
> 
> *Your* goal.

I certainly hope there are other people interested in that. 


> For others - like me - sandboxing is not that important. Sandboxing might be
> very important for an ISP. However, these days, more and more "private" web
> servers exist.

More and more insecure web servers exist too. 


> In my company, we control all the code that is placed in the Web Server. 

Well, you may think so, and I hope you are right. I would be happy to
accept all your flames and "narrow-minded" if I can make you have just a
little doubt about this statement. 

Do you control your own code ( from a security point of view ) ? Do you
control the code ( including jsp pages, etc) written by all programmers
that work for your organization? And all the code that is included in
various APIs and packges you use ?

> On
> the other hand, the applications that we have to develop are becoming more
> and more complex.

And you still think you are controlling it? 

 
> Then, versatility and fast problem solving becomes much more important than
> sandboxing.

I hope you're not running an e-commerce site :-)


> Prioritizing sandboxing is very nice for a campus server used by students:
> many users are trying to hack the thing and they do not have great
> complexity demands.
> 
> Are you giving any priority to commercial users needs?

I certainly hope so - I spent some time running servers that many users
tryed to hack, and I have an ideea about how many commercial sites are
run. And indeed, there is a huge difference. 


Do you believe that sandbox is required more for "campus" servers ( where
a traditional experience exists with hacking and admins are well aware of
the risks, and more important - the possible damage is minor) 

And on commercial sites ( where most effort goes into creating bloated
sites and delivering on short schedules, security policies don't exist or
are ignored and a break in means lots of $$$ losts ) it's less important ?

What's your security policy ? How much experience do you have on detecting
break-ins ? How much % time do you spend reviewing the code ( that is more
and more complex ) for security ?
( do you use XSL ? Are you aware that in a stylesheet you can have embeded
scripts - and you can call exec("rm -rf ~") from inside ? Or 
"exec( mail hacker < creditcard_numbers )" ? ) Same for jsps btw.
  

> > I am also interested in performance - and I don't like the idea of running
> > 3-4 compiler instances at the same time ( or even 1 ! ) on a production
> > site. In most cases this can be avoided by compiling at deployment time.
> 
> "In most cases" is the key expression.
> 
> Well, in most cases you can build a site with static HTML... so, let's kill
> both Servlet and JSP technologies too, right?

Using them in a secure way isn't the same as killing them. In fact
servlets and Jsps are the only technologies today that may provide this. 
( but that's changing, with less and less interest in security and
performance and more and more interest in feature bloat ) 
  

> Is Tomcat supposed to be ONLY a production server? Without being good for
> development???

I certainly hope tomcat will not be the only production server interested
in security and performance. 

And so far we put a lot of effort into making it good for development. 


> Besides, I do not have a problem with most "compile on first demand"
> performance costs. Many other people do not have problems with that too.
>  - JSP use "compile on first demand";

and it does that without including a compiler in the webapp. And it have (
thanks to Danno ) a simpe tool to allow you to precompile the jsps ( and
if you do a simple benchmark you'll notice the difference - and not only
on the first request). 

That means it is possible to satisfy both sides.  

> Do you also disaprove Hot Spot?

I will quote Stefano - sometimes it's better to delete the mail before
sending it and start again. I did that with this mail - 2 times :-)

> Well, I wonder if it is not the other way around:
>   I understand your concerns (even if they are not mine too), but do you
>   understand mine?

If I'm not too narrow-minded for that :-).

Costin

RE: WEB-INF classloading and on the fly compilation

Posted by Paulo Gaspar <pa...@krankikom.de>.

> -----Original Message-----
> From: cmanolache@yahoo.com [mailto:cmanolache@yahoo.com]
>
> Doesn't it ? Is it simpler to change both the compiler and probably all
> the tools ? Compiling from streams to streams is simple to implement ?

Well, in the end of my posting there was this sentence:
  "You do not have to implement it, but why not encourage those who do?"

(Yeah... there was a typo on "those".)

>
> I just tried to point the differences between an web app environment and a
> general development environment.

I never found so many nice uses for dynamic code generation as in web app
development.


> My goal is to implement a server where webapps can be deployed in real
> sandboxes, like applets in browsers. If Cocoon and other web-development
> environments will go this way ( putting too much complexity
> inside the web app and expecting it to work ) than the whole security will
> be useless - you'll have a sandbox but nothing will be able to
> run inside.

*Your* goal.

For others - like me - sandboxing is not that important. Sandboxing might be
very important for an ISP. However, these days, more and more "private" web
servers exist.

In my company, we control all the code that is placed in the Web Server. On
the other hand, the applications that we have to develop are becoming more
and more complex.

Then, versatility and fast problem solving becomes much more important than
sandboxing.

Prioritizing sandboxing is very nice for a campus server used by students:
many users are trying to hack the thing and they do not have great
complexity demands.

Are you giving any priority to commercial users needs?


> I am also interested in performance - and I don't like the idea of running
> 3-4 compiler instances at the same time ( or even 1 ! ) on a production
> site. In most cases this can be avoided by compiling at deployment time.

"In most cases" is the key expression.

Well, in most cases you can build a site with static HTML... so, let's kill
both Servlet and JSP technologies too, right?


> Yes, I use to compile Apache from sources - but I wouldn't do it at
> runtime while it is serving pages.

I have cases where designers will be redefining stuff in development and
production servers trough templates and simple GUI interfaces.

Developing this logic in the Web Server is much simpler than outside. And it
is no problem to do it in a Development Server.

Is Tomcat supposed to be ONLY a production server? Without being good for
development???


Besides, I do not have a problem with most "compile on first demand"
performance costs. Many other people do not have problems with that too.
 - JSP use "compile on first demand";
 - C# and the next generation of the M$ Scripting Engine (used by Active
   Server Pages) uses "compile on first demand";
 - Other technologies do it too.

Are they all wrong?
Do you also disaprove Hot Spot?


Notice that I am talking about "compile on first demand" and NOT about
"compile everytime you f***ing need it". I have a problem with this last
one too.


> I hope that letting people know about my concerns doesn't hurt - it
> may be something they haven't considered. Or they can easily ignore me.
>
> Costin

Well, I wonder if it is not the other way around:
  I understand your concerns (even if they are not mine too), but do you
  understand mine?

  And I bet I am not alone on my concerns.


I think that Web Servers are very generic general tools with a very broad
range of use models that must be supported. At least if the Web Server
wants to be taken seriously.


Have fun,

Paulo Gaspar

RE: WEB-INF classloading and on the fly compilation

Posted by cm...@yahoo.com.

> Sorry, but that really sounds narrow minded. 
> 
> There are just too many problems that can be solved trough dynamic 
> code generation (besides JSPs, XSLTs and template mechanisms) and 
> having to put source code on the disk does not make things easier.

Doesn't it ? Is it simpler to change both the compiler and probably all
the tools ? Compiling from streams to streams is simple to implement ? 
 
 
> 
> Such an abstraction (e.g.: compiling from streams into streams) is
> simple to implement in an existing Java compiler and could bring 
> great benefits.
> 
> Why dismiss it?
> 
> You do not have to implement it, but why not encourage does who do?

I just tried to point the differences between an web app environment and a
general development environment.


My goal is to implement a server where webapps can be deployed in real
sandboxes, like applets in browsers. If Cocoon and other web-development
environments will go this way ( putting too much complexity 
inside the web app and expecting it to work ) than the whole security will
be useless - you'll have a sandbox but nothing will be able to run inside. 


I am also interested in performance - and I don't like the idea of running
3-4 compiler instances at the same time ( or even 1 ! ) on a production
site. In most cases this can be avoided by compiling at deployment time. 
Yes, I use to compile Apache from sources - but I wouldn't do it at
runtime while it is serving pages. 


I hope that letting people know about my concerns doesn't hurt - it
may be something they haven't considered. Or they can easily ignore me.

Costin

RE: WEB-INF classloading and on the fly compilation

Posted by Paulo Gaspar <pa...@krankikom.de>.

Sorry, but that really sounds narrow minded. 

There are just too many problems that can be solved trough dynamic 
code generation (besides JSPs, XSLTs and template mechanisms) and 
having to put source code on the disk does not make things easier.

Such an abstraction (e.g.: compiling from streams into streams) is
simple to implement in an existing Java compiler and could bring 
great benefits.

Why dismiss it?

You do not have to implement it, but why not encourage does who do?


Have fun,
Paulo Gaspar


> -----Original Message-----
> From: cmanolache@yahoo.com [mailto:cmanolache@yahoo.com]
> Sent: Friday, September 15, 2000 01:27
> 
> Deployment != Runtime.
> 
> A compiler is a very complex tool, with a very narrow goal - to produce
> bytecode from a java source. It doesn't try to be the coolest
> distributable-location-independent-load-from-network app. 
> 
> I would be more than happy if the compiler would do a good job at
> generating a reasonable good code at a reasonable speed. 
> 
> 
> Costin

Re: WEB-INF classloading and on the fly compilation

Posted by cm...@yahoo.com.

> That's a problem related to the compiler, relying on files instead on
> classes loaded by the classloaders... I know that at Olliance they're
> thinking about distributing WEB-APPs on a set of different servlet
> engines (a some sort of load balancing but with more guts :). This will
> surely add lots of hacks in class-loading mechanisms, since WEB-APPs
> WARs will be loaded from the network and so on...
> If the compiler relies on files on the disk, then, we're screwed,
> because we cannot possibly have a portable servlet using on-the-fly
> compilation...

Deployment != Runtime.

A compiler is a very complex tool, with a very narrow goal - to produce
bytecode from a java source. It doesn't try to be the coolest
distributable-location-independent-load-from-network app. 

I would be more than happy if the compiler would do a good job at
generating a reasonable good code at a reasonable speed. 


Costin

Re: WEB-INF classloading and on the fly compilation

Posted by "Pier P. Fumagalli" <pi...@eng.sun.com>.

cmanolache@yahoo.com wrote:
> 
> :-)
> 
> > Just imagine that this thing we have to compile DEPENDS on another JAR
> > included in our little web-app (maybe a set of utility classes, maybe
> > whatever, I have the classes somewhere in the WEB-APP).
> 
> AFAIK jasper is passing CLASSPATH explicitely ( using -classpath
> parameter). I don't know what's wrong with javac ( or if something is
> wrong at all - after all we are doing a hack to load javac, I don't think
> there is any documentation saying that calling sun.tools.javac.Main is
> supposed to work ). Same for Ant - it creates a -classpath argument.
> 
> Since you may want to support jikes and out-of-process compilation (with
> exec ) I think you'll have to pass the class path explicitely.

Really, I wouldn't give a crap to Jikes if I was able to compile from
within a standard WEB-APP...

> Of course, it is not possible from a "standard" servlet to do that - since
> the classpath is not available in any standard way. Tomcat provides an
> attribute for that in order to hack jasper in.

That's a problem related to the compiler, relying on files instead on
classes loaded by the classloaders... I know that at Olliance they're
thinking about distributing WEB-APPs on a set of different servlet
engines (a some sort of load balancing but with more guts :). This will
surely add lots of hacks in class-loading mechanisms, since WEB-APPs
WARs will be loaded from the network and so on...
If the compiler relies on files on the disk, then, we're screwed,
because we cannot possibly have a portable servlet using on-the-fly
compilation...

> You could also find the webapp root ( getRealPath("/") ) and compute the
> classpath yourself - but I would say this is worse, since you'll rely on a
> run-time directory structure that may not exist ( again, war file format
> is a deployment format, not a run-time format ).

Yep... 

	Pier

Re: WEB-INF classloading and on the fly compilation

Posted by Peter Donald <do...@mad.scientist.com>.

At 03:42  14/9/00 -0700, you wrote:
>Stefano Mazzocchi wrote:
>> 
>> The WAR packages were created for that right? Good, but this idea is
>> broken by the fact that all existing java compilers assume that classes
>> are loaded from disk.
>> 
>> Damn, now I need to write a java compiler :(
>
>What about the thing you were telling me on the phone? "Pizza" ???

optionally you could modify kopi - unfortunately that is GPL and you would
have to either ship it seperately or else convince them to dual license it.

Cheers,

Pete

*------------------------------------------------------*
| "Nearly all men can stand adversity, but if you want |
| to test a man's character, give him power."          |
|       -Abraham Lincoln                               |
*------------------------------------------------------*

Re: WEB-INF classloading and on the fly compilation

Posted by "Pier P. Fumagalli" <pi...@eng.sun.com>.

Stefano Mazzocchi wrote:
> 
> "Pier P. Fumagalli" wrote:
> >
> > Stefano Mazzocchi wrote:
> > >
> > > The WAR packages were created for that right? Good, but this idea is
> > > broken by the fact that all existing java compilers assume that classes
> > > are loaded from disk.
> > >
> > > Damn, now I need to write a java compiler :(
> >
> > What about the thing you were telling me on the phone? "Pizza" ???
> 
> I checked all existing Java compiler and none of them work thru resource
> abstraction but they read the files directly from the disk. :(

If we find a suitable license, I can hack the compiler itself...

	Per

Re: WEB-INF classloading and on the fly compilation

Posted by "Pier P. Fumagalli" <pi...@eng.sun.com>.

Stefano Mazzocchi wrote:
> 
> "Pier P. Fumagalli" wrote:
> >
> > Stefano Mazzocchi wrote:
> > >
> > > The WAR packages were created for that right? Good, but this idea is
> > > broken by the fact that all existing java compilers assume that classes
> > > are loaded from disk.
> > >
> > > Damn, now I need to write a java compiler :(
> >
> > What about the thing you were telling me on the phone? "Pizza" ???
> 
> I checked all existing Java compiler and none of them work thru resource
> abstraction but they read the files directly from the disk. :(

If we find a suitable license, I can hack the compiler itself...

	Per

Re: WEB-INF classloading and on the fly compilation

Posted by Stefano Mazzocchi <st...@apache.org>.

"Pier P. Fumagalli" wrote:
> 
> Stefano Mazzocchi wrote:
> >
> > The WAR packages were created for that right? Good, but this idea is
> > broken by the fact that all existing java compilers assume that classes
> > are loaded from disk.
> >
> > Damn, now I need to write a java compiler :(
> 
> What about the thing you were telling me on the phone? "Pizza" ???

I checked all existing Java compiler and none of them work thru resource
abstraction but they read the files directly from the disk. :(

-- 
Stefano Mazzocchi      One must still have chaos in oneself to be
                          able to give birth to a dancing star.
<st...@apache.org>                             Friedrich Nietzsche
--------------------------------------------------------------------
 Missed us in Orlando? Make it up with ApacheCON Europe in London!
------------------------- http://ApacheCon.Com ---------------------

Re: WEB-INF classloading and on the fly compilation

Posted by Stefano Mazzocchi <st...@apache.org>.

"Pier P. Fumagalli" wrote:
> 
> Stefano Mazzocchi wrote:
> >
> > The WAR packages were created for that right? Good, but this idea is
> > broken by the fact that all existing java compilers assume that classes
> > are loaded from disk.
> >
> > Damn, now I need to write a java compiler :(
> 
> What about the thing you were telling me on the phone? "Pizza" ???

I checked all existing Java compiler and none of them work thru resource
abstraction but they read the files directly from the disk. :(

-- 
Stefano Mazzocchi      One must still have chaos in oneself to be
                          able to give birth to a dancing star.
<st...@apache.org>                             Friedrich Nietzsche
--------------------------------------------------------------------
 Missed us in Orlando? Make it up with ApacheCON Europe in London!
------------------------- http://ApacheCon.Com ---------------------

Re: WEB-INF classloading and on the fly compilation

Posted by "Pier P. Fumagalli" <pi...@eng.sun.com>.

Stefano Mazzocchi wrote:
> 
> The WAR packages were created for that right? Good, but this idea is
> broken by the fact that all existing java compilers assume that classes
> are loaded from disk.
> 
> Damn, now I need to write a java compiler :(

What about the thing you were telling me on the phone? "Pizza" ???

	Pier

Re: WEB-INF classloading and on the fly compilation

Posted by "Pier P. Fumagalli" <pi...@eng.sun.com>.

Stefano Mazzocchi wrote:
> 
> The WAR packages were created for that right? Good, but this idea is
> broken by the fact that all existing java compilers assume that classes
> are loaded from disk.
> 
> Damn, now I need to write a java compiler :(

What about the thing you were telling me on the phone? "Pizza" ???

	Pier

Re: WEB-INF classloading and on the fly compilation

Posted by "Pier P. Fumagalli" <pi...@eng.sun.com>.

Vadim Gritsenko wrote:
> 
> Sources are available at www.sun.com; and even if they would be unavailable
> there are tools like JAD.
> 
> You just need to inherit ( or re-write ) from sun.tools.javac.Main and
> replace call
> 
>         // Create our Environment.
>         BatchEnvironment env = BatchEnvironment.create(out,
>                                                        sourcePathArg,
>                                                        classPathArg,
>                                                        sysClassPathArg,
>                                                        extDirsArg);
> 
> with something like this:
> 
>         BatchEnvironment env = new BatchEnvironment(out, srcClassPath,
> binClassPath);
> 
> where srcClassPahth and binClassPath - instances of your class which
> inherits from ClassPath,
> and this new ClassPath should load classes from where you want - from file
> system, RDBMS, network, ...

BUT... From my WEB-APP I don't know where the JARs I place in
WEB-INF/lib are... They cannot even be on the filesystems...
The problem when compiling is basically resolving "import ...;"
statements in java code of classes included WITHIN the WEB-APP...
There's no portable way of doing that...

The only solution would be to have a compiler that, instead of relying
on files, could rely on Class object instances as loaded by the
classloader...

	Pier

RE: WEB-INF classloading and on the fly compilation

Posted by Vadim Gritsenko <vg...@hns.com>.

Sources are available at www.sun.com; and even if they would be unavailable
there are tools like JAD.

You just need to inherit ( or re-write ) from sun.tools.javac.Main and
replace call

	// Create our Environment.
	BatchEnvironment env = BatchEnvironment.create(out,
						       sourcePathArg,
						       classPathArg,
						       sysClassPathArg,
						       extDirsArg);

with something like this:

        BatchEnvironment env = new BatchEnvironment(out, srcClassPath,
binClassPath);

where srcClassPahth and binClassPath - instances of your class which
inherits from ClassPath,
and this new ClassPath should load classes from where you want - from file
system, RDBMS, network, ...


But - I don't know where to get sources for OS - sorry! :)

Vadim


> -----Original Message-----
> From: Pier P. Fumagalli [mailto:pier.fumagalli@eng.sun.com]
> Sent: Thursday, September 14, 2000 6:49 PM
> To: tomcat-dev@jakarta.apache.org
> Cc: Stefano Mazzocchi; cmanolache@yahoo.com;
> pier@betaversion.org; cocoon-dev@xml.apache.org
> Subject: Re: WEB-INF classloading and on the fly compilation
>
>
> cmanolache@yahoo.com wrote:
> >
> > On Thu, 14 Sep 2000, Stefano Mazzocchi wrote:
> >
> > > cmanolache@yahoo.com wrote:
> > > >
> > > > :-)
> > > >
> > > > You could also find the webapp root ( getRealPath("/") )
> and compute the
> > > > classpath yourself - but I would say this is worse, since
> you'll rely on a
> > > > run-time directory structure that may not exist ( again,
> war file format
> > > > is a deployment format, not a run-time format ).
> >
> > > This works _ONLY_ if classloading is performed thru files... what
> > > happens if I have a NetworkClassLoader a JDBCClassLoader or a
> > > OnTheFlyAssembledClassLoader? no compiler existing today (I
> just checked
> > > ALL the existing java compiler I could find!) would be able to compile
> > > against classes loaded thru a classloader.
> >
> > One would argue that compilation should happen at deployment, not at
> > runtime.
>
> Let's get real... More and more we're using on the fly compilation
> because it has been proven to be an effective techniques... Now you're
> saying that I can't do it within a servlet? And just because the
> compiler works in a different way?
>
> > > everything, ranging from a database to a compiler... so you don't go
> > > shopping for tons of jar, but you just drop  the WAR on the webapp
> > > folder and you're done. Period.
> > >
> > > The WAR packages were created for that right? Good, but this idea is
> > > broken by the fact that all existing java compilers assume
> that classes
> > > are loaded from disk.
> >
> > You may have additional problems - even if you include a javac
> compiler in
> > your WAR, the sandbox will not allow you to use it too much.
> Some servers
> > may care about security ( well, I'm dreaming, nobody cares about that
> > today :-).
>
> What Cocoon does today is to write the JAVA files in the servlet work
> directory (and I believe that's given in the specs), and perform a bunch
> of security-legal operations (like constructing a ClassLoader with the
> WEB-APP classloader as a parent... blablabla)...
> The only "security constraint" such a WEB-APP would require is to be
> able to construct  a classloader, and work in the work directory (even
> if this last one can be avoided, doing basically everything in
> memory)...
>
> > > Damn, now I need to write a java compiler :(
> >
> > Maybe an OS too - all those existing OSes that are using this bad
> > filesystem concept ... :-)
>
> We can hack them, though... If we have the sources.... :) :) :)
>
> 	Pier

Re: WEB-INF classloading and on the fly compilation

Posted by cm...@yahoo.com.

> What Cocoon does today is to write the JAVA files in the servlet work
> directory (and I believe that's given in the specs), and perform a bunch
> of security-legal operations (like constructing a ClassLoader with the
> WEB-APP classloader as a parent... blablabla)...
> The only "security constraint" such a WEB-APP would require is to be
> able to construct  a classloader, and work in the work directory (even
> if this last one can be avoided, doing basically everything in
> memory)...

Creating a ClassLoader is not a security-legal operation for a sandboxed
application ( unless you use a special case of URLClassLoader ). In fact
it's considered a dangerous operation. 

Javac itself has a number of insecure operations inside - and jikes ( or
other C-based java compilers) are clearly out. 
It works with jasper because doPriviledged() is used - and jasper is
granted AllPermissions.

Of course, you can just disable security and grant all permissions to
all applications - the real problem is ( IMHO ) that you stretch too much
the servlet. Not everything must be a servlet. 

Or - write a simple interface for what you need and adapters for common
servers, and if someone has a very special setup he can create an adapter. 
Install cocoon ( or any system that is complex enough to require a
complier ) as a server extension. ( at least that's how jasper works )

Or you can put everything in a WAR box, and rewrite the compiler and
everything else to fit the init()/service() model. 


Costin

Re: WEB-INF classloading and on the fly compilation

Posted by "Pier P. Fumagalli" <pi...@eng.sun.com>.

cmanolache@yahoo.com wrote:
> 
> On Thu, 14 Sep 2000, Stefano Mazzocchi wrote:
> 
> > cmanolache@yahoo.com wrote:
> > >
> > > :-)
> > >
> > > You could also find the webapp root ( getRealPath("/") ) and compute the
> > > classpath yourself - but I would say this is worse, since you'll rely on a
> > > run-time directory structure that may not exist ( again, war file format
> > > is a deployment format, not a run-time format ).
> 
> > This works _ONLY_ if classloading is performed thru files... what
> > happens if I have a NetworkClassLoader a JDBCClassLoader or a
> > OnTheFlyAssembledClassLoader? no compiler existing today (I just checked
> > ALL the existing java compiler I could find!) would be able to compile
> > against classes loaded thru a classloader.
> 
> One would argue that compilation should happen at deployment, not at
> runtime.

Let's get real... More and more we're using on the fly compilation
because it has been proven to be an effective techniques... Now you're
saying that I can't do it within a servlet? And just because the
compiler works in a different way?

> > everything, ranging from a database to a compiler... so you don't go
> > shopping for tons of jar, but you just drop  the WAR on the webapp
> > folder and you're done. Period.
> >
> > The WAR packages were created for that right? Good, but this idea is
> > broken by the fact that all existing java compilers assume that classes
> > are loaded from disk.
> 
> You may have additional problems - even if you include a javac compiler in
> your WAR, the sandbox will not allow you to use it too much. Some servers
> may care about security ( well, I'm dreaming, nobody cares about that
> today :-).

What Cocoon does today is to write the JAVA files in the servlet work
directory (and I believe that's given in the specs), and perform a bunch
of security-legal operations (like constructing a ClassLoader with the
WEB-APP classloader as a parent... blablabla)...
The only "security constraint" such a WEB-APP would require is to be
able to construct  a classloader, and work in the work directory (even
if this last one can be avoided, doing basically everything in
memory)...

> > Damn, now I need to write a java compiler :(
> 
> Maybe an OS too - all those existing OSes that are using this bad
> filesystem concept ... :-)

We can hack them, though... If we have the sources.... :) :) :)

	Pier

Re: WEB-INF classloading and on the fly compilation

Posted by "Pier P. Fumagalli" <pi...@eng.sun.com>.

cmanolache@yahoo.com wrote:
> 
> On Thu, 14 Sep 2000, Stefano Mazzocchi wrote:
> 
> > cmanolache@yahoo.com wrote:
> > >
> > > :-)
> > >
> > > You could also find the webapp root ( getRealPath("/") ) and compute the
> > > classpath yourself - but I would say this is worse, since you'll rely on a
> > > run-time directory structure that may not exist ( again, war file format
> > > is a deployment format, not a run-time format ).
> 
> > This works _ONLY_ if classloading is performed thru files... what
> > happens if I have a NetworkClassLoader a JDBCClassLoader or a
> > OnTheFlyAssembledClassLoader? no compiler existing today (I just checked
> > ALL the existing java compiler I could find!) would be able to compile
> > against classes loaded thru a classloader.
> 
> One would argue that compilation should happen at deployment, not at
> runtime.

Let's get real... More and more we're using on the fly compilation
because it has been proven to be an effective techniques... Now you're
saying that I can't do it within a servlet? And just because the
compiler works in a different way?

> > everything, ranging from a database to a compiler... so you don't go
> > shopping for tons of jar, but you just drop  the WAR on the webapp
> > folder and you're done. Period.
> >
> > The WAR packages were created for that right? Good, but this idea is
> > broken by the fact that all existing java compilers assume that classes
> > are loaded from disk.
> 
> You may have additional problems - even if you include a javac compiler in
> your WAR, the sandbox will not allow you to use it too much. Some servers
> may care about security ( well, I'm dreaming, nobody cares about that
> today :-).

What Cocoon does today is to write the JAVA files in the servlet work
directory (and I believe that's given in the specs), and perform a bunch
of security-legal operations (like constructing a ClassLoader with the
WEB-APP classloader as a parent... blablabla)...
The only "security constraint" such a WEB-APP would require is to be
able to construct  a classloader, and work in the work directory (even
if this last one can be avoided, doing basically everything in
memory)...

> > Damn, now I need to write a java compiler :(
> 
> Maybe an OS too - all those existing OSes that are using this bad
> filesystem concept ... :-)

We can hack them, though... If we have the sources.... :) :) :)

	Pier

Re: WEB-INF classloading and on the fly compilation

Posted by cm...@yahoo.com.

On Thu, 14 Sep 2000, Stefano Mazzocchi wrote:

> cmanolache@yahoo.com wrote:
> > 
> > :-)
> > 
> > You could also find the webapp root ( getRealPath("/") ) and compute the
> > classpath yourself - but I would say this is worse, since you'll rely on a
> > run-time directory structure that may not exist ( again, war file format
> > is a deployment format, not a run-time format ).


> This works _ONLY_ if classloading is performed thru files... what
> happens if I have a NetworkClassLoader a JDBCClassLoader or a
> OnTheFlyAssembledClassLoader? no compiler existing today (I just checked
> ALL the existing java compiler I could find!) would be able to compile
> against classes loaded thru a classloader.

One would argue that compilation should happen at deployment, not at
runtime. 


> everything, ranging from a database to a compiler... so you don't go
> shopping for tons of jar, but you just drop  the WAR on the webapp
> folder and you're done. Period.
> 
> The WAR packages were created for that right? Good, but this idea is
> broken by the fact that all existing java compilers assume that classes
> are loaded from disk.

You may have additional problems - even if you include a javac compiler in
your WAR, the sandbox will not allow you to use it too much. Some servers
may care about security ( well, I'm dreaming, nobody cares about that
today :-). 


> Damn, now I need to write a java compiler :(

Maybe an OS too - all those existing OSes that are using this bad
filesystem concept ... :-)

Costin

Re: WEB-INF classloading and on the fly compilation

Posted by cm...@yahoo.com.

On Thu, 14 Sep 2000, Stefano Mazzocchi wrote:

> cmanolache@yahoo.com wrote:
> > 
> > :-)
> > 
> > You could also find the webapp root ( getRealPath("/") ) and compute the
> > classpath yourself - but I would say this is worse, since you'll rely on a
> > run-time directory structure that may not exist ( again, war file format
> > is a deployment format, not a run-time format ).


> This works _ONLY_ if classloading is performed thru files... what
> happens if I have a NetworkClassLoader a JDBCClassLoader or a
> OnTheFlyAssembledClassLoader? no compiler existing today (I just checked
> ALL the existing java compiler I could find!) would be able to compile
> against classes loaded thru a classloader.

One would argue that compilation should happen at deployment, not at
runtime. 


> everything, ranging from a database to a compiler... so you don't go
> shopping for tons of jar, but you just drop  the WAR on the webapp
> folder and you're done. Period.
> 
> The WAR packages were created for that right? Good, but this idea is
> broken by the fact that all existing java compilers assume that classes
> are loaded from disk.

You may have additional problems - even if you include a javac compiler in
your WAR, the sandbox will not allow you to use it too much. Some servers
may care about security ( well, I'm dreaming, nobody cares about that
today :-). 


> Damn, now I need to write a java compiler :(

Maybe an OS too - all those existing OSes that are using this bad
filesystem concept ... :-)

Costin

Re: WEB-INF classloading and on the fly compilation

Posted by Stefano Mazzocchi <st...@apache.org>.

cmanolache@yahoo.com wrote:
> 
> :-)
> 
> > Just imagine that this thing we have to compile DEPENDS on another JAR
> > included in our little web-app (maybe a set of utility classes, maybe
> > whatever, I have the classes somewhere in the WEB-APP).
> 
> AFAIK jasper is passing CLASSPATH explicitely ( using -classpath
> parameter). I don't know what's wrong with javac ( or if something is
> wrong at all - after all we are doing a hack to load javac, I don't think
> there is any documentation saying that calling sun.tools.javac.Main is
> supposed to work ). Same for Ant - it creates a -classpath argument.
> 
> Since you may want to support jikes and out-of-process compilation (with
> exec ) I think you'll have to pass the class path explicitely.
> 
> Of course, it is not possible from a "standard" servlet to do that - since
> the classpath is not available in any standard way. Tomcat provides an
> attribute for that in order to hack jasper in.
> 
> You could also find the webapp root ( getRealPath("/") ) and compute the
> classpath yourself - but I would say this is worse, since you'll rely on a
> run-time directory structure that may not exist ( again, war file format
> is a deployment format, not a run-time format ).

This works _ONLY_ if classloading is performed thru files... what
happens if I have a NetworkClassLoader a JDBCClassLoader or a
OnTheFlyAssembledClassLoader? no compiler existing today (I just checked
ALL the existing java compiler I could find!) would be able to compile
against classes loaded thru a classloader.

And I think that considering classloading equivalent to classpath is THE
major usability problem that Java faces today.

Pier and I are working on providing a big Cocoon.war file that includes
everything, ranging from a database to a compiler... so you don't go
shopping for tons of jar, but you just drop  the WAR on the webapp
folder and you're done. Period.

The WAR packages were created for that right? Good, but this idea is
broken by the fact that all existing java compilers assume that classes
are loaded from disk.

Damn, now I need to write a java compiler :(

-- 
Stefano Mazzocchi      One must still have chaos in oneself to be
                          able to give birth to a dancing star.
<st...@apache.org>                             Friedrich Nietzsche
--------------------------------------------------------------------
 Missed us in Orlando? Make it up with ApacheCON Europe in London!
------------------------- http://ApacheCon.Com ---------------------

Re: WEB-INF classloading and on the fly compilation

Posted by Stefano Mazzocchi <st...@apache.org>.

cmanolache@yahoo.com wrote:
> 
> :-)
> 
> > Just imagine that this thing we have to compile DEPENDS on another JAR
> > included in our little web-app (maybe a set of utility classes, maybe
> > whatever, I have the classes somewhere in the WEB-APP).
> 
> AFAIK jasper is passing CLASSPATH explicitely ( using -classpath
> parameter). I don't know what's wrong with javac ( or if something is
> wrong at all - after all we are doing a hack to load javac, I don't think
> there is any documentation saying that calling sun.tools.javac.Main is
> supposed to work ). Same for Ant - it creates a -classpath argument.
> 
> Since you may want to support jikes and out-of-process compilation (with
> exec ) I think you'll have to pass the class path explicitely.
> 
> Of course, it is not possible from a "standard" servlet to do that - since
> the classpath is not available in any standard way. Tomcat provides an
> attribute for that in order to hack jasper in.
> 
> You could also find the webapp root ( getRealPath("/") ) and compute the
> classpath yourself - but I would say this is worse, since you'll rely on a
> run-time directory structure that may not exist ( again, war file format
> is a deployment format, not a run-time format ).

This works _ONLY_ if classloading is performed thru files... what
happens if I have a NetworkClassLoader a JDBCClassLoader or a
OnTheFlyAssembledClassLoader? no compiler existing today (I just checked
ALL the existing java compiler I could find!) would be able to compile
against classes loaded thru a classloader.

And I think that considering classloading equivalent to classpath is THE
major usability problem that Java faces today.

Pier and I are working on providing a big Cocoon.war file that includes
everything, ranging from a database to a compiler... so you don't go
shopping for tons of jar, but you just drop  the WAR on the webapp
folder and you're done. Period.

The WAR packages were created for that right? Good, but this idea is
broken by the fact that all existing java compilers assume that classes
are loaded from disk.

Damn, now I need to write a java compiler :(

-- 
Stefano Mazzocchi      One must still have chaos in oneself to be
                          able to give birth to a dancing star.
<st...@apache.org>                             Friedrich Nietzsche
--------------------------------------------------------------------
 Missed us in Orlando? Make it up with ApacheCON Europe in London!
------------------------- http://ApacheCon.Com ---------------------

Re: WEB-INF classloading and on the fly compilation

Posted by Nick Bauman <ni...@cortexity.com>.

On Thu, 14 Sep 2000 cmanolache@yahoo.com wrote:

> > 
> > > parameter). I don't know what's wrong with javac ( or if something is
> > > wrong at all - after all we are doing a hack to load javac, I don't think
> > 
> > FYI My javac says:
> > 
> > [nick@www nick]$ javac -encoding
> > javac: -encoding requires argument
> > use: javac [-g][-O][-debug][-depend][-nowarn][-verbose][-classpath
> > path][-nowrite][-deprecation][-d dir][-J<runtime flag>] file.java...
> > 
> > Seem like that javac is broken, no?
> 
> ???
> 
> Costin
> 

In the original post, the javac the user was using from within
Jasper complained that the encoding flag was meaningless, and AFAIK,
calling the main() from within Java should behave the same, no?

Re: WEB-INF classloading and on the fly compilation

Posted by cm...@yahoo.com.

> 
> > parameter). I don't know what's wrong with javac ( or if something is
> > wrong at all - after all we are doing a hack to load javac, I don't think
> 
> FYI My javac says:
> 
> [nick@www nick]$ javac -encoding
> javac: -encoding requires argument
> use: javac [-g][-O][-debug][-depend][-nowarn][-verbose][-classpath
> path][-nowrite][-deprecation][-d dir][-J<runtime flag>] file.java...
> 
> Seem like that javac is broken, no?

???

Costin

Re: WEB-INF classloading and on the fly compilation

Posted by Nick Bauman <ni...@cortexity.com>.

On Thu, 14 Sep 2000 cmanolache@yahoo.com wrote:

> parameter). I don't know what's wrong with javac ( or if something is
> wrong at all - after all we are doing a hack to load javac, I don't think

FYI My javac says:

[nick@www nick]$ javac -encoding
javac: -encoding requires argument
use: javac [-g][-O][-debug][-depend][-nowarn][-verbose][-classpath
path][-nowrite][-deprecation][-d dir][-J<runtime flag>] file.java...

Seem like that javac is broken, no?

Re: WEB-INF classloading and on the fly compilation

Posted by cm...@yahoo.com.

:-)

> Just imagine that this thing we have to compile DEPENDS on another JAR
> included in our little web-app (maybe a set of utility classes, maybe
> whatever, I have the classes somewhere in the WEB-APP).

AFAIK jasper is passing CLASSPATH explicitely ( using -classpath
parameter). I don't know what's wrong with javac ( or if something is
wrong at all - after all we are doing a hack to load javac, I don't think
there is any documentation saying that calling sun.tools.javac.Main is
supposed to work ). Same for Ant - it creates a -classpath argument.

Since you may want to support jikes and out-of-process compilation (with
exec ) I think you'll have to pass the class path explicitely.

Of course, it is not possible from a "standard" servlet to do that - since
the classpath is not available in any standard way. Tomcat provides an
attribute for that in order to hack jasper in.

You could also find the webapp root ( getRealPath("/") ) and compute the
classpath yourself - but I would say this is worse, since you'll rely on a
run-time directory structure that may not exist ( again, war file format
is a deployment format, not a run-time format ).


Costin

Re: WEB-INF classloading and on the fly compilation

Posted by cm...@yahoo.com.

:-)

> Just imagine that this thing we have to compile DEPENDS on another JAR
> included in our little web-app (maybe a set of utility classes, maybe
> whatever, I have the classes somewhere in the WEB-APP).

AFAIK jasper is passing CLASSPATH explicitely ( using -classpath
parameter). I don't know what's wrong with javac ( or if something is
wrong at all - after all we are doing a hack to load javac, I don't think
there is any documentation saying that calling sun.tools.javac.Main is
supposed to work ). Same for Ant - it creates a -classpath argument.

Since you may want to support jikes and out-of-process compilation (with
exec ) I think you'll have to pass the class path explicitely.

Of course, it is not possible from a "standard" servlet to do that - since
the classpath is not available in any standard way. Tomcat provides an
attribute for that in order to hack jasper in.

You could also find the webapp root ( getRealPath("/") ) and compute the
classpath yourself - but I would say this is worse, since you'll rely on a
run-time directory structure that may not exist ( again, war file format
is a deployment format, not a run-time format ).


Costin

WEB-INF classloading and on the fly compilation

Posted by "Pier P. Fumagalli" <pi...@apache.org>.

<?xml version="1.0"?>

<rant spam="everyone">

<target audience="cocoon-people">
  It seems that all our classloading related problems are now fixed...
  BUT (yes, there's always a creepy BUT)...
</target>

This is a wider problem, that's why I'm spamming basically all mailing
list on the subject...

We have to compile ON-THE-FLY (at runtime) a file generated and/or
included in the WEB-APP, stupid, indeed. If we see it from Cocoon, we
can see that the Sitemap needs to be compiled before execution, if we
see it from the JSP side of things, we want to compile a JSP into a
.class

Just imagine that this thing we have to compile DEPENDS on another JAR
included in our little web-app (maybe a set of utility classes, maybe
whatever, I have the classes somewhere in the WEB-APP).

I am _SURE_ that the JAVAC classes and the WEB-APP are loaded by the
same classloader (I moved tools.jar from my jkd/lib to
WEB-APP/WEB-INF/lib and checked with System.identityHashCode()), just in
case, but I also ran the same test having the tools.jar loaded by the
system classloader and the WEB-APP loaded by it's own classloader...

What happens when I compile? All the references I put to all the objects
in my WEB-APP bundled jar (those utility classes I want to distribute
with my WEB-APP) are not resolved. JavaC doesn't find a pre-compiled
class existent in my WEB-APP library...

Now, either am I a complete MORON or JavaC resolves everything using the
System classloader, completely ignoring whatever classloader loaded
itself? NOTICE: If this is true, we have a bunch of problems basically
everywhere we want to do some on-the-fly compilation (tomcat people
read: JSP!).

Well, now I'm tired of bashing my head on this (4:30 am and still
coding) so, I'd better get off to bed...

<signed author="Pier" comment="I'm tired"/>

</rant>

WEB-INF classloading and on the fly compilation

Posted by "Pier P. Fumagalli" <pi...@apache.org>.

<?xml version="1.0"?>

<rant spam="everyone">

<target audience="cocoon-people">
  It seems that all our classloading related problems are now fixed...
  BUT (yes, there's always a creepy BUT)...
</target>

This is a wider problem, that's why I'm spamming basically all mailing
list on the subject...

We have to compile ON-THE-FLY (at runtime) a file generated and/or
included in the WEB-APP, stupid, indeed. If we see it from Cocoon, we
can see that the Sitemap needs to be compiled before execution, if we
see it from the JSP side of things, we want to compile a JSP into a
.class

Just imagine that this thing we have to compile DEPENDS on another JAR
included in our little web-app (maybe a set of utility classes, maybe
whatever, I have the classes somewhere in the WEB-APP).

I am _SURE_ that the JAVAC classes and the WEB-APP are loaded by the
same classloader (I moved tools.jar from my jkd/lib to
WEB-APP/WEB-INF/lib and checked with System.identityHashCode()), just in
case, but I also ran the same test having the tools.jar loaded by the
system classloader and the WEB-APP loaded by it's own classloader...

What happens when I compile? All the references I put to all the objects
in my WEB-APP bundled jar (those utility classes I want to distribute
with my WEB-APP) are not resolved. JavaC doesn't find a pre-compiled
class existent in my WEB-APP library...

Now, either am I a complete MORON or JavaC resolves everything using the
System classloader, completely ignoring whatever classloader loaded
itself? NOTICE: If this is true, we have a bunch of problems basically
everywhere we want to do some on-the-fly compilation (tomcat people
read: JSP!).

Well, now I'm tired of bashing my head on this (4:30 am and still
coding) so, I'd better get off to bed...

<signed author="Pier" comment="I'm tired"/>

</rant>