Problem with o.a.c.webresources and packed WARs

[Mark Thomas <ma...@apache.org>]

I started to look at Bug 52489 [1] which is requesting code signing
support for WARs. I decided to work with an unpacked version of the
examples application. During testing, I noticed that WEB-INF/classes and
WEB-INF/lib were always being unpacked to the work directory.

As well as looking to be unnecessary, always unpacking WEB-INF/classes
and loading the classes from the work directory makes it possible to
bypass any signature checks that are added to the WAR. Therefore, I
removed the code that extracted WEB-INF/classes and added a TODO to do
the same for WEB-INF/lib. [2]

I wanted to be sure that there wasn't a good reason for extracting
WEB-INF/classes so I did some more testing. It was at this point I
discovered a much bigger issue. When packed, URLs to the JARs inside the
WAR look like this:
jar:file:/D:/repos/asf-public/tomcat/trunk/output/build/webapps/examples.war!/WEB-INF/lib/standard.jar

If you obtain a JarFile from this URL, it is for examples.war rather
than standard.jar. In short, jar URLs are not designed / intended /
implemented to support access of JARs inside WARs (which are just JARs
with a different name). This creates a number of issues internally
within Tomcat as URLs are often used to refer to resources.

I started to work-around this by implementing a third version of
o.a.t.util.scan.Jar - FileUrlNestedJar. This overcame the first few
obstacles but I then quickly reached the point where it was going to be
necessary to modify o.a.jasper.compiler.JarResource. It would be
necessary to remove all the current methods on that interface and
replace them with something that worked with these 'nested' JARs -
probably something that provided access to the input stream.

Before starting on what looked to be a fair amount of work, I wanted to
think about the problem more widely. At the back of my mind was the fact
that I removed the Tomcat specific jndi url scheme that was previously
used for accessing resources.

The Javadoc for ServletContext#getResource() [3] is clear. The container
must provide a URL for every resource and that URL must be usable. There
is an immediate issue with resources located inside the
META-INF/resources directory of a JAR which is itself located inside a
WAR when unpacking WARs is disabled. There is no support from the JRE
for accessing those resources with a URL.

I see three possible solutions to this:
a) Unpack JAR resources to the work dir and return a URL to the unpacked
location
b) Restore something like the jndi url scheme (renamed to tomcat to
reduce the likelihood of clashes)
c) Create a war url scheme only for accessing resources inside WARs

a) Strikes me as a bit of a hack and one that requires a writeable work
directory which might not always be available.

b) Would address a concern Konstantin raised with the new webresources
that once a URL had been obtained it bypassed the cache. On the down
side it is likely to be more work than c) to implement.

c) Would fix the issue at hand.


In trying to decide between b) and c) I came to the conclusion that it
would be better to keep the war:// url scheme and generic tomcat:// url
schemes separated even if we went for option b). Therefore, I intend to
go ahead with implementing c) and keep an eye on the users and dev list
to see if a case for implementing b) emerges.

Mark


[1] https://issues.apache.org/bugzilla/show_bug.cgi?id=52489
[2] http://svn.apache.org/r1522704
[3]
http://docs.oracle.com/javaee/7/api/javax/servlet/ServletContext.html#getResource(java.lang.String)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org

Re: Problem with o.a.c.webresources and packed WARs

[Mark Thomas <ma...@apache.org>]

On 13/09/2013 17:23, Christopher Schultz wrote:
> Mark,
> 
> On 9/13/13 9:16 AM, Mark Thomas wrote:
>> The Javadoc for ServletContext#getResource() [3] is clear. The
>> container must provide a URL for every resource and that URL must
>> be usable.
> 
> Usable by whom? (Sounds like a dumb question, but please see
> below)

My starting position is any other code running in the same JVM.

>> b) Restore something like the jndi url scheme (renamed to tomcat
>> to reduce the likelihood of clashes)
> 
> You could use any kind of URL scheme, right? There's no reason that
> the scheme needs to be conflated with JNDI -- unless there is
> something that makes the combination of the two advantageous.

Correct. I'd also most certainly use tomcat://... if I went down this
road.

Mark


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org

Re: Problem with o.a.c.webresources and packed WARs

[Christopher Schultz <ch...@christopherschultz.net>]

Mark,

On 9/13/13 9:16 AM, Mark Thomas wrote:
> The Javadoc for ServletContext#getResource() [3] is clear. The container
> must provide a URL for every resource and that URL must be usable.

Usable by whom? (Sounds like a dumb question, but please see below)

> There
> is an immediate issue with resources located inside the
> META-INF/resources directory of a JAR which is itself located inside a
> WAR when unpacking WARs is disabled. There is no support from the JRE
> for accessing those resources with a URL.

If the JRE does not support this, then I don't believe it's possible to
return a URL to the caller that is "usable" unless the URL is
accompanied by a URL handler that Tomcat also provides. If the URL must
be universally applicable (e.g. works outside of Tomcat, works with
custom URL handlers that subvert Tomcat's hooks, etc.), then I believe
this is an impossible situation.

> I see three possible solutions to this:
> a) Unpack JAR resources to the work dir and return a URL to the unpacked
> location

-1 due to security considerations plus it really should be possible to
run without expanding WAR files.

> b) Restore something like the jndi url scheme (renamed to tomcat to
> reduce the likelihood of clashes)

You could use any kind of URL scheme, right? There's no reason that the
scheme needs to be conflated with JNDI -- unless there is something that
makes the combination of the two advantageous.

> c) Create a war url scheme only for accessing resources inside WARs

I kind of like this: it just seems like architecturally the right
decision (the problem is the URL, so fix the URL/handler instead of
writing a whole new subsystem to get around it). Plus, it's going to be
much less work and will probably handle all the currently-desired cases.
If it doesn't stand the test of time, you can always invest the time in
re-writing a whole bunch of stuff.

> In trying to decide between b) and c) I came to the conclusion that it
> would be better to keep the war:// url scheme and generic tomcat:// url
> schemes separated even if we went for option b). Therefore, I intend to
> go ahead with implementing c) and keep an eye on the users and dev list
> to see if a case for implementing b) emerges.

+1

-chris

Re: Problem with o.a.c.webresources and packed WARs

[Mark Thomas <ma...@apache.org>]

Konstantin Kolinko <kn...@gmail.com> wrote:
>2013/9/13 Mark Thomas <ma...@apache.org>:
>> I started to look at Bug 52489 [1] which is requesting code signing
>> support for WARs. I decided to work with an unpacked version of the
>> examples application. During testing, I noticed that WEB-INF/classes
>and
>> WEB-INF/lib were always being unpacked to the work directory.
>>
>
>I have not verified it, but my guess was that that trick was needed to
>provide classpath to java compiler in Jasper,  be it Eclipse compiler
>or Sun's javac.
>
>Have you tested that those still work?

It was testing this that highlighted the issues with WebResources and packed WARs. The issue was that TLDs weren't found in a packed WAR (irrespective of this change).

The Eclipse compiler seems fine with this change. I haven't checked javac yet. 

Mark



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org

Re: Problem with o.a.c.webresources and packed WARs

[Konstantin Kolinko <kn...@gmail.com>]

2013/9/13 Mark Thomas <ma...@apache.org>:
> I started to look at Bug 52489 [1] which is requesting code signing
> support for WARs. I decided to work with an unpacked version of the
> examples application. During testing, I noticed that WEB-INF/classes and
> WEB-INF/lib were always being unpacked to the work directory.
>

I have not verified it, but my guess was that that trick was needed to
provide classpath to java compiler in Jasper,  be it Eclipse compiler
or Sun's javac.

Have you tested that those still work?

Best regards,
Konstantin Kolinko

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org

Re: Problem with o.a.c.webresources and packed WARs

[Violeta Georgieva <mi...@gmail.com>]

Hi Mark,

2013/9/13 Mark Thomas <ma...@apache.org>
>
> a) Unpack JAR resources to the work dir and return a URL to the unpacked
> location
-1

>
> In trying to decide between b) and c) I came to the conclusion that it
> would be better to keep the war:// url scheme and generic tomcat:// url
> schemes separated even if we went for option b). Therefore, I intend to
> go ahead with implementing c) and keep an eye on the users and dev list
> to see if a case for implementing b) emerges.
>
I just started to integrate Tomcat 8 Web Resources in Gemini Web (OSGi Web
Container).
There we use this jndi scheme. We have a special URLResource implementation
that returns resource from the bundle archive.
At a first glance I'm for b) but I need some time to become more familiar
with the new Web Resources and whether we can remove the usage of
jndischeme (at that point I do not think we can).
In OSGi the typical use case is to run directly the web application from
the bundle archive and the more rare use case is an expanded directory.


Regards
Violeta