You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tapestry.apache.org by David White <dw...@onemail.at> on 2005/03/21 10:52:40 UTC
PATCH: fix for asset service (branch-3-0 2005-03-16)
So, here is the diff from branch-3-0 as of 2005-03-16.
The basic idea is to enter the path to PrivateAssets into the engine's
Pool when the PrivateAsset is rendered. Instead of simply using the path
provided by the user, the AssetService looks up the given path in the
Pool to see if it has been "declared" in a page or component
specification. If the path has not been pooled, the request is treated
as a request for a missing resource.
Thinking about this matter carefully, I think I can see some real use
cases for MD5 content validation; consider ExternalAssets. It would be
nice to know that the URL is still pointing to the desired content, and
some mischievious person hasn't copied tubgirl.jpg over the URL's
referent. So a good solution would permit MD5 hashing for some
resources, but not require it for all assets.
Questions? Comments?
Thanks,
David WHITE
--
Index: framework/src/org/apache/tapestry/asset/AssetService.java
===================================================================
RCS
file: /home/cvspublic/jakarta-tapestry/framework/src/org/apache/tapestry/asset/Attic/AssetService.java,v
retrieving revision 1.8
diff -r1.8 AssetService.java
28a29
> import org.apache.tapestry.IEngine;
140,147c141,151
< String resourcePath = (String) parameters[0];
<
< URL resourceURL =
cycle.getEngine().getResourceResolver().getResource(resourcePath);
<
< if (resourceURL == null)
< throw new ApplicationRuntimeException(
< Tapestry.format("missing-resource", resourcePath));
<
---
> IEngine theEngine = cycle.getEngine();
> String resourceHandle = (String) parameters[0];
> String resourcePath =
(String)theEngine.getPool().retrieve(resourceHandle);
> URL resourceURL = resourcePath != null ?
>
theEngine.getResourceResolver().getResource(resourcePath) :
> null;
>
> if (resourcePath == null || resourceURL == null)
> throw new ApplicationRuntimeException(
>
Tapestry.format("missing-resource", resourceHandle));
>
Index: framework/src/org/apache/tapestry/asset/PrivateAsset.java
===================================================================
RCS
file: /home/cvspublic/jakarta-tapestry/framework/src/org/apache/tapestry/asset/Attic/PrivateAsset.java,v
retrieving revision 1.7
diff -r1.7 PrivateAsset.java
20a21
> import org.apache.tapestry.IEngine;
73a75,76
> IEngine engine = cycle.getEngine();
> engine.getPool().store(path,path);
--
David White <dw...@onemail.at>
---------------------------------------------------------------------
To unsubscribe, e-mail: tapestry-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tapestry-dev-help@jakarta.apache.org
Re: PATCH: fix for asset service (branch-3-0 2005-03-16)
Posted by David Taylor <Da...@cox.net>.
Interesting point regarding cached pages. Unfortunately, Howard's
approach also has a similar problem when assets are updated at runtime
changing their hash value. As luck would have it, one of my applications
uses (abuses?) the asset service to serve JPEG images from a database.
Whenever an image is replaced in the database its hash value changes.
This is particularly troublesome since the application makes heavy use
of remote scripting (i.e. AJAX) and is not easily refreshed..
My solution to the problem was to abandon the asset service for that
part of the application. I am currently building a new resource
management service that uses a declarative security mechanism to grant
access to resources. Using this approach I can limit access to a list of
named resources, those in specified "folders" or those matching a
regular expression. The first version is targeted at Tapestry 3.0.2
since that is what I currently have in production. Ideally I would like
to make this more generalized and extensible to support more resource
locations (i.e. file system, classpath, database, dynamically
generated...). Some sort of namespace specifier might also be in order
to keep these logically separate.
David
Mind Bridge wrote:
>Hi,
>
>We have been thinking about that approach earlier, but it has a problem:
>when the application is restarted, the pool will no longer contain the
>asset, but its URL will be out there, on cached/bookmarked pages or the
>like. If the asset is not tied to a page, which is possible as well as it
>can be declared in code for example, it gets interesting too.
>
>In short, it is possible to receive "access denied" errors after a server
>restart even though the asset is completely valid. Howard's patch is a
>solution that covers the general case and is always valid. We just need to
>make sure that the absolute path of the asset (case insensitive in some
>cases) is used as a key for the checksum to ensure that there is no DoS
>possibility.
>
>
>----- Original Message -----
>From: "David White" <dw...@onemail.at>
>To: <ta...@jakarta.apache.org>
>Sent: Monday, March 21, 2005 11:52 AM
>Subject: PATCH: fix for asset service (branch-3-0 2005-03-16)
>
>
>
>
>>So, here is the diff from branch-3-0 as of 2005-03-16.
>>
>>The basic idea is to enter the path to PrivateAssets into the engine's
>>Pool when the PrivateAsset is rendered. Instead of simply using the path
>>provided by the user, the AssetService looks up the given path in the
>>Pool to see if it has been "declared" in a page or component
>>specification. If the path has not been pooled, the request is treated
>>as a request for a missing resource.
>>
>>Thinking about this matter carefully, I think I can see some real use
>>cases for MD5 content validation; consider ExternalAssets. It would be
>>nice to know that the URL is still pointing to the desired content, and
>>some mischievious person hasn't copied tubgirl.jpg over the URL's
>>referent. So a good solution would permit MD5 hashing for some
>>resources, but not require it for all assets.
>>
>>Questions? Comments?
>>
>>Thanks,
>>
>>David WHITE
>>
>>--
>>Index: framework/src/org/apache/tapestry/asset/AssetService.java
>>===================================================================
>>RCS
>>file:
>>
>>
>/home/cvspublic/jakarta-tapestry/framework/src/org/apache/tapestry/asset/Att
>ic/AssetService.java,v
>
>
>>retrieving revision 1.8
>>diff -r1.8 AssetService.java
>>28a29
>>
>>
>>>import org.apache.tapestry.IEngine;
>>>
>>>
>>140,147c141,151
>>< String resourcePath = (String) parameters[0];
>><
>>< URL resourceURL =
>>cycle.getEngine().getResourceResolver().getResource(resourcePath);
>><
>>< if (resourceURL == null)
>>< throw new ApplicationRuntimeException(
>>< Tapestry.format("missing-resource", resourcePath));
>><
>>---
>>
>>
>>> IEngine theEngine = cycle.getEngine();
>>> String resourceHandle = (String) parameters[0];
>>> String resourcePath =
>>>
>>>
>>(String)theEngine.getPool().retrieve(resourceHandle);
>>
>>
>>> URL resourceURL = resourcePath != null ?
>>>
>>>
>>>
>>theEngine.getResourceResolver().getResource(resourcePath) :
>>
>>
>>> null;
>>>
>>> if (resourcePath == null || resourceURL == null)
>>> throw new ApplicationRuntimeException(
>>>
>>>
>>>
>>Tapestry.format("missing-resource", resourceHandle));
>>
>>
>>Index: framework/src/org/apache/tapestry/asset/PrivateAsset.java
>>===================================================================
>>RCS
>>file:
>>
>>
>/home/cvspublic/jakarta-tapestry/framework/src/org/apache/tapestry/asset/Att
>ic/PrivateAsset.java,v
>
>
>>retrieving revision 1.7
>>diff -r1.7 PrivateAsset.java
>>20a21
>>
>>
>>>import org.apache.tapestry.IEngine;
>>>
>>>
>>73a75,76
>>
>>
>>> IEngine engine = cycle.getEngine();
>>> engine.getPool().store(path,path);
>>>
>>>
>>--
>>David White <dw...@onemail.at>
>>
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: tapestry-dev-unsubscribe@jakarta.apache.org
>>For additional commands, e-mail: tapestry-dev-help@jakarta.apache.org
>>
>>
>>
>>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: tapestry-dev-unsubscribe@jakarta.apache.org
>For additional commands, e-mail: tapestry-dev-help@jakarta.apache.org
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tapestry-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tapestry-dev-help@jakarta.apache.org
Re: PATCH: fix for asset service (branch-3-0 2005-03-16)
Posted by David White <dw...@onemail.at>.
On Mon, 2005-03-21 at 10:00 -0500, Howard Lewis Ship wrote:
> Another solution is to change the asset service to reject all incoming
> requests if externalization of assets is enabled. In this scenario,
> assets are copies out of JAR files onto the file system (mapped to a
> web folder). Currently, the asset service continues to execute. By
> disabling it, you close the gap ... and have an effective whitelist
> because files are extenalized when a URL referencing them is generated
> (as part of a page render).
>
That seems quite reasonable, and would work well. The whitelist would
generate itself, and be persistent across server restarts.
> However, this has some issues in a cluster, unless the file system /
> web folder is shared or shadowed between servers in the cluster.
SAN/NAS's are a cluster's best friend. :)
David WHITE
---------------------------------------------------------------------
To unsubscribe, e-mail: tapestry-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tapestry-dev-help@jakarta.apache.org
Re: PATCH: fix for asset service (branch-3-0 2005-03-16)
Posted by Howard Lewis Ship <hl...@gmail.com>.
Another solution is to change the asset service to reject all incoming
requests if externalization of assets is enabled. In this scenario,
assets are copies out of JAR files onto the file system (mapped to a
web folder). Currently, the asset service continues to execute. By
disabling it, you close the gap ... and have an effective whitelist
because files are extenalized when a URL referencing them is generated
(as part of a page render).
However, this has some issues in a cluster, unless the file system /
web folder is shared or shadowed between servers in the cluster.
On Mon, 21 Mar 2005 15:56:34 +0100, David White <dw...@onemail.at> wrote:
> On Mon, 2005-03-21 at 09:11 -0500, Howard Lewis Ship wrote:
> > > All a CHF can do is confirm that the content that has
> > been hashed is almost certainly identical with another piece of content
> > that produced the same digest.
> >
>
> > Exactly. And that's all that's necessary. That's a near perfect
> > credential because it is non-trivial (i.e., virtually impossible) to
> > generate the digest without the file.
>
> An UNIX box may be considered well and truly compromised if an attacker
> gets access to the hashed passwords in /etc/shadow or /etc/passwd. The
> same applies for the SAM in Windows. Why, if the CHF is truly one-way?
> Because the bad guy can e.g. use a dictionary and end-user stupidity to
> dramatically reduce the search space.
>
> You ignored my argument about using the asset service to scan for
> vulnerable Java classes or other resources. For an attacker, finding out
> that a resource in www.someserver.com/TapestryAppContext is identical to
> a resource that has been acquired by other means is useful information.
> This attack is isomorphic to a dictionary attack on a password hash.
>
> > The digest is long enough that
> > you aren't going to guess it (not in the lifespan of the universe). As
> > far as I understand it, this is exactly the kind of thing CHF are for.
> >
>
> No, a CHF is most commonly used to ensure integrity of a message, thus
> the name "message digest". The use of a CHF in an authentication routine
> is deprecated when better mechanisms (e.g. Kerebos, smart cards,
> challenge-response with asymmetric crypto, et. al.) are available. These
> mechanisms may well use CHFs, but a CHF alone is *not* such a mechanism.
>
> > You can overide the asset service in both 3.0 and 3.1 to fit your
> > needs ... but all our energies are better spent elsewhere.
> >
>
> The patch I sent serves the needs of the project I am working on. I
> would be willing to contribute a further patch to handle clustered
> environments. I also see value in supporting the use of a message digest
> to verify content assets provided by untrusted sources.
>
> Cordially,
>
> David WHITE
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tapestry-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tapestry-dev-help@jakarta.apache.org
>
>
--
Howard M. Lewis Ship
Independent J2EE / Open-Source Java Consultant
Creator, Jakarta Tapestry
Creator, Jakarta HiveMind
Professional Tapestry training, mentoring, support
and project work. http://howardlewisship.com
---------------------------------------------------------------------
To unsubscribe, e-mail: tapestry-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tapestry-dev-help@jakarta.apache.org
Re: PATCH: fix for asset service (branch-3-0 2005-03-16)
Posted by David White <dw...@onemail.at>.
On Mon, 2005-03-21 at 09:11 -0500, Howard Lewis Ship wrote:
> > All a CHF can do is confirm that the content that has
> been hashed is almost certainly identical with another piece of content
> that produced the same digest.
>
> Exactly. And that's all that's necessary. That's a near perfect
> credential because it is non-trivial (i.e., virtually impossible) to
> generate the digest without the file.
An UNIX box may be considered well and truly compromised if an attacker
gets access to the hashed passwords in /etc/shadow or /etc/passwd. The
same applies for the SAM in Windows. Why, if the CHF is truly one-way?
Because the bad guy can e.g. use a dictionary and end-user stupidity to
dramatically reduce the search space.
You ignored my argument about using the asset service to scan for
vulnerable Java classes or other resources. For an attacker, finding out
that a resource in www.someserver.com/TapestryAppContext is identical to
a resource that has been acquired by other means is useful information.
This attack is isomorphic to a dictionary attack on a password hash.
> The digest is long enough that
> you aren't going to guess it (not in the lifespan of the universe). As
> far as I understand it, this is exactly the kind of thing CHF are for.
>
No, a CHF is most commonly used to ensure integrity of a message, thus
the name "message digest". The use of a CHF in an authentication routine
is deprecated when better mechanisms (e.g. Kerebos, smart cards,
challenge-response with asymmetric crypto, et. al.) are available. These
mechanisms may well use CHFs, but a CHF alone is *not* such a mechanism.
> You can overide the asset service in both 3.0 and 3.1 to fit your
> needs ... but all our energies are better spent elsewhere.
>
The patch I sent serves the needs of the project I am working on. I
would be willing to contribute a further patch to handle clustered
environments. I also see value in supporting the use of a message digest
to verify content assets provided by untrusted sources.
Cordially,
David WHITE
---------------------------------------------------------------------
To unsubscribe, e-mail: tapestry-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tapestry-dev-help@jakarta.apache.org
Re: PATCH: fix for asset service (branch-3-0 2005-03-16)
Posted by Howard Lewis Ship <hl...@gmail.com>.
> All a CHF can do is confirm that the content that has
been hashed is almost certainly identical with another piece of content
that produced the same digest.
Exactly. And that's all that's necessary. That's a near perfect
credential because it is non-trivial (i.e., virtually impossible) to
generate the digest without the file. The digest is long enough that
you aren't going to guess it (not in the lifespan of the universe). As
far as I understand it, this is exactly the kind of thing CHF are for.
You can overide the asset service in both 3.0 and 3.1 to fit your
needs ... but all our energies are better spent elsewhere.
On Mon, 21 Mar 2005 14:42:53 +0100, David White <dw...@onemail.at> wrote:
> On Mon, 2005-03-21 at 07:58 -0500, Howard Lewis Ship wrote:
> > In a cluster, there's no guarantee that every request will be
> > processed by the same server. Especially for the asset service, which
> > is not URL encoded (no session id is passed in the URL, though it may
> > come up in a cookie).
> >
> > Therefore server A may render the page and read the spec, and server B
> > may receive the asset request and fail because it's not in your
> > registry of valid assets.
> >
> > The correct approach is the one I developed for 3.1 and Paul back
> > ported to 3.0.3. It's basic security ... you are allowed access if
> > you can prove that you should have access, by having a credential only
> > the server can provide. The MD5 digest of the file can only be
> > generated by the server (which has the file), so that's a good
> > credential.
>
> The asset service has access to every single resource on the classpath
> as it stands.
>
> Say there were a security problem in a particular revision of a
> publically released Java class. An attacker can now use Tapestry's asset
> service to create an effective vulnerablity scanner that can provide a
> high degree of confidence about the presence of a particular version of
> a vulnerable class.
>
> My problem with this solution is that it is a misuse of cryptographic
> hash functions. All a CHF can do is confirm that the content that has
> been hashed is almost certainly identical with another piece of content
> that produced the same digest. That's it, it serves NO other purpose.
> Your are ignoring the possibility that the attacker may already know the
> content by some other mechanism.
>
> This problem won't go away until the asset service loses the ability to
> serve any resource reachable by the ClassLoader. An even more basic
> security principle is to deny everything, and permit only well-known,
> well-defined behaviors.
>
> Your objection is valid for a clustered environment. This implies that
> the asset service must be stateful if it is to be secure. A patch will
> be forthcoming.
>
> Thanks again,
>
> David WHITE
>
>
--
Howard M. Lewis Ship
Independent J2EE / Open-Source Java Consultant
Creator, Jakarta Tapestry
Creator, Jakarta HiveMind
Professional Tapestry training, mentoring, support
and project work. http://howardlewisship.com
---------------------------------------------------------------------
To unsubscribe, e-mail: tapestry-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tapestry-dev-help@jakarta.apache.org
Re: PATCH: fix for asset service (branch-3-0 2005-03-16)
Posted by Howard Lewis Ship <hl...@gmail.com>.
In a cluster, there's no guarantee that every request will be
processed by the same server. Especially for the asset service, which
is not URL encoded (no session id is passed in the URL, though it may
come up in a cookie).
Therefore server A may render the page and read the spec, and server B
may receive the asset request and fail because it's not in your
registry of valid assets.
The correct approach is the one I developed for 3.1 and Paul back
ported to 3.0.3. It's basic security ... you are allowed access if
you can prove that you should have access, by having a credential only
the server can provide. The MD5 digest of the file can only be
generated by the server (which has the file), so that's a good
credential.
On Mon, 21 Mar 2005 11:22:35 +0100, David White <dw...@onemail.at> wrote:
> On Mon, 2005-03-21 at 12:12 +0200, Mind Bridge wrote:
> > Hi,
> >
> > We have been thinking about that approach earlier, but it has a problem:
> > when the application is restarted, the pool will no longer contain the
> > asset, but its URL will be out there, on cached/bookmarked pages or the
> > like. If the asset is not tied to a page, which is possible as well as it
> > can be declared in code for example, it gets interesting too.
> >
> > In short, it is possible to receive "access denied" errors after a server
> > restart even though the asset is completely valid.
>
> No, it isn't. If the server is restarted, the specification will have to
> be reloaded and rendered.
>
> David WHITE
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tapestry-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tapestry-dev-help@jakarta.apache.org
>
>
--
Howard M. Lewis Ship
Independent J2EE / Open-Source Java Consultant
Creator, Jakarta Tapestry
Creator, Jakarta HiveMind
Professional Tapestry training, mentoring, support
and project work. http://howardlewisship.com
---------------------------------------------------------------------
To unsubscribe, e-mail: tapestry-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tapestry-dev-help@jakarta.apache.org
Re: PATCH: fix for asset service (branch-3-0 2005-03-16)
Posted by David White <dw...@onemail.at>.
On Mon, 2005-03-21 at 12:12 +0200, Mind Bridge wrote:
> Hi,
>
> We have been thinking about that approach earlier, but it has a problem:
> when the application is restarted, the pool will no longer contain the
> asset, but its URL will be out there, on cached/bookmarked pages or the
> like. If the asset is not tied to a page, which is possible as well as it
> can be declared in code for example, it gets interesting too.
>
> In short, it is possible to receive "access denied" errors after a server
> restart even though the asset is completely valid.
No, it isn't. If the server is restarted, the specification will have to
be reloaded and rendered.
David WHITE
---------------------------------------------------------------------
To unsubscribe, e-mail: tapestry-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tapestry-dev-help@jakarta.apache.org
Re: PATCH: fix for asset service (branch-3-0 2005-03-16)
Posted by Mind Bridge <mi...@yahoo.com>.
Hi,
We have been thinking about that approach earlier, but it has a problem:
when the application is restarted, the pool will no longer contain the
asset, but its URL will be out there, on cached/bookmarked pages or the
like. If the asset is not tied to a page, which is possible as well as it
can be declared in code for example, it gets interesting too.
In short, it is possible to receive "access denied" errors after a server
restart even though the asset is completely valid. Howard's patch is a
solution that covers the general case and is always valid. We just need to
make sure that the absolute path of the asset (case insensitive in some
cases) is used as a key for the checksum to ensure that there is no DoS
possibility.
----- Original Message -----
From: "David White" <dw...@onemail.at>
To: <ta...@jakarta.apache.org>
Sent: Monday, March 21, 2005 11:52 AM
Subject: PATCH: fix for asset service (branch-3-0 2005-03-16)
> So, here is the diff from branch-3-0 as of 2005-03-16.
>
> The basic idea is to enter the path to PrivateAssets into the engine's
> Pool when the PrivateAsset is rendered. Instead of simply using the path
> provided by the user, the AssetService looks up the given path in the
> Pool to see if it has been "declared" in a page or component
> specification. If the path has not been pooled, the request is treated
> as a request for a missing resource.
>
> Thinking about this matter carefully, I think I can see some real use
> cases for MD5 content validation; consider ExternalAssets. It would be
> nice to know that the URL is still pointing to the desired content, and
> some mischievious person hasn't copied tubgirl.jpg over the URL's
> referent. So a good solution would permit MD5 hashing for some
> resources, but not require it for all assets.
>
> Questions? Comments?
>
> Thanks,
>
> David WHITE
>
> --
> Index: framework/src/org/apache/tapestry/asset/AssetService.java
> ===================================================================
> RCS
> file:
/home/cvspublic/jakarta-tapestry/framework/src/org/apache/tapestry/asset/Att
ic/AssetService.java,v
> retrieving revision 1.8
> diff -r1.8 AssetService.java
> 28a29
> > import org.apache.tapestry.IEngine;
> 140,147c141,151
> < String resourcePath = (String) parameters[0];
> <
> < URL resourceURL =
> cycle.getEngine().getResourceResolver().getResource(resourcePath);
> <
> < if (resourceURL == null)
> < throw new ApplicationRuntimeException(
> < Tapestry.format("missing-resource", resourcePath));
> <
> ---
> > IEngine theEngine = cycle.getEngine();
> > String resourceHandle = (String) parameters[0];
> > String resourcePath =
> (String)theEngine.getPool().retrieve(resourceHandle);
> > URL resourceURL = resourcePath != null ?
> >
> theEngine.getResourceResolver().getResource(resourcePath) :
> > null;
> >
> > if (resourcePath == null || resourceURL == null)
> > throw new ApplicationRuntimeException(
> >
> Tapestry.format("missing-resource", resourceHandle));
> >
> Index: framework/src/org/apache/tapestry/asset/PrivateAsset.java
> ===================================================================
> RCS
> file:
/home/cvspublic/jakarta-tapestry/framework/src/org/apache/tapestry/asset/Att
ic/PrivateAsset.java,v
> retrieving revision 1.7
> diff -r1.7 PrivateAsset.java
> 20a21
> > import org.apache.tapestry.IEngine;
> 73a75,76
> > IEngine engine = cycle.getEngine();
> > engine.getPool().store(path,path);
>
> --
> David White <dw...@onemail.at>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tapestry-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tapestry-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tapestry-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tapestry-dev-help@jakarta.apache.org