You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by "William A. Rowe, Jr." <wr...@rowe-clan.net> on 2004/02/04 23:54:01 UTC
FileSystem v.s. Other Resources [was configurable Location?]
At 03:39 PM 2/4/2004, gregames@apache.org wrote:
>But then if I play devil's advocate, someone could see the new directive and turn it on when it's not appropriate and cause Bad Things to happen. Mainly I'm looking for comments on whether this should be configurable or not.
Yes, I'm one who will agree with your devil's position :) I know the problem
you are trying to solve, and changing the behavior of <Directory > isn't quite
the solution...
Issue 1:
Walking the filesystem (the stat aspect of Directory and File) for something
that doesn't live there is stupid :) We agree, and this must be fixed in 2.1
Issue 2:
Matching directories when something is outside of the filesystem is simply
wasteful, and here too we agree.
The problem is that you want to add layers of additional directives, which
would change the behavior of <Directory > or <Location >, in ways that would
allow the user to seriously compromise what they had explicitly configured
for security. This is unwise. Let's look at the two issues above;
Effect/Issue 1:
Bypassing the filesystem canonicalization would be very bad on certain
platforms such as windows, depending on case sensitivity, etc. It would
also bypass *user configured* options such as avoiding symlinks.
Effect/Issue 2:
Bypassing the directory block walk would ignore the very <Directory >
sections that the user explicitly put in their config. This too is bad.
There is only one way to avoid these consequences, and still save ourselves
the pain of the filesystem walk and <Directory > block handling...
=== Break out file handling as a separate component ===
I've proposed in the past the simple directive
FilesystemMount /path/to/files
Consider the existing directive, DocumentRoot, as shorthand for;
<Location />
FilesystemMount /old/root/
</Location>
Also consider the Alias /foo /path/to/bar directive as shorthand for;
<Location /foo>
FilesystemMount /path/to/bar
</Location>
But wait, there is more :) We have a default handler. Simply modify it to
be the default_file_handler. Insert it ONLY if the appropriate FilesystemMount
directive exists in the particular location block.
What happens to requests that don't have a FilesystemMount block,
and don't have another handler (e.g. SetHandler info-handler or whatever)?
These requests would error out with a 500 - there is no way for the server
to handle them.
What happens to our old default handler (inserted at all times?) It becomes
a simple error-500 handler if nothing else intercepts the request.
Note that the exact semantics could change (my naming isn't all that great
either) - but the key detail is that mount directives within a given <Location >
are much safer and easier to follow than arbitrary Mount src dst directives
which may be order and/or length dependent, creating confusion for the user.
So in short, veto on the original proposal to give users gun to point at foot.
+1 on the concepts that we quit interrogating the filesystem when the user
wants to serve non-files, and quit walking directory blocks for things that
do not live in directories. Let's do this in 2.1 by splitting out the file system,
and if the filesystem module isn't handling a request, it won't be serving
content but also won't be invoking the directory walk or stat-ing files.
Oh last observation - it should become (in 2.1) nearly impossible for folks
to just bork around with the contents of r->filename and r->finfo, first by
stripping them from the request rec, and second by providing an API to
the filesystem module that lets another module link into another file.
That API would prevent module authors from bypassing the filesystem
module's internal security. If they want different behavior, they can plug
in their own backend handler to deal with the whole request process.
Bill
Re: FileSystem v.s. Other Resources [was configurable Location?]
Posted by Geoffrey Young <ge...@modperlcookbook.org>.
> Let's do this in 2.1 by splitting out the file system,
> and if the filesystem module isn't handling a request, it won't be serving
> content but also won't be invoking the directory walk or stat-ing files.
this all sounds kinda interesting, and similar to the way auth has been set
up in 2.1 - more directives and modules, but more flexibility and power.
>
> Oh last observation - it should become (in 2.1) nearly impossible for folks
> to just bork around with the contents of r->filename and r->finfo, first by
> stripping them from the request rec, and second by providing an API to
> the filesystem module that lets another module link into another file.
> That API would prevent module authors from bypassing the filesystem
> module's internal security.
this has come up before, and I'd love to see an API that prevents accidental
disagreement between r->filename and r->finfo (for one). IIRC, there was
supposed to be some discussion at the hackathon about this, but it sounds
like I didn't miss it :)
--Geoff
Re: FileSystem v.s. Other Resources [was configurable
Location?]
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
At 09:22 AM 2/5/2004, gregames@apache.org wrote:
>>Effect/Issue 1:
>>Bypassing the filesystem canonicalization would be very bad on certain platforms such as windows, depending on case sensitivity, etc. It would
>>also bypass *user configured* options such as avoiding symlinks.
>
>only for <Location > If the admin is telling us the content is outside the filesystem, I don't think filesystem canonicalization or symlinks are relevant.
But the question is how the admin communicates that to us. You are
suggesting we allow <Location virtual /not/a/file> to say it's outside of the
file system. *However* are you preventing a file from being served?
Guess that is the gist of my concern.
>>Effect/Issue 2:
>>Bypassing the directory block walk would ignore the very <Directory >
>>sections that the user explicitly put in their config. This too is bad.
>
>This is only for <Location > . There is no change to the behavior of <Directory >.
>
>>to avoid these consequences, and still save ourselves
>>the pain of the filesystem walk and <Directory > block handling...
>
>I'm only changing <Location >
Wait - so you say. But the effect is to disable <Directory > block handling,
am I confused on this point??? If you ignore valid <Directory > blocks because
somewhere else the admin says "Wait - this location is virtual!" then are you
also preventing those files from being served?
If not this is what I suggest is a violation of the principal of least surprise.
>>=== Break out file handling as a separate component ===
>>I've proposed in the past the simple directive
>>FilesystemMount /path/to/files
>
>At first glance it looks like a key difference is that you are approaching it by saying that it's OK to have <Location > map to a real filesystem and providing config to explicitly specify the mapping. I'm advocating going the other way and allowing <Location > to be treated as if it is really outside of the filesystem.
Here's what I'm suggesting;
1. Everything in Apache is at some location. Location is a predictable factor
because every URI becomes a location. A Location block will always
override all other configurations, location blocks are always applied
as least-to-most significant if they are absolute, and then pattern matching
location blocks are applied after that.
2. Some things in Apache are in the Filesystem, some are in JK, some are
in an application (a cgi with PATH_INFO). These aren't always 1:1 to the
filesystem, except in the case of filesystem-based modules such as our
default handler and the current cgi modules.
3. If something is in the Filesystem, then <Directory > and <Files > should
be applied. If something is not in the filesystem, than no, it should not, and
that other backend (e.g. proxy) should have it's own container blocks to
deal with the resource (e.g. <Proxy > blocks.) Overloading can become
a very confusing a dangerous game for admins to properly configure their
servers.
4. If something is not in the Filesystem, there should be no way that our own
CGI handler or default file handler should be allowed to handle the request.
If Apache
5. Whatever resource (Filesystem or what have you) that claims to own the
resource for config block processing should be the only available handler
to deal with the resource on the back end. Jumping from one resource
context in the configuration phase to another resource resource in the
handler phase is very dangerous.
Beyond what I've detailed above, I'm flexible about how it works/what the various
directives are, etc.
Bill
Re: FileSystem v.s. Other Resources [was configurable
Location?]
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
At 09:47 AM 2/6/2004, gregames@apache.org wrote:
>William A. Rowe, Jr. wrote:
>>At 12:17 PM 2/5/2004, Joshua Slive wrote:
>>
>>>>I do, however, agree that doing a directory-walk on virtual resources is
>>>>not nice. But my opinion is that "virtualness" is a property of the
>>>>resource, and hence should be designated when selecting the resource.
>>>>That is why I suggested changing SetHandler rather than <Location>.
>>>
>>>And perhaps I'm going way off in left field here, but why should this be
>>>user-configurable at all? Shouldn't the (for example) server-status
>>>handler know itself that it is a virtual handler, and therefore indicate
>>>that the directory-walk should be skipped?
>>
>>For example, yes. But on the other hand, what prevents someone from
>>removing the server-status handler in the fixups phase and tricking us into
>>serving a file.
>
>sounds like we're getting into defense mechanisms against hypothetical malicious modules... a loosing battle IMO
Malice? Nah, defending against badly written or inherently insecure modules.
But much more to the discussion at hand...
...will administrators grok when <Location virtual "/foo"> is a safe bet? Will
it proliferate in example configurations in the wild, in unsafe ways? I think
we can all presume it will. Look at our frustrations with users that have
essentially open proxy configs, which they set up by following examples
that proliferate out there.
Just playing defense here.
Bill
Re: FileSystem v.s. Other Resources [was configurable Location?]
Posted by gr...@apache.org.
William A. Rowe, Jr. wrote:
> At 12:17 PM 2/5/2004, Joshua Slive wrote:
>
>
>>>I do, however, agree that doing a directory-walk on virtual resources is
>>>not nice. But my opinion is that "virtualness" is a property of the
>>>resource, and hence should be designated when selecting the resource.
>>>That is why I suggested changing SetHandler rather than <Location>.
>>
>>And perhaps I'm going way off in left field here, but why should this be
>>user-configurable at all? Shouldn't the (for example) server-status
>>handler know itself that it is a virtual handler, and therefore indicate
>>that the directory-walk should be skipped?
>
>
> For example, yes. But on the other hand, what prevents someone from
> removing the server-status handler in the fixups phase and tricking us into
> serving a file.
sounds like we're getting into defense mechanisms against hypothetical malicious
modules... a loosing battle IMO
> I'd like to see the default handler entirely detached from the request if the
> dir_walk code has been bypassed.
if it's zero cost/dirt cheap in mainline code, I don't mind.
Greg
Re: FileSystem v.s. Other Resources [was configurable
Location?]
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
At 12:17 PM 2/5/2004, Joshua Slive wrote:
>> I do, however, agree that doing a directory-walk on virtual resources is
>> not nice. But my opinion is that "virtualness" is a property of the
>> resource, and hence should be designated when selecting the resource.
>> That is why I suggested changing SetHandler rather than <Location>.
>
>And perhaps I'm going way off in left field here, but why should this be
>user-configurable at all? Shouldn't the (for example) server-status
>handler know itself that it is a virtual handler, and therefore indicate
>that the directory-walk should be skipped?
For example, yes. But on the other hand, what prevents someone from
removing the server-status handler in the fixups phase and tricking us into
serving a file.
I'd like to see the default handler entirely detached from the request if the
dir_walk code has been bypassed.
Bill
Re: FileSystem v.s. Other Resources [was configurable Location?]
Posted by gr...@apache.org.
William A. Rowe, Jr. wrote:
> At 02:11 PM 2/9/2004, gregames@apache.org wrote:
>
>>William A. Rowe, Jr. wrote:
>>>Modules can do that today with some very trivial code...
>>
>>I think I see a problem. No doubt it could be made to work with a simple tweak.
>>
>>SetHandler in the location container sets the handler field in the core "dir" conf during location walk.
>
>
> ??? I'm not seeing that, the dir/location walk doesn't set up r->handler at all.
right. From merge_core_dir_configs():
if (new->handler) {
conf->handler = new->handler;
}
...so currently it only lives in config land until the fixups phase if I'm
reading the code correctly. That's the problem.
> You can use the SetHandler directive in location or dir contexts. I suppose
> if it is set in a location, we set r->handler it in the translate_name phase.
sounds good
> Even if it's overridden in a <Directory > block, the <Location > block it
> points at would still override it. So this change seems harmless.
I'll buy it
>>map_to_storage runs, but r->handler hasn't been updated due to the SetHandler yet. virtual_map_location doesn't know that it owns this URI.
>>
>>An unnecessary directory walk happens.
>
>
> Yes - the module needs to determine ownership in the translate_name phase.
>
>
>>core_override_type runs during the fixups phase and sets r->handler to the handler specified in the <Location > block from the handler field in the core "dir" config. It's too late to affect the directory walk.
>>
>>Another nit is that every module with virtual content would repeat essentially the same code, other than checking the unique handler name of course, and add to mainline path length on every request by increasing the number of map_to_storage routines. We would depend on each module zapping r->filename. This doesn't bother me for 2.0 if we can save a directory walk.
>
>
> I guess I'm really concerned, after some of the problems and confusion this
> thread has revealed, about changing the behavior at all in 2.0.
Maybe you're right.
> If we can get
> our own core "not a file" handlers to behave nicely (by fixing r->handler back
> in the translate_name phase) I would feel better about a minor change. That's
> a pretty restrictive change that would only affect, mod_info, mod_status etc.
> Other authors could do likewise if they understand the ramifications.
The performance of mod_info and mod_status doesn't bother me. Seeing the system
calls they generate in a trace does, but I'll get over it. Isn't mod_dav
virtual content though from the core's point of view (pls excuse my ignorance)?
mod_proxy is, but it already has working special case code scattered
throughout the core. (hmmm, might be some hints as to where the hits are for a
more general solution)
> But it seems after reading all of the comments, we aught to make some
> appropriate design changes in 2.2 so this entire solution is robust without
> breaking existing modules and possibly introducing security issues. After
> all, Apache 2.0 should be stable, and we should spend some time breaking
> the server in 2.1-dev with the eye to releasing a faster, more optimized 2.2.
Amen
> And yup - *every* handler author aught to make the appropriate change,
> because only they understand what's up with their content, and if it should
> be walked by dir and file sections, or not. We know mod_info and mod_status
> don't need to be, so we can code that ourselves.
right
Greg
Re: FileSystem v.s. Other Resources [was configurable
Location?]
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
At 02:11 PM 2/9/2004, gregames@apache.org wrote:
>William A. Rowe, Jr. wrote:
>>At 04:34 PM 2/6/2004, gregames@apache.org wrote:
>>
>>>Joshua Slive wrote:
>
>>>>And perhaps I'm going way off in left field here, but why should this be
>>>>user-configurable at all? Shouldn't the (for example) server-status
>>>>handler know itself that it is a virtual handler, and therefore indicate
>>>>that the directory-walk should be skipped?
>>>
>>>I like it! assuming we tweak it to be the module's responsibility to declare this property, vs. strictly the handler's (too late)
>>Modules can do that today with some very trivial code...
>
>I think I see a problem. No doubt it could be made to work with a simple tweak.
>
>SetHandler in the location container sets the handler field in the core "dir" conf during location walk.
??? I'm not seeing that, the dir/location walk doesn't set up r->handler at all.
You can use the SetHandler directive in location or dir contexts. I suppose
if it is set in a location, we set r->handler it in the translate_name phase.
Even if it's overridden in a <Directory > block, the <Location > block it
points at would still override it. So this change seems harmless.
>map_to_storage runs, but r->handler hasn't been updated due to the SetHandler yet. virtual_map_location doesn't know that it owns this URI.
>
>An unnecessary directory walk happens.
Yes - the module needs to determine ownership in the translate_name phase.
>core_override_type runs during the fixups phase and sets r->handler to the handler specified in the <Location > block from the handler field in the core "dir" config. It's too late to affect the directory walk.
>
>Another nit is that every module with virtual content would repeat essentially the same code, other than checking the unique handler name of course, and add to mainline path length on every request by increasing the number of map_to_storage routines. We would depend on each module zapping r->filename. This doesn't bother me for 2.0 if we can save a directory walk.
I guess I'm really concerned, after some of the problems and confusion this
thread has revealed, about changing the behavior at all in 2.0. If we can get
our own core "not a file" handlers to behave nicely (by fixing r->handler back
in the translate_name phase) I would feel better about a minor change. That's
a pretty restrictive change that would only affect, mod_info, mod_status etc.
Other authors could do likewise if they understand the ramifications.
But it seems after reading all of the comments, we aught to make some
appropriate design changes in 2.2 so this entire solution is robust without
breaking existing modules and possibly introducing security issues. After
all, Apache 2.0 should be stable, and we should spend some time breaking
the server in 2.1-dev with the eye to releasing a faster, more optimized 2.2.
And yup - *every* handler author aught to make the appropriate change,
because only they understand what's up with their content, and if it should
be walked by dir and file sections, or not. We know mod_info and mod_status
don't need to be, so we can code that ourselves.
Bill
Re: FileSystem v.s. Other Resources [was configurable Location?]
Posted by gr...@apache.org.
William A. Rowe, Jr. wrote:
> At 04:34 PM 2/6/2004, gregames@apache.org wrote:
>
>>Joshua Slive wrote:
>>>And perhaps I'm going way off in left field here, but why should this be
>>>user-configurable at all? Shouldn't the (for example) server-status
>>>handler know itself that it is a virtual handler, and therefore indicate
>>>that the directory-walk should be skipped?
>>
>>I like it! assuming we tweak it to be the module's responsibility to declare this property, vs. strictly the handler's (too late)
>
> Modules can do that today with some very trivial code...
I think I see a problem. No doubt it could be made to work with a simple tweak.
SetHandler in the location container sets the handler field in the core "dir"
conf during location walk.
map_to_storage runs, but r->handler hasn't been updated due to the SetHandler
yet. virtual_map_location doesn't know that it owns this URI.
An unnecessary directory walk happens.
core_override_type runs during the fixups phase and sets r->handler to the
handler specified in the <Location > block from the handler field in the core
"dir" config. It's too late to affect the directory walk.
Another nit is that every module with virtual content would repeat essentially
the same code, other than checking the unique handler name of course, and add to
mainline path length on every request by increasing the number of map_to_storage
routines. We would depend on each module zapping r->filename. This doesn't
bother me for 2.0 if we can save a directory walk.
Greg
> static int virtual_map_location(request_rec *r)
> {
> int access_status;
>
> /* Test that SetHandler forced to this content
> * otherwise skip it.
> */
> if (strncmp(r->handler, "info-handler") != 0)
> return DECLINED;
>
> /* this info-handler won't deal with the filename
> * so null the filename to ensure no file is served.
> */
> r->filename = "";
>
> /* Don't let the core map_to_storage hook handle this,
> * We don't need directory/file_walk. See mod_proxy.c
> * if our own custom <Restrictions > blocks were needed.
> */
> return OK;
> }
>
>
> static void register_hooks(apr_pool_t *p)
> {
> ...
> /* Suppress default dir_walk behavior, our module's
> * content is virtual.
> */
> ap_hook_map_to_storage(virtual_map_location, NULL, NULL, APR_HOOK_MIDDLE);
>
> }
>
>
>
Re: FileSystem v.s. Other Resources [was configurable
Location?]
Posted by André Malo <nd...@perlig.de>.
* "William A. Rowe, Jr." <wr...@rowe-clan.net> wrote:
> >The only issue in mod_rewrite could be the Type/Handler forcing which
> >occurs also in fixup, where is the right place for such things (to force
> >something).
>
> Don't they result in internal redirects? In that case there is nothing
> wrong when the internal redirect starts a new request cycle with the
> modified uri/filename.
Type or handler forcing just does ap_set_content_type or sets r->handler
respectively. (like ForceType does).
nd
Re: FileSystem v.s. Other Resources [was configurable
Location?]
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
At 01:37 PM 2/9/2004, André Malo wrote:
>* "William A. Rowe, Jr." <wr...@rowe-clan.net> wrote:
>
>> You hit the nail on the head - Alias (and mod_rewrite) cause us the greatest
>> grief in fixing this set of issues. *IF* they all are parsed in the
>> translate name phase we are fine, since map_to_storage is run after
>> translate_name is done.
>>
>> If they are not handled up front we have problems (some translations can
>> occur in the fixup phase, IIRC, which is wrong IMHO.)
>
>No. Alias translations and RewriteRules in servercontext are handled in the
>translate_name hook. Fixup rewrites (->internal redirects) or
>mod_alias-redirects in directory context are only triggered by per-directory
>configuration, which would be skipped in that case.
Ahhh of course.
>The only issue in mod_rewrite could be the Type/Handler forcing which occurs
>also in fixup, where is the right place for such things (to force something).
Don't they result in internal redirects? In that case there is nothing wrong when
the internal redirect starts a new request cycle with the modified uri/filename.
Bill
Re: FileSystem v.s. Other Resources [was configurable Location?]
Posted by André Malo <nd...@perlig.de>.
* "William A. Rowe, Jr." <wr...@rowe-clan.net> wrote:
> You hit the nail on the head - Alias (and mod_rewrite) cause us the greatest
> grief in fixing this set of issues. *IF* they all are parsed in the
> translate name phase we are fine, since map_to_storage is run after
> translate_name is done.
>
> If they are not handled up front we have problems (some translations can
> occur in the fixup phase, IIRC, which is wrong IMHO.)
No. Alias translations and RewriteRules in servercontext are handled in the
translate_name hook. Fixup rewrites (->internal redirects) or
mod_alias-redirects in directory context are only triggered by per-directory
configuration, which would be skipped in that case.
The only issue in mod_rewrite could be the Type/Handler forcing which occurs
also in fixup, where is the right place for such things (to force something).
nd
Re: FileSystem v.s. Other Resources [was configurable
Location?]
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
You hit the nail on the head - Alias (and mod_rewrite) cause us the greatest
grief in fixing this set of issues. *IF* they all are parsed in the translate name
phase we are fine, since map_to_storage is run after translate_name is done.
If they are not handled up front we have problems (some translations can
occur in the fixup phase, IIRC, which is wrong IMHO.)
The other risk, that we reset filename because we might hit the core handler,
could be much cleaner if we just failed 500 right out of the core handler any
time that dir_walk isn't run against the resource. I recall some arguments
to that fix in 2.0 - I'd like to force the hand on that issue in 2.2. Unsetting
the r->filename would become unnecessary in this trivial example.
Bill
At 08:08 AM 2/9/2004, Brian Akins wrote:
>William A. Rowe, Jr. wrote:
>>At 04:34 PM 2/6/2004, gregames@apache.org wrote:
>
>> /* this info-handler won't deal with the filename
>> * so null the filename to ensure no file is served.
>> */
>> r->filename = "";
>>\
>
>
>What if you have something like:
>
><Location /dynamic-stuff>
> SetHandler virtual-dynamic-handler
></Location>
>
>Alias /normal/stuff/dyn /dynamic-stuff
>
>
>How would all this affect this situation? IE, we would still need to know that /normal/stuff/dyn/1/2/test.html became /dynamic-stuff/1/2/test.html
Re: FileSystem v.s. Other Resources [was configurable Location?]
Posted by Brian Akins <ba...@web.turner.com>.
Joshua Slive wrote:
> On Mon, 9 Feb 2004, Brian Akins wrote:
>
>><Location /dynamic-stuff>
>> SetHandler virtual-dynamic-handler
>></Location>
>>
>>Alias /normal/stuff/dyn /dynamic-stuff
>>
>>
>>How would all this affect this situation? IE, we would still need to
>>know that /normal/stuff/dyn/1/2/test.html became
>>/dynamic-stuff/1/2/test.html
>
>
> Alias maps URLs to the filesystem. The second argument must be a
> filesystem path, and should not be mapped against any <Location>.
>
> Joshua.
>
OK. So in my example, in the proposed system, I should use Rewrite? So
the alias would become:
RewriteRule /normal/stuff/dyn(.*) /dynamic-stuff/$1
I am agreement with the need to not do uneccesarry stats, but I do'nt
want to have to write my own "alias" style routine...
--
Brian Akins
Senior Systems Engineer
CNN Internet Technologies
Re: FileSystem v.s. Other Resources [was configurable Location?]
Posted by Joshua Slive <jo...@slive.ca>.
On Mon, 9 Feb 2004, Brian Akins wrote:
> <Location /dynamic-stuff>
> SetHandler virtual-dynamic-handler
> </Location>
>
> Alias /normal/stuff/dyn /dynamic-stuff
>
>
> How would all this affect this situation? IE, we would still need to
> know that /normal/stuff/dyn/1/2/test.html became
> /dynamic-stuff/1/2/test.html
Alias maps URLs to the filesystem. The second argument must be a
filesystem path, and should not be mapped against any <Location>.
Joshua.
Re: FileSystem v.s. Other Resources [was configurable Location?]
Posted by Brian Akins <ba...@web.turner.com>.
William A. Rowe, Jr. wrote:
> At 04:34 PM 2/6/2004, gregames@apache.org wrote:
> /* this info-handler won't deal with the filename
> * so null the filename to ensure no file is served.
> */
> r->filename = "";
>
>\
What if you have something like:
<Location /dynamic-stuff>
SetHandler virtual-dynamic-handler
</Location>
Alias /normal/stuff/dyn /dynamic-stuff
How would all this affect this situation? IE, we would still need to
know that /normal/stuff/dyn/1/2/test.html became
/dynamic-stuff/1/2/test.html
--
Brian Akins
Senior Systems Engineer
CNN Internet Technologies
Re: FileSystem v.s. Other Resources [was configurable
Location?]
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
At 04:34 PM 2/6/2004, gregames@apache.org wrote:
>Joshua Slive wrote:
>>>I do, however, agree that doing a directory-walk on virtual resources is
>>>not nice. But my opinion is that "virtualness" is a property of the
>>>resource, and hence should be designated when selecting the resource.
>>>That is why I suggested changing SetHandler rather than <Location>.
>>
>>And perhaps I'm going way off in left field here, but why should this be
>>user-configurable at all? Shouldn't the (for example) server-status
>>handler know itself that it is a virtual handler, and therefore indicate
>>that the directory-walk should be skipped?
>
>I like it! assuming we tweak it to be the module's responsibility to declare this property, vs. strictly the handler's (too late).
Modules can do that today with some very trivial code...
static int virtual_map_location(request_rec *r)
{
int access_status;
/* Test that SetHandler forced to this content
* otherwise skip it.
*/
if (strncmp(r->handler, "info-handler") != 0)
return DECLINED;
/* this info-handler won't deal with the filename
* so null the filename to ensure no file is served.
*/
r->filename = "";
/* Don't let the core map_to_storage hook handle this,
* We don't need directory/file_walk. See mod_proxy.c
* if our own custom <Restrictions > blocks were needed.
*/
return OK;
}
static void register_hooks(apr_pool_t *p)
{
...
/* Suppress default dir_walk behavior, our module's
* content is virtual.
*/
ap_hook_map_to_storage(virtual_map_location, NULL, NULL, APR_HOOK_MIDDLE);
}
Re: FileSystem v.s. Other Resources [was configurable Location?]
Posted by gr...@apache.org.
Joshua Slive wrote:
>>I do, however, agree that doing a directory-walk on virtual resources is
>>not nice. But my opinion is that "virtualness" is a property of the
>>resource, and hence should be designated when selecting the resource.
>>That is why I suggested changing SetHandler rather than <Location>.
>
>
> And perhaps I'm going way off in left field here, but why should this be
> user-configurable at all? Shouldn't the (for example) server-status
> handler know itself that it is a virtual handler, and therefore indicate
> that the directory-walk should be skipped?
I like it! assuming we tweak it to be the module's responsibility to declare
this property, vs. strictly the handler's (too late).
Greg
Re: FileSystem v.s. Other Resources [was configurable Location?]
Posted by Joshua Slive <jo...@slive.ca>.
> I do, however, agree that doing a directory-walk on virtual resources is
> not nice. But my opinion is that "virtualness" is a property of the
> resource, and hence should be designated when selecting the resource.
> That is why I suggested changing SetHandler rather than <Location>.
And perhaps I'm going way off in left field here, but why should this be
user-configurable at all? Shouldn't the (for example) server-status
handler know itself that it is a virtual handler, and therefore indicate
that the directory-walk should be skipped?
Joshua.
Re: FileSystem v.s. Other Resources [was configurable Location?]
Posted by Joshua Slive <jo...@slive.ca>.
I just want to add a couple notes here on what I see as user-expectations.
> > The problem is that you want to add layers of additional directives, which
> > would change the behavior of <Directory > or <Location >,
>
> only <Location >, in a way that IMO is consistent with the existing
> documentation, but not the existing code of course.
[...]
> A complication to treating <Location > as if it's outside the filesystem
> is the doc that states <Location /> is a quick way to apply a config to
> the whole server. It will indeed do that today, unfortunately. I
> believe another quick way to get the same result is to put your config
> directives outside of any container. If that's true, I would prefer to
> see that doc for <Location /> removed. If the change to the behavior of
> <Location > is configurable, this is not an issue. If it's not
> configurable, then we probably should treat <Location /> as if it's in
> the filesystem - a special case for backwards compatibility.
I wrote the particular doc you quoted earlier, and I meant it more as a
recommendation (for security reasons) than a documentation of how apache
works. The true description of how apache is supposed to function is
here:
http://httpd.apache.org/docs-2.0/sections.html
(But it is a little vauge, because my knowledge of the internals of
configuration merging was not sufficient to clarify it the last time I
went through.)
Many people do, in fact, use <Location> for filesystem-related things.
See the examples in the mod_dav docs for one:
http://httpd.apache.org/docs-2.0/mod/mod_dav.html
Although I don't think this is esthetically correct, it is not
particularly dangerous. Using <Location> to provide *extra* access is
okay, because if you circumvent the <Location> using filesystem tricks,
you just lose the access. Using <Location> to *restrict* access is not
safe, because the restrictions may be circumvented.
The other important thing to note is that <Location> is often used for a
reason that is particular to apache configuration merging:
those sections are evaluated last. That means that
<Location />
Order deny,allow
Deny from all
Allow from local.net
</Location>
has a very different effect than the same directives placed in <Directory
/> because it will be evaluated after all <Directory> and <Files>
sections which could potentially override directives.
I'm not saying that this is a strictly correct use of <Location>, but it
is very useful and there is no other easy way to accomplish this "apply
this directive after everything else" effect.
I do, however, agree that doing a directory-walk on virtual resources is
not nice. But my opinion is that "virtualness" is a property of the
resource, and hence should be designated when selecting the resource.
That is why I suggested changing SetHandler rather than <Location>.
Joshua.
Re: FileSystem v.s. Other Resources [was configurable Location?]
Posted by Greg Ames <gr...@apache.org>.
Greg Marr wrote:
>> I'm only changing <Location > ... <Directory > is unaffected.
>
>
> Well, that's not entirely true. The <Directory> is affected indirectly,
> because it no longer applies. The behavior currently is: it applies to
> everything it matches. This would change it to: it applies to
> everything it matches unless it also matches a <Location>, in which case
> it doesn't apply.
OK, got it. Thanks for the clear explanation.
This could possibly be handled by checking for overlaps between <Location > and
<Directory > at config time, especially if they are sorted in least-to-most
specific order as wrowe suggests.
But my current favorite idea is Joshua's suggestion for the module to declare
that the content is virtual, pure and simple. In that case, any such overlaps
are an admin screwup IMO. GET /server-status/index.html doesn't make any sense
to me even if such a directory exists.
Greg
Re: FileSystem v.s. Other Resources [was configurable
Location?]
Posted by "William A. Rowe, Jr." <wr...@rowe-clan.net>.
At 10:43 AM 2/5/2004, Greg Marr wrote:
>At 10:22 AM 2/5/2004, gregames@apache.org wrote:
>>Thanks for the feedback, Will.
>>
>>William A. Rowe, Jr. wrote:
>>>At 03:39 PM 2/4/2004, gregames@apache.org wrote:
>>>
>>>>But then if I play devil's advocate, someone could see the new directive and turn it on when it's not appropriate and cause Bad Things to happen. Mainly I'm looking for comments on whether this should be configurable or not.
>>>
>>>Yes, I'm one who will agree with your devil's position :) I know the problem
>>>you are trying to solve, and changing the behavior of <Directory > isn't quite the solution...
>>
>>I'm only changing <Location > ... <Directory > is unaffected.
>
>Well, that's not entirely true. The <Directory> is affected indirectly, because it no longer applies. The behavior currently is: it applies to everything it matches. This would change it to: it applies to everything it matches unless it also matches a <Location>, in which case it doesn't apply.
Right - I understand.
I'm saying (as you pointed out) that it's too dangerous :) Defies the principle
of least surprise. They configure <Dir > blocks only to have them ignored.
The only safe way to do what you are suggesting, is to yank the default file
handler from under the request anytime the user has overridden the <Dir >
checks. Simplest way to do that is bind them together as a filesystem
module.
Oh, the same problems (of yanking <Dir > out from under the user) would be
true of the CGI handler, as well as third party modules that do things with
the filesystem. It would be important that they could explicitly enable the
module to add <Dir > handling.
Which brings us back to square one. I'd actually argued a long time ago
that a module that handles requests outside of the filesystem should be
responsible for bypassing the <Dir > handler itself. For example, mod_proxy
satisfies the map_to_storage hook on it's own, therefore never hitting <Dir >.
Bill
Re: FileSystem v.s. Other Resources [was configurable
Location?]
Posted by Greg Marr <gr...@alum.wpi.edu>.
At 10:22 AM 2/5/2004, gregames@apache.org wrote:
>Thanks for the feedback, Will.
>
>William A. Rowe, Jr. wrote:
>>At 03:39 PM 2/4/2004, gregames@apache.org wrote:
>>
>>>But then if I play devil's advocate, someone could see the new directive
>>>and turn it on when it's not appropriate and cause Bad Things to
>>>happen. Mainly I'm looking for comments on whether this should be
>>>configurable or not.
>>
>>Yes, I'm one who will agree with your devil's position :) I know the problem
>>you are trying to solve, and changing the behavior of <Directory > isn't
>>quite the solution...
>
>I'm only changing <Location > ... <Directory > is unaffected.
Well, that's not entirely true. The <Directory> is affected indirectly,
because it no longer applies. The behavior currently is: it applies to
everything it matches. This would change it to: it applies to everything
it matches unless it also matches a <Location>, in which case it doesn't apply.
Re: FileSystem v.s. Other Resources [was configurable Location?]
Posted by gr...@apache.org.
Thanks for the feedback, Will.
William A. Rowe, Jr. wrote:
> At 03:39 PM 2/4/2004, gregames@apache.org wrote:
>
>>But then if I play devil's advocate, someone could see the new directive and turn it on when it's not appropriate and cause Bad Things to happen. Mainly I'm looking for comments on whether this should be configurable or not.
>
>
> Yes, I'm one who will agree with your devil's position :) I know the problem
> you are trying to solve,
great!
> and changing the behavior of <Directory > isn't quite the solution...
I'm only changing <Location > ... <Directory > is unaffected.
>
> Issue 1:
>
> Walking the filesystem (the stat aspect of Directory and File) for something
> that doesn't live there is stupid :) We agree, and this must be fixed in 2.1
cool!
> Issue 2:
>
> Matching directories when something is outside of the filesystem is simply
> wasteful, and here too we agree.
>
> The problem is that you want to add layers of additional directives, which
> would change the behavior of <Directory > or <Location >,
only <Location >, in a way that IMO is consistent with the existing
documentation, but not the existing code of course.
> in ways that would
> allow the user to seriously compromise what they had explicitly configured
> for security. This is unwise. Let's look at the two issues above;
>
> Effect/Issue 1:
>
> Bypassing the filesystem canonicalization would be very bad on certain
> platforms such as windows, depending on case sensitivity, etc. It would
> also bypass *user configured* options such as avoiding symlinks.
only for <Location > If the admin is telling us the content is outside the
filesystem, I don't think filesystem canonicalization or symlinks are relevant.
> Effect/Issue 2:
>
> Bypassing the directory block walk would ignore the very <Directory >
> sections that the user explicitly put in their config. This too is bad.
This is only for <Location > . There is no change to the behavior of <Directory >.
>
> There is only one way
that sounds kinda religious...in software there are generally a number of ways
to solve a problem :-)
> to avoid these consequences, and still save ourselves
> the pain of the filesystem walk and <Directory > block handling...
I'm only changing <Location >
>
> === Break out file handling as a separate component ===
>
> I've proposed in the past the simple directive
>
> FilesystemMount /path/to/files
At first glance it looks like a key difference is that you are approaching it by
saying that it's OK to have <Location > map to a real filesystem and providing
config to explicitly specify the mapping. I'm advocating going the other way
and allowing <Location > to be treated as if it is really outside of the filesystem.
A complication to treating <Location > as if it's outside the filesystem is the
doc that states <Location /> is a quick way to apply a config to the whole
server. It will indeed do that today, unfortunately. I believe another quick
way to get the same result is to put your config directives outside of any
container. If that's true, I would prefer to see that doc for <Location />
removed. If the change to the behavior of <Location > is configurable, this is
not an issue. If it's not configurable, then we probably should treat <Location
/> as if it's in the filesystem - a special case for backwards compatibility.
I'd like to study your comments more after my coffee kicks in :-) Obviously
you've given a lot of thought to this issue so I want to see if I can understand
where you're coming from.
Greg