You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by "Leslie R. Lait" <lr...@ertel.gsfc.nasa.gov> on 2003/10/16 00:22:51 UTC
[users@httpd] mod_autoindex and filenames containing "%"
I'm having problems using mod_autoindex to provide directory listings
of collections of data files that follow a certain local file naming
convention.
The autoindex module in Apache 2.0.47 seems to ignore certain
files which contain percent signs ("%") in the filename. The problem
appears to be that the server is interpreting those filename
percents as the beginning of hex escape sequences.
Symptoms:
Set up a directory somewhere under the server document root
with the following files:
test00
test01%w
test02%0
test03%00
test04%25
test05%26
test06%10
test07%01
The web server has autoindexing enabled, of course. The problem
appears to be independent of whether various indexing options
are enabled or disabled. The symptoms appear when running
under IRIX and Linux, and from the code this looks like an
architecture-independent issue.
An attempt to GET the directory will yield a directory listing
that contains only:
test00
test04%25
test05%26
test06%10
test07%01
Filenames which have a valid hex escape sequence are listed
correctly; "test05%26" appears as "test05%26" and not "test05&".
But filenames which contain a "%" character followed by an
invalid hex number are rejected and consequently ignored.
"test03%00" also fails, apparently because the "%00" gets
unescaped to the null character.
After looking at the server source code and testing execution paths,
this is what I have found:
The index_directory() function in mod_autoindex.c examines and
assembles a list of the directory entries. To be entered into
that list, the ap_process_request_internal() function---called
by way of make_autoindex_entry()---must return an "OK" status.
But one of the first things that ap_process_request_internal()
does is pass the request URI through the ap_unescape_url() function
to unescape any "%hh" sequences in the string. This does not
work so well when "%" characters are actually in the filenames
being examined.
Has anyone else encountered this problem? Is this a known issue?
--
----------------------------------|
Leslie Robert Lait |
An SSAI employee |
lrlait@code916.gsfc.nasa.gov |
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] mod_autoindex and filenames containing "%"
Posted by Leif W <wa...@usa.net>.
Here's an update I just recevied from bugzilla.
Leif
directory listing bug with % symbol in directory or file names
nd@perlig.de changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution| |FIXED
------- Additional Comments From nd@perlig.de 2003-11-01 22:33 -------
Fixed in 2.1 and proposed for backport into the 2.0 stable branch.
Thanks for your report and thanks for using Apache!
----- Original Message -----
From: "Robert Andersson" <ro...@profundis.nu>
To: <us...@httpd.apache.org>
Sent: Wednesday, October 29, 2003 1:53 AM
Subject: Re: [users@httpd] mod_autoindex and filenames containing "%"
> Leslie R. Lait wrote:
> > I'm having problems using mod_autoindex to provide directory listings
> > of collections of data files that follow a certain local file naming
> > convention.
>
> I have been able to fix this behaviour. See the bug report that Leif
> found/updated:
> http://nagoya.apache.org/bugzilla/show_bug.cgi?id=13598
>
> I've been too busy to find time to figure out how to propose it to be
fixed,
> and sadly will continue to be so.
>
> If you are in need of having mod_autoindex work the way you want, a quick
> fix is to change a line in the source and recompile. I cannot make a patch
> now, so locate this function:
>
> // in server/request.c (around line 1664)
> AP_DECLARE(request_rec *) ap_sub_req_lookup_dirent(...)
> {
> ...
> if (r->path_info && *r->path_info) {
> ...
> }
> else {
> udir = ap_make_dirstr_parent(rnew->pool, r->uri);
> rnew->uri = ap_make_full_path(rnew->pool, udir, dirent->name);
> }
> ...
> }
>
> The last statement (rnew->uri ...) should be changed to:
> rnew->uri = ap_make_full_path(rnew->pool, udir,
> ap_escape_path_segment(rnew->pool, dirent->name));
>
> This will escape the URI passed to ap_process_request_internal(), which
then
> will behave as expected.
>
> Regards,
> Robert Andersson
>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] mod_autoindex and filenames containing "%"
Posted by Robert Andersson <ro...@profundis.nu>.
Leslie R. Lait wrote:
> I'm having problems using mod_autoindex to provide directory listings
> of collections of data files that follow a certain local file naming
> convention.
I have been able to fix this behaviour. See the bug report that Leif
found/updated:
http://nagoya.apache.org/bugzilla/show_bug.cgi?id=13598
I've been too busy to find time to figure out how to propose it to be fixed,
and sadly will continue to be so.
If you are in need of having mod_autoindex work the way you want, a quick
fix is to change a line in the source and recompile. I cannot make a patch
now, so locate this function:
// in server/request.c (around line 1664)
AP_DECLARE(request_rec *) ap_sub_req_lookup_dirent(...)
{
...
if (r->path_info && *r->path_info) {
...
}
else {
udir = ap_make_dirstr_parent(rnew->pool, r->uri);
rnew->uri = ap_make_full_path(rnew->pool, udir, dirent->name);
}
...
}
The last statement (rnew->uri ...) should be changed to:
rnew->uri = ap_make_full_path(rnew->pool, udir,
ap_escape_path_segment(rnew->pool, dirent->name));
This will escape the URI passed to ap_process_request_internal(), which then
will behave as expected.
Regards,
Robert Andersson
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] mod_autoindex and filenames containing "%"
Posted by Robert Andersson <ro...@profundis.nu>.
Leif W wrote:
> Anyone feel like trying to hack the source? This seems straightforward
> enough for me to give it a shot. Just, I can't seem to find (grep -niH
> sub_request `find . -name "*.[ch]"`) a function named sub_request.
> There's a make_sub_request... but it doesn't seem related (all calls are
in
> request.c).
Sorry, I think I meant process_request_internal(), that performs a "sub
request".
> I'm guessing it's just mod_autoindex.c somewhere in the index_directory
> function, somewhere in this chunk of code:
>
> else {
> pstring = apr_pstrndup(r->pool, qstring, eos -
qstring);
> --> if (ap_unescape_url(pstring) != OK) {
> /* ignore the pattern, if it's bad. */
> pstring = NULL;
> }
> else {
> ppre = ";P=";
> /* be correct */
> epattern = ap_escape_uri(r->pool, pstring);
> }
> }
No, this piece does checking on the query string (for sorting etc), and not
related to the issue at hand, I think.
My theory was based mostly on Leslie's description and quick glance at the
relevant code. I did look a little more now, and cannot really find anything
conclusive. A little "call stack":
// mod_autoindex.c -- index_directory()
while (apr_dir_read(&dirent, APR_FINFO_MIN | APR_FINFO_NAME,
thedir) == APR_SUCCESS) {
// ...
p = make_autoindex_entry(&dirent, autoindex_opts, autoindex_conf, r,
keyid, direction, pstring);
if (p != NULL) {
p->next = head;
head = p;
num_ent++;
}
}
// mod_autoindex.c -- make_autoindex_entry()
if (!(rr = ap_sub_req_lookup_dirent(dirent, r, AP_SUBREQ_NO_ARGS,
NULL))) {
return (NULL);
}
// server/request.c -- ap_sub_req_lookup_dirent()
if ((res = ap_process_request_internal(rnew))) {
rnew->status = res;
}
// server/request.c -- ap_process_request_internal()
access_status = ap_unescape_url(r->parsed_uri.path);
But, I'm not sure that this last call is executed in our case, as there are
quite a few conditionals. I will see if I can hook up a debugger and see
where it goes wrong. I don't know if ap_process_request_internal() expects
an escaped or unescaped URI, either.
Regards,
Robert Andersson
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] mod_autoindex and filenames containing "%"
Posted by Leif W <wa...@usa.net>.
> ----- Original Message -----
> From: "Robert Andersson" <ro...@profundis.nu>
> To: <us...@httpd.apache.org>
> Sent: Monday, October 20, 2003 1:24 AM
> Subject: Re: [users@httpd] mod_autoindex and filenames containing "%"
>
> > Leif W:
> > This seems like a bug to me.
>
> I think it is safe to file this as a bug, if it isn't already in the
> database;
Done. I modified an existing bug report. It was a narrower scope
(directory name only), so I broaded it to include directory AND file names.
It was also only on Win2k/NT, so I changed the OS to ALL. And finally, I
obviously changed the Apache version to the most current. But the bug has
already been in the database over a year and never been assigned to anyone,
still marked as NEW. Included a URL back to this thread for examples and
verified reproducibility. Quoted your estimation of what might be the cause
of the bug.
> As I see it, the "bug" in mod_autoindex would be that it doesn't
> URI-escape the filename before sending it to the sub_request function.
Anyone feel like trying to hack the source? This seems straightforward
enough for me to give it a shot. Just, I can't seem to find (grep -niH
sub_request `find . -name "*.[ch]"`) a function named sub_request. There's
a make_sub_request... but it doesn't seem related (all calls are in
request.c).
Just to be clear, please restate what IS happening, and what is SUPPOSED to
happen. I'll look at which files need modifying, and which function(s).
I'm guessing it's just mod_autoindex.c somewhere in the index_directory
function, somewhere in this chunk of code:
else {
pstring = apr_pstrndup(r->pool, qstring, eos - qstring);
--> if (ap_unescape_url(pstring) != OK) {
/* ignore the pattern, if it's bad. */
pstring = NULL;
}
else {
ppre = ";P=";
/* be correct */
epattern = ap_escape_uri(r->pool, pstring);
}
}
Leif
P.S. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=13598
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] mod_autoindex and filenames containing "%"
Posted by Robert Andersson <ro...@profundis.nu>.
Leif W:
> This seems like a bug to me. Why does mod_autoindex care about what the
> filenames are? And why does it bother tinkering with an unescape, munging
the
> "status" of a file's string, and swallow up something it deems unsafe?
As I understood the OP's conclusions, and from my own source examination,
mod_autoindex posts the request as a sub_request in order to get necessary
information about the file/directory. This one is expecting a URI, so of
course it unescapes it before further processing. As I see it, the "bug" in
mod_autoindex would be that it doesn't URI-escape the filename before
sending it to the sub_request function.
I think it is safe to file this as a bug, if it isn't already in the
database; I was also able to reproduce this incorrect behaviour on both
Linux and Windows.
Regards,
Robert Andersson
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] mod_autoindex and filenames containing "%"
Posted by Leif W <wa...@usa.net>.
Just writing to say I've tested this and observed the same thing in a Debian
GNU/Linux system. Also, adding that the function ap_unescape_url resides in
httpd-2.0.47/server/util.c line 1563. This seems like a bug to me. Why
does mod_autoindex care about what the filenames are? And why does it
bother tinkering with an unescape, munging the "status" of a file's string,
and swallow up something it deems unsafe? If a file name is file%00, why
can't mod_autoindex just correctly generate the HTML link to the file as
HREF="file%2500" like it's supposed to?
Is there some justification for "escaping" certain hex codes, like if
someone names their file something like the escaped equivalent of
"../../../../usr/bin/adduser" (
%25%2E%25%2E%25%2F%25%2E%25%2E%25%2F%25%2E%25%2E%25%2F%25%2E%25%2E%25%2Fusr%
25%2Fbin%25%2Fadduser ) or something else that I can't see?
Leif
----- Original Message -----
From: "Leslie R. Lait" <lr...@ertel.gsfc.nasa.gov>
To: <us...@httpd.apache.org>
Sent: Wednesday, October 15, 2003 6:22 PM
Subject: [users@httpd] mod_autoindex and filenames containing "%"
>
>
> I'm having problems using mod_autoindex to provide directory listings
> of collections of data files that follow a certain local file naming
> convention.
>
> The autoindex module in Apache 2.0.47 seems to ignore certain
> files which contain percent signs ("%") in the filename. The problem
> appears to be that the server is interpreting those filename
> percents as the beginning of hex escape sequences.
>
> Symptoms:
> Set up a directory somewhere under the server document root
> with the following files:
> test00
> test01%w
> test02%0
> test03%00
> test04%25
> test05%26
> test06%10
> test07%01
> The web server has autoindexing enabled, of course. The problem
> appears to be independent of whether various indexing options
> are enabled or disabled. The symptoms appear when running
> under IRIX and Linux, and from the code this looks like an
> architecture-independent issue.
>
> An attempt to GET the directory will yield a directory listing
> that contains only:
> test00
> test04%25
> test05%26
> test06%10
> test07%01
>
> Filenames which have a valid hex escape sequence are listed
> correctly; "test05%26" appears as "test05%26" and not "test05&".
> But filenames which contain a "%" character followed by an
> invalid hex number are rejected and consequently ignored.
> "test03%00" also fails, apparently because the "%00" gets
> unescaped to the null character.
>
> After looking at the server source code and testing execution paths,
> this is what I have found:
>
> The index_directory() function in mod_autoindex.c examines and
> assembles a list of the directory entries. To be entered into
> that list, the ap_process_request_internal() function---called
> by way of make_autoindex_entry()---must return an "OK" status.
>
> But one of the first things that ap_process_request_internal()
> does is pass the request URI through the ap_unescape_url() function
> to unescape any "%hh" sequences in the string. This does not
> work so well when "%" characters are actually in the filenames
> being examined.
>
> Has anyone else encountered this problem? Is this a known issue?
>
> --
> ----------------------------------|
> Leslie Robert Lait |
> An SSAI employee |
> lrlait@code916.gsfc.nasa.gov |
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org