You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Eygene Ryabinkin <re...@codelabs.ru> on 2007/04/02 10:56:12 UTC
Reverse proxy mode and DAV protocol
Good day!
Sorry for rather long letter, but this is the summary from numerous
discuissions and I really tried to make it short.
I happened to notice the wrong behaviour of the Apache proxying
modules (originally, mod_proxy, but later the mod_rewrite and
essentially, all clients of mod_proxy rewriting) when it is doing
reverse-proxying for a DAV server.
The setup is following: HTTPS server is used as the reverse proxy
(RP) and the DAV server is sitting on the localhost and using the
HTTP transport. The problem is that the 'Destination' header is not
rewritten by the RP server and the DAV server refuses to serve the
MOVE and COPY requests. The problem is rather well understood:
http://silmor.de/49
http://svn.haxx.se/users/archive-2003-08/0763.shtml
http://svn.haxx.se/users/archive-2006-03/0549.shtml
and there are some workarounds for it ranging from the manual
patching the Apache mod_dav sources to get rid from the schema and
port checking (modules/dav/main/util.c:204 from the snapshot
httpd_20070330041747.tar.gz) to using the header rewriting routines
appeared (but not yet documented) in the 2.2.4.
My thought is that modules that are doing the rewriting stuff (in
the reverse proxy mode) should care for the 'Destination' header
themselves. So, I wrote the patch for the mod_proxy and, later, for
the mod_rewrite:
http://issues.apache.org/bugzilla/show_bug.cgi?id=41698
But Ruediger Pluem was kindly pointed that all modules that are
using the mod_proxy rewriting routines should be fixed by the patch.
I am not sure that it can be done in the generic case, because
mod_proxy currently have no idea about the other modules rewriting
procedures, it is just passed the 'proxy:whatever' URL. Please,
see the bug's comments for the complete discuission. (Comments 13-16
are the most relevant to my and Ruediger's points of view).
There are at least three solutions:
- patch the Apache's own modules to rewrite 'Destination' header
and leave other third-party modules patching to their respective
authors.
- invent the generic way to rewrite the 'Destination' header (and
others) in the mod_proxy.
- leave the things 'as is' and make the notes in the mod_rewrite
and mod_proxy documentation about the DAV and header rewriting,
saying that header rewriting will help.
I am somewhat against the third case, because my mind tells me that
if programmers can do something with this problem, they should try.
Can anyone comment on this topic?
Thanks for your patience! ;))
PS: talking about the latest patch version. I am not entirely sure
that the 'dirpath' argument in the mod_rewrite part is getting the
right value. The intent was to has the directory prefix for the
current location. I will research into is a bit later, but may be
someone will enlight me with the knowledge on how to get the correct
value of 'dirpath'?
--
Eygene
Re: Reverse proxy mode and DAV protocol
Posted by Henrik Nordstrom <he...@henriknordstrom.net>.
ons 2007-04-04 klockan 13:12 +0200 skrev Julian Reschke:
> What I meant by "reason" was the fact that the "Destination" header (and
> some aspects of the "If" header) require absolute URIs, which is
> problematic when there's a reverse proxy in the transmission path. All
> the issues around to rewrite or not to rewrite headers go away once
> these headers use absolute paths (well, as long as the reverse proxy
> doesn't also rewrite paths, but I would claim that this is nearly
> impossible to get right with WebDAV).
Rewriting is nearly impossible to get right. Even when limited to
"simple" rewrites of just the host component.
There is many aspects of HTTP and HTTP applications which depend on the
URIs being known on both sides (client and server), and the more
rewrites you do the higher the risk that some of these is overlooked and
things starts to break.
Most reverse proxies fail even the simple host:port based rewrites,
forgetting or wrongly mapping the internal URIs in some random headers
(i.e. Location, Content-Location, Destination, etc) or generated
response entities.
WebDAV in particular has a lot of URIs embedded in generated XML
request/response entities.
If you do rewrite then you better make sure you have a clear view of how
the backend URIs should be mapped back to external URIs, and make sure
your rewrites is applied on the complete HTTP messages headers & body,
both requests and responses, and not just the request line.
Regards
Henrik
Re: Reverse proxy mode and DAV protocol
Posted by Julian Reschke <ju...@gmx.de>.
Eygene Ryabinkin schrieb:
> Julian,
>
> Wed, Apr 04, 2007 at 03:22:53PM +0200, Julian Reschke wrote:
>>> So, your point is that Apache should make no attempts to rewrite
>>> the 'Destination' header for DAV and clients should use absolute
>>> paths for DAV. Or we still need 'Destination' rewriting? In the
>>> reverse proxy mode, I mean.
>>> Thank you!
>> I'd say Path rewriting is very hard, host rewriting is possible.
>
> So, if Apache will be able to do the rewriting in its modules, then
> it should try. OK, thank you.
>
>> 1) Servers, such as Apache/moddav should start supporting absolute paths in the
>> "Destination" and "If" headers ASAP.
>
> Again, talking about the current situation when URI are absolute:
> do you think that 'If' header automatical rewriting is possible and
> viable for the Apache's modules? I see no technical problems in
> such rewriting, but may be I am missing something.
>
> Thank you!
"If" header rewriting should be possible, as long as the code doing the
rewriting does the rewriting carefully (for instance, scanning for
"<.*>" will probably not be good enough...).
Best regards, Julian
Re: Reverse proxy mode and DAV protocol
Posted by Eygene Ryabinkin <re...@codelabs.ru>.
Julian,
Wed, Apr 04, 2007 at 03:22:53PM +0200, Julian Reschke wrote:
> >So, your point is that Apache should make no attempts to rewrite
> >the 'Destination' header for DAV and clients should use absolute
> >paths for DAV. Or we still need 'Destination' rewriting? In the
> >reverse proxy mode, I mean.
> >Thank you!
>
> I'd say Path rewriting is very hard, host rewriting is possible.
So, if Apache will be able to do the rewriting in its modules, then
it should try. OK, thank you.
>
>1) Servers, such as Apache/moddav should start supporting absolute paths in the
> "Destination" and "If" headers ASAP.
Again, talking about the current situation when URI are absolute:
do you think that 'If' header automatical rewriting is possible and
viable for the Apache's modules? I see no technical problems in
such rewriting, but may be I am missing something.
Thank you!
--
Eygene
Re: Reverse proxy mode and DAV protocol
Posted by Julian Reschke <ju...@gmx.de>.
Eygene Ryabinkin schrieb:
> Julian,
>
> Wed, Apr 04, 2007 at 01:12:54PM +0200, Julian Reschke wrote:
>>> Pardon for my stupideness, but what 'that reason' you are talking
>>> about? The previous letter was rather long and I fail to identify
>>> the exact point you're commenting. Could you, please, elaborate
>>> a bit?
>> Sorry.
>>
>> What I meant by "reason" was the fact that the "Destination" header (and some
>> aspects of the "If" header) require absolute URIs, which is problematic when
>> there's a reverse proxy in the transmission path. All the issues around to
>> rewrite or not to rewrite headers go away once these headers use absolute paths
>> (well, as long as the reverse proxy doesn't also rewrite paths, but I would
>> claim that this is nearly impossible to get right with WebDAV).
>
> Ahh, OK, thanks for the clarification. I had just tried to do path
> rewriting and I see that things are bad: I am using Git over HTTP
> and it fails badly with the path rewriting. May be will try to
> understand why, but I am not sure.
>
> So, your point is that Apache should make no attempts to rewrite
> the 'Destination' header for DAV and clients should use absolute
> paths for DAV. Or we still need 'Destination' rewriting? In the
> reverse proxy mode, I mean.
>
> Thank you!
I'd say Path rewriting is very hard, host rewriting is possible.
With RFC2518bis, clients can use absolute paths in some places where
before an absolute URI was required. This makes host rewriting much
easier, but of course that change needs to get deployed.
1) Servers, such as Apache/moddav should start supporting absolute paths
in the "Destination" and "If" headers ASAP. That's a backwards
compatible change and should be easy to deploy. Unfortunately, there's
no simple way to check whether a server allows path notation, except by
explicitly checking for DAV compliance level "3"
(<http://greenbytes.de/tech/webdav/draft-ietf-webdav-rfc2518bis-18.html#rfc.section.18.3>).
2) Clients should use absolute paths as soon as they see that the remote
server allows this.
Of course all of this is of little value until widely deployed
servers/clients start using it.
Best regards, Julian
Re: Reverse proxy mode and DAV protocol
Posted by Eygene Ryabinkin <re...@codelabs.ru>.
Julian,
Wed, Apr 04, 2007 at 01:12:54PM +0200, Julian Reschke wrote:
> >Pardon for my stupideness, but what 'that reason' you are talking
> >about? The previous letter was rather long and I fail to identify
> >the exact point you're commenting. Could you, please, elaborate
> >a bit?
>
> Sorry.
>
> What I meant by "reason" was the fact that the "Destination" header (and some
> aspects of the "If" header) require absolute URIs, which is problematic when
> there's a reverse proxy in the transmission path. All the issues around to
>rewrite or not to rewrite headers go away once these headers use absolute paths
> (well, as long as the reverse proxy doesn't also rewrite paths, but I would
> claim that this is nearly impossible to get right with WebDAV).
Ahh, OK, thanks for the clarification. I had just tried to do path
rewriting and I see that things are bad: I am using Git over HTTP
and it fails badly with the path rewriting. May be will try to
understand why, but I am not sure.
So, your point is that Apache should make no attempts to rewrite
the 'Destination' header for DAV and clients should use absolute
paths for DAV. Or we still need 'Destination' rewriting? In the
reverse proxy mode, I mean.
Thank you!
--
Eygene
Re: Reverse proxy mode and DAV protocol
Posted by Julian Reschke <ju...@gmx.de>.
Eygene Ryabinkin schrieb:
> Julian, good day.
>
> Tue, Apr 03, 2007 at 06:52:59PM +0200, Julian Reschke wrote:
>> Eygene Ryabinkin schrieb:
>>> Good day!
>>> Sorry for rather long letter, but this is the summary from numerous
>>> discuissions and I really tried to make it short.
>>> ...
>> Hi.
>>
>> RFC2518bis allows the Destination header to be just an absolute path for
>> exactly that reason (see
>> <http://greenbytes.de/tech/webdav/draft-ietf-webdav-rfc2518bis-18.html#destination-header>).
>
> Pardon for my stupideness, but what 'that reason' you are talking
> about? The previous letter was rather long and I fail to identify
> the exact point you're commenting. Could you, please, elaborate
> a bit?
Sorry.
What I meant by "reason" was the fact that the "Destination" header (and
some aspects of the "If" header) require absolute URIs, which is
problematic when there's a reverse proxy in the transmission path. All
the issues around to rewrite or not to rewrite headers go away once
these headers use absolute paths (well, as long as the reverse proxy
doesn't also rewrite paths, but I would claim that this is nearly
impossible to get right with WebDAV).
>> So this is really one of the few things why RFC2518bis support would be useful
>> in practice (if clients start to take advantage of it).
>
> But even if the 'Destination' is just the absolute path, then in
> the reversy proxying from the http://frontend/dir-one to the
> http://backend/some-long-path/dir-two we will still need to rewrite
> that absolute path. It is not very uncommon to see the reverse
Right (see above). But if you also rewrite paths, you will also need to
rewrite all multistatus response bodies. Thus is why I assumed nobody is
traing that.
> proxying for the different absolute paths on the frontend and backend
> servers: I've seen some that were due to some software weirdness
> (for example, two proxied Wiki instances that require the /wiki
> path to be present in both of them) or some legacy configuration
> that can not be readily changed due to the poor design. Am I missing
> some point?
I haven't seen reverse proxying with path rewriting working for WebDAV
so far. My assumption was that the easy variant (host/port) is already
hard enough, and the intent of those changes in RFC2518bis is to make at
least this case simpler.
Best regards, Julian
Re: Reverse proxy mode and DAV protocol
Posted by Eygene Ryabinkin <re...@codelabs.ru>.
Julian, good day.
Tue, Apr 03, 2007 at 06:52:59PM +0200, Julian Reschke wrote:
> Eygene Ryabinkin schrieb:
> >Good day!
> >Sorry for rather long letter, but this is the summary from numerous
> >discuissions and I really tried to make it short.
> >...
>
> Hi.
>
> RFC2518bis allows the Destination header to be just an absolute path for
> exactly that reason (see
> <http://greenbytes.de/tech/webdav/draft-ietf-webdav-rfc2518bis-18.html#destination-header>).
Pardon for my stupideness, but what 'that reason' you are talking
about? The previous letter was rather long and I fail to identify
the exact point you're commenting. Could you, please, elaborate
a bit?
> So this is really one of the few things why RFC2518bis support would be useful
> in practice (if clients start to take advantage of it).
But even if the 'Destination' is just the absolute path, then in
the reversy proxying from the http://frontend/dir-one to the
http://backend/some-long-path/dir-two we will still need to rewrite
that absolute path. It is not very uncommon to see the reverse
proxying for the different absolute paths on the frontend and backend
servers: I've seen some that were due to some software weirdness
(for example, two proxied Wiki instances that require the /wiki
path to be present in both of them) or some legacy configuration
that can not be readily changed due to the poor design. Am I missing
some point?
--
Eygene
Re: Reverse proxy mode and DAV protocol
Posted by Julian Reschke <ju...@gmx.de>.
Eygene Ryabinkin schrieb:
> Good day!
>
> Sorry for rather long letter, but this is the summary from numerous
> discuissions and I really tried to make it short.
> ...
Hi.
RFC2518bis allows the Destination header to be just an absolute path for
exactly that reason (see
<http://greenbytes.de/tech/webdav/draft-ietf-webdav-rfc2518bis-18.html#destination-header>).
So this is really one of the few things why RFC2518bis support would be
useful in practice (if clients start to take advantage of it).
Best regards, Julian