You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Graham Leggett <mi...@sharp.fm> on 2001/04/08 18:01:32 UTC
A problem with the CORE filter and tunnelling
Hi all,
I am having a problem with the CONNECT part of the proxy, and need some
help to solve this one.
The SSL tunnel functionality is currently working great - a
bidirectional tunnel is set up between client and remote server, and
data flows normally in both directions over the link until either side
breaks the connection.
The trouble is that being a tunnel, the CONNECT module handles all it's
own data transfer using APR. (None of the filters should be touching the
data, so they are bypassed). Once the connection is finished, the CORE
filter suddenly jumps in and tacks the as-unsent HTTP/1.1 response line
to the end of the request, like this:
(Using an ftp remote server as an example)
[root@jessica httpd-2.0]# telnet 127.0.0.1 80
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
CONNECT 127.0.0.1:21 HTTP/1.0
User-Agent: Fred
HTTP/1.0 200 Connection Established
Proxy-agent: Apache/2.0.16-dev (Unix)
220 ProFTPD 1.2.0pre10 Server (ProFTPD) [jessica]
USER ftp
331 Anonymous login ok, send your complete e-mail address as password.
--> at this point the ftp server is killed, and thus the remote
connection
is closed. Apache now adds this: >--
HTTP/1.1 200 OK
Date: Sun, 08 Apr 2001 15:45:11 GMT
Server: Apache/2.0.16-dev (Unix)
Content-Length: 0
Connection: close
Content-Type: text/plain; charset=ISO-8859-1
Connection closed by foreign host.
Obviously the CORE filter, having never got to do it's "send headers"
thing, is jumping in (too late of course) and finishing the job. What
needs to happen is that the filter be removed or disabled somehow, the
question is, how?
a) signal somehow to the core filter that it must not send anything - if
so, how?
b) remove all the filters from the output filter stack, and replace them
with a NULL filter that does this:
static int null_filter() {
return APR_SUCCESS;
}
c) something else?
Regards,
Graham
--
-----------------------------------------
minfrin@sharp.fm "There's a moon
over Bourbon Street
tonight..."
Re: A problem with the CORE filter and tunnelling
Posted by Graham Leggett <mi...@sharp.fm>.
rbb@covalent.net wrote:
> Make sure that it is actually working, and we aren't just returning an
> error that isn't being detected.
Using FTP is an example of a protocol that could be tunnelled, I issued
a QUIT command. The FTP daemon closed the remote connection, which the
CONNECT code picked up correctly, and returned OK to indicate it had
served the request. Apache then closed the connection (as it should
have). No errors were logged in the error log, nor sent back to the
client.
Regards,
Graham
--
-----------------------------------------
minfrin@sharp.fm "There's a moon
over Bourbon Street
tonight..."
Re: A problem with the CORE filter and tunnelling
Posted by rb...@covalent.net.
On Tue, 10 Apr 2001, Graham Leggett wrote:
> rbb@covalent.net wrote:
>
> > No it shouldn't. That is an error condition. The error returned is
> > NobodyWrote. It basically says that nobody ever sent the data to the
> > network.
>
> It works without the NULL filter though - should it be there?
Make sure that it is actually working, and we aren't just returning an
error that isn't being detected.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: A problem with the CORE filter and tunnelling
Posted by Graham Leggett <mi...@sharp.fm>.
rbb@covalent.net wrote:
> No it shouldn't. That is an error condition. The error returned is
> NobodyWrote. It basically says that nobody ever sent the data to the
> network.
It works without the NULL filter though - should it be there?
Regards,
Graham
--
-----------------------------------------
minfrin@sharp.fm "There's a moon
over Bourbon Street
tonight..."
Re: A problem with the CORE filter and tunnelling
Posted by rb...@covalent.net.
On Tue, 10 Apr 2001, Greg Stein wrote:
> On Tue, Apr 10, 2001 at 01:02:35AM +0200, Graham Leggett wrote:
> > "Roy T. Fielding" wrote:
> >
> > > Definitely remove it -- after the CONNECT response, we should be acting
> > > like a transport router and know nothing about HTTP. This is essentially
> > > a new (null) protocol taking over the connection.
> >
> > That's what I've done at the moment - the filter stack is replaced with
> > a NULL filter (just returns APR_SUCCESS) so the rest of Apache doesn't
> > fiddle with the connection.
>
> Why are you inserting a "null" filter at all? Why doesn't removing all the
> filters work? It should.
No it shouldn't. That is an error condition. The error returned is
NobodyWrote. It basically says that nobody ever sent the data to the
network.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: A problem with the CORE filter and tunnelling
Posted by Greg Stein <gs...@lyra.org>.
On Tue, Apr 10, 2001 at 01:02:35AM +0200, Graham Leggett wrote:
> "Roy T. Fielding" wrote:
>
> > Definitely remove it -- after the CONNECT response, we should be acting
> > like a transport router and know nothing about HTTP. This is essentially
> > a new (null) protocol taking over the connection.
>
> That's what I've done at the moment - the filter stack is replaced with
> a NULL filter (just returns APR_SUCCESS) so the rest of Apache doesn't
> fiddle with the connection.
Why are you inserting a "null" filter at all? Why doesn't removing all the
filters work? It should.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: A problem with the CORE filter and tunnelling
Posted by Graham Leggett <mi...@sharp.fm>.
"Roy T. Fielding" wrote:
> Definitely remove it -- after the CONNECT response, we should be acting
> like a transport router and know nothing about HTTP. This is essentially
> a new (null) protocol taking over the connection.
That's what I've done at the moment - the filter stack is replaced with
a NULL filter (just returns APR_SUCCESS) so the rest of Apache doesn't
fiddle with the connection.
Regards,
Graham
--
-----------------------------------------
minfrin@sharp.fm "There's a moon
over Bourbon Street
tonight..."
Re: A problem with the CORE filter and tunnelling
Posted by rb...@covalent.net.
On Mon, 9 Apr 2001, Roy T. Fielding wrote:
> > I'd say that you want to remove the HTTP_HEADER filter, or keep it in and
> > make sure it has the data to operate properly.
>
> Definitely remove it -- after the CONNECT response, we should be acting
> like a transport router and know nothing about HTTP. This is essentially
> a new (null) protocol taking over the connection.
I agree with Roy. Removing filters is fine to do, assuming it is done
correctly. We don't want to die if the filter isn't there, we just want
to remove it if it is.
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: A problem with the CORE filter and tunnelling
Posted by "Roy T. Fielding" <fi...@ebuilt.com>.
> I'd say that you want to remove the HTTP_HEADER filter, or keep it in and
> make sure it has the data to operate properly.
Definitely remove it -- after the CONNECT response, we should be acting
like a transport router and know nothing about HTTP. This is essentially
a new (null) protocol taking over the connection.
....Roy
Re: A problem with the CORE filter and tunnelling
Posted by Greg Stein <gs...@lyra.org>.
On Sun, Apr 08, 2001 at 06:01:32PM +0200, Graham Leggett wrote:
>...
> The trouble is that being a tunnel, the CONNECT module handles all it's
> own data transfer using APR. (None of the filters should be touching the
> data, so they are bypassed). Once the connection is finished, the CORE
> filter suddenly jumps in and tacks the as-unsent HTTP/1.1 response line
> to the end of the request, like this:
Actually, it sounds like you've got the HTTP_HEADER filter in there. It is
the one which sends the headers.
In this particular case, I'd guess that an EOS bucket comes along and the
HTTP_HEADER filter pops the headers into the output.
I'd say that you want to remove the HTTP_HEADER filter, or keep it in and
make sure it has the data to operate properly.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: A problem with the CORE filter and tunnelling
Posted by Greg Stein <gs...@lyra.org>.
On Sun, Apr 08, 2001 at 06:01:32PM +0200, Graham Leggett wrote:
>...
> The trouble is that being a tunnel, the CONNECT module handles all it's
> own data transfer using APR. (None of the filters should be touching the
> data, so they are bypassed). Once the connection is finished, the CORE
> filter suddenly jumps in and tacks the as-unsent HTTP/1.1 response line
> to the end of the request, like this:
Actually, it sounds like you've got the HTTP_HEADER filter in there. It is
the one which sends the headers.
In this particular case, I'd guess that an EOS bucket comes along and the
HTTP_HEADER filter pops the headers into the output.
I'd say that you want to remove the HTTP_HEADER filter, or keep it in and
make sure it has the data to operate properly.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: A problem with the CORE filter and tunnelling
Posted by Greg Stein <gs...@lyra.org>.
On Sun, Apr 08, 2001 at 06:41:40PM +0200, Graham Leggett wrote:
> rbb@covalent.net wrote:
>...
> > If you want to remove a filter, look at the reset_filters function. It
> > should show you how to do it.
>
> Will take a look.
Removing a filter is almost always a bad idea. Try to avoid putting the
filter into the stack in the first place.
[ by the simple fact that you "know" a filter is present, and you go to
remove it, that means that you've constrained the system to have that
thing present and have it removed. think "coupling" or "encapsulation" ...
one piece of the code shouldn't know what the other guy is doing, and it
should be independent of whatever they've done. ]
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: A problem with the CORE filter and tunnelling
Posted by Greg Stein <gs...@lyra.org>.
On Sun, Apr 08, 2001 at 06:41:40PM +0200, Graham Leggett wrote:
> rbb@covalent.net wrote:
>...
> > If you want to remove a filter, look at the reset_filters function. It
> > should show you how to do it.
>
> Will take a look.
Removing a filter is almost always a bad idea. Try to avoid putting the
filter into the stack in the first place.
[ by the simple fact that you "know" a filter is present, and you go to
remove it, that means that you've constrained the system to have that
thing present and have it removed. think "coupling" or "encapsulation" ...
one piece of the code shouldn't know what the other guy is doing, and it
should be independent of whatever they've done. ]
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
Re: A problem with the CORE filter and tunnelling
Posted by rb...@covalent.net.
On Sun, 8 Apr 2001, Graham Leggett wrote:
> rbb@covalent.net wrote:
>
> > Why is the CONNECT proxy not using filters? It seems to me that filters
> > are almost ideally matched for the CONNECT requests.
>
> Aren't filters half duplex though? The CONNECT tunnel is bidirectional.
Filters are half-duplex, but just use input and output filters, and the
problem goes away. When you need to write data, just create the brigade
to write, and call ap_pass_brigade().
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: A problem with the CORE filter and tunnelling
Posted by rb...@covalent.net.
On Sun, 8 Apr 2001, Graham Leggett wrote:
> rbb@covalent.net wrote:
>
> > Why is the CONNECT proxy not using filters? It seems to me that filters
> > are almost ideally matched for the CONNECT requests.
>
> Aren't filters half duplex though? The CONNECT tunnel is bidirectional.
Filters are half-duplex, but just use input and output filters, and the
problem goes away. When you need to write data, just create the brigade
to write, and call ap_pass_brigade().
Ryan
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: A problem with the CORE filter and tunnelling
Posted by Graham Leggett <mi...@sharp.fm>.
rbb@covalent.net wrote:
> Why is the CONNECT proxy not using filters? It seems to me that filters
> are almost ideally matched for the CONNECT requests.
Aren't filters half duplex though? The CONNECT tunnel is bidirectional.
> If you want to remove a filter, look at the reset_filters function. It
> should show you how to do it.
Will take a look.
Regards,
Graham
--
-----------------------------------------
minfrin@sharp.fm "There's a moon
over Bourbon Street
tonight..."
Re: A problem with the CORE filter and tunnelling
Posted by Graham Leggett <mi...@sharp.fm>.
rbb@covalent.net wrote:
> Why is the CONNECT proxy not using filters? It seems to me that filters
> are almost ideally matched for the CONNECT requests.
Aren't filters half duplex though? The CONNECT tunnel is bidirectional.
> If you want to remove a filter, look at the reset_filters function. It
> should show you how to do it.
Will take a look.
Regards,
Graham
--
-----------------------------------------
minfrin@sharp.fm "There's a moon
over Bourbon Street
tonight..."
Re: A problem with the CORE filter and tunnelling
Posted by rb...@covalent.net.
Why is the CONNECT proxy not using filters? It seems to me that filters
are almost ideally matched for the CONNECT requests.
If you want to remove a filter, look at the reset_filters function. It
should show you how to do it.
Ryan
On Sun, 8 Apr 2001, Graham Leggett wrote:
> Hi all,
>
> I am having a problem with the CONNECT part of the proxy, and need some
> help to solve this one.
>
> The SSL tunnel functionality is currently working great - a
> bidirectional tunnel is set up between client and remote server, and
> data flows normally in both directions over the link until either side
> breaks the connection.
>
> The trouble is that being a tunnel, the CONNECT module handles all it's
> own data transfer using APR. (None of the filters should be touching the
> data, so they are bypassed). Once the connection is finished, the CORE
> filter suddenly jumps in and tacks the as-unsent HTTP/1.1 response line
> to the end of the request, like this:
>
> (Using an ftp remote server as an example)
>
> [root@jessica httpd-2.0]# telnet 127.0.0.1 80
> Trying 127.0.0.1...
> Connected to 127.0.0.1.
> Escape character is '^]'.
> CONNECT 127.0.0.1:21 HTTP/1.0
> User-Agent: Fred
>
> HTTP/1.0 200 Connection Established
> Proxy-agent: Apache/2.0.16-dev (Unix)
>
> 220 ProFTPD 1.2.0pre10 Server (ProFTPD) [jessica]
> USER ftp
> 331 Anonymous login ok, send your complete e-mail address as password.
> --> at this point the ftp server is killed, and thus the remote
> connection
> is closed. Apache now adds this: >--
> HTTP/1.1 200 OK
> Date: Sun, 08 Apr 2001 15:45:11 GMT
> Server: Apache/2.0.16-dev (Unix)
> Content-Length: 0
> Connection: close
> Content-Type: text/plain; charset=ISO-8859-1
>
> Connection closed by foreign host.
>
> Obviously the CORE filter, having never got to do it's "send headers"
> thing, is jumping in (too late of course) and finishing the job. What
> needs to happen is that the filter be removed or disabled somehow, the
> question is, how?
>
> a) signal somehow to the core filter that it must not send anything - if
> so, how?
>
> b) remove all the filters from the output filter stack, and replace them
> with a NULL filter that does this:
>
> static int null_filter() {
> return APR_SUCCESS;
> }
>
> c) something else?
>
> Regards,
> Graham
> --
> -----------------------------------------
> minfrin@sharp.fm "There's a moon
> over Bourbon Street
> tonight..."
>
>
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------
Re: A problem with the CORE filter and tunnelling
Posted by rb...@covalent.net.
Why is the CONNECT proxy not using filters? It seems to me that filters
are almost ideally matched for the CONNECT requests.
If you want to remove a filter, look at the reset_filters function. It
should show you how to do it.
Ryan
On Sun, 8 Apr 2001, Graham Leggett wrote:
> Hi all,
>
> I am having a problem with the CONNECT part of the proxy, and need some
> help to solve this one.
>
> The SSL tunnel functionality is currently working great - a
> bidirectional tunnel is set up between client and remote server, and
> data flows normally in both directions over the link until either side
> breaks the connection.
>
> The trouble is that being a tunnel, the CONNECT module handles all it's
> own data transfer using APR. (None of the filters should be touching the
> data, so they are bypassed). Once the connection is finished, the CORE
> filter suddenly jumps in and tacks the as-unsent HTTP/1.1 response line
> to the end of the request, like this:
>
> (Using an ftp remote server as an example)
>
> [root@jessica httpd-2.0]# telnet 127.0.0.1 80
> Trying 127.0.0.1...
> Connected to 127.0.0.1.
> Escape character is '^]'.
> CONNECT 127.0.0.1:21 HTTP/1.0
> User-Agent: Fred
>
> HTTP/1.0 200 Connection Established
> Proxy-agent: Apache/2.0.16-dev (Unix)
>
> 220 ProFTPD 1.2.0pre10 Server (ProFTPD) [jessica]
> USER ftp
> 331 Anonymous login ok, send your complete e-mail address as password.
> --> at this point the ftp server is killed, and thus the remote
> connection
> is closed. Apache now adds this: >--
> HTTP/1.1 200 OK
> Date: Sun, 08 Apr 2001 15:45:11 GMT
> Server: Apache/2.0.16-dev (Unix)
> Content-Length: 0
> Connection: close
> Content-Type: text/plain; charset=ISO-8859-1
>
> Connection closed by foreign host.
>
> Obviously the CORE filter, having never got to do it's "send headers"
> thing, is jumping in (too late of course) and finishing the job. What
> needs to happen is that the filter be removed or disabled somehow, the
> question is, how?
>
> a) signal somehow to the core filter that it must not send anything - if
> so, how?
>
> b) remove all the filters from the output filter stack, and replace them
> with a NULL filter that does this:
>
> static int null_filter() {
> return APR_SUCCESS;
> }
>
> c) something else?
>
> Regards,
> Graham
> --
> -----------------------------------------
> minfrin@sharp.fm "There's a moon
> over Bourbon Street
> tonight..."
>
>
_______________________________________________________________________________
Ryan Bloom rbb@apache.org
406 29th St.
San Francisco, CA 94131
-------------------------------------------------------------------------------