You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by steve labar <st...@gmail.com> on 2011/11/18 05:48:16 UTC
SSL and HTTPService
I'm trying to create a http/https proxy server leveraging httpclient and
httpcore. I can get the http portion of the proxy to work just fine.
However, when I try to implement the ssl section I have major trouble. It
looks like its failing within HTTPService.handleRequest(). I expect its due
to ssl but hard to really tell what i'm doing wrong. Here is what i'm
currently doing:
Setup socket to listen on 8080
serversocket = new ServerSocket(port);
this.params = new SyncBasicHttpParams();
this.params.setBooleanParameter(ClientPNames.HANDLE_REDIRECTS,
true).setIntParameter(CoreConnectionPNames.SO_TIMEOUT, 50000)
.setIntParameter(CoreConnectionPNames.SOCKET_BUFFER_SIZE,
8 * 1024)
.setBooleanParameter(
CoreConnectionPNames.STALE_CONNECTION_CHECK,
false)
.setBooleanParameter(CoreConnectionPNames.TCP_NODELAY,
true)
.setParameter(CoreProtocolPNames.ORIGIN_SERVER,
"HttpComponents/1.1");
// Set up the HTTP protocol processor
HttpProcessor httpproc = new ImmutableHttpProcessor(
new HttpResponseInterceptor[] { new ResponseDate(),
new ResponseServer(), new ResponseContent(),
new ResponseConnControl() });
// Set up request handlers
HttpRequestHandlerRegistry reqistry = new
HttpRequestHandlerRegistry();
reqistry.register("*", new DefaultHttpRequestHandler());
// Set up the HTTP service
this.httpService = new HttpService(httpproc,
new DefaultConnectionReuseStrategy(),
new DefaultHttpResponseFactory(), reqistry,
this.params);
Then on every incoming accept() on that socket i send that data to a worker
thread that calls httpservice.handlerequest() here is that code snippet:
public void run() {
System.out.println("New connection thread");
HttpContext context = new BasicHttpContext(null);
try {
while (!Thread.interrupted() && this.conn.isOpen()) {
//HttpRequest req = this.conn.receiveRequestHeader();
this.httpservice.handleRequest(this.conn, context);
}
} catch (ConnectionClosedException ex) {
System.err.println("Client closed connection");
} catch (IOException ex) {
System.err.println("I/O error: " + ex.getMessage());
ex.printStackTrace();
} catch (HttpException ex) {
System.err.println("Unrecoverable HTTP protocol violation: "
+ ex.getMessage());
} finally {
try {
this.conn.shutdown();
} catch (IOException ignore) {
}
}
Then each request gets handled by my DefaultHttpRequestHandler which
extends HttpRequestHandler here is the method called anytime
httpService.handleRequest() is called. You will notice I peek at the
http(s) method if not a CONNECT. Its super simple make request to server
and set the response object and let the method handle sending back to
browser on socket listening. The problem comes up if i hit as an example
https://google.com. When i debug it comes in gets set to handle() by the
httpservice.handlerequest(). As soon as it reaches the handle method I peek
and see its a Connect www.google.com:443. So, i make request with
httpclient right here and get a 200 response so i know my proxy server and
the server(google) can accept ssl so at this point i set the response to be
HTTP/1.0 200 Connection established\r\nProxy-agent: proxy client\r\n\r\n"
just as RFC says you do to tell browser we can start talking in ssl. But as
soon as i set this as response and it leaves the method it shuts connection
down and Firefox says:
Secure Connection Failed
An error occurred during a connection to www.google.com.
SSL received a record that exceeded the maximum permissible length.
(Error code: ssl_error_rx_record_too_long)
I have read online and it says that this happens if you try to send ssl to
port 80 but i'm pretty sure that is not problem. Any ideas?
public void handle(HttpRequest request, HttpResponse response,
HttpContext context) throws HttpException, IOException {
System.out.println("Request in handle: " + request);
RequestLine reqLine = request.getRequestLine();
if(reqLine.getMethod().equalsIgnoreCase("CONNECT"))
{
String uri = reqLine.getUri();
String[] uriParts = uri.split(":");
String url ="https://"+ uriParts[0];
request = new BasicHttpRequest("GET",url);
HttpResponse connectRes = getResponse(request);
StatusLine responseLine = connectRes.getStatusLine();
if(responseLine.getStatusCode() == 200)
{
response.setEntity(new BufferedHttpEntity(new
StringEntity("HTTP/1.0 200 Connection established\r\nProxy-agent: proxy
client\r\n\r\n")));
}
response.setEntity(new BufferedHttpEntity(new
StringEntity("HTTP/1.0 200 Connection established\r\nProxy-agent: proxy
client\r\n\r\n")));
}
else
{
try {
HttpResponse clientResponse = null;
HttpEntity entity = null;
clientResponse = httpClient.execute(new
RequestWrapper(request));
entity = clientResponse.getEntity();
if (entity != null) {
response.setEntity(new BufferedHttpEntity(entity));
}
} catch (Exception e) {
System.err.println(e.getMessage());
e.printStackTrace();
}
}
}
Re: SSL and HTTPService
Posted by Bill Speirs <bi...@gmail.com>.
You might be able to learn on your own by doing a bit more research,
maybe this can get you started.
You need to configure the browser so that HTTP traffic goes to your
proxy server on one port, and HTTPS traffic on another port. Proxying
the HTTP traffic is "easy", you just need to be careful with the
hop-by-hop headers.
Handling SSL traffic is slightly harder. Fidder and other such
programs install a self-signed cert into your browser (or system's)
certificate store. This enables them to properly sign request/response
that your browser will trust. This takes a bit more setup and
configuration, but it is doable.
Hope this points you down the right path...
Bill-
On Sun, Nov 20, 2011 at 4:51 PM, steve labar
<st...@gmail.com> wrote:
> So that is why the proxy becomes the endpoint for the browser. The SSL
> handshake is done between the two of them. I know it can be done because
> there are a number of proxy tools like fiddler,web scarab,paros proxy,burp,
> etc that do just this. I wanted to try to create my own for learning and
> practical purposes. Although I do admit it's a little bit over my head thus
> far.
>
> I know the key is to establish an SSL handshake with the browser on a
> specified port. Therefore to it you are considered the endpoint. Once you
> receive the http request you can inspect it and look at it and then
> establish the connection to the origin server on behalf of browser get the
> data than pass it back to the browser just on the trusted channel.
>
> My problem is that initially when I establish the connection it is a
> standard socket. So, when you receive connect request after everything is
> negotiated the browser at some point is going to be passing you encrypted
> data. now because the initial socket listening on 8080 is a normal socket
> I'm assuming it will fail. However, when I tried to switch the socket SSL
> on port 8080 other non-SSL traffic was also throwing exception. So I
> absolutely know I'm doing something wrong just not sure exactly what it is.
> I'm thinking it might be time to possibly higher someone to do that work.
> I'm assuming it would only be an hour to work. Not entirely sure but I
> figure this is the only way I'm going to learn is by having someone else do
> it and look at their work. I'm open for suggestions!
>
> On Sat, Nov 19, 2011 at 10:28 PM, Asankha C. Perera <as...@apache.org>wrote:
>
>> Hi Steve
>>
>> 1. one thing that I might have failed to mention is this proxy needs to be
>>> able to intercept and look at the request before it is being sent to the
>>> origin server. The whole idea behind this proxy is to be a security tool
>>> to
>>> be able to look and manipulate the request that has been sent by the
>>> browser before it gets sent to the origin server. Now having said that in
>>> this case wouldn't the proxy server need to establish an SSL handshake
>>> with
>>> the browser so that the browser will trust and send that encrypted request
>>> and your proxy will be able to decrypt the encrypted request?
>>>
>> The way SSL operates is that end to end the path would be secured from the
>> client making the request to the actual endpoint its talking to. Hence,
>> there is no possibility for the proxy to look at the actual request or
>> manipulate it - as it violates the whole purpose of SSL.
>>
>> I am not sure of your exact requirement - but for example if your clients
>> are within an intranet wanting to talk to an external endpoint, maybe a
>> compromise is that they "explicitly" talk to a well known proxy server over
>> SSL (for security), which can then look at or manipulate the
>> requests/responses and forward them to the external proxy again over
>> *another* SSL connection. Is this acceptable?
>>
>> cheers
>> asankha
>>
>> --
>> Asankha C. Perera
>> AdroitLogic, http://adroitlogic.org
>>
>> http://esbmagic.blogspot.com
>>
>>
>>
>>
>>
>>
>> ------------------------------**------------------------------**---------
>> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
>> For additional commands, e-mail: dev-help@hc.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: SSL and HTTPService
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sun, Nov 20, 2011 at 01:51:54PM -0800, steve labar wrote:
> So that is why the proxy becomes the endpoint for the browser. The SSL
> handshake is done between the two of them. I know it can be done because
> there are a number of proxy tools like fiddler,web scarab,paros proxy,burp,
> etc that do just this. I wanted to try to create my own for learning and
> practical purposes. Although I do admit it's a little bit over my head thus
> far.
>
> I know the key is to establish an SSL handshake with the browser on a
> specified port. Therefore to it you are considered the endpoint. Once you
> receive the http request you can inspect it and look at it and then
> establish the connection to the origin server on behalf of browser get the
> data than pass it back to the browser just on the trusted channel.
>
> My problem is that initially when I establish the connection it is a
> standard socket. So, when you receive connect request after everything is
> negotiated the browser at some point is going to be passing you encrypted
> data. now because the initial socket listening on 8080 is a normal socket
> I'm assuming it will fail. However, when I tried to switch the socket SSL
> on port 8080 other non-SSL traffic was also throwing exception. So I
> absolutely know I'm doing something wrong just not sure exactly what it is.
> I'm thinking it might be time to possibly higher someone to do that work.
> I'm assuming it would only be an hour to work. Not entirely sure but I
> figure this is the only way I'm going to learn is by having someone else do
> it and look at their work. I'm open for suggestions!
>
One cannot intercept SSL traffic and make it completely transparent to the client being eavesdropped upon. HTTP proxies can intercept encrypted data _ONLY_ if clients are OK with being monitored and explicitly trust the proxy. Effectively the proxy needs to intercept SSL/TLS protocol handshake and send its own certificate to the client (most likely generated on-the-fly) instead of that of the origin server. The proxy would have to receive encrypted data from the client, decrypt it, look at the message content, encrypt it and send it to the origin server and the reverse process would have to done with data transmitted in the opposite direction.
One would effectively need to re-implement the entire SSL/TLS protocol handling engine. In my option, that would be an enormously difficult task, especially with Java NIO.
Oleg
> On Sat, Nov 19, 2011 at 10:28 PM, Asankha C. Perera <as...@apache.org>wrote:
>
> > Hi Steve
> >
> > 1. one thing that I might have failed to mention is this proxy needs to be
> >> able to intercept and look at the request before it is being sent to the
> >> origin server. The whole idea behind this proxy is to be a security tool
> >> to
> >> be able to look and manipulate the request that has been sent by the
> >> browser before it gets sent to the origin server. Now having said that in
> >> this case wouldn't the proxy server need to establish an SSL handshake
> >> with
> >> the browser so that the browser will trust and send that encrypted request
> >> and your proxy will be able to decrypt the encrypted request?
> >>
> > The way SSL operates is that end to end the path would be secured from the
> > client making the request to the actual endpoint its talking to. Hence,
> > there is no possibility for the proxy to look at the actual request or
> > manipulate it - as it violates the whole purpose of SSL.
> >
> > I am not sure of your exact requirement - but for example if your clients
> > are within an intranet wanting to talk to an external endpoint, maybe a
> > compromise is that they "explicitly" talk to a well known proxy server over
> > SSL (for security), which can then look at or manipulate the
> > requests/responses and forward them to the external proxy again over
> > *another* SSL connection. Is this acceptable?
> >
> > cheers
> > asankha
> >
> > --
> > Asankha C. Perera
> > AdroitLogic, http://adroitlogic.org
> >
> > http://esbmagic.blogspot.com
> >
> >
> >
> >
> >
> >
> > ------------------------------**------------------------------**---------
> > To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> > For additional commands, e-mail: dev-help@hc.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: SSL and HTTPService
Posted by steve labar <st...@gmail.com>.
So that is why the proxy becomes the endpoint for the browser. The SSL
handshake is done between the two of them. I know it can be done because
there are a number of proxy tools like fiddler,web scarab,paros proxy,burp,
etc that do just this. I wanted to try to create my own for learning and
practical purposes. Although I do admit it's a little bit over my head thus
far.
I know the key is to establish an SSL handshake with the browser on a
specified port. Therefore to it you are considered the endpoint. Once you
receive the http request you can inspect it and look at it and then
establish the connection to the origin server on behalf of browser get the
data than pass it back to the browser just on the trusted channel.
My problem is that initially when I establish the connection it is a
standard socket. So, when you receive connect request after everything is
negotiated the browser at some point is going to be passing you encrypted
data. now because the initial socket listening on 8080 is a normal socket
I'm assuming it will fail. However, when I tried to switch the socket SSL
on port 8080 other non-SSL traffic was also throwing exception. So I
absolutely know I'm doing something wrong just not sure exactly what it is.
I'm thinking it might be time to possibly higher someone to do that work.
I'm assuming it would only be an hour to work. Not entirely sure but I
figure this is the only way I'm going to learn is by having someone else do
it and look at their work. I'm open for suggestions!
On Sat, Nov 19, 2011 at 10:28 PM, Asankha C. Perera <as...@apache.org>wrote:
> Hi Steve
>
> 1. one thing that I might have failed to mention is this proxy needs to be
>> able to intercept and look at the request before it is being sent to the
>> origin server. The whole idea behind this proxy is to be a security tool
>> to
>> be able to look and manipulate the request that has been sent by the
>> browser before it gets sent to the origin server. Now having said that in
>> this case wouldn't the proxy server need to establish an SSL handshake
>> with
>> the browser so that the browser will trust and send that encrypted request
>> and your proxy will be able to decrypt the encrypted request?
>>
> The way SSL operates is that end to end the path would be secured from the
> client making the request to the actual endpoint its talking to. Hence,
> there is no possibility for the proxy to look at the actual request or
> manipulate it - as it violates the whole purpose of SSL.
>
> I am not sure of your exact requirement - but for example if your clients
> are within an intranet wanting to talk to an external endpoint, maybe a
> compromise is that they "explicitly" talk to a well known proxy server over
> SSL (for security), which can then look at or manipulate the
> requests/responses and forward them to the external proxy again over
> *another* SSL connection. Is this acceptable?
>
> cheers
> asankha
>
> --
> Asankha C. Perera
> AdroitLogic, http://adroitlogic.org
>
> http://esbmagic.blogspot.com
>
>
>
>
>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
>
Re: SSL and HTTPService
Posted by "Asankha C. Perera" <as...@apache.org>.
Hi Steve
> 1. one thing that I might have failed to mention is this proxy needs to be
> able to intercept and look at the request before it is being sent to the
> origin server. The whole idea behind this proxy is to be a security tool to
> be able to look and manipulate the request that has been sent by the
> browser before it gets sent to the origin server. Now having said that in
> this case wouldn't the proxy server need to establish an SSL handshake with
> the browser so that the browser will trust and send that encrypted request
> and your proxy will be able to decrypt the encrypted request?
The way SSL operates is that end to end the path would be secured from
the client making the request to the actual endpoint its talking to.
Hence, there is no possibility for the proxy to look at the actual
request or manipulate it - as it violates the whole purpose of SSL.
I am not sure of your exact requirement - but for example if your
clients are within an intranet wanting to talk to an external endpoint,
maybe a compromise is that they "explicitly" talk to a well known proxy
server over SSL (for security), which can then look at or manipulate the
requests/responses and forward them to the external proxy again over
*another* SSL connection. Is this acceptable?
cheers
asankha
--
Asankha C. Perera
AdroitLogic, http://adroitlogic.org
http://esbmagic.blogspot.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: SSL and HTTPService
Posted by steve labar <st...@gmail.com>.
Oleg,
I actually have few questions regarding your response after having thought
about this for a little while:
1. one thing that I might have failed to mention is this proxy needs to be
able to intercept and look at the request before it is being sent to the
origin server. The whole idea behind this proxy is to be a security tool to
be able to look and manipulate the request that has been sent by the
browser before it gets sent to the origin server. Now having said that in
this case wouldn't the proxy server need to establish an SSL handshake with
the browser so that the browser will trust and send that encrypted request
and your proxy will be able to decrypt the encrypted request?
2.if an NIO is going to be the best alternative I decided to use a class
called NHttpSSLServer which I found online that looks to be a good template
for trying to build this proxy that looks to be using Httpcomponents. The
question i I have is regarding how to explicitly implement the connect
request you mention as a part of number four in your response. Initially
upon creating the proxy I used the following snippet of code taken from
that class. The only difference is I use the non-ssl classes for my socket
and listeners:
HttpRequestHandlerRegistry reqistry = new HttpRequestHandlerRegistry();
reqistry.register("*", new DefaultHttpRequestHandler(this));
handler.setHandlerResolver(reqistry);
// Provide an event logger
handler.setEventListener(new EventLogger());
IOEventDispatch ioEventDispatch = new
DefaultServerIOEventDispatch(
handler,
params);
ListeningIOReactor ioReactor = new
DefaultListeningIOReactor(2, params);
try {
ioReactor.listen(new InetSocketAddress(8080));
ioReactor.execute(ioEventDispatch);
} catch (InterruptedIOException ex) {
System.err.println("Interrupted");
} catch (IOException e) {
System.err.println("I/O error: " + e.getMessage());
}
at this point everything over http being sent to my proxy is working
perfectly. Now the hard part that I don't quite fully understand the
"Connect" request. Currently, I have a class called
DefaultHttpRequestHandler this is the class that currently looks at each of
the request that come in. initially when creating the proxy I pass a
reference of the proxy to that DefaultHttpRequestHandler.The idea behind
this is when I see the CONNECT inside my requesthandler i can take the
reference to the proxy and add another listener for the ssl traffic.
However how do i handle sorting what goes to the tunneled socket and what
does not? Still everything will always be coming in from the browser over
that initial socket over 8080 which was set when initially starting proxy.
So then even the newly encrypted traffic will be coming in from the 8080
socket then hitting that DefaultHttpRequestHandler. So do i track the
traffic inside the .handle() and route to that newly created ssl tunel
based on the host i see maybe and if i have created a tunnel w/ that host?
Because the browser on 8080 is not going to be able to ever send to a new
socket its always coming in from 8080. Can you see where i'm not clear?
So to reiterate.
1. Connect gets recognized within my
DefaultHttpRequestHandler.handle(HttpRequest request, HttpResponse response,
HttpContext context) method so i do the following steps.
a. create sslsocket with origin server pulled from CONNECT
request message.
b. if connection can be established now go ahead send the
"HTTP/1.1 200 Connection established\r\nProxy-agent: proxy client\r\n\r\n"
to the browser.
2. Now this is where stuff gets fuzzy. At this point browser is closing the
connection I'm maybe assuming this is due to the fact that i may need to
establish a ssl trusted handshake before it will allow my proxy to receive
its encrypted data? Problem is the browser is sending its data to 8080 and
the serversockket setup is just a DefaultServerIOEventDispatch not a
SSLServerIOEventDispatch. This was what I thought my initial issues were.
but then it seemed like you were stating a normal proxy treats everything
it sees transparently. Which in that case whether encrypted or not is just
data you pass along to origin server. However, in my case i need to be able
to view that encrypted data then if necessary make changes then send to
origin. In essence I establish trusted handshakes from client (browser) to
my server and from my server to origin server.
Is this a big job? I think i'm way over my head here. Maybe I need to hire
out help! Is this a very complex task you think.?
On Fri, Nov 18, 2011 at 11:38 AM, steve labar <steve.labarbera.one@gmail.com
> wrote:
> Thank you for your response. You have got to get an award for how quickly
> and reliably you respond to peoples problems! You my friend deserve a medal!
>
> Thank again,
> Steve
>
>
> On Fri, Nov 18, 2011 at 7:32 AM, Oleg Kalnichevski <ol...@apache.org>wrote:
>
>> On Thu, Nov 17, 2011 at 08:48:16PM -0800, steve labar wrote:
>> > I'm trying to create a http/https proxy server leveraging httpclient and
>> > httpcore. I can get the http portion of the proxy to work just fine.
>> > However, when I try to implement the ssl section I have major trouble.
>> It
>> > looks like its failing within HTTPService.handleRequest(). I expect its
>> due
>> > to ssl but hard to really tell what i'm doing wrong. Here is what i'm
>> > currently doing:
>> >
>> > Setup socket to listen on 8080
>> >
>> > serversocket = new ServerSocket(port);
>> > this.params = new SyncBasicHttpParams();
>> >
>> this.params.setBooleanParameter(ClientPNames.HANDLE_REDIRECTS,
>> > true).setIntParameter(CoreConnectionPNames.SO_TIMEOUT, 50000)
>> >
>> > .setIntParameter(CoreConnectionPNames.SOCKET_BUFFER_SIZE,
>> > 8 * 1024)
>> > .setBooleanParameter(
>> > CoreConnectionPNames.STALE_CONNECTION_CHECK,
>> > false)
>> >
>> .setBooleanParameter(CoreConnectionPNames.TCP_NODELAY,
>> > true)
>> > .setParameter(CoreProtocolPNames.ORIGIN_SERVER,
>> > "HttpComponents/1.1");
>> > // Set up the HTTP protocol processor
>> > HttpProcessor httpproc = new ImmutableHttpProcessor(
>> > new HttpResponseInterceptor[] { new ResponseDate(),
>> > new ResponseServer(), new ResponseContent(),
>> > new ResponseConnControl() });
>> >
>> > // Set up request handlers
>> > HttpRequestHandlerRegistry reqistry = new
>> > HttpRequestHandlerRegistry();
>> > reqistry.register("*", new DefaultHttpRequestHandler());
>> > // Set up the HTTP service
>> > this.httpService = new HttpService(httpproc,
>> > new DefaultConnectionReuseStrategy(),
>> > new DefaultHttpResponseFactory(), reqistry,
>> > this.params);
>> >
>> > Then on every incoming accept() on that socket i send that data to a
>> worker
>> > thread that calls httpservice.handlerequest() here is that code snippet:
>> >
>> > public void run() {
>> > System.out.println("New connection thread");
>> > HttpContext context = new BasicHttpContext(null);
>> >
>> > try {
>> > while (!Thread.interrupted() && this.conn.isOpen()) {
>> > //HttpRequest req =
>> this.conn.receiveRequestHeader();
>> > this.httpservice.handleRequest(this.conn, context);
>> > }
>> > } catch (ConnectionClosedException ex) {
>> > System.err.println("Client closed connection");
>> > } catch (IOException ex) {
>> > System.err.println("I/O error: " + ex.getMessage());
>> > ex.printStackTrace();
>> > } catch (HttpException ex) {
>> > System.err.println("Unrecoverable HTTP protocol
>> violation: "
>> > + ex.getMessage());
>> > } finally {
>> > try {
>> > this.conn.shutdown();
>> > } catch (IOException ignore) {
>> > }
>> > }
>> >
>> > Then each request gets handled by my DefaultHttpRequestHandler which
>> > extends HttpRequestHandler here is the method called anytime
>> > httpService.handleRequest() is called. You will notice I peek at the
>> > http(s) method if not a CONNECT. Its super simple make request to server
>> > and set the response object and let the method handle sending back to
>> > browser on socket listening. The problem comes up if i hit as an example
>> > https://google.com. When i debug it comes in gets set to handle() by
>> the
>> > httpservice.handlerequest(). As soon as it reaches the handle method I
>> peek
>> > and see its a Connect www.google.com:443. So, i make request with
>> > httpclient right here and get a 200 response so i know my proxy server
>> and
>> > the server(google) can accept ssl so at this point i set the response
>> to be
>> > HTTP/1.0 200 Connection established\r\nProxy-agent: proxy
>> client\r\n\r\n"
>> > just as RFC says you do to tell browser we can start talking in ssl.
>> But as
>> > soon as i set this as response and it leaves the method it shuts
>> connection
>> > down and Firefox says:
>> >
>> > Secure Connection Failed
>> >
>> > An error occurred during a connection to www.google.com.
>> >
>> > SSL received a record that exceeded the maximum permissible length.
>> >
>> > (Error code: ssl_error_rx_record_too_long)
>> >
>> > I have read online and it says that this happens if you try to send ssl
>> to
>> > port 80 but i'm pretty sure that is not problem. Any ideas?
>> >
>>
>> Hi Steve
>>
>> SSL tunneling via an HTTP proxy simply does not work like that. Upon
>> receipt of a CONNECT request, the proxy is expected to do the following:
>> (1) optionally authenticate the user and respond with 407 status if user
>> credentials are not valid
>> (2) open a TCP connection to the origin server, but send NO data to the
>> origin
>> (3) if connection is successful, respond to the client with 200 status
>> (4) at this point the tunnel can be considered established and
>> operational. From hence on the proxy is expected to read whatever data
>> comes from the client and relay it to the origin and the other way around.
>> The content passed between the client and the origin is completely opaque
>> to the proxy.
>>
>> For proper specification of the protocol please see
>>
>> http://www.ietf.org/rfc/rfc2817.txt
>>
>> The trouble is that connection tunneling is not easy to implement with
>> the classic (blocking) I/O. One would need to employ two treads to be able
>> to read and write in both directions simultaneously or use one thread and
>> constantly switch between read and write mode which may be not very
>> efficient. Usually NIO tends to be a better model for proxies.
>>
>> Oleg
>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
>> For additional commands, e-mail: dev-help@hc.apache.org
>>
>>
>
Re: SSL and HTTPService
Posted by steve labar <st...@gmail.com>.
Thank you for your response. You have got to get an award for how quickly
and reliably you respond to peoples problems! You my friend deserve a medal!
Thank again,
Steve
On Fri, Nov 18, 2011 at 7:32 AM, Oleg Kalnichevski <ol...@apache.org> wrote:
> On Thu, Nov 17, 2011 at 08:48:16PM -0800, steve labar wrote:
> > I'm trying to create a http/https proxy server leveraging httpclient and
> > httpcore. I can get the http portion of the proxy to work just fine.
> > However, when I try to implement the ssl section I have major trouble. It
> > looks like its failing within HTTPService.handleRequest(). I expect its
> due
> > to ssl but hard to really tell what i'm doing wrong. Here is what i'm
> > currently doing:
> >
> > Setup socket to listen on 8080
> >
> > serversocket = new ServerSocket(port);
> > this.params = new SyncBasicHttpParams();
> >
> this.params.setBooleanParameter(ClientPNames.HANDLE_REDIRECTS,
> > true).setIntParameter(CoreConnectionPNames.SO_TIMEOUT, 50000)
> >
> > .setIntParameter(CoreConnectionPNames.SOCKET_BUFFER_SIZE,
> > 8 * 1024)
> > .setBooleanParameter(
> > CoreConnectionPNames.STALE_CONNECTION_CHECK,
> > false)
> >
> .setBooleanParameter(CoreConnectionPNames.TCP_NODELAY,
> > true)
> > .setParameter(CoreProtocolPNames.ORIGIN_SERVER,
> > "HttpComponents/1.1");
> > // Set up the HTTP protocol processor
> > HttpProcessor httpproc = new ImmutableHttpProcessor(
> > new HttpResponseInterceptor[] { new ResponseDate(),
> > new ResponseServer(), new ResponseContent(),
> > new ResponseConnControl() });
> >
> > // Set up request handlers
> > HttpRequestHandlerRegistry reqistry = new
> > HttpRequestHandlerRegistry();
> > reqistry.register("*", new DefaultHttpRequestHandler());
> > // Set up the HTTP service
> > this.httpService = new HttpService(httpproc,
> > new DefaultConnectionReuseStrategy(),
> > new DefaultHttpResponseFactory(), reqistry,
> > this.params);
> >
> > Then on every incoming accept() on that socket i send that data to a
> worker
> > thread that calls httpservice.handlerequest() here is that code snippet:
> >
> > public void run() {
> > System.out.println("New connection thread");
> > HttpContext context = new BasicHttpContext(null);
> >
> > try {
> > while (!Thread.interrupted() && this.conn.isOpen()) {
> > //HttpRequest req = this.conn.receiveRequestHeader();
> > this.httpservice.handleRequest(this.conn, context);
> > }
> > } catch (ConnectionClosedException ex) {
> > System.err.println("Client closed connection");
> > } catch (IOException ex) {
> > System.err.println("I/O error: " + ex.getMessage());
> > ex.printStackTrace();
> > } catch (HttpException ex) {
> > System.err.println("Unrecoverable HTTP protocol
> violation: "
> > + ex.getMessage());
> > } finally {
> > try {
> > this.conn.shutdown();
> > } catch (IOException ignore) {
> > }
> > }
> >
> > Then each request gets handled by my DefaultHttpRequestHandler which
> > extends HttpRequestHandler here is the method called anytime
> > httpService.handleRequest() is called. You will notice I peek at the
> > http(s) method if not a CONNECT. Its super simple make request to server
> > and set the response object and let the method handle sending back to
> > browser on socket listening. The problem comes up if i hit as an example
> > https://google.com. When i debug it comes in gets set to handle() by the
> > httpservice.handlerequest(). As soon as it reaches the handle method I
> peek
> > and see its a Connect www.google.com:443. So, i make request with
> > httpclient right here and get a 200 response so i know my proxy server
> and
> > the server(google) can accept ssl so at this point i set the response to
> be
> > HTTP/1.0 200 Connection established\r\nProxy-agent: proxy client\r\n\r\n"
> > just as RFC says you do to tell browser we can start talking in ssl. But
> as
> > soon as i set this as response and it leaves the method it shuts
> connection
> > down and Firefox says:
> >
> > Secure Connection Failed
> >
> > An error occurred during a connection to www.google.com.
> >
> > SSL received a record that exceeded the maximum permissible length.
> >
> > (Error code: ssl_error_rx_record_too_long)
> >
> > I have read online and it says that this happens if you try to send ssl
> to
> > port 80 but i'm pretty sure that is not problem. Any ideas?
> >
>
> Hi Steve
>
> SSL tunneling via an HTTP proxy simply does not work like that. Upon
> receipt of a CONNECT request, the proxy is expected to do the following:
> (1) optionally authenticate the user and respond with 407 status if user
> credentials are not valid
> (2) open a TCP connection to the origin server, but send NO data to the
> origin
> (3) if connection is successful, respond to the client with 200 status
> (4) at this point the tunnel can be considered established and
> operational. From hence on the proxy is expected to read whatever data
> comes from the client and relay it to the origin and the other way around.
> The content passed between the client and the origin is completely opaque
> to the proxy.
>
> For proper specification of the protocol please see
>
> http://www.ietf.org/rfc/rfc2817.txt
>
> The trouble is that connection tunneling is not easy to implement with the
> classic (blocking) I/O. One would need to employ two treads to be able to
> read and write in both directions simultaneously or use one thread and
> constantly switch between read and write mode which may be not very
> efficient. Usually NIO tends to be a better model for proxies.
>
> Oleg
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
>
Re: SSL and HTTPService
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Thu, Nov 17, 2011 at 08:48:16PM -0800, steve labar wrote:
> I'm trying to create a http/https proxy server leveraging httpclient and
> httpcore. I can get the http portion of the proxy to work just fine.
> However, when I try to implement the ssl section I have major trouble. It
> looks like its failing within HTTPService.handleRequest(). I expect its due
> to ssl but hard to really tell what i'm doing wrong. Here is what i'm
> currently doing:
>
> Setup socket to listen on 8080
>
> serversocket = new ServerSocket(port);
> this.params = new SyncBasicHttpParams();
> this.params.setBooleanParameter(ClientPNames.HANDLE_REDIRECTS,
> true).setIntParameter(CoreConnectionPNames.SO_TIMEOUT, 50000)
>
> .setIntParameter(CoreConnectionPNames.SOCKET_BUFFER_SIZE,
> 8 * 1024)
> .setBooleanParameter(
> CoreConnectionPNames.STALE_CONNECTION_CHECK,
> false)
> .setBooleanParameter(CoreConnectionPNames.TCP_NODELAY,
> true)
> .setParameter(CoreProtocolPNames.ORIGIN_SERVER,
> "HttpComponents/1.1");
> // Set up the HTTP protocol processor
> HttpProcessor httpproc = new ImmutableHttpProcessor(
> new HttpResponseInterceptor[] { new ResponseDate(),
> new ResponseServer(), new ResponseContent(),
> new ResponseConnControl() });
>
> // Set up request handlers
> HttpRequestHandlerRegistry reqistry = new
> HttpRequestHandlerRegistry();
> reqistry.register("*", new DefaultHttpRequestHandler());
> // Set up the HTTP service
> this.httpService = new HttpService(httpproc,
> new DefaultConnectionReuseStrategy(),
> new DefaultHttpResponseFactory(), reqistry,
> this.params);
>
> Then on every incoming accept() on that socket i send that data to a worker
> thread that calls httpservice.handlerequest() here is that code snippet:
>
> public void run() {
> System.out.println("New connection thread");
> HttpContext context = new BasicHttpContext(null);
>
> try {
> while (!Thread.interrupted() && this.conn.isOpen()) {
> //HttpRequest req = this.conn.receiveRequestHeader();
> this.httpservice.handleRequest(this.conn, context);
> }
> } catch (ConnectionClosedException ex) {
> System.err.println("Client closed connection");
> } catch (IOException ex) {
> System.err.println("I/O error: " + ex.getMessage());
> ex.printStackTrace();
> } catch (HttpException ex) {
> System.err.println("Unrecoverable HTTP protocol violation: "
> + ex.getMessage());
> } finally {
> try {
> this.conn.shutdown();
> } catch (IOException ignore) {
> }
> }
>
> Then each request gets handled by my DefaultHttpRequestHandler which
> extends HttpRequestHandler here is the method called anytime
> httpService.handleRequest() is called. You will notice I peek at the
> http(s) method if not a CONNECT. Its super simple make request to server
> and set the response object and let the method handle sending back to
> browser on socket listening. The problem comes up if i hit as an example
> https://google.com. When i debug it comes in gets set to handle() by the
> httpservice.handlerequest(). As soon as it reaches the handle method I peek
> and see its a Connect www.google.com:443. So, i make request with
> httpclient right here and get a 200 response so i know my proxy server and
> the server(google) can accept ssl so at this point i set the response to be
> HTTP/1.0 200 Connection established\r\nProxy-agent: proxy client\r\n\r\n"
> just as RFC says you do to tell browser we can start talking in ssl. But as
> soon as i set this as response and it leaves the method it shuts connection
> down and Firefox says:
>
> Secure Connection Failed
>
> An error occurred during a connection to www.google.com.
>
> SSL received a record that exceeded the maximum permissible length.
>
> (Error code: ssl_error_rx_record_too_long)
>
> I have read online and it says that this happens if you try to send ssl to
> port 80 but i'm pretty sure that is not problem. Any ideas?
>
Hi Steve
SSL tunneling via an HTTP proxy simply does not work like that. Upon receipt of a CONNECT request, the proxy is expected to do the following:
(1) optionally authenticate the user and respond with 407 status if user credentials are not valid
(2) open a TCP connection to the origin server, but send NO data to the origin
(3) if connection is successful, respond to the client with 200 status
(4) at this point the tunnel can be considered established and operational. From hence on the proxy is expected to read whatever data comes from the client and relay it to the origin and the other way around. The content passed between the client and the origin is completely opaque to the proxy.
For proper specification of the protocol please see
http://www.ietf.org/rfc/rfc2817.txt
The trouble is that connection tunneling is not easy to implement with the classic (blocking) I/O. One would need to employ two treads to be able to read and write in both directions simultaneously or use one thread and constantly switch between read and write mode which may be not very efficient. Usually NIO tends to be a better model for proxies.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org