You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Arya <ar...@gmail.com> on 2012/09/08 23:11:17 UTC
httpclient socks dns not working correctly
Hello
When using socks proxy with httpclient the dns gets resolved on the local
machine instead of on the proxy machine. socks 5 should not be behaving
like this and it causes security issues. When I use TOR with httpclient set
to use it as a SOCKS proxy TOR gives out warning messages about DNS not
being resolved on the remote proxy server which it should, but when I use
Java's native socks proxy support everything works as expected with no
warning messages about dns not being resolved on the proxy server. Are
there any plans to fix this issue?
Regards!
Re: Servlet getInputStream getting truncated
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Mon, 2012-11-05 at 15:06 -0600, Godbey, David J. (HQ-LM020)[DIGITAL
MANAGEMENT INC.] wrote:
> This is probably a good place to pose this question. I'm guessing someone will know what is going on, and perhaps http-client has a solution.
>
> You will recall that recently I created a "proxy" servlet to deal with NTLMv2 authentication because not java nor jaxws can manage NTLMv2 authentication. http-client provides the authentication layer for me, and that is working perfectly.
>
> Now I have a new problem. When the servlet payload gets too large (and not all that large, mind you) the input stream gets truncated and the Exchange EWS request fails. Here are the details, see code below if needed:
>
> The request.getContentLenght() method returns 9,219 bytes.
> When I unload the request.getInputStream() into the buffer and count the bytes the logger.debug spits out, I count 7,776 bytes. And indeed, the SOAP XML package is malformed. What happened to the other 1,443 bytes?
>
> App server=JBoss 5.1.
>
> Anyone know what is going on and how to fix it?
> Thanks,
> Dave
>
> public void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
> response.setContentType(CONTENT_XML);
> PrintWriter out = response.getWriter();
> try {
> int len = request.getContentLength();
> byte[ ] buf = new byte[len];
> _logger.debug("SOAP request length="+len+" buffer size="+buf.length);
> BufferedInputStream in = new BufferedInputStream(request.getInputStream(), len);
> in.read(buf, 0, len);
I believe the way you are reading message content is wrong.
InputStream#read() method is not guaranteed to return the request amount
of data. The operation can terminate as soon as there is _some_ data to
be read
0 < n <= len
Not to mention that request#getContentLength() can return -1 in case the
message body is chunk coded.
So, one must read from the stream in a loop until a read operation
terminates with -1 (end of stream).
> _logger.debug("SOAP resquest :");
> _logger.debug(new String(buf));
> String soap = doEws(buf, request);
> _logger.debug("SOAP response :");
> _logger.debug(soap);
> out.print(soap);
> } catch (Exception e) {
> throw new RuntimeException(e);
> } finally {
> out.close();
> }
> }
>
> private String doEws(byte[] inSoap, HttpServletRequest r) {
> DefaultHttpClient httpclient = new DefaultHttpClient();
> String soap = "";
> try {
> String localIp = Inet4Address.getLocalHost().getHostAddress();
> HttpHost httpHost = new HttpHost(_host, 443, "https");
> httpclient.getCredentialsProvider().setCredentials(new
> AuthScope(httpHost), new NTCredentials(_user, _password,
> localIp, _domain));
>
> AuthCache authCache = new BasicAuthCache();
> HttpParams params = new BasicHttpParams();
> HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
> HttpProtocolParams.setContentCharset(params, "UTF-8");
> HttpProtocolParams.setUseExpectContinue(params, true);
>
> NTLMSchemeFactory nsf = new NTLMSchemeFactory();
> AuthScheme authScheme = nsf.newInstance(params);
> authCache.put(httpHost, authScheme);
>
NTLM auth scheme cannot be used preemptively. There is no point caching
it.
> // Add AuthCache to the execution context
> BasicHttpContext localcontext = new BasicHttpContext();
> localcontext.setAttribute(ClientContext.AUTH_CACHE, authCache);
>
> HttpPost http = new HttpPost(_serviceEndpoint);
> for (Enumeration en=r.getHeaderNames();en.hasMoreElements();) {
> String name = (String) en.nextElement();
> if (!(name.equals("host") || name.equals("authorization") ||
> name.equals("content-length"))) {
> _logger.debug("Adding header="+name);
> try {
> http.addHeader(name, r.getHeader(name));
> } catch (Exception e) {
> // Don't worry, keep going
> }
> }
> }
> http.setEntity(new ByteArrayEntity(inSoap));
>
> HttpResponse response = httpclient.execute(httpHost, http,
> localcontext);
> HttpEntity entity = response.getEntity();
>
> if (entity != null) {
> soap = EntityUtils.toString(entity);
> }
> EntityUtils.consume(entity);
>
> } catch (Exception e) {
> throw new RuntimeException(e);
> } finally {
> httpclient.getConnectionManager().shutdown();
> }
> return soap;
> }
>
You might want to consider reusing HttpClient instance in order to
improve performance and resource utilization.
Hope this helps
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Servlet getInputStream getting truncated
Posted by "Godbey, David J. (HQ-LM020)[DIGITAL MANAGEMENT INC.]" <da...@nasa.gov>.
This is probably a good place to pose this question. I'm guessing someone will know what is going on, and perhaps http-client has a solution.
You will recall that recently I created a "proxy" servlet to deal with NTLMv2 authentication because not java nor jaxws can manage NTLMv2 authentication. http-client provides the authentication layer for me, and that is working perfectly.
Now I have a new problem. When the servlet payload gets too large (and not all that large, mind you) the input stream gets truncated and the Exchange EWS request fails. Here are the details, see code below if needed:
The request.getContentLenght() method returns 9,219 bytes.
When I unload the request.getInputStream() into the buffer and count the bytes the logger.debug spits out, I count 7,776 bytes. And indeed, the SOAP XML package is malformed. What happened to the other 1,443 bytes?
App server=JBoss 5.1.
Anyone know what is going on and how to fix it?
Thanks,
Dave
public void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
response.setContentType(CONTENT_XML);
PrintWriter out = response.getWriter();
try {
int len = request.getContentLength();
byte[ ] buf = new byte[len];
_logger.debug("SOAP request length="+len+" buffer size="+buf.length);
BufferedInputStream in = new BufferedInputStream(request.getInputStream(), len);
in.read(buf, 0, len);
_logger.debug("SOAP resquest :");
_logger.debug(new String(buf));
String soap = doEws(buf, request);
_logger.debug("SOAP response :");
_logger.debug(soap);
out.print(soap);
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
out.close();
}
}
private String doEws(byte[] inSoap, HttpServletRequest r) {
DefaultHttpClient httpclient = new DefaultHttpClient();
String soap = "";
try {
String localIp = Inet4Address.getLocalHost().getHostAddress();
HttpHost httpHost = new HttpHost(_host, 443, "https");
httpclient.getCredentialsProvider().setCredentials(new
AuthScope(httpHost), new NTCredentials(_user, _password,
localIp, _domain));
AuthCache authCache = new BasicAuthCache();
HttpParams params = new BasicHttpParams();
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
HttpProtocolParams.setContentCharset(params, "UTF-8");
HttpProtocolParams.setUseExpectContinue(params, true);
NTLMSchemeFactory nsf = new NTLMSchemeFactory();
AuthScheme authScheme = nsf.newInstance(params);
authCache.put(httpHost, authScheme);
// Add AuthCache to the execution context
BasicHttpContext localcontext = new BasicHttpContext();
localcontext.setAttribute(ClientContext.AUTH_CACHE, authCache);
HttpPost http = new HttpPost(_serviceEndpoint);
for (Enumeration en=r.getHeaderNames();en.hasMoreElements();) {
String name = (String) en.nextElement();
if (!(name.equals("host") || name.equals("authorization") ||
name.equals("content-length"))) {
_logger.debug("Adding header="+name);
try {
http.addHeader(name, r.getHeader(name));
} catch (Exception e) {
// Don't worry, keep going
}
}
}
http.setEntity(new ByteArrayEntity(inSoap));
HttpResponse response = httpclient.execute(httpHost, http,
localcontext);
HttpEntity entity = response.getEntity();
if (entity != null) {
soap = EntityUtils.toString(entity);
}
EntityUtils.consume(entity);
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
httpclient.getConnectionManager().shutdown();
}
return soap;
}
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: httpclient socks dns not working correctly
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sun, 2012-09-09 at 18:53 -0400, Arya wrote:
> Hi Oleg
>
> I could be mistaken but as far as I know HTTPClient has built in socks
> support since version 4? I use HTMLUnit and one of the devs said that they
> were able to add socks support to HTMLUnit after upgrading to httpclient 4.
> Also take a look at this example
> http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/httpclient/src/examples/org/apache/http/examples/client/ClientExecuteSOCKS.java
>
HttpClient 4.x made it easier to customize the process of network socket
initialization. One can leverage this mechanism to apply custom SOCKS
configurations, but otherwise HttpClient 4.x remains completely SOCKS
unaware.
Oleg
> On Sun, Sep 9, 2012 at 1:52 PM, Oleg Kalnichevski <ol...@apache.org> wrote:
>
> > On Sat, 2012-09-08 at 17:11 -0400, Arya wrote:
> > > Hello
> > >
> > > When using socks proxy with httpclient the dns gets resolved on the local
> > > machine instead of on the proxy machine. socks 5 should not be behaving
> > > like this and it causes security issues. When I use TOR with httpclient
> > set
> > > to use it as a SOCKS proxy TOR gives out warning messages about DNS not
> > > being resolved on the remote proxy server which it should, but when I use
> > > Java's native socks proxy support everything works as expected with no
> > > warning messages about dns not being resolved on the proxy server. Are
> > > there any plans to fix this issue?
> > >
> > > Regards!
> >
> > SOCKS is a TCP/IP level proxy mechanism, not an HTTP level one.
> > HttpClient doesn't have any SOCKS specific logic and it simply relies on
> > the functionality provided by JRE. I'll happily review and commit a
> > patch improving SOCKS support in HttpClient but I have no good knowledge
> > of the SOCKS protocol myself and have no intention of investing my time
> > into learning it.
> >
> > Oleg
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: httpclient socks dns not working correctly
Posted by Arya <ar...@gmail.com>.
Hi Oleg
I could be mistaken but as far as I know HTTPClient has built in socks
support since version 4? I use HTMLUnit and one of the devs said that they
were able to add socks support to HTMLUnit after upgrading to httpclient 4.
Also take a look at this example
http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/httpclient/src/examples/org/apache/http/examples/client/ClientExecuteSOCKS.java
On Sun, Sep 9, 2012 at 1:52 PM, Oleg Kalnichevski <ol...@apache.org> wrote:
> On Sat, 2012-09-08 at 17:11 -0400, Arya wrote:
> > Hello
> >
> > When using socks proxy with httpclient the dns gets resolved on the local
> > machine instead of on the proxy machine. socks 5 should not be behaving
> > like this and it causes security issues. When I use TOR with httpclient
> set
> > to use it as a SOCKS proxy TOR gives out warning messages about DNS not
> > being resolved on the remote proxy server which it should, but when I use
> > Java's native socks proxy support everything works as expected with no
> > warning messages about dns not being resolved on the proxy server. Are
> > there any plans to fix this issue?
> >
> > Regards!
>
> SOCKS is a TCP/IP level proxy mechanism, not an HTTP level one.
> HttpClient doesn't have any SOCKS specific logic and it simply relies on
> the functionality provided by JRE. I'll happily review and commit a
> patch improving SOCKS support in HttpClient but I have no good knowledge
> of the SOCKS protocol myself and have no intention of investing my time
> into learning it.
>
> Oleg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
Re: httpclient socks dns not working correctly
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sat, 2012-09-08 at 17:11 -0400, Arya wrote:
> Hello
>
> When using socks proxy with httpclient the dns gets resolved on the local
> machine instead of on the proxy machine. socks 5 should not be behaving
> like this and it causes security issues. When I use TOR with httpclient set
> to use it as a SOCKS proxy TOR gives out warning messages about DNS not
> being resolved on the remote proxy server which it should, but when I use
> Java's native socks proxy support everything works as expected with no
> warning messages about dns not being resolved on the proxy server. Are
> there any plans to fix this issue?
>
> Regards!
SOCKS is a TCP/IP level proxy mechanism, not an HTTP level one.
HttpClient doesn't have any SOCKS specific logic and it simply relies on
the functionality provided by JRE. I'll happily review and commit a
patch improving SOCKS support in HttpClient but I have no good knowledge
of the SOCKS protocol myself and have no intention of investing my time
into learning it.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org