RE: [HttpClient] getting the http connection or setting the param s

[Gustavo Hexsel <gh...@sagebrushcorp.com>]

  Hi Michael,

  thanks for the prompt answer.

  By connection params I meant the HttpConnection.setParams().  I saw the
javadoc for that class, and it states that if I set SO_TIMEOUT there, it
will change the value for open sockets as well.  If I just set the method
parameters, by calling HttpBaseMethod.setParams(), it will change the
parameter only when opening new sockets.  Why do I need to set the
SO_TIMEOUT again?  Because I have, let's say, 10 secs to retrieve the whole
page, including any redirection I might need.  So for connection
establishment, I have 10 secs, but if it takes 5 secs to connect, I only
have 5 secs left for the first read on the input stream.  Again, if it takes
2 secs to read, I only have 3 seconds for all subsequent reads, until all
redirections are followed and the page is fetched, or until I time out.

  Below is my method's code (I simplified a little, and the format might be
little odd here):

private String executeMethod(
    String urlString,
    HttpMethodBase method,
    ITimeoutChecker timeoutChecker,
    HttpState state
    ) throws IOException, TimeoutException {

	HttpURL url = new HttpURL(urlString);
	method.setURI(url);
	method.setFollowRedirects(false);
	method.getParams().setSoTimeout(remainingTime);

	HostConfiguration hostConfig = new HostConfiguration();
	hostConfig.setHost(url);
	method.setHostConfiguration(hostConfig);

	int statusCode = client.executeMethod(hostConfig, method, state);
	String pageContent;
	if (isRedirect(statusCode)) {
		if (timeoutChecker.isTimeout()) {
			throw new TimeoutException("Total execution time for
fetch exceeded timeout parameter");
		} else {
			Header locationHeader =
method.getResponseHeader("location");
			HttpURL nextLocation = new
HttpURL(locationHeader.getValue().toCharArray());
			pageContent = fetchGet(nextLocation.getEscapedURI(),
addressHolder, timeoutChecker, state);
		}
	} else if (isSuccess(statusCode)) {
		// at least 4K buffers, might be as big as the webpage
		int responseSize = Math.max(getResponseSize(method),
DEFAULT_RESPONSE_SIZE);
		InputStream response = method.getResponseBodyAsStream();
		if (response != null) {
			ByteArrayOutputStream outstream = new
ByteArrayOutputStream(responseSize);
			byte[] buffer = new byte[responseSize];
			int len;
			while (((len = response.read(buffer)) > 0) &&
!timeoutChecker.isTimeout()) {
				outstream.write(buffer, 0, len);
			}
			outstream.close();
			pageContent = EncodingUtil.getString(
					outstream.toByteArray(),
					method.getResponseCharSet()
			);
			response.close();
		} else {
			throw new HttpException("No response stream was
available");
		}
	} else {
		throw new HttpException("HTTP status returned was not
success nor redirect. It was: " + statusCode);
	}
	return pageContent;
}


-----Original Message-----
From: Michael Becke [mailto:becke@u.washington.edu]
Sent: October 7, 2004 7:35 PM
To: Jakarta Commons Users List
Subject: Re: [HttpClient] getting the http connection or setting the
params


Hi Gustavo,

Not sure what you mean by "setting the connection parameters".  All 
timeout params available on the connection are configurable via some 
HttpClient param.  Which timeout do you want to set?

Though it is possible to get access to the actual connection it is 
highly discouraged.

It sounds like you may want a method to abort a request after some 
time.  HttpClient does not have a built-in "request timeout" param, but 
it can be simulated.  You would need a thread separate from the one 
executing the method that keeps track of the method execution time.  If 
it went over a timeout value you could call HttpMethod.abort().

Mike

On Oct 7, 2004, at 6:17 PM, Gustavo Hexsel wrote:

>   I'm using HttpClient to read a variable number of pages in sequence. 
>  I
> have a "time frame" by which the page accesses (method execution and 
> all the
> input stream reads) have to be done.  I tried calling
>
> method.getParams().setSoTimeout(remainingTime)
>
>   but it only sets the timeout once, just before opening the 
> connection.  Is
> there a way of getting the http connection behind an executing http 
> method?
> Or setting the connection parameters?
>
>   Thank you!
>
>     []s Gustavo
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-user-help@jakarta.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org

RE: [HttpClient] getting the http connection or setting the param s

[Oleg Kalnichevski <ol...@bluewin.ch>]

On Fri, 2004-10-08 at 17:41, Gustavo Hexsel wrote:
>   Hi Michael,
> 
>   thanks for the prompt answer.
> 
>   By connection params I meant the HttpConnection.setParams().  I saw the
> javadoc for that class, and it states that if I set SO_TIMEOUT there, it
> will change the value for open sockets as well.  If I just set the method
> parameters, by calling HttpBaseMethod.setParams(), it will change the
> parameter only when opening new sockets. 

Gustavo,
This is not the case. HttpClient resets SO_TIMEOUT every time the
connection is obtained from the connection manager, because the
connection can be still left open ('alive') and may contain an arbitrary
SO_TIMEOUT value set by the previous method. 

When SO_TIMEOUT is defined at the HTTP method, the method value is used,
otherwise, default value defined at the HTTP connection/connection
manager level is used.

http://jakarta.apache.org/commons/httpclient/3.0/xref/org/apache/commons/httpclient/HttpMethodDirector.html#379

Oleg


---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org