You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Donald Fernandes <do...@gmail.com> on 2011/04/20 15:00:00 UTC
help with EofSensorInputStream..
I have recently moved from Httpclient 3.1 to HttpClient 4.2, mainly to
resolve the inputstream autoclose problem. I need help in creating the
EofSensorInputStream..
I have a http connection pool amanager using ThreadSafeClientConnManager
using..
when i post an url and get the response using getEntity().getContent() i
believe this method by default closes the stream. and to override this we
need to use the EofSensorInputStream
I'm facing problems in creating the constructor for the same, because it
take ManagedClientConnection as a parameter and i cannot Cast this to
ThreadSafeClientConnManager...
THe only way i see , i can use this is by using BasicEofSensorWatcher..but
the constructor is a tedious process.
EofSensorWatcher watcher = new BasicEofSensorWatcher(new
BasicPooledConnAdapter(httpConnectionPoolMgr.getMgr(), new BasicPoolEntry
(new DefaultClientConnectionOperator(new SchemeRegistry()), new
HttpRoute(new HttpHost("hostname")))) , false );
EofSensorInputStream result = new
EofSensorInputStream(status.getEntity().getContent(), watcher);
But this also gives me a compilation problems. Is there any easier way i can
use this? any help here would be really appreciated.
--
View this message in context: http://old.nabble.com/help-with-EofSensorInputStream..-tp31441020p31441020.html
Sent from the HttpClient-User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: help with EofSensorInputStream..
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, 2011-04-20 at 09:32 -0700, Donald Fernandes wrote:
> Hi oleg,
>
> Let me explain the problem first.
> when using httpClient 3.1 we were using the getResponseBody api, which was
> resulting in huge jvm consumption for large data files and at time throwing
> out OutOfMemory exceptions. A bit on investigation suggested to use
> getResponseBodyAsStream. But after using the API, I was getting
> AutoCloseInputStream exception...
> Attempted read on closed stream.
> java.io.IOException: Attempted read on closed stream.
> at
> org.apache.commons.httpclient.AutoCloseInputStream.isReadAllowed(AutoCloseInputStream.java:183)
> at
> org.apache.commons.httpclient.AutoCloseInputStream.read(AutoCloseInputStream.java:126)
>
> Hence i moved to HttpClient 4.1, but with no luck... i had read that you can
> use the EofSensorInputStream not to close the stream..
>
> any pointers on getting this rectified would be of immense help
>
> Regards,
> Donald
>
I am sorry I do not understand the problem. Either do not close the
stream or do not read from the stream if you have closed it.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: help with EofSensorInputStream..
Posted by Donald Fernandes <do...@gmail.com>.
Hi oleg,
Let me explain the problem first.
when using httpClient 3.1 we were using the getResponseBody api, which was
resulting in huge jvm consumption for large data files and at time throwing
out OutOfMemory exceptions. A bit on investigation suggested to use
getResponseBodyAsStream. But after using the API, I was getting
AutoCloseInputStream exception...
Attempted read on closed stream.
java.io.IOException: Attempted read on closed stream.
at
org.apache.commons.httpclient.AutoCloseInputStream.isReadAllowed(AutoCloseInputStream.java:183)
at
org.apache.commons.httpclient.AutoCloseInputStream.read(AutoCloseInputStream.java:126)
Hence i moved to HttpClient 4.1, but with no luck... i had read that you can
use the EofSensorInputStream not to close the stream..
any pointers on getting this rectified would be of immense help
Regards,
Donald
olegk wrote:
>
> On Wed, 2011-04-20 at 06:32 -0700, Donald Fernandes wrote:
>> Hi Oleg,
>>
>> I want the stream to be open since i passing the response i receive to
>> another method, which parses the stream to do additional work....hence i
>> want it to be open. Later we are explcitly closing the stream...
>>
>> Here is my code snippet ::
>> HttpPool :
>>
>> public void init()
>> {
>> ThreadSafeClientConnManager mgr = new ThreadSafeClientConnManager();
>> mgr.setDefaultMaxPerRoute(MAX_CONNECTION_PERHOST);
>> mgr.setMaxTotal(HTTP_CONNECTIONPOOL_SIZE);
>> mgr.closeIdleConnections(CONNECTION_IDLE_TIME*1000L, TimeUnit.MINUTES);
>> client = new DefaultHttpClient(mgr);
>> // client.getParams().setParameter(HttpMethodParams.RETRY_HANDLER, new
>> DefaultHttpRequestRetryHandler(1,false));
>> client.getParams().setParameter(HttpConnectionParams.SO_TIMEOUT,new
>> Integer(SO_TIMEOUT*60*1000));
>>
>> }
>>
>>
>> Code which uses this :
>>
>> HttpPost postMethod =
>> createPostMethod(PropertyHandler.getStringProperty("URL"));
>> try {
>> HttpEntity requestEntity = new StringEntity(wrdJaxbString); // This is
>> an
>> XML marshlled to String
>> postMethod.setEntity(requestEntity);
>> HttpResponse status = client.execute(postMethod);
>>
>> /* Does not work :( */
>> EofSensorWatcher watcher = new BasicEofSensorWatcher(new
>> BasicPooledConnAdapter(httpConnectionPoolMgr.getMgr(), new BasicPoolEntry
>> (new DefaultClientConnectionOperator(new SchemeRegistry()), new
>> HttpRoute(new HttpHost("hostname")))) , false );
>> EofSensorInputStream result = new
>> EofSensorInputStream(status.getEntity().getContent(), watcher);
>> /* Does not work :( */
>>
>> response = status.getEntity().getContent();
>> } catch (IOException e) {}
>> finally {
>> postMethod.abort();
>> }
>> return response;
>>
>> This is a REST call, the response is sent to another module for further
>> processing of the stream
>>
>>
>
> I still do not understand.
>
> InputStream instance obtained from the response HttpEntity is
> self-contained and is perfectly capable of deallocating all underlying
> resources (such as pooled HTTP connection) by itself. Why do you need to
> mess with EOF sensors and such? Just pass that input stream to the other
> (processing, I assume) module and just make sure it always closes the
> stream once it is done processing it. Resource deallocation will happen
> automatically when InputStream#close() is executed.
>
> Hope this helps
>
> Oleg
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
>
--
View this message in context: http://old.nabble.com/help-with-EofSensorInputStream..-tp31441020p31442799.html
Sent from the HttpClient-User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: help with EofSensorInputStream..
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, 2011-04-20 at 06:32 -0700, Donald Fernandes wrote:
> Hi Oleg,
>
> I want the stream to be open since i passing the response i receive to
> another method, which parses the stream to do additional work....hence i
> want it to be open. Later we are explcitly closing the stream...
>
> Here is my code snippet ::
> HttpPool :
>
> public void init()
> {
> ThreadSafeClientConnManager mgr = new ThreadSafeClientConnManager();
> mgr.setDefaultMaxPerRoute(MAX_CONNECTION_PERHOST);
> mgr.setMaxTotal(HTTP_CONNECTIONPOOL_SIZE);
> mgr.closeIdleConnections(CONNECTION_IDLE_TIME*1000L, TimeUnit.MINUTES);
> client = new DefaultHttpClient(mgr);
> // client.getParams().setParameter(HttpMethodParams.RETRY_HANDLER, new
> DefaultHttpRequestRetryHandler(1,false));
> client.getParams().setParameter(HttpConnectionParams.SO_TIMEOUT,new
> Integer(SO_TIMEOUT*60*1000));
>
> }
>
>
> Code which uses this :
>
> HttpPost postMethod =
> createPostMethod(PropertyHandler.getStringProperty("URL"));
> try {
> HttpEntity requestEntity = new StringEntity(wrdJaxbString); // This is an
> XML marshlled to String
> postMethod.setEntity(requestEntity);
> HttpResponse status = client.execute(postMethod);
>
> /* Does not work :( */
> EofSensorWatcher watcher = new BasicEofSensorWatcher(new
> BasicPooledConnAdapter(httpConnectionPoolMgr.getMgr(), new BasicPoolEntry
> (new DefaultClientConnectionOperator(new SchemeRegistry()), new
> HttpRoute(new HttpHost("hostname")))) , false );
> EofSensorInputStream result = new
> EofSensorInputStream(status.getEntity().getContent(), watcher);
> /* Does not work :( */
>
> response = status.getEntity().getContent();
> } catch (IOException e) {}
> finally {
> postMethod.abort();
> }
> return response;
>
> This is a REST call, the response is sent to another module for further
> processing of the stream
>
>
I still do not understand.
InputStream instance obtained from the response HttpEntity is
self-contained and is perfectly capable of deallocating all underlying
resources (such as pooled HTTP connection) by itself. Why do you need to
mess with EOF sensors and such? Just pass that input stream to the other
(processing, I assume) module and just make sure it always closes the
stream once it is done processing it. Resource deallocation will happen
automatically when InputStream#close() is executed.
Hope this helps
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: help with EofSensorInputStream..
Posted by Donald Fernandes <do...@gmail.com>.
Hi Oleg,
I want the stream to be open since i passing the response i receive to
another method, which parses the stream to do additional work....hence i
want it to be open. Later we are explcitly closing the stream...
Here is my code snippet ::
HttpPool :
public void init()
{
ThreadSafeClientConnManager mgr = new ThreadSafeClientConnManager();
mgr.setDefaultMaxPerRoute(MAX_CONNECTION_PERHOST);
mgr.setMaxTotal(HTTP_CONNECTIONPOOL_SIZE);
mgr.closeIdleConnections(CONNECTION_IDLE_TIME*1000L, TimeUnit.MINUTES);
client = new DefaultHttpClient(mgr);
// client.getParams().setParameter(HttpMethodParams.RETRY_HANDLER, new
DefaultHttpRequestRetryHandler(1,false));
client.getParams().setParameter(HttpConnectionParams.SO_TIMEOUT,new
Integer(SO_TIMEOUT*60*1000));
}
Code which uses this :
HttpPost postMethod =
createPostMethod(PropertyHandler.getStringProperty("URL"));
try {
HttpEntity requestEntity = new StringEntity(wrdJaxbString); // This is an
XML marshlled to String
postMethod.setEntity(requestEntity);
HttpResponse status = client.execute(postMethod);
/* Does not work :( */
EofSensorWatcher watcher = new BasicEofSensorWatcher(new
BasicPooledConnAdapter(httpConnectionPoolMgr.getMgr(), new BasicPoolEntry
(new DefaultClientConnectionOperator(new SchemeRegistry()), new
HttpRoute(new HttpHost("hostname")))) , false );
EofSensorInputStream result = new
EofSensorInputStream(status.getEntity().getContent(), watcher);
/* Does not work :( */
response = status.getEntity().getContent();
} catch (IOException e) {}
finally {
postMethod.abort();
}
return response;
This is a REST call, the response is sent to another module for further
processing of the stream
olegk wrote:
>
> On Wed, 2011-04-20 at 06:00 -0700, Donald Fernandes wrote:
>> I have recently moved from Httpclient 3.1 to HttpClient 4.2, mainly to
>> resolve the inputstream autoclose problem. I need help in creating the
>> EofSensorInputStream..
>>
>> I have a http connection pool amanager using ThreadSafeClientConnManager
>> using..
>>
>> when i post an url and get the response using getEntity().getContent() i
>> believe this method by default closes the stream. and to override this we
>> need to use the EofSensorInputStream
>>
>
> Donald
>
> I am not sure I understand the problem.
>
> What is it _exactly_ you are trying to do and why do you need to prevent
> the content stream from being closed? What is the requirement that
> drives this?
>
> Oleg
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>
>
--
View this message in context: http://old.nabble.com/help-with-EofSensorInputStream..-tp31441020p31441208.html
Sent from the HttpClient-User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org
Re: help with EofSensorInputStream..
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, 2011-04-20 at 06:00 -0700, Donald Fernandes wrote:
> I have recently moved from Httpclient 3.1 to HttpClient 4.2, mainly to
> resolve the inputstream autoclose problem. I need help in creating the
> EofSensorInputStream..
>
> I have a http connection pool amanager using ThreadSafeClientConnManager
> using..
>
> when i post an url and get the response using getEntity().getContent() i
> believe this method by default closes the stream. and to override this we
> need to use the EofSensorInputStream
>
Donald
I am not sure I understand the problem.
What is it _exactly_ you are trying to do and why do you need to prevent
the content stream from being closed? What is the requirement that
drives this?
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org