Thread safety and resource usage of AbderaClient

[Morten Bendiksen <MB...@vizrt.com>]

Hi,

I am using Abdera in a project where different threads use the AbderaClient to read and post ducuments from/to a server. I am wondering what are the recommended way to do this safely and without consuming too many resources. Can I safely share one instance of AberaClient between multiple threads? Should I always call the ClientResponse.release() method after I'm done with a response?

When I blindly followed the Getting started guide, I quickly ran into problems of performance and even ran out of connections after a while. Now I call the .teardown() method when I'm no longer using the AbderaClient, and things improved. I could not find any documentation about this though, so I am therefore asking here, since I really want to ensure I'm doing things the right way, so I don't bump into other problems later.

Best regards,
Morten Bendiksen

RE: Thread safety and resource usage of AbderaClient

[Morten Bendiksen <MB...@vizrt.com>]

Good to know, thanks for the answer.

I did write up a test to check what happened with multiple AbderaClient instances. I got some weird results. On some machines it crashes with a "SocketException: No buffer space available" after it has run through all available ports. But on others it works fine and just cycles the available ports over and over. I made a selfcontained project with a mock http server, so I know it is not the servers fault. Here is the relevant code.

        Abdera abdera = Abdera.getInstance();
        for (int i = 0; i < 10000000; i++)
        {
            AbderaClient client = new AbderaClient(abdera);
            ClientResponse res = client.get("http://localhost:8580/");
            if (res.getStatus() == 200)
                System.out.println("OK");
            else
                System.out.println("NOT OK!");
            res.release();
            client.teardown();
        }

Is there something I am doing wrong here or might there be a bug?

On the classpath I have:
abdera-1.1.2.jar
axiom.jar
commons-codec-1.3.jar
commons-httpclient-3.0.1.jar
commons-logging-1.0.4.jar
simple-4.1.21.jar(for a mock http-server)

- Morten

-----Original Message-----
From: James M Snell [mailto:jasnell@gmail.com] 
Sent: 24. mai 2012 20:35
To: user@abdera.apache.org
Subject: Re: Thread safety and resource usage of AbderaClient

On Thu, May 24, 2012 at 3:36 AM, Morten Bendiksen <MB...@vizrt.com> wrote:
> Hi,
>
> I am using Abdera in a project where different threads use the AbderaClient to read and post ducuments from/to a server. I am wondering what are the recommended way to do this safely and without consuming too many resources. Can I safely share one instance of AberaClient between multiple threads? Should I always call the ClientResponse.release() method after I'm done with a response?
>

Instances of AbderaClient are generally threadsafe as long as you're careful not to be changing configuration properties within individual threads. It does use the multithreaded connection manager under the covers. The Response objects, however, are not threadsafe and must only ever be used by a single thread; and yes, you need to call
release() when you're done with the response. Note, however, that because of the lazy parsing model, you need to ensure that the stream is completely parsed or buffered before calling release or you will see IOExceptions.

> When I blindly followed the Getting started guide, I quickly ran into problems of performance and even ran out of connections after a while. Now I call the .teardown() method when I'm no longer using the AbderaClient, and things improved. I could not find any documentation about this though, so I am therefore asking here, since I really want to ensure I'm doing things the right way, so I don't bump into other problems later.

Yes, this particular part of the code was limited by the design of the apache http client 3.x and definitely could have been better documented. Always call teardown and release to free up the resources as the http client definitely likes to hold on to things.

>
> Best regards,
> Morten Bendiksen

Re: Thread safety and resource usage of AbderaClient

[James M Snell <ja...@gmail.com>]

On Thu, May 24, 2012 at 3:36 AM, Morten Bendiksen <MB...@vizrt.com> wrote:
> Hi,
>
> I am using Abdera in a project where different threads use the AbderaClient to read and post ducuments from/to a server. I am wondering what are the recommended way to do this safely and without consuming too many resources. Can I safely share one instance of AberaClient between multiple threads? Should I always call the ClientResponse.release() method after I'm done with a response?
>

Instances of AbderaClient are generally threadsafe as long as you're
careful not to be changing configuration properties within individual
threads. It does use the multithreaded connection manager under the
covers. The Response objects, however, are not threadsafe and must
only ever be used by a single thread; and yes, you need to call
release() when you're done with the response. Note, however, that
because of the lazy parsing model, you need to ensure that the stream
is completely parsed or buffered before calling release or you will
see IOExceptions.

> When I blindly followed the Getting started guide, I quickly ran into problems of performance and even ran out of connections after a while. Now I call the .teardown() method when I'm no longer using the AbderaClient, and things improved. I could not find any documentation about this though, so I am therefore asking here, since I really want to ensure I'm doing things the right way, so I don't bump into other problems later.

Yes, this particular part of the code was limited by the design of the
apache http client 3.x and definitely could have been better
documented. Always call teardown and release to free up the resources
as the http client definitely likes to hold on to things.

>
> Best regards,
> Morten Bendiksen