Is there any difference between jclouds swift client and phython swiftclient?

[Naga Viswanathan Malepati <na...@yahoo.com.INVALID>]

Hi,

We have been doing some performance testing on our jclouds client code and we noticed that the throughput is much lower than python client. To validate here is the experiment we ran using a simple implementation using jclouds & swift-bench (python based bench marking tool)

We can 60000 puts with 100 concurrent users (threads). What we noticed was swift-bench had a throughput of about 380 puts/s where as the jclouds based client had a max of 100 puts/s. First, has anybody compared the two before and seen similar behavior?

Secondly, we were wondering if there are any tuning params for jclouds swiftclient that we can tweak and play with.

Thanks in advance.

-Naga

Re: Is there any difference between jclouds swift client and phython swiftclient?

[Aled Sage <al...@gmail.com>]

Hi Naga,

First quick sanity check: is jclouds wire logging etc all turned off?

Can you share your jclouds test class?
Is your test just doing puts or also gets - on the same blob or 
different blob names? Or are you also deleting the blob (as recommended 
in [1])?
What size is each blob?

Do you know what was maxing out in jclouds? Was your CPU at 100%?

I hacked together a quick test but am currently network-bound so 
couldn't get above about 45 per second.

Aled


On 31/07/2014 22:42, Andrew Phillips wrote:
> Hi Naga
>
> You may want to have a look at and perhaps pick up this thread [1], 
> which touches pretty much the same topic.
>
> Regards
>
> ap
>
> [1] http://markmail.org/thread/jizyq43naajigjt7

Re: Is there any difference between jclouds swift client and phython swiftclient?

[Andrew Phillips <ap...@qrmedia.com>]

Hi Naga

You may want to have a look at and perhaps pick up this thread [1],  
which touches pretty much the same topic.

Regards

ap

[1] http://markmail.org/thread/jizyq43naajigjt7

RE: Is there any difference between jclouds swift client and phython swiftclient?

[Zack Shoylev <za...@RACKSPACE.COM>]

If you have wire level logging enabled, you would be able to see what was sent over the wire by jclouds. Then these logs can probably be compared with the python logs.
Then we can probably start narrowing down on what exactly is causing the discrepancy.
It would also help if you can share any of the code used for testing.
Some connection tuning parameters are available, but they usually focus on timeouts and exponential backoff (which might be causing what you see).
You can see how some of these properties are used in this test:
https://github.com/rackerlabs/jclouds-labs-openstack/blob/master/openstack-swift/src/test/java/org/jclouds/openstack/swift/v1/features/ObjectApiMockTest.java#L310

Thanks!
________________________________________
From: Naga Viswanathan Malepati [naga.malepati@yahoo.com.INVALID]
Sent: Thursday, July 31, 2014 4:28 PM
To: dev@jclouds.apache.org
Subject: Is there any difference between jclouds swift client and phython swiftclient?

Hi,

We have been doing some performance testing on our jclouds client code and we noticed that the throughput is much lower than python client. To validate here is the experiment we ran using a simple implementation using jclouds & swift-bench (python based bench marking tool)

We can 60000 puts with 100 concurrent users (threads). What we noticed was swift-bench had a throughput of about 380 puts/s where as the jclouds based client had a max of 100 puts/s. First, has anybody compared the two before and seen similar behavior?

Secondly, we were wondering if there are any tuning params for jclouds swiftclient that we can tweak and play with.

Thanks in advance.

-Naga