You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@couchdb.apache.org by "Knudsen, Ken" <Ke...@imaginecommunications.com> on 2014/04/04 04:40:22 UTC
Bench marking a simple 10k write
....to a CouchDB server....I'm literally just executing a unit test 10k times that open an HTTP connection, post the data and close... CouchDB is performing very slowly in this. I thought it'd be very fast. Any ideas?
Thanks for the help,
Ken
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________
Re: Bench marking a simple 10k write
Posted by Wendall Cada <we...@apache.org>.
Are you updating the same doc 10k times? Or writing a new doc each time?
If it's the former, this isn't a really good test for CouchDB, and
certainly going to be slow.
Wendall
On 04/03/2014 07:40 PM, Knudsen, Ken wrote:
> ....to a CouchDB server....I'm literally just executing a unit test 10k times that open an HTTP connection, post the data and close... CouchDB is performing very slowly in this. I thought it'd be very fast. Any ideas?
>
> Thanks for the help,
>
> Ken
>
> ______________________________________________________________________
> This email has been scanned by the Symantec Email Security.cloud service.
> For more information please visit http://www.symanteccloud.com
> ______________________________________________________________________
RE: Bench marking a simple 10k write
Posted by Stanley Iriele <si...@gmail.com>.
Well for known bulk input you should use the bulk API but in your
case...every test is recreating your client and making that call. I would
try making the request over and over again with the same http client
instead of re running the request over and over again.
In a real live server you would be sharing the http client and taking
advantage of the connection pooling taking place under the hood
On Apr 3, 2014 8:12 PM, "Knudsen, Ken" <
Ken.Knudsen@imaginecommunications.com> wrote:
> Oh sure.. I was just following the same pattern I did for SQL (all be it a
> System.Data.SqlClient driver - C#).. So NUnit Test... Repeat the test 10k
> times (RepeatAttribute(10000) )... code is..
>
> var httpWebRequest = (HttpWebRequest)WebRequest.Create("
> http://137.237.134.94:5984/inserttest/");
>
> httpWebRequest.ContentType = "application/json";
> httpWebRequest.Method = "POST";
>
> using (var streamWriter = new
> StreamWriter(httpWebRequest.GetRequestStream()))
> {
> streamWriter.Write(jsonVal);
> streamWriter.Flush();
> streamWriter.Close();
>
> var httpResponse =
> (HttpWebResponse)httpWebRequest.GetResponse();
> using (var streamReader = new
> StreamReader(httpResponse.GetResponseStream()))
> {
> var result = streamReader.ReadToEnd();
> }
> }
>
> -----Original Message-----
> From: Stanley Iriele [mailto:siriele2x3@gmail.com]
> Sent: April-03-14 11:04 PM
> To: user@couchdb.apache.org
> Subject: RE: Bench marking a simple 10k write
>
> That's fine..I definitely relate to your situation.. If you want to do 10k
> writes...use the bulk_docs API that should do the trick... Bit tearing down
> a connection per request will probably not work so well...I just wanted to
> see your actual loop or code to verify if its tearing down the connections
> or not.. Is there anyway to post it?
> On Apr 3, 2014 7:58 PM, "Knudsen, Ken" <
> Ken.Knudsen@imaginecommunications.com> wrote:
>
> > System is Windows, 64 bit, V-7...
> >
> > With delayed_commits on: roughly 40 seconds for 10k
> >
> > With delayed_commits off...don't ask, way to long.
> >
> >
> > Why I'm asking (as I've asked in IRC as well)....
> >
> > So why am I doing this... well, the Gods that be don't believe NoSQL
> > (MongoDB, CouchDB, ArangoDB and so on) can compete with SQL Server..
> > 10k SQLServer writes get's done in 8 seconds ...pretend there's
> > nothing else you could convince them of until you show them that 10k
> > writes in CouchDB can happen as fast..
> >
> > ArangoDB is extremely fast, as I expected (waitforsync=false)... I
> > thought CouchDB would have been just as fast.
> >
> > -----Original Message-----
> > From: Stanley Iriele [mailto:siriele2x3@gmail.com]
> > Sent: April-03-14 10:46 PM
> > To: user@couchdb.apache.org
> > Subject: Re: Bench marking a simple 10k write
> >
> > Are you tearing down the connection and reestablishing it every time?
> > Or have you played with the TCP_no_delay settings?... Could you define
> > slow and maybe post your function minus credentials?
> > On Apr 3, 2014 7:41 PM, "Knudsen, Ken" <
> > Ken.Knudsen@imaginecommunications.com> wrote:
> >
> > > ....to a CouchDB server....I'm literally just executing a unit test
> > > 10k times that open an HTTP connection, post the data and close...
> > > CouchDB is performing very slowly in this. I thought it'd be very fast.
> > Any ideas?
> > >
> > > Thanks for the help,
> > >
> > > Ken
> > >
> > > ____________________________________________________________________
> > > __ This email has been scanned by the Symantec Email Security.cloud
> > > service.
> > > For more information please visit http://www.symanteccloud.com
> > > ____________________________________________________________________
> > > __
> >
> >
> > ______________________________________________________________________
> > This email has been scanned by the Symantec Email Security.cloud service.
> > For more information please visit
> > http://www.symanteccloud.com__________________________________________
> > ____________________________
> >
> > ______________________________________________________________________
> > This email has been scanned by the Symantec Email Security.cloud service.
> > For more information please visit http://www.symanteccloud.com
> > ______________________________________________________________________
> >
>
>
> ______________________________________________________________________
> This email has been scanned by the Symantec Email Security.cloud service.
> For more information please visit http://www.symanteccloud.com______________________________________________________________________
>
> ______________________________________________________________________
> This email has been scanned by the Symantec Email Security.cloud service.
> For more information please visit http://www.symanteccloud.com
> ______________________________________________________________________
>
Re: Bench marking a simple 10k write
Posted by Alexander Shorin <kx...@gmail.com>.
On Fri, Apr 4, 2014 at 9:02 AM, Jens Alfke <je...@couchbase.com> wrote:
> On Apr 3, 2014, at 8:11 PM, Knudsen, Ken <Ke...@imaginecommunications.com> wrote:
>
>> Oh sure.. I was just following the same pattern I did for SQL
>
> As the name implies, NoSQL databases don’t work the same way as SQL/relational databases. Part of that is that the APIs are different, and require some different patterns for working with them efficiently.
>
> In CouchDB’s case, if you want to write a lot of docs at once, you use a _bulk_docs request.
Actually, for SQL there is the same rule to run a lot of write
operations (insert/update/delete) in single transaction (read as "in
bulk") for better performance.
--
,,,^..^,,,
Re: Bench marking a simple 10k write
Posted by Jens Alfke <je...@couchbase.com>.
On Apr 3, 2014, at 8:11 PM, Knudsen, Ken <Ke...@imaginecommunications.com> wrote:
> Oh sure.. I was just following the same pattern I did for SQL
As the name implies, NoSQL databases don’t work the same way as SQL/relational databases. Part of that is that the APIs are different, and require some different patterns for working with them efficiently.
In CouchDB’s case, if you want to write a lot of docs at once, you use a _bulk_docs request.
—Jens
RE: Bench marking a simple 10k write
Posted by "Knudsen, Ken" <Ke...@imaginecommunications.com>.
Oh sure.. I was just following the same pattern I did for SQL (all be it a System.Data.SqlClient driver - C#).. So NUnit Test... Repeat the test 10k times (RepeatAttribute(10000) )... code is..
var httpWebRequest = (HttpWebRequest)WebRequest.Create("http://137.237.134.94:5984/inserttest/");
httpWebRequest.ContentType = "application/json";
httpWebRequest.Method = "POST";
using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
{
streamWriter.Write(jsonVal);
streamWriter.Flush();
streamWriter.Close();
var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
var result = streamReader.ReadToEnd();
}
}
-----Original Message-----
From: Stanley Iriele [mailto:siriele2x3@gmail.com]
Sent: April-03-14 11:04 PM
To: user@couchdb.apache.org
Subject: RE: Bench marking a simple 10k write
That's fine..I definitely relate to your situation.. If you want to do 10k writes...use the bulk_docs API that should do the trick... Bit tearing down a connection per request will probably not work so well...I just wanted to see your actual loop or code to verify if its tearing down the connections or not.. Is there anyway to post it?
On Apr 3, 2014 7:58 PM, "Knudsen, Ken" < Ken.Knudsen@imaginecommunications.com> wrote:
> System is Windows, 64 bit, V-7...
>
> With delayed_commits on: roughly 40 seconds for 10k
>
> With delayed_commits off...don't ask, way to long.
>
>
> Why I'm asking (as I've asked in IRC as well)....
>
> So why am I doing this... well, the Gods that be don't believe NoSQL
> (MongoDB, CouchDB, ArangoDB and so on) can compete with SQL Server..
> 10k SQLServer writes get's done in 8 seconds ...pretend there's
> nothing else you could convince them of until you show them that 10k
> writes in CouchDB can happen as fast..
>
> ArangoDB is extremely fast, as I expected (waitforsync=false)... I
> thought CouchDB would have been just as fast.
>
> -----Original Message-----
> From: Stanley Iriele [mailto:siriele2x3@gmail.com]
> Sent: April-03-14 10:46 PM
> To: user@couchdb.apache.org
> Subject: Re: Bench marking a simple 10k write
>
> Are you tearing down the connection and reestablishing it every time?
> Or have you played with the TCP_no_delay settings?... Could you define
> slow and maybe post your function minus credentials?
> On Apr 3, 2014 7:41 PM, "Knudsen, Ken" <
> Ken.Knudsen@imaginecommunications.com> wrote:
>
> > ....to a CouchDB server....I'm literally just executing a unit test
> > 10k times that open an HTTP connection, post the data and close...
> > CouchDB is performing very slowly in this. I thought it'd be very fast.
> Any ideas?
> >
> > Thanks for the help,
> >
> > Ken
> >
> > ____________________________________________________________________
> > __ This email has been scanned by the Symantec Email Security.cloud
> > service.
> > For more information please visit http://www.symanteccloud.com
> > ____________________________________________________________________
> > __
>
>
> ______________________________________________________________________
> This email has been scanned by the Symantec Email Security.cloud service.
> For more information please visit
> http://www.symanteccloud.com__________________________________________
> ____________________________
>
> ______________________________________________________________________
> This email has been scanned by the Symantec Email Security.cloud service.
> For more information please visit http://www.symanteccloud.com
> ______________________________________________________________________
>
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com ______________________________________________________________________
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________
RE: Bench marking a simple 10k write
Posted by Stanley Iriele <si...@gmail.com>.
That's fine..I definitely relate to your situation.. If you want to do 10k
writes...use the bulk_docs API that should do the trick... Bit tearing down
a connection per request will probably not work so well...I just wanted to
see your actual loop or code to verify if its tearing down the connections
or not.. Is there anyway to post it?
On Apr 3, 2014 7:58 PM, "Knudsen, Ken" <
Ken.Knudsen@imaginecommunications.com> wrote:
> System is Windows, 64 bit, V-7...
>
> With delayed_commits on: roughly 40 seconds for 10k
>
> With delayed_commits off...don't ask, way to long.
>
>
> Why I'm asking (as I've asked in IRC as well)....
>
> So why am I doing this... well, the Gods that be don't believe NoSQL
> (MongoDB, CouchDB, ArangoDB and so on) can compete with SQL Server.. 10k
> SQLServer writes get's done in 8 seconds ...pretend there's nothing else
> you could convince them of until you show them that 10k writes in CouchDB
> can happen as fast..
>
> ArangoDB is extremely fast, as I expected (waitforsync=false)... I thought
> CouchDB would have been just as fast.
>
> -----Original Message-----
> From: Stanley Iriele [mailto:siriele2x3@gmail.com]
> Sent: April-03-14 10:46 PM
> To: user@couchdb.apache.org
> Subject: Re: Bench marking a simple 10k write
>
> Are you tearing down the connection and reestablishing it every time? Or
> have you played with the TCP_no_delay settings?... Could you define slow
> and maybe post your function minus credentials?
> On Apr 3, 2014 7:41 PM, "Knudsen, Ken" <
> Ken.Knudsen@imaginecommunications.com> wrote:
>
> > ....to a CouchDB server....I'm literally just executing a unit test
> > 10k times that open an HTTP connection, post the data and close...
> > CouchDB is performing very slowly in this. I thought it'd be very fast.
> Any ideas?
> >
> > Thanks for the help,
> >
> > Ken
> >
> > ______________________________________________________________________
> > This email has been scanned by the Symantec Email Security.cloud service.
> > For more information please visit http://www.symanteccloud.com
> > ______________________________________________________________________
>
>
> ______________________________________________________________________
> This email has been scanned by the Symantec Email Security.cloud service.
> For more information please visit http://www.symanteccloud.com______________________________________________________________________
>
> ______________________________________________________________________
> This email has been scanned by the Symantec Email Security.cloud service.
> For more information please visit http://www.symanteccloud.com
> ______________________________________________________________________
>
RE: Bench marking a simple 10k write
Posted by "Knudsen, Ken" <Ke...@imaginecommunications.com>.
System is Windows, 64 bit, V-7...
With delayed_commits on: roughly 40 seconds for 10k
With delayed_commits off...don't ask, way to long.
Why I'm asking (as I've asked in IRC as well)....
So why am I doing this... well, the Gods that be don't believe NoSQL (MongoDB, CouchDB, ArangoDB and so on) can compete with SQL Server.. 10k SQLServer writes get's done in 8 seconds ...pretend there's nothing else you could convince them of until you show them that 10k writes in CouchDB can happen as fast..
ArangoDB is extremely fast, as I expected (waitforsync=false)... I thought CouchDB would have been just as fast.
-----Original Message-----
From: Stanley Iriele [mailto:siriele2x3@gmail.com]
Sent: April-03-14 10:46 PM
To: user@couchdb.apache.org
Subject: Re: Bench marking a simple 10k write
Are you tearing down the connection and reestablishing it every time? Or have you played with the TCP_no_delay settings?... Could you define slow and maybe post your function minus credentials?
On Apr 3, 2014 7:41 PM, "Knudsen, Ken" < Ken.Knudsen@imaginecommunications.com> wrote:
> ....to a CouchDB server....I'm literally just executing a unit test
> 10k times that open an HTTP connection, post the data and close...
> CouchDB is performing very slowly in this. I thought it'd be very fast. Any ideas?
>
> Thanks for the help,
>
> Ken
>
> ______________________________________________________________________
> This email has been scanned by the Symantec Email Security.cloud service.
> For more information please visit http://www.symanteccloud.com
> ______________________________________________________________________
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com ______________________________________________________________________
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________
Re: Bench marking a simple 10k write
Posted by Stanley Iriele <si...@gmail.com>.
Are you tearing down the connection and reestablishing it every time? Or
have you played with the TCP_no_delay settings?... Could you define slow
and maybe post your function minus credentials?
On Apr 3, 2014 7:41 PM, "Knudsen, Ken" <
Ken.Knudsen@imaginecommunications.com> wrote:
> ....to a CouchDB server....I'm literally just executing a unit test 10k
> times that open an HTTP connection, post the data and close... CouchDB is
> performing very slowly in this. I thought it'd be very fast. Any ideas?
>
> Thanks for the help,
>
> Ken
>
> ______________________________________________________________________
> This email has been scanned by the Symantec Email Security.cloud service.
> For more information please visit http://www.symanteccloud.com
> ______________________________________________________________________
Re: Bench marking a simple 10k write
Posted by Jens Alfke <je...@couchbase.com>.
On Apr 4, 2014, at 8:19 AM, Robert Samuel Newson <rn...@apache.org> wrote:
> Ok, that’s too far. This is the couchdb user mailing list not an ad channel.
Sorry, everyone; in hindsight I should have sent that directly to Ken, or left the post as just “don’t forget to try Couchbase Server too”.
—Jens
RE: Bench marking a simple 10k write
Posted by "Knudsen, Ken" <Ke...@imaginecommunications.com>.
Haha.. Thanks for that info Jens... I should have pointed out as well... currently it's about Apple and oranges... SQL Server is allowed to have money allocated to it and everything in the NoSQL space for comparison (so far) has been free...and even at that bench mark level, magic happens... I haven't even begun getting into Couchbase, Neo4J and paid models of other NoSQL database (MongoDB, etc) ... I'm purely demonstrating how the current free open source community can stand up to the big giants first.
-----Original Message-----
From: matt j. sorenson [mailto:matt@sorensonbros.net]
Sent: April-04-14 11:21 AM
To: user@couchdb.apache.org
Subject: Re: Bench marking a simple 10k write
but I totally just went out and bought a surface and downloaded sql server because of this thread...
On Fri, Apr 4, 2014 at 10:19 AM, Robert Samuel Newson <rn...@apache.org>wrote:
> Ok, that's too far. This is the couchdb user mailing list not an ad
> channel.
>
> B.
>
> On 4 Apr 2014, at 15:46, Jens Alfke <je...@couchbase.com> wrote:
>
> > {Getting off-topic, but as long as people are already talking about
> non-CouchDB databases...}
> >
> > If you're kicking the tires of NoSQL databases with performance in
> > mind,
> you really should be looking at Couchbase Server, whose calling card
> is insane speed and scalability. For example, Amadeus (the world's
> largest processor of airline reservations) is using it to serve "over
> 20 terabytes of data. Peak traffic is two million reads of objects per
> second, and 400K writes per second". [1] LinkedIn achieves "400K
> ops/second with their four-node cluster, and loads 16 million entries
> into Couchbase every 5 minutes."[2]
> >
> > (In case there's confusion: Couchbase's map/reduce view engine is
> derived from CouchDB, but its JSON document storage and general API
> are entirely different, being descended from memcached. There is a
> Sync Gateway add-on, currently in beta, that offers a
> CouchDB-compatible API.)
> >
> > --Jens
> >
> > [1]:
> http://www.couchbase.com/customer-stories/couchbase-helps-amadeus-powe
> r-travel-industry
> > [2]:
> http://www.couchbase.com/customer-stories/linkedin-monitors-massive-da
> ta-couchbase
>
>
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com ______________________________________________________________________
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________
Re: Bench marking a simple 10k write
Posted by "matt j. sorenson" <ma...@sorensonbros.net>.
but I totally just went out and bought a surface and downloaded sql server
because of this thread...
On Fri, Apr 4, 2014 at 10:19 AM, Robert Samuel Newson <rn...@apache.org>wrote:
> Ok, that's too far. This is the couchdb user mailing list not an ad
> channel.
>
> B.
>
> On 4 Apr 2014, at 15:46, Jens Alfke <je...@couchbase.com> wrote:
>
> > {Getting off-topic, but as long as people are already talking about
> non-CouchDB databases...}
> >
> > If you're kicking the tires of NoSQL databases with performance in mind,
> you really should be looking at Couchbase Server, whose calling card is
> insane speed and scalability. For example, Amadeus (the world's largest
> processor of airline reservations) is using it to serve "over 20 terabytes
> of data. Peak traffic is two million reads of objects per second, and 400K
> writes per second". [1] LinkedIn achieves "400K ops/second with their
> four-node cluster, and loads 16 million entries into Couchbase every 5
> minutes."[2]
> >
> > (In case there's confusion: Couchbase's map/reduce view engine is
> derived from CouchDB, but its JSON document storage and general API are
> entirely different, being descended from memcached. There is a Sync Gateway
> add-on, currently in beta, that offers a CouchDB-compatible API.)
> >
> > --Jens
> >
> > [1]:
> http://www.couchbase.com/customer-stories/couchbase-helps-amadeus-power-travel-industry
> > [2]:
> http://www.couchbase.com/customer-stories/linkedin-monitors-massive-data-couchbase
>
>
Re: Bench marking a simple 10k write
Posted by Robert Samuel Newson <rn...@apache.org>.
Ok, that’s too far. This is the couchdb user mailing list not an ad channel.
B.
On 4 Apr 2014, at 15:46, Jens Alfke <je...@couchbase.com> wrote:
> {Getting off-topic, but as long as people are already talking about non-CouchDB databases…}
>
> If you’re kicking the tires of NoSQL databases with performance in mind, you really should be looking at Couchbase Server, whose calling card is insane speed and scalability. For example, Amadeus (the world's largest processor of airline reservations) is using it to serve "over 20 terabytes of data. Peak traffic is two million reads of objects per second, and 400K writes per second”. [1] LinkedIn achieves "400K ops/second with their four-node cluster, and loads 16 million entries into Couchbase every 5 minutes.”[2]
>
> (In case there’s confusion: Couchbase’s map/reduce view engine is derived from CouchDB, but its JSON document storage and general API are entirely different, being descended from memcached. There is a Sync Gateway add-on, currently in beta, that offers a CouchDB-compatible API.)
>
> —Jens
>
> [1]: http://www.couchbase.com/customer-stories/couchbase-helps-amadeus-power-travel-industry
> [2]: http://www.couchbase.com/customer-stories/linkedin-monitors-massive-data-couchbase
Re: Bench marking a simple 10k write
Posted by Jens Alfke <je...@couchbase.com>.
{Getting off-topic, but as long as people are already talking about non-CouchDB databases…}
If you’re kicking the tires of NoSQL databases with performance in mind, you really should be looking at Couchbase Server, whose calling card is insane speed and scalability. For example, Amadeus (the world's largest processor of airline reservations) is using it to serve "over 20 terabytes of data. Peak traffic is two million reads of objects per second, and 400K writes per second”. [1] LinkedIn achieves "400K ops/second with their four-node cluster, and loads 16 million entries into Couchbase every 5 minutes.”[2]
(In case there’s confusion: Couchbase’s map/reduce view engine is derived from CouchDB, but its JSON document storage and general API are entirely different, being descended from memcached. There is a Sync Gateway add-on, currently in beta, that offers a CouchDB-compatible API.)
—Jens
[1]: http://www.couchbase.com/customer-stories/couchbase-helps-amadeus-power-travel-industry
[2]: http://www.couchbase.com/customer-stories/linkedin-monitors-massive-data-couchbase
RE: Bench marking a simple 10k write
Posted by "Knudsen, Ken" <Ke...@imaginecommunications.com>.
Wow, that's great Daniel, thank you very much!
What do you guys think of these small bench marks ArangoDB did with couchDB ... I think they are fair and solid enough to use as numbers for pretty pictures...thoughts? My next steps would be then to do similar tests against SQL Server.. get those numbers.. then with all the numbers, my manager can plug them into excel to make more charts. The first goal here is to break that outright wall of denial with some hard numbers and get us into 'talk and prove it' land.
https://www.arangodb.org/2012/08/24/benchmarking-arangodb-and-couchdb
https://www.arangodb.org/2012/09/04/bulk-inserts-mongodb-couchdb-arangodb
https://www.arangodb.org/2012/08/08/more-datafile-sizes
-----Original Message-----
From: Daniel Wertheim [mailto:daniel@wertheim.se]
Sent: April-04-14 8:56 AM
To: user@couchdb.apache.org
Subject: Re: Bench marking a simple 10k write
Just did some measurements in .Net using MyCouch. If I create a new client each time, with no bulk and no batch for 10k I get 25s and not 40s. That's a default, local install of CouchDb 1.5 on Windows 8.1 i7 quad core, 8GB RAM 64bit, running a simple release compiled console app.
New Db was created for each lap.
Note! I did not use the async behaviors of MyCouch, I awaited each PUT.
Hence, if you tweak the ServicePointManager.DefaultConnectionLimit (
http://msdn.microsoft.com/en-us/library/system.net.servicepointmanager.defaultconnectionlimit.aspx)
and allow async requests......
Did this using PUT so I generate the id (simple running int).
One PUT was executed as first to get "Warmup" behavior.
Cheers,
//Daniel
==========================
Using same Client no bulk and no batch
==========================
Total timings: 10000
Total (ms): 20886,0011000001
Avg (ms): 2,08860011000001
==========================
Using NEW Client no bulk and no batch
Also timing the time it takes to construct the client ========================== Total timings: 10000 Total (ms): 25062,9446000001 Avg (ms): 2,50629446000001
==========================
Using same Client with batch
==========================
Total timings: 10000
Total (ms): 9909,26200000002
Avg (ms): 0,990926200000002
==========================
Using NEW Client with batch
Also timing the time it takes to construct the client ========================== Total timings: 10000 Total (ms): 13414,0004 Avg (ms): 1,34140004
==========================
Using same Client with bulk of 100 in each ========================== Total timings: 100 Total (ms): 2259,0868 Avg (ms): 22,590868
==========================
Using NEW Client with bulk of 100 in each Also timing the time it takes to construct the client ========================== Total timings: 100 Total (ms): 2430,1235 Avg (ms): 24,301235
On 4 April 2014 12:41, Knudsen, Ken
<Ke...@imaginecommunications.com>wrote:
> Thanks for the comments guys, I appreciate it. I understand to use
> Bulk-Import, I've done my reading (just in case you think I'm being
> lazy :)
> ) ... what I'm trying to run tests for (get some numbers, throw them
> into a chart), a persons current thinking pattern, is someone high up
> that's either had a bad experience with the early days of NoSQL
> (MongoDB to be exact)...and because of this, is dead set against the whole word 'NoSQL'...
> at that level, charts and numbers is all they pay attention too. So
> I'm just trying to put together an enlightening sea of pictures that
> will show that's not the case. I know it's not the case.
>
> The micro tests are just one set of the pictures.. I do have a full
> TSung test bed happening as well... A simple rest service with a few methods...
> Each testing 1 call to their respective database (SQL, ArangoDB,
> CouchDB, MongoDB)...Then in TSung I setup a test client to make 10k
> iteration of calls to the REST service (simple initial setup, 1
> user)...where each call inserts 1 document...Now, under this senario,
> the SQL Server comes in last....It takes 2.33 mins to execute 10k of
> calls. CouchDB takes less then a minute and ArangoDB comes in at about
> 33 seconds. I find it interesting as a more real distributed test bed
> is put into play, that the power of the NoSQL databases start to come alive.
>
> thanks again!
>
> Ken
> ________________________________________
> From: Stanley Iriele [siriele2x3@gmail.com]
> Sent: Friday, April 04, 2014 6:23 AM
> To: user@couchdb.apache.org
> Subject: Re: Bench marking a simple 10k write
>
> I believe most clients use keep alive by default but what I'm saying
> is that he's running the test in nunit (to my understanding) of its in
> the same loop or whatever then it should be using keep alive...but the
> nunits tear down the class and its resources per run. So even if the
> default library uses keep a lives its still freshly created each time
> On Apr 4, 2014 2:37 AM, "Nick North" <no...@gmail.com> wrote:
>
> > On second thoughts, while my previous claim is true, I'm not sure
> > what happens when you use WebRequest.Create for each write, as in the example.
> > So maybe keep-alive is not enabled after all.
> >
> > Nick
> >
> >
> > On 4 April 2014 10:35, Nick North <no...@gmail.com> wrote:
> >
> > > The HttpWebRequest class has a KeepAlive <
> >
> http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.keep
> alive.aspx
> > >property
> > > that defaults to true under HTTP 1.1, so it should be enabled in
> > >the
> OP's
> > > example.
> > >
> > > Nick
> > >
> > >
> > > On 4 April 2014 10:24, Stanley Iriele <si...@gmail.com> wrote:
> > >
> > >> On Fri, Apr 4, 2014 at 1:28 AM, Benoit Chesneau
> > >> <bc...@gmail.com>
> > >> wrote:
> > >>
> > >> > On Fri, Apr 4, 2014 at 10:25 AM, Will Holley
> > >> > <wi...@gmail.com>
> > >> wrote:
> > >> >
> > >> > > .NET should set keep-alive by default.
> > >> > >
> > >> >
> > >>
> > >> I have a hard time believing that because its the library you're
> > >> using
> > not
> > >> NET itself that decides that. Also If the test is truly being
> > >> torn
> down
> > >> rerun I don't see how it could use keep-alives...that would
> > >> invalidate
> > the
> > >> test wouldn't it? being that the 1st test makes a connection and
> > >> subsequent tests reuse the same connection
> > >>
> > >
> > >
> >
>
>
> ______________________________________________________________________
> This email has been scanned by the Symantec Email Security.cloud service.
> For more information please visit http://www.symanteccloud.com
> ______________________________________________________________________
>
> ______________________________________________________________________
> This email has been scanned by the Symantec Email Security.cloud service.
> For more information please visit http://www.symanteccloud.com
> ______________________________________________________________________
>
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________
Re: Bench marking a simple 10k write
Posted by Daniel Wertheim <da...@wertheim.se>.
Just did some measurements in .Net using MyCouch. If I create a new client
each time, with no bulk and no batch for 10k I get 25s and not 40s. That's
a default, local install of CouchDb 1.5 on Windows 8.1 i7 quad core, 8GB
RAM 64bit, running a simple release compiled console app.
New Db was created for each lap.
Note! I did not use the async behaviors of MyCouch, I awaited each PUT.
Hence, if you tweak the ServicePointManager.DefaultConnectionLimit (
http://msdn.microsoft.com/en-us/library/system.net.servicepointmanager.defaultconnectionlimit.aspx)
and allow async requests......
Did this using PUT so I generate the id (simple running int).
One PUT was executed as first to get "Warmup" behavior.
Cheers,
//Daniel
==========================
Using same Client no bulk and no batch
==========================
Total timings: 10000
Total (ms): 20886,0011000001
Avg (ms): 2,08860011000001
==========================
Using NEW Client no bulk and no batch
Also timing the time it takes to construct the client
==========================
Total timings: 10000
Total (ms): 25062,9446000001
Avg (ms): 2,50629446000001
==========================
Using same Client with batch
==========================
Total timings: 10000
Total (ms): 9909,26200000002
Avg (ms): 0,990926200000002
==========================
Using NEW Client with batch
Also timing the time it takes to construct the client
==========================
Total timings: 10000
Total (ms): 13414,0004
Avg (ms): 1,34140004
==========================
Using same Client with bulk of 100 in each
==========================
Total timings: 100
Total (ms): 2259,0868
Avg (ms): 22,590868
==========================
Using NEW Client with bulk of 100 in each
Also timing the time it takes to construct the client
==========================
Total timings: 100
Total (ms): 2430,1235
Avg (ms): 24,301235
On 4 April 2014 12:41, Knudsen, Ken
<Ke...@imaginecommunications.com>wrote:
> Thanks for the comments guys, I appreciate it. I understand to use
> Bulk-Import, I've done my reading (just in case you think I'm being lazy :)
> ) ... what I'm trying to run tests for (get some numbers, throw them into a
> chart), a persons current thinking pattern, is someone high up that's
> either had a bad experience with the early days of NoSQL (MongoDB to be
> exact)...and because of this, is dead set against the whole word 'NoSQL'...
> at that level, charts and numbers is all they pay attention too. So I'm
> just trying to put together an enlightening sea of pictures that will show
> that's not the case. I know it's not the case.
>
> The micro tests are just one set of the pictures.. I do have a full TSung
> test bed happening as well... A simple rest service with a few methods...
> Each testing 1 call to their respective database (SQL, ArangoDB, CouchDB,
> MongoDB)...Then in TSung I setup a test client to make 10k iteration of
> calls to the REST service (simple initial setup, 1 user)...where each call
> inserts 1 document...Now, under this senario, the SQL Server comes in
> last....It takes 2.33 mins to execute 10k of calls. CouchDB takes less then
> a minute and ArangoDB comes in at about 33 seconds. I find it interesting
> as a more real distributed test bed is put into play, that the power of the
> NoSQL databases start to come alive.
>
> thanks again!
>
> Ken
> ________________________________________
> From: Stanley Iriele [siriele2x3@gmail.com]
> Sent: Friday, April 04, 2014 6:23 AM
> To: user@couchdb.apache.org
> Subject: Re: Bench marking a simple 10k write
>
> I believe most clients use keep alive by default but what I'm saying is
> that he's running the test in nunit (to my understanding) of its in the
> same loop or whatever then it should be using keep alive...but the nunits
> tear down the class and its resources per run. So even if the default
> library uses keep a lives its still freshly created each time
> On Apr 4, 2014 2:37 AM, "Nick North" <no...@gmail.com> wrote:
>
> > On second thoughts, while my previous claim is true, I'm not sure what
> > happens when you use WebRequest.Create for each write, as in the example.
> > So maybe keep-alive is not enabled after all.
> >
> > Nick
> >
> >
> > On 4 April 2014 10:35, Nick North <no...@gmail.com> wrote:
> >
> > > The HttpWebRequest class has a KeepAlive
> > > <
> >
> http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.keepalive.aspx
> > >property
> > > that defaults to true under HTTP 1.1, so it should be enabled in the
> OP's
> > > example.
> > >
> > > Nick
> > >
> > >
> > > On 4 April 2014 10:24, Stanley Iriele <si...@gmail.com> wrote:
> > >
> > >> On Fri, Apr 4, 2014 at 1:28 AM, Benoit Chesneau <bc...@gmail.com>
> > >> wrote:
> > >>
> > >> > On Fri, Apr 4, 2014 at 10:25 AM, Will Holley <wi...@gmail.com>
> > >> wrote:
> > >> >
> > >> > > .NET should set keep-alive by default.
> > >> > >
> > >> >
> > >>
> > >> I have a hard time believing that because its the library you're using
> > not
> > >> NET itself that decides that. Also If the test is truly being torn
> down
> > >> rerun I don't see how it could use keep-alives...that would invalidate
> > the
> > >> test wouldn't it? being that the 1st test makes a connection and
> > >> subsequent
> > >> tests reuse the same connection
> > >>
> > >
> > >
> >
>
>
> ______________________________________________________________________
> This email has been scanned by the Symantec Email Security.cloud service.
> For more information please visit http://www.symanteccloud.com
> ______________________________________________________________________
>
> ______________________________________________________________________
> This email has been scanned by the Symantec Email Security.cloud service.
> For more information please visit http://www.symanteccloud.com
> ______________________________________________________________________
>
RE: Bench marking a simple 10k write
Posted by "Knudsen, Ken" <Ke...@imaginecommunications.com>.
Thanks for the comments guys, I appreciate it. I understand to use Bulk-Import, I've done my reading (just in case you think I'm being lazy :) ) ... what I'm trying to run tests for (get some numbers, throw them into a chart), a persons current thinking pattern, is someone high up that's either had a bad experience with the early days of NoSQL (MongoDB to be exact)...and because of this, is dead set against the whole word 'NoSQL'... at that level, charts and numbers is all they pay attention too. So I'm just trying to put together an enlightening sea of pictures that will show that's not the case. I know it's not the case.
The micro tests are just one set of the pictures.. I do have a full TSung test bed happening as well... A simple rest service with a few methods... Each testing 1 call to their respective database (SQL, ArangoDB, CouchDB, MongoDB)...Then in TSung I setup a test client to make 10k iteration of calls to the REST service (simple initial setup, 1 user)...where each call inserts 1 document...Now, under this senario, the SQL Server comes in last....It takes 2.33 mins to execute 10k of calls. CouchDB takes less then a minute and ArangoDB comes in at about 33 seconds. I find it interesting as a more real distributed test bed is put into play, that the power of the NoSQL databases start to come alive.
thanks again!
Ken
________________________________________
From: Stanley Iriele [siriele2x3@gmail.com]
Sent: Friday, April 04, 2014 6:23 AM
To: user@couchdb.apache.org
Subject: Re: Bench marking a simple 10k write
I believe most clients use keep alive by default but what I'm saying is
that he's running the test in nunit (to my understanding) of its in the
same loop or whatever then it should be using keep alive...but the nunits
tear down the class and its resources per run. So even if the default
library uses keep a lives its still freshly created each time
On Apr 4, 2014 2:37 AM, "Nick North" <no...@gmail.com> wrote:
> On second thoughts, while my previous claim is true, I'm not sure what
> happens when you use WebRequest.Create for each write, as in the example.
> So maybe keep-alive is not enabled after all.
>
> Nick
>
>
> On 4 April 2014 10:35, Nick North <no...@gmail.com> wrote:
>
> > The HttpWebRequest class has a KeepAlive
> > <
> http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.keepalive.aspx
> >property
> > that defaults to true under HTTP 1.1, so it should be enabled in the OP's
> > example.
> >
> > Nick
> >
> >
> > On 4 April 2014 10:24, Stanley Iriele <si...@gmail.com> wrote:
> >
> >> On Fri, Apr 4, 2014 at 1:28 AM, Benoit Chesneau <bc...@gmail.com>
> >> wrote:
> >>
> >> > On Fri, Apr 4, 2014 at 10:25 AM, Will Holley <wi...@gmail.com>
> >> wrote:
> >> >
> >> > > .NET should set keep-alive by default.
> >> > >
> >> >
> >>
> >> I have a hard time believing that because its the library you're using
> not
> >> NET itself that decides that. Also If the test is truly being torn down
> >> rerun I don't see how it could use keep-alives...that would invalidate
> the
> >> test wouldn't it? being that the 1st test makes a connection and
> >> subsequent
> >> tests reuse the same connection
> >>
> >
> >
>
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________
Re: Bench marking a simple 10k write
Posted by Stanley Iriele <si...@gmail.com>.
I believe most clients use keep alive by default but what I'm saying is
that he's running the test in nunit (to my understanding) of its in the
same loop or whatever then it should be using keep alive...but the nunits
tear down the class and its resources per run. So even if the default
library uses keep a lives its still freshly created each time
On Apr 4, 2014 2:37 AM, "Nick North" <no...@gmail.com> wrote:
> On second thoughts, while my previous claim is true, I'm not sure what
> happens when you use WebRequest.Create for each write, as in the example.
> So maybe keep-alive is not enabled after all.
>
> Nick
>
>
> On 4 April 2014 10:35, Nick North <no...@gmail.com> wrote:
>
> > The HttpWebRequest class has a KeepAlive
> > <
> http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.keepalive.aspx
> >property
> > that defaults to true under HTTP 1.1, so it should be enabled in the OP's
> > example.
> >
> > Nick
> >
> >
> > On 4 April 2014 10:24, Stanley Iriele <si...@gmail.com> wrote:
> >
> >> On Fri, Apr 4, 2014 at 1:28 AM, Benoit Chesneau <bc...@gmail.com>
> >> wrote:
> >>
> >> > On Fri, Apr 4, 2014 at 10:25 AM, Will Holley <wi...@gmail.com>
> >> wrote:
> >> >
> >> > > .NET should set keep-alive by default.
> >> > >
> >> >
> >>
> >> I have a hard time believing that because its the library you're using
> not
> >> NET itself that decides that. Also If the test is truly being torn down
> >> rerun I don't see how it could use keep-alives...that would invalidate
> the
> >> test wouldn't it? being that the 1st test makes a connection and
> >> subsequent
> >> tests reuse the same connection
> >>
> >
> >
>
Re: Bench marking a simple 10k write
Posted by Nick North <no...@gmail.com>.
On second thoughts, while my previous claim is true, I'm not sure what
happens when you use WebRequest.Create for each write, as in the example.
So maybe keep-alive is not enabled after all.
Nick
On 4 April 2014 10:35, Nick North <no...@gmail.com> wrote:
> The HttpWebRequest class has a KeepAlive
> <http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.keepalive.aspx>property
> that defaults to true under HTTP 1.1, so it should be enabled in the OP's
> example.
>
> Nick
>
>
> On 4 April 2014 10:24, Stanley Iriele <si...@gmail.com> wrote:
>
>> On Fri, Apr 4, 2014 at 1:28 AM, Benoit Chesneau <bc...@gmail.com>
>> wrote:
>>
>> > On Fri, Apr 4, 2014 at 10:25 AM, Will Holley <wi...@gmail.com>
>> wrote:
>> >
>> > > .NET should set keep-alive by default.
>> > >
>> >
>>
>> I have a hard time believing that because its the library you're using not
>> NET itself that decides that. Also If the test is truly being torn down
>> rerun I don't see how it could use keep-alives...that would invalidate the
>> test wouldn't it? being that the 1st test makes a connection and
>> subsequent
>> tests reuse the same connection
>>
>
>
Re: Bench marking a simple 10k write
Posted by Nick North <no...@gmail.com>.
The HttpWebRequest class has a KeepAlive
<http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.keepalive.aspx>property
that defaults to true under HTTP 1.1, so it should be enabled in the OP's
example.
Nick
On 4 April 2014 10:24, Stanley Iriele <si...@gmail.com> wrote:
> On Fri, Apr 4, 2014 at 1:28 AM, Benoit Chesneau <bc...@gmail.com>
> wrote:
>
> > On Fri, Apr 4, 2014 at 10:25 AM, Will Holley <wi...@gmail.com>
> wrote:
> >
> > > .NET should set keep-alive by default.
> > >
> >
>
> I have a hard time believing that because its the library you're using not
> NET itself that decides that. Also If the test is truly being torn down
> rerun I don't see how it could use keep-alives...that would invalidate the
> test wouldn't it? being that the 1st test makes a connection and subsequent
> tests reuse the same connection
>
Re: Bench marking a simple 10k write
Posted by Stanley Iriele <si...@gmail.com>.
On Fri, Apr 4, 2014 at 1:28 AM, Benoit Chesneau <bc...@gmail.com> wrote:
> On Fri, Apr 4, 2014 at 10:25 AM, Will Holley <wi...@gmail.com> wrote:
>
> > .NET should set keep-alive by default.
> >
>
I have a hard time believing that because its the library you're using not
NET itself that decides that. Also If the test is truly being torn down
rerun I don't see how it could use keep-alives...that would invalidate the
test wouldn't it? being that the 1st test makes a connection and subsequent
tests reuse the same connection
Re: Bench marking a simple 10k write
Posted by Benoit Chesneau <bc...@gmail.com>.
On Fri, Apr 4, 2014 at 10:25 AM, Will Holley <wi...@gmail.com> wrote:
> .NET should set keep-alive by default.
>
uh? any link about it?
Re: Bench marking a simple 10k write
Posted by Will Holley <wi...@gmail.com>.
.NET should set keep-alive by default.
What's the latency to the CouchDB server that you're testing against?
On 4 April 2014 07:57, Benoit Chesneau <bc...@gmail.com> wrote:
> On Fri, Apr 4, 2014 at 4:40 AM, Knudsen, Ken <
> Ken.Knudsen@imaginecommunications.com> wrote:
>
>> ....to a CouchDB server....I'm literally just executing a unit test 10k
>> times that open an HTTP connection, post the data and close... CouchDB is
>> performing very slowly in this. I thought it'd be very fast. Any ideas?
>>
>> Did you reuse the connectons ? ( with HTTP keepalive)
>
> - benoit
Re: Bench marking a simple 10k write
Posted by Benoit Chesneau <bc...@gmail.com>.
On Fri, Apr 4, 2014 at 4:40 AM, Knudsen, Ken <
Ken.Knudsen@imaginecommunications.com> wrote:
> ....to a CouchDB server....I'm literally just executing a unit test 10k
> times that open an HTTP connection, post the data and close... CouchDB is
> performing very slowly in this. I thought it'd be very fast. Any ideas?
>
> Did you reuse the connectons ? ( with HTTP keepalive)
- benoit