New user // CouchD 3.5.2 Slow bulk insert on Windows 11

[Momchil Bozhinov <mo...@bozhinov.eu>]

> Hello,
>
> This is my first day of trying out CouchDb (1 node). Looking for a 
> MongoDb alternative.
>
> First thing I noticed - no collections. That's fine though.
>
> I tried bulk inserting 110 Mb list of dictionaries (~250 000 entries) 
> using pycouchdb and couchdb3
>
Failed with timeout after 30 minutes and failed to insert the last ~10k 
events.
>
> Tried disabling the compression - that did not help
>
> Looked for the option to enable Delayed commit but I could not find it.
>
> ref https://guide.couchdb.org/draft/performance.html
>
Google AI suggested I add some config options to use the kernel directly 
and increase the number of processes.

Upon checking those turned out old and obsolete.

> Seems to be creating a bunch of stuff (keys, ids) for each entry. I 
> guess that is taking a while.
>
My setup is a xen vm running on top of an old HP DL360 G8. Maybe I lack 
CPU instructions ?
>
> Please help ?
>
> Momchil
>
>
>

Re: New user // CouchD 3.5.2 Slow bulk insert on Windows 11

[Momchil Bozhinov <mo...@bozhinov.eu>]

Hello,

Same operation in MongoDb takes 7 seconds.

CouchDb fails after 30 minutes with only 94% inserts successful

Momchil

On 5/23/2026 12:33 PM, Momchil Bozhinov wrote:
>
> Am sure, it's 1 line
>
> self.db.database(self.db_name).save_bulk(data)
>
> Where data is a list of dictionaries, each with 6 members of short 
> length strings
>
>
> On 5/23/2026 11:34 AM, Cluxter wrote:
>> Hi,
>>
>> Could you share the code that you have been using to insert your 
>> data? It doesn't have to be your whole codebase, just the part the 
>> takes care of the inserts would be enough.
>>
>>
>> Le ven. 22 mai 2026 à 08:37, Momchil Bozhinov <mo...@bozhinov.eu> a 
>> écrit :
>>
>>     Hi,
>>
>>     working around the timeout is the easy part. the problem is that
>>     it took
>>     30 minutes and that is not acceptable
>>
>>     Compared to the rest of the solutions I have, it's not even the same
>>     ball park.
>>
>>     Is there no way to turn off revisions or pick my own indexes ?
>>
>>     It would also seem to write to the disk a lot.
>>
>>     Momchil
>>
>>     On 5/21/2026 10:24 PM, Cluxter wrote:
>>     > Since you know that it almost got inserted entirely, you could
>>     simply split
>>     > your entries in 2 files of 125,000 entries each and insert
>>     these 2 files
>>     > one after the other.
>>     >
>>     > If your need a reliable way to do these kinds of insertions on
>>     a regular
>>     > basis, then there is probably a way to adjust the timeout of
>>     the HTTP
>>     > connection somewhere to extend it to more than 30 minutes.
>>     >
>>     > It depends on your real need.
>>     >
>>     > Google AI is talking nonsense. Erlang already uses one process
>>     for each CPU
>>     > core and maxes out the usage of your CPU, so what Google AI
>>     told you is
>>     > nothing here or there.
>>     >
>>     > Le jeu. 21 mai 2026 à 21:10, Momchil Bozhinov
>>     <mo...@bozhinov.eu> a
>>     > écrit :
>>     >
>>     >>> Hello,
>>     >>>
>>     >>> This is my first day of trying out CouchDb (1 node). Looking
>>     for a
>>     >>> MongoDb alternative.
>>     >>>
>>     >>> First thing I noticed - no collections. That's fine though.
>>     >>>
>>     >>> I tried bulk inserting 110 Mb list of dictionaries (~250 000
>>     entries)
>>     >>> using pycouchdb and couchdb3
>>     >>>
>>     >> Failed with timeout after 30 minutes and failed to insert the
>>     last ~10k
>>     >> events.
>>     >>> Tried disabling the compression - that did not help
>>     >>>
>>     >>> Looked for the option to enable Delayed commit but I could
>>     not find it.
>>     >>>
>>     >>> ref https://guide.couchdb.org/draft/performance.html
>>     >>>
>>     >> Google AI suggested I add some config options to use the
>>     kernel directly
>>     >> and increase the number of processes.
>>     >>
>>     >> Upon checking those turned out old and obsolete.
>>     >>
>>     >>> Seems to be creating a bunch of stuff (keys, ids) for each
>>     entry. I
>>     >>> guess that is taking a while.
>>     >>>
>>     >> My setup is a xen vm running on top of an old HP DL360 G8.
>>     Maybe I lack
>>     >> CPU instructions ?
>>     >>> Please help ?
>>     >>>
>>     >>> Momchil
>>     >>>
>>     >>>
>>     >>>
>>

Re: New user // CouchD 3.5.2 Slow bulk insert on Windows 11

[Cluxter <co...@cluxter.org>]

Since you know that it almost got inserted entirely, you could simply split
your entries in 2 files of 125,000 entries each and insert these 2 files
one after the other.

If your need a reliable way to do these kinds of insertions on a regular
basis, then there is probably a way to adjust the timeout of the HTTP
connection somewhere to extend it to more than 30 minutes.

It depends on your real need.

Google AI is talking nonsense. Erlang already uses one process for each CPU
core and maxes out the usage of your CPU, so what Google AI told you is
nothing here or there.

Le jeu. 21 mai 2026 à 21:10, Momchil Bozhinov <mo...@bozhinov.eu> a
écrit :

> > Hello,
> >
> > This is my first day of trying out CouchDb (1 node). Looking for a
> > MongoDb alternative.
> >
> > First thing I noticed - no collections. That's fine though.
> >
> > I tried bulk inserting 110 Mb list of dictionaries (~250 000 entries)
> > using pycouchdb and couchdb3
> >
> Failed with timeout after 30 minutes and failed to insert the last ~10k
> events.
> >
> > Tried disabling the compression - that did not help
> >
> > Looked for the option to enable Delayed commit but I could not find it.
> >
> > ref https://guide.couchdb.org/draft/performance.html
> >
> Google AI suggested I add some config options to use the kernel directly
> and increase the number of processes.
>
> Upon checking those turned out old and obsolete.
>
> > Seems to be creating a bunch of stuff (keys, ids) for each entry. I
> > guess that is taking a while.
> >
> My setup is a xen vm running on top of an old HP DL360 G8. Maybe I lack
> CPU instructions ?
> >
> > Please help ?
> >
> > Momchil
> >
> >
> >