You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by ermouth <er...@gmail.com> on 2019/10/04 21:40:48 UTC
Re: CouchDB and future
This tuned out to be interesting, results vary heavily for different
platforms.
I included `perftest.html` into Photon design doc. The utility gives good
insight how q,n impact doc creation, doc update, validate + update, and
viewindex create/update – for a particular CouchDB instance.
Common observations:
* q=1 is only ok for rare special cases (like DBs with dozen docs)
* it’s ok to have q==cores or q==cores+1 or even cores+2: write perf drops
a little, but viewindex update is bit faster
* excessive q (say, more than cores×2) negatively impacts direct write
perf: having q8 for 2core at least halves write perf for most
commodity/lean platforms and hw cfg.
ermouth
чт, 11 июл. 2019 г. в 18:17, Joan Touzet <wo...@apache.org>:
> On 2019-07-11 8:05, ermouth wrote:
> >> to help with the documentation step
> >
> > Probably. Please give me a hint what you need.
>
> What default settings right now are wrong for a system with low RAM, low
> CPU, and slow disk - such as a RaspberryPi v1, or a $15/mo AWS server?
>
> -Joan
>
>
Re: CouchDB and future
Posted by Robert Samuel Newson <rn...@apache.org>.
Not even that. :) If you have one node, then your dbs are N=1 even if N is set to 3 in config. the code takes the lower of the n setting and the number of known nodes (which don't have to be up at the time of db creation).
B.
> On 5 Oct 2019, at 21:00, Joan Touzet <wo...@apache.org> wrote:
>
> Agree - couldn't find the reference from earlier, but I believe the
> rules should be:
>
> If you have 1 node, set n=1.
> If you have 2 nodes, set n=2.
> If you have 3 or more nodes, set n=3.
>
> Anything else is a special case and isn't recommended.
>
> -Joan
>
> On 2019-10-05 15:20, Robert Samuel Newson wrote:
>> Pulling this sentence out from much earlier in the thread;
>>
>> "N should be set to the number of nodes."
>>
>> That's not true and our docs should not say it today or be changed to say it in the future.
>>
>> N is how many copies of a document we'll maintain. If you had a 20 node cluster, you would not want 20 copies of every document. The default of 3 copies would be sufficient to prevent accidental loss.
>>
>> B.
>>
>>> On 4 Oct 2019, at 23:55, Joan Touzet <wo...@apache.org> wrote:
>>>
>>> Thanks for the follow-up. Thinking of >1 active DB per host makes this
>>> calculation a lot more challenging.
>>>
>>> Sounds like our recently merged default change of q=2 will be fine.
>>>
>>> -Joan
>>>
>>>
>>> On 2019-10-04 17:40, ermouth wrote:
>>>> This tuned out to be interesting, results vary heavily for different
>>>> platforms.
>>>>
>>>> I included `perftest.html` into Photon design doc. The utility gives good
>>>> insight how q,n impact doc creation, doc update, validate + update, and
>>>> viewindex create/update – for a particular CouchDB instance.
>>>>
>>>> Common observations:
>>>> * q=1 is only ok for rare special cases (like DBs with dozen docs)
>>>> * it’s ok to have q==cores or q==cores+1 or even cores+2: write perf drops
>>>> a little, but viewindex update is bit faster
>>>> * excessive q (say, more than cores×2) negatively impacts direct write
>>>> perf: having q8 for 2core at least halves write perf for most
>>>> commodity/lean platforms and hw cfg.
>>>>
>>>> ermouth
>>>>
>>>>
>>>> чт, 11 июл. 2019 г. в 18:17, Joan Touzet <wo...@apache.org>:
>>>>
>>>>> On 2019-07-11 8:05, ermouth wrote:
>>>>>>> to help with the documentation step
>>>>>>
>>>>>> Probably. Please give me a hint what you need.
>>>>>
>>>>> What default settings right now are wrong for a system with low RAM, low
>>>>> CPU, and slow disk - such as a RaspberryPi v1, or a $15/mo AWS server?
>>>>>
>>>>> -Joan
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: CouchDB and future
Posted by Joan Touzet <wo...@apache.org>.
Agree - couldn't find the reference from earlier, but I believe the
rules should be:
If you have 1 node, set n=1.
If you have 2 nodes, set n=2.
If you have 3 or more nodes, set n=3.
Anything else is a special case and isn't recommended.
-Joan
On 2019-10-05 15:20, Robert Samuel Newson wrote:
> Pulling this sentence out from much earlier in the thread;
>
> "N should be set to the number of nodes."
>
> That's not true and our docs should not say it today or be changed to say it in the future.
>
> N is how many copies of a document we'll maintain. If you had a 20 node cluster, you would not want 20 copies of every document. The default of 3 copies would be sufficient to prevent accidental loss.
>
> B.
>
>> On 4 Oct 2019, at 23:55, Joan Touzet <wo...@apache.org> wrote:
>>
>> Thanks for the follow-up. Thinking of >1 active DB per host makes this
>> calculation a lot more challenging.
>>
>> Sounds like our recently merged default change of q=2 will be fine.
>>
>> -Joan
>>
>>
>> On 2019-10-04 17:40, ermouth wrote:
>>> This tuned out to be interesting, results vary heavily for different
>>> platforms.
>>>
>>> I included `perftest.html` into Photon design doc. The utility gives good
>>> insight how q,n impact doc creation, doc update, validate + update, and
>>> viewindex create/update – for a particular CouchDB instance.
>>>
>>> Common observations:
>>> * q=1 is only ok for rare special cases (like DBs with dozen docs)
>>> * it’s ok to have q==cores or q==cores+1 or even cores+2: write perf drops
>>> a little, but viewindex update is bit faster
>>> * excessive q (say, more than cores×2) negatively impacts direct write
>>> perf: having q8 for 2core at least halves write perf for most
>>> commodity/lean platforms and hw cfg.
>>>
>>> ermouth
>>>
>>>
>>> чт, 11 июл. 2019 г. в 18:17, Joan Touzet <wo...@apache.org>:
>>>
>>>> On 2019-07-11 8:05, ermouth wrote:
>>>>>> to help with the documentation step
>>>>>
>>>>> Probably. Please give me a hint what you need.
>>>>
>>>> What default settings right now are wrong for a system with low RAM, low
>>>> CPU, and slow disk - such as a RaspberryPi v1, or a $15/mo AWS server?
>>>>
>>>> -Joan
>>>>
>>>>
>>>
>>
>
Re: CouchDB and future
Posted by Robert Samuel Newson <rn...@apache.org>.
Pulling this sentence out from much earlier in the thread;
"N should be set to the number of nodes."
That's not true and our docs should not say it today or be changed to say it in the future.
N is how many copies of a document we'll maintain. If you had a 20 node cluster, you would not want 20 copies of every document. The default of 3 copies would be sufficient to prevent accidental loss.
B.
> On 4 Oct 2019, at 23:55, Joan Touzet <wo...@apache.org> wrote:
>
> Thanks for the follow-up. Thinking of >1 active DB per host makes this
> calculation a lot more challenging.
>
> Sounds like our recently merged default change of q=2 will be fine.
>
> -Joan
>
>
> On 2019-10-04 17:40, ermouth wrote:
>> This tuned out to be interesting, results vary heavily for different
>> platforms.
>>
>> I included `perftest.html` into Photon design doc. The utility gives good
>> insight how q,n impact doc creation, doc update, validate + update, and
>> viewindex create/update – for a particular CouchDB instance.
>>
>> Common observations:
>> * q=1 is only ok for rare special cases (like DBs with dozen docs)
>> * it’s ok to have q==cores or q==cores+1 or even cores+2: write perf drops
>> a little, but viewindex update is bit faster
>> * excessive q (say, more than cores×2) negatively impacts direct write
>> perf: having q8 for 2core at least halves write perf for most
>> commodity/lean platforms and hw cfg.
>>
>> ermouth
>>
>>
>> чт, 11 июл. 2019 г. в 18:17, Joan Touzet <wo...@apache.org>:
>>
>>> On 2019-07-11 8:05, ermouth wrote:
>>>>> to help with the documentation step
>>>>
>>>> Probably. Please give me a hint what you need.
>>>
>>> What default settings right now are wrong for a system with low RAM, low
>>> CPU, and slow disk - such as a RaspberryPi v1, or a $15/mo AWS server?
>>>
>>> -Joan
>>>
>>>
>>
>
Re: CouchDB and future
Posted by Joan Touzet <wo...@apache.org>.
Thanks for the follow-up. Thinking of >1 active DB per host makes this
calculation a lot more challenging.
Sounds like our recently merged default change of q=2 will be fine.
-Joan
On 2019-10-04 17:40, ermouth wrote:
> This tuned out to be interesting, results vary heavily for different
> platforms.
>
> I included `perftest.html` into Photon design doc. The utility gives good
> insight how q,n impact doc creation, doc update, validate + update, and
> viewindex create/update – for a particular CouchDB instance.
>
> Common observations:
> * q=1 is only ok for rare special cases (like DBs with dozen docs)
> * it’s ok to have q==cores or q==cores+1 or even cores+2: write perf drops
> a little, but viewindex update is bit faster
> * excessive q (say, more than cores×2) negatively impacts direct write
> perf: having q8 for 2core at least halves write perf for most
> commodity/lean platforms and hw cfg.
>
> ermouth
>
>
> чт, 11 июл. 2019 г. в 18:17, Joan Touzet <wo...@apache.org>:
>
>> On 2019-07-11 8:05, ermouth wrote:
>>>> to help with the documentation step
>>>
>>> Probably. Please give me a hint what you need.
>>
>> What default settings right now are wrong for a system with low RAM, low
>> CPU, and slow disk - such as a RaspberryPi v1, or a $15/mo AWS server?
>>
>> -Joan
>>
>>
>
Re: CouchDB and future
Posted by Jan Lehnardt <ja...@apache.org>.
Hi ermouth,
thanks for providing these. Do you have a feeling for a scenario with q>1 on a single-core. single-thread system?
Best
Jan
—
> On 4. Oct 2019, at 23:40, ermouth <er...@gmail.com> wrote:
>
> This tuned out to be interesting, results vary heavily for different
> platforms.
>
> I included `perftest.html` into Photon design doc. The utility gives good
> insight how q,n impact doc creation, doc update, validate + update, and
> viewindex create/update – for a particular CouchDB instance.
>
> Common observations:
> * q=1 is only ok for rare special cases (like DBs with dozen docs)
> * it’s ok to have q==cores or q==cores+1 or even cores+2: write perf drops
> a little, but viewindex update is bit faster
> * excessive q (say, more than cores×2) negatively impacts direct write
> perf: having q8 for 2core at least halves write perf for most
> commodity/lean platforms and hw cfg.
>
> ermouth
>
>
> чт, 11 июл. 2019 г. в 18:17, Joan Touzet <wo...@apache.org>:
>
>> On 2019-07-11 8:05, ermouth wrote:
>>>> to help with the documentation step
>>>
>>> Probably. Please give me a hint what you need.
>>
>> What default settings right now are wrong for a system with low RAM, low
>> CPU, and slow disk - such as a RaspberryPi v1, or a $15/mo AWS server?
>>
>> -Joan
>>
>>
--
Professional Support for Apache CouchDB:
https://neighbourhood.ie/couchdb-support/