SetPropertyTest / test result distortion by background activity

[Julian Reschke <ju...@gmx.de>]

Hi there,

as a quick check, I have bumped up the number of iterations per test 
from 1000 to 5000. In this case I see

> # SetPropertyTest                  C     min     10%     50%     90%     max
>    N
> Oak-Mongo                          1    6720    6767    7006    8031    8802
>   42

where the ratio between best and worst values is 8802/6729 ~ 130%.

Comparison: for 1000 iterations, I see

> # SetPropertyTest                  C     min     10%     50%     90%     max
>    N
> Oak-Mongo                          1    1250    1324    1460    1663    2514
>  202

with a ratio of ~ 200%.

(For RDB-in-Postgres, I see 220% and 400%)

Conclusion:

a) the longer tests run, the smaller is the variation

b) DocumentMK with RDB varies more than DocumentMK with Mongo


Best regards, Julian