SetPropertyTest progress

[Jukka Zitting <ju...@gmail.com>]

Hi,

SetPropertyTest is a simple benchmark that measures how long it takes
to execute the following snippet of code:

    for (int i = 0; i < 1000; i++) {
        node.setProperty("count", i);
        session.save();
    }

The idea behind this benchmark is to get an estimate of the minimum
amount of time it takes to write even the simplest changes to the
repository.

Last month I started looking at this benchmark as it highlights many
of our current bottlenecks, like slow access to node type information
and repetitive commit hooks, that affect also many other operations.

I'm glad to report that our combined efforts on these fronts are
paying off. Here's how we did a month ago:

    # SetPropertyTest                min     10%     50%     90%     max       N
    Oak-Default                      599     608     638     764     975      91
    Oak-Segment                     4124    4141    4206    4608    4699      14

And here's the latest status now:

    # SetPropertyTest                min     10%     50%     90%     max       N
    Oak-Default                      318     325     350     420     791     162
    Oak-Segment                     1030    1057    1070    1123    1264      56

In other words, the performance boosts for this benchmark are roughly
2x and 4x for the H2 and segment MKs, respectively.

For comparison, here's how Jackrabbit is doing:

    # SetPropertyTest                min     10%     50%     90%     max       N
    Jackrabbit                       260     276     299     351     650     192

It's still faster than Oak, but we're getting pretty close and there's
still a whole bunch of further optimizations we can do.

PS. This benchmark is unfortunately particularly ill-suited for use
with the MongoMK, as it ends up blowing up the revision history of a
single node:

    # SetPropertyTest                min     10%     50%     90%     max       N
    Oak-Mongo                      31442   32098   58855   92934   93871      10

BR,

Jukka Zitting

Re: SetPropertyTest progress

[Jukka Zitting <ju...@gmail.com>]

Hi,

On Wed, Mar 27, 2013 at 12:25 PM, Thomas Mueller <mu...@adobe.com> wrote:
> Yes, that's a problem. The plan is to split documents once they are too
> large (if they contain many properties, large properties, or many
> updates). This is partially implemented, but disabled currently. When
> enabled, it shouldn't be a problem.

Cool. The SetPropertyTest is a rather extreme edge case with its rapid
sequence of thousands of changes to a single node, that shouldn't
occur too often in practice (only something like a page-view counter
might behave similarly), but it's good to be prepared when something
like that eventually does happen.

BR,

Jukka Zitting

Re: SetPropertyTest progress

[Thomas Mueller <mu...@adobe.com>]

Hi,

Thanks a lot!

>PS. This benchmark is unfortunately particularly ill-suited for use
>with the MongoMK, as it ends up blowing up the revision history of a
>single node:
>
>    # SetPropertyTest                min     10%     50%     90%     max
>     N
>    Oak-Mongo                      31442   32098   58855   92934   93871
>    10

Yes, that's a problem. The plan is to split documents once they are too
large (if they contain many properties, large properties, or many
updates). This is partially implemented, but disabled currently. When
enabled, it shouldn't be a problem.

Regards,
Thomas