Metrics Framework: Nested Durations and Multi-threading

[Bob Briody <bo...@datastax.com>]

The Metrics framework provides vendors the ability to enhance the
information about a Traversal's behavior by adding nested Metrics that
include timers/durations and annotations.

2 areas of confusion arose when Titan began utilizing this capability, so
I'd like to set out some guidelines to clear things up. I'm starting the
discussion here with the expectation that the decisions we make will result
in some formal documentation.

The goal is to strike a balance between
- ease of use and interpretation from the user perspective
- TinkerPop Metrics Framework generality and extensibillity
- vendor implementation burden


*#1 Nested Metrics Durations*

The duration of a child Metrics will be included/captured in its parent’s
duration. Thus, the sum of the durations of all child Metrics will never be
greater than the duration of the parent. This includes the duration of
tasks that are executed in separate threads (covered below).

Note: This should be familiar since it aligns with the approach used by
profilers such as VisualVM
http://www.infoq.com/resource/articles/java-profiling-with-open-source/en/resources/figure8large.jpg


*#2 Handling parallel execution*

In cases of parallel execution/multi-threading, vendors should create a
separate Metrics object for each thread. The Metrics Framework will provide
a mechanism for nesting a collection of these parallel Metrics, and this
group of parallel Metrics will be automatically condensed into a single
Metrics "row". A user will then be able to easily obtain the
longest-running Metrics, all individual Metrics in the group, as well as
the number of Metrics/threads and any annotations that were added to the
group.

The duration of the parent Metrics must include the time spent waiting for
all threads to finish executing (thus adhering to the previous guideline
regarding nested Metrics duration).


*Some notes on #2*
I am assuming that in cases of parallel execution, users will be most
interested in the longest-running duration, and secondarily interested in
inspecting information about all threads in aggregate (probably to identify
signs of potentially high CPU load). I'm also assuming that it will not be
a deal-breaker for vendor's to create a Metrics object for each thread.
Feedback here is especially appreciated.

-Bob

Re: Metrics Framework: Nested Durations and Multi-threading

[Bob Briody <bo...@datastax.com>]

Sorry, I meant to say that the Parent duration has to be "greater than or
equal to 8", and I couldn't let it go.

Bob

On Mon, Jun 1, 2015 at 8:04 PM, Bob Briody <bo...@datastax.com> wrote:

> For a group of aggregated parallel metrics the duration would be equal to
> the greatest duration of the group.
>
> So a set of Metrics like this:
>
> Parent                -Dur 8*
> Parallel Child 0  -Dur 3
> Parallel Child 1  -Dur 3
> Parallel Child 2  -Dur 8
>
> (*The Parent duration doesn't have to be exactly 8, but it does have to be
> greater than 8.)
>
> Would condense to this:
>
> Parent                                        -Dur   8
> Child (actually a Metrics group) -Dur   8
>
> ...and the Child Metrics would provide getters to obtain the individual
> parallel Metrics for detailed inspection.
>
>
> Bob
>
> On Mon, Jun 1, 2015 at 5:59 PM, Matthias Broecheler <me...@matthiasb.com>
> wrote:
>
>> >
>> > *#2 Handling parallel execution*
>> >
>> > In cases of parallel execution/multi-threading, vendors should create a
>> > separate Metrics object for each thread. The Metrics Framework will
>> provide
>> > a mechanism for nesting a collection of these parallel Metrics, and this
>> > group of parallel Metrics will be automatically condensed into a single
>> > Metrics "row". A user will then be able to easily obtain the
>> > longest-running Metrics, all individual Metrics in the group, as well as
>> > the number of Metrics/threads and any annotations that were added to the
>> > group.
>> >
>>
>> Does this mean that #1 does not apply for this group or would you still
>> have to sum all those parallel executions up to yield the parent metric?
>> If the latter, wouldn't that be confusing since the resulting number can
>> be
>> larger than the total query time? If the former, what is the aggregate
>> function used on multiple parallel execution metrics?
>>
>>
>>
>> >
>> > The duration of the parent Metrics must include the time spent waiting
>> for
>> > all threads to finish executing (thus adhering to the previous guideline
>> > regarding nested Metrics duration).
>> >
>> >
>> > *Some notes on #2*
>> > I am assuming that in cases of parallel execution, users will be most
>> > interested in the longest-running duration, and secondarily interested
>> in
>> > inspecting information about all threads in aggregate (probably to
>> identify
>> > signs of potentially high CPU load). I'm also assuming that it will not
>> be
>> > a deal-breaker for vendor's to create a Metrics object for each thread.
>> > Feedback here is especially appreciated.
>> >
>> > -Bob
>> >
>>
>
>

Re: Metrics Framework: Nested Durations and Multi-threading

[Matthias Broecheler <me...@matthiasb.com>]

>
> *#2 Handling parallel execution*
>
> In cases of parallel execution/multi-threading, vendors should create a
> separate Metrics object for each thread. The Metrics Framework will provide
> a mechanism for nesting a collection of these parallel Metrics, and this
> group of parallel Metrics will be automatically condensed into a single
> Metrics "row". A user will then be able to easily obtain the
> longest-running Metrics, all individual Metrics in the group, as well as
> the number of Metrics/threads and any annotations that were added to the
> group.
>

Does this mean that #1 does not apply for this group or would you still
have to sum all those parallel executions up to yield the parent metric?
If the latter, wouldn't that be confusing since the resulting number can be
larger than the total query time? If the former, what is the aggregate
function used on multiple parallel execution metrics?



>
> The duration of the parent Metrics must include the time spent waiting for
> all threads to finish executing (thus adhering to the previous guideline
> regarding nested Metrics duration).
>
>
> *Some notes on #2*
> I am assuming that in cases of parallel execution, users will be most
> interested in the longest-running duration, and secondarily interested in
> inspecting information about all threads in aggregate (probably to identify
> signs of potentially high CPU load). I'm also assuming that it will not be
> a deal-breaker for vendor's to create a Metrics object for each thread.
> Feedback here is especially appreciated.
>
> -Bob
>