[jira] [Created] (YARN-4716) TimelineClient to implement Flushable; propagate to writer

["Steve Loughran (JIRA)" <ji...@apache.org>]

Steve Loughran created YARN-4716:
------------------------------------

             Summary: TimelineClient to implement Flushable; propagate to writer
                 Key: YARN-4716
                 URL: https://issues.apache.org/jira/browse/YARN-4716
             Project: Hadoop YARN
          Issue Type: Sub-task
          Components: timelineserver
    Affects Versions: 2.8.0
            Reporter: Steve Loughran


I need a {{flush()}} operation in the timeline client.  Knowing the lifecycle of my app, I do explicitly want to flush up events at certain points (app start event, app end, without relying on an async flush client)

Right now, in tests, the time it takes for an event to propagate is: flush delay + scan delay, you are looking at 2 seconds minimum, per test case, with less deterministic outcomes.

In production, those big app lifecycle events are so important my client code currently explicitly flushes my own event queue, and expects them to reach the destination

With the filesystem writer, I've lost those durability guarantees.

Implementing {{Flushable.flush()}} would let my app tell the timeline client to write; it'd be a no-op on the http client, but for the filesystem writer (or any other async writer), it'd be expected to force a write to the durable medium.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)