[jira] [Resolved] (NIP-19) Add Support for Gauges to Processor Metrics

["Pierre Villard (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIP-19?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Pierre Villard resolved NIP-19.
-------------------------------
    Resolution: Implemented

> Add Support for Gauges to Processor Metrics
> -------------------------------------------
>
>                 Key: NIP-19
>                 URL: https://issues.apache.org/jira/browse/NIP-19
>             Project: NiFi Improvement Proposal
>          Issue Type: New Feature
>            Reporter: David Handermann
>            Assignee: David Handermann
>            Priority: Major
>
> h2. Motivation
> The framework has supported Counters as a general method for tracking selected behaviors in Processors since project inception. Although Counters can be used for a number of scenarios, they are limited to reporting values that accumulate over time.
> Adding support for Gauges would enable Processors to report values at a given point in time. Gauges support tracking for numbers such as the processing rate of an operation, or the size of temporary buffers used for an operation, among other metrics.
> h2. Scope
> The initial scope involves adding a new method to the {{ProcessSession}} interface in the public Apache NiFi API. The initial implementation should include updates to the standard implementations of {{ProcessSession}} in the framework, along with changes to the {{StatusHistoryRepository}} and other framework components to capture and process Gauges.
> Subsequent implementation could include updates to the user interface to present captured Gauge metrics. User interface changes may involve additional design consideration for optimal presentation in the NiFi application. For these reasons, any changes to the user interface should be evaluated after the initial NiFi API and framework level changes.
> h2. Compatibility
> As an addition to the {{ProcessSession}} interface, support for Gauges does not present any potential break in backward compatibility, and thus does not require a new major version of either the Apache NiFi API or the Apache NiFi framework.
> h2. Description
> The implementation strategy consists of NiFi API additions, NiFi framework updates, NiFi REST API updates, and NiFi UI changes.
> h3. NiFi API
> Adding support for Gauges to Processor metrics begins with targeted updates to the NiFi API, adding a method to the {{ProcessSession}} interface. The method should accept a name, a number, and flag indicating commit timing.
> {{void recordGauge(String name, double value, CommitTiming commitTiming);}}
> The {{CommitTiming}} argument is a new {{enum}} that should have the following values:
>  * NOW
>  * SESSION_COMMITTED
> The {{CommitTiming}} enumeration follows the pattern of the Boolean {{immediate}} flag in the existing {{adjustCounter}} method, but using an {{enum}} provides clearer semantics when writing and reading code for implementing Processors.
> h3. NiFi Framework
> Implementation in the NiFi framework involves implementing the new {{ProcessSession}} method in both standard classes and the {{nifi-mock}} library. The {{StandardProcessSession}} is the primary point of implementation, requiring new instance variables for storing recored Gauge values analogous to current Maps for counters.
> Additional framework changes include updates to the {{RepositoryContext}} interface, the addition of an intermediate {{GaugeRepository}} analogous to the {{{}CounterRepository{}}}, and potential changes to the {{{}StatusHistoryRepository{}}}. As framework interfaces, these changes can be coordinated within the scope of implementation for general framework updates.
> Framework implementation may include the introduction of new application properties for configuring settings specific to Gauges.
> h3. NiFi REST API
> Following basic implementation at the framework level, a new {{GaugesResource}} should be created, analogous to the current {{{}CountersResource{}}}, providing REST API methods for listing current Gauges and values.
> h3. NiFi UI
> Potential NiFi user interface changes include integrating Gauges with Status History information. With certain Node information such as CPU and memory already rendered based on gauges, any new implementation should follow the same general pattern.
> Unlike Counters that show an accumulated number, rendering a simple list of Gauges could limit the presentation to the most recent value recorded.
> h2. Verification
> Changes to the {{ProcessSession}} interface in NiFi API are self-contained. Updates to the NiFi framework and other components should follow standard unit test patterns. As a framework feature, Gauges would also be a candidate for system testing.
> h2. Alternatives
> Possible alternatives include custom implementation in each Processor that requires the ability to record metrics using Gauges. This might work for a small number of Processors, but would not be suitable as a pattern for implementation across the Apache NiFi project.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)