You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Kevin Doran <kd...@apache.org> on 2026/05/27 19:34:44 UTC

[lazy-consensus vote] NIP-30: Connector Observability

Team,

I'd like to start a vote thread for NIP-30 [1] which aims to extend
existing observability features to the new Connectors feature. Mainly,
it will allow Connectors to have custom MDC logging attributes and
have their statuses included in the interfaces used by reporting
tasks.

This is a small change, so using the updated NIP process recently
adopted, this will be a 24-hour lazy consensus vote [2].

I already have a draft PR up for these changes [3] for those that want
a feel for what the changes will look like.

Cheers,
Kevin

[1] https://issues.apache.org/jira/browse/NIP-30
[2] https://apache.org/foundation/voting.html#LazyConsensus
[3] https://github.com/apache/nifi-api/pull/89

Re: [lazy-consensus vote] NIP-30: Connector Observability

Posted by Kevin Doran <kd...@apache.org>.
The vote passes. I will follow the normal RTC process to make sure the
details of the API changes are peer reviewed before merging to main.

Thanks all!
Kevin

On Mon, Jun 1, 2026 at 9:34 AM David Handermann
<ex...@apache.org> wrote:
>
> Kevin,
>
> Thanks for reconsidering the approach and proposing the adjustments on
> the Connector Configuration Provider. That looks like a much better
> strategy, keeping the logging attributes within the boundary of the
> framework, and extensible through the nifi-framework-api.
>
> With that change, +1 from my side, and looking forward to moving ahead
> with implementation.
>
> Regards,
> David Handermann
>
> On Mon, Jun 1, 2026 at 8:03 AM Kevin Doran <kd...@apache.org> wrote:
> >
> > Hey David,
> >
> > Thanks for the review. I agree with your feedback, allowing
> > per-connector sourced MDC attributes via the
> > ConnectorInitializationContext is too broad. I have modified the NIP
> > to propose that attributes always come from the framework. This would
> > include a default set of attributes such as Connector Id, Name, and
> > Component. Extensibility would only be via Framework extensions
> > (nifi-framework-api), such as ConnectorConfigurationProvider, rather
> > than at the component level in nifi-api.
> >
> > The other suggested improvements to surface Connector Statuses in
> > Reporting Task APIs remains unchanged.
> >
> > I think this is a better and safer approach, so thanks for calling our
> > the potential issues with the original proposal. Hopefully this is
> > agreeable.
> >
> > I have extended this NIP vote for 24 hours.
> >
> > On Fri, May 29, 2026 at 8:29 PM David Handermann
> > <ex...@apache.org> wrote:
> > >
> > > Kevin,
> > >
> > > Thanks for the reply!
> > >
> > > I provided some feedback on the corresponding pull request for
> > > nifi-api [1]. I agree with all of the changes, except for the
> > > setLoggingAttributes addition to the ConnectorInitializationContext.
> > >
> > > Based on that concern, I am reluctantly voting -1 on the proposal as it stands.
> > >
> > > I think it would be better to make changes to the MDC at the framework
> > > level. If there are features of a Connector that should be included,
> > > I'm certainly open to considering additions in that direction.
> > >
> > > I would also be glad to move forward with a 24 hour vote on this topic
> > > once we have an agreeable way forward.
> > >
> > > Regards,
> > > David Handermann
> > >
> > > [1] https://github.com/apache/nifi-api/pull/89
> > >
> > > On Thu, May 28, 2026 at 10:00 AM Kevin Doran <kd...@apache.org> wrote:
> > > >
> > > > Sounds good, thanks for the input, David!
> > > >
> > > > The vote has been extended to 72 hours.
> > > >
> > > > On Wed, May 27, 2026 at 9:11 PM David Handermann
> > > > <ex...@apache.org> wrote:
> > > > >
> > > > > Kevin,
> > > > >
> > > > > Thanks for proposing these improvements to support observability at
> > > > > the Connector level.
> > > > >
> > > > > Providing status tracking and attribution at the Connector level makes
> > > > > sense, and provides a very useful way of tracking behavior. On initial
> > > > > review, I'm concerned about introducing the generic Logging Attributes
> > > > > concept at the public API level on the Connector Initialization
> > > > > Context. As it exists now at the framework level, there are a
> > > > > different set of concerns around exposing it in the public API.
> > > > >
> > > > > With that background, I'm not ready to vote for or against the
> > > > > proposal right now. Given the concern, I believe this needs to become
> > > > > a 72 hour vote. I plan to review the proposal in more detail and
> > > > > follow up soon.
> > > > >
> > > > > Regards,
> > > > > David Handermann
> > > > >
> > > > > On Wed, May 27, 2026 at 2:35 PM Kevin Doran <kd...@apache.org> wrote:
> > > > > >
> > > > > > Team,
> > > > > >
> > > > > > I'd like to start a vote thread for NIP-30 [1] which aims to extend
> > > > > > existing observability features to the new Connectors feature. Mainly,
> > > > > > it will allow Connectors to have custom MDC logging attributes and
> > > > > > have their statuses included in the interfaces used by reporting
> > > > > > tasks.
> > > > > >
> > > > > > This is a small change, so using the updated NIP process recently
> > > > > > adopted, this will be a 24-hour lazy consensus vote [2].
> > > > > >
> > > > > > I already have a draft PR up for these changes [3] for those that want
> > > > > > a feel for what the changes will look like.
> > > > > >
> > > > > > Cheers,
> > > > > > Kevin
> > > > > >
> > > > > > [1] https://issues.apache.org/jira/browse/NIP-30
> > > > > > [2] https://apache.org/foundation/voting.html#LazyConsensus
> > > > > > [3] https://github.com/apache/nifi-api/pull/89

Re: [lazy-consensus vote] NIP-30: Connector Observability

Posted by David Handermann <ex...@apache.org>.
Kevin,

Thanks for reconsidering the approach and proposing the adjustments on
the Connector Configuration Provider. That looks like a much better
strategy, keeping the logging attributes within the boundary of the
framework, and extensible through the nifi-framework-api.

With that change, +1 from my side, and looking forward to moving ahead
with implementation.

Regards,
David Handermann

On Mon, Jun 1, 2026 at 8:03 AM Kevin Doran <kd...@apache.org> wrote:
>
> Hey David,
>
> Thanks for the review. I agree with your feedback, allowing
> per-connector sourced MDC attributes via the
> ConnectorInitializationContext is too broad. I have modified the NIP
> to propose that attributes always come from the framework. This would
> include a default set of attributes such as Connector Id, Name, and
> Component. Extensibility would only be via Framework extensions
> (nifi-framework-api), such as ConnectorConfigurationProvider, rather
> than at the component level in nifi-api.
>
> The other suggested improvements to surface Connector Statuses in
> Reporting Task APIs remains unchanged.
>
> I think this is a better and safer approach, so thanks for calling our
> the potential issues with the original proposal. Hopefully this is
> agreeable.
>
> I have extended this NIP vote for 24 hours.
>
> On Fri, May 29, 2026 at 8:29 PM David Handermann
> <ex...@apache.org> wrote:
> >
> > Kevin,
> >
> > Thanks for the reply!
> >
> > I provided some feedback on the corresponding pull request for
> > nifi-api [1]. I agree with all of the changes, except for the
> > setLoggingAttributes addition to the ConnectorInitializationContext.
> >
> > Based on that concern, I am reluctantly voting -1 on the proposal as it stands.
> >
> > I think it would be better to make changes to the MDC at the framework
> > level. If there are features of a Connector that should be included,
> > I'm certainly open to considering additions in that direction.
> >
> > I would also be glad to move forward with a 24 hour vote on this topic
> > once we have an agreeable way forward.
> >
> > Regards,
> > David Handermann
> >
> > [1] https://github.com/apache/nifi-api/pull/89
> >
> > On Thu, May 28, 2026 at 10:00 AM Kevin Doran <kd...@apache.org> wrote:
> > >
> > > Sounds good, thanks for the input, David!
> > >
> > > The vote has been extended to 72 hours.
> > >
> > > On Wed, May 27, 2026 at 9:11 PM David Handermann
> > > <ex...@apache.org> wrote:
> > > >
> > > > Kevin,
> > > >
> > > > Thanks for proposing these improvements to support observability at
> > > > the Connector level.
> > > >
> > > > Providing status tracking and attribution at the Connector level makes
> > > > sense, and provides a very useful way of tracking behavior. On initial
> > > > review, I'm concerned about introducing the generic Logging Attributes
> > > > concept at the public API level on the Connector Initialization
> > > > Context. As it exists now at the framework level, there are a
> > > > different set of concerns around exposing it in the public API.
> > > >
> > > > With that background, I'm not ready to vote for or against the
> > > > proposal right now. Given the concern, I believe this needs to become
> > > > a 72 hour vote. I plan to review the proposal in more detail and
> > > > follow up soon.
> > > >
> > > > Regards,
> > > > David Handermann
> > > >
> > > > On Wed, May 27, 2026 at 2:35 PM Kevin Doran <kd...@apache.org> wrote:
> > > > >
> > > > > Team,
> > > > >
> > > > > I'd like to start a vote thread for NIP-30 [1] which aims to extend
> > > > > existing observability features to the new Connectors feature. Mainly,
> > > > > it will allow Connectors to have custom MDC logging attributes and
> > > > > have their statuses included in the interfaces used by reporting
> > > > > tasks.
> > > > >
> > > > > This is a small change, so using the updated NIP process recently
> > > > > adopted, this will be a 24-hour lazy consensus vote [2].
> > > > >
> > > > > I already have a draft PR up for these changes [3] for those that want
> > > > > a feel for what the changes will look like.
> > > > >
> > > > > Cheers,
> > > > > Kevin
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/NIP-30
> > > > > [2] https://apache.org/foundation/voting.html#LazyConsensus
> > > > > [3] https://github.com/apache/nifi-api/pull/89

Re: [lazy-consensus vote] NIP-30: Connector Observability

Posted by Kevin Doran <kd...@apache.org>.
Hey David,

Thanks for the review. I agree with your feedback, allowing
per-connector sourced MDC attributes via the
ConnectorInitializationContext is too broad. I have modified the NIP
to propose that attributes always come from the framework. This would
include a default set of attributes such as Connector Id, Name, and
Component. Extensibility would only be via Framework extensions
(nifi-framework-api), such as ConnectorConfigurationProvider, rather
than at the component level in nifi-api.

The other suggested improvements to surface Connector Statuses in
Reporting Task APIs remains unchanged.

I think this is a better and safer approach, so thanks for calling our
the potential issues with the original proposal. Hopefully this is
agreeable.

I have extended this NIP vote for 24 hours.

On Fri, May 29, 2026 at 8:29 PM David Handermann
<ex...@apache.org> wrote:
>
> Kevin,
>
> Thanks for the reply!
>
> I provided some feedback on the corresponding pull request for
> nifi-api [1]. I agree with all of the changes, except for the
> setLoggingAttributes addition to the ConnectorInitializationContext.
>
> Based on that concern, I am reluctantly voting -1 on the proposal as it stands.
>
> I think it would be better to make changes to the MDC at the framework
> level. If there are features of a Connector that should be included,
> I'm certainly open to considering additions in that direction.
>
> I would also be glad to move forward with a 24 hour vote on this topic
> once we have an agreeable way forward.
>
> Regards,
> David Handermann
>
> [1] https://github.com/apache/nifi-api/pull/89
>
> On Thu, May 28, 2026 at 10:00 AM Kevin Doran <kd...@apache.org> wrote:
> >
> > Sounds good, thanks for the input, David!
> >
> > The vote has been extended to 72 hours.
> >
> > On Wed, May 27, 2026 at 9:11 PM David Handermann
> > <ex...@apache.org> wrote:
> > >
> > > Kevin,
> > >
> > > Thanks for proposing these improvements to support observability at
> > > the Connector level.
> > >
> > > Providing status tracking and attribution at the Connector level makes
> > > sense, and provides a very useful way of tracking behavior. On initial
> > > review, I'm concerned about introducing the generic Logging Attributes
> > > concept at the public API level on the Connector Initialization
> > > Context. As it exists now at the framework level, there are a
> > > different set of concerns around exposing it in the public API.
> > >
> > > With that background, I'm not ready to vote for or against the
> > > proposal right now. Given the concern, I believe this needs to become
> > > a 72 hour vote. I plan to review the proposal in more detail and
> > > follow up soon.
> > >
> > > Regards,
> > > David Handermann
> > >
> > > On Wed, May 27, 2026 at 2:35 PM Kevin Doran <kd...@apache.org> wrote:
> > > >
> > > > Team,
> > > >
> > > > I'd like to start a vote thread for NIP-30 [1] which aims to extend
> > > > existing observability features to the new Connectors feature. Mainly,
> > > > it will allow Connectors to have custom MDC logging attributes and
> > > > have their statuses included in the interfaces used by reporting
> > > > tasks.
> > > >
> > > > This is a small change, so using the updated NIP process recently
> > > > adopted, this will be a 24-hour lazy consensus vote [2].
> > > >
> > > > I already have a draft PR up for these changes [3] for those that want
> > > > a feel for what the changes will look like.
> > > >
> > > > Cheers,
> > > > Kevin
> > > >
> > > > [1] https://issues.apache.org/jira/browse/NIP-30
> > > > [2] https://apache.org/foundation/voting.html#LazyConsensus
> > > > [3] https://github.com/apache/nifi-api/pull/89

Re: [lazy-consensus vote] NIP-30: Connector Observability

Posted by David Handermann <ex...@apache.org>.
Kevin,

Thanks for the reply!

I provided some feedback on the corresponding pull request for
nifi-api [1]. I agree with all of the changes, except for the
setLoggingAttributes addition to the ConnectorInitializationContext.

Based on that concern, I am reluctantly voting -1 on the proposal as it stands.

I think it would be better to make changes to the MDC at the framework
level. If there are features of a Connector that should be included,
I'm certainly open to considering additions in that direction.

I would also be glad to move forward with a 24 hour vote on this topic
once we have an agreeable way forward.

Regards,
David Handermann

[1] https://github.com/apache/nifi-api/pull/89

On Thu, May 28, 2026 at 10:00 AM Kevin Doran <kd...@apache.org> wrote:
>
> Sounds good, thanks for the input, David!
>
> The vote has been extended to 72 hours.
>
> On Wed, May 27, 2026 at 9:11 PM David Handermann
> <ex...@apache.org> wrote:
> >
> > Kevin,
> >
> > Thanks for proposing these improvements to support observability at
> > the Connector level.
> >
> > Providing status tracking and attribution at the Connector level makes
> > sense, and provides a very useful way of tracking behavior. On initial
> > review, I'm concerned about introducing the generic Logging Attributes
> > concept at the public API level on the Connector Initialization
> > Context. As it exists now at the framework level, there are a
> > different set of concerns around exposing it in the public API.
> >
> > With that background, I'm not ready to vote for or against the
> > proposal right now. Given the concern, I believe this needs to become
> > a 72 hour vote. I plan to review the proposal in more detail and
> > follow up soon.
> >
> > Regards,
> > David Handermann
> >
> > On Wed, May 27, 2026 at 2:35 PM Kevin Doran <kd...@apache.org> wrote:
> > >
> > > Team,
> > >
> > > I'd like to start a vote thread for NIP-30 [1] which aims to extend
> > > existing observability features to the new Connectors feature. Mainly,
> > > it will allow Connectors to have custom MDC logging attributes and
> > > have their statuses included in the interfaces used by reporting
> > > tasks.
> > >
> > > This is a small change, so using the updated NIP process recently
> > > adopted, this will be a 24-hour lazy consensus vote [2].
> > >
> > > I already have a draft PR up for these changes [3] for those that want
> > > a feel for what the changes will look like.
> > >
> > > Cheers,
> > > Kevin
> > >
> > > [1] https://issues.apache.org/jira/browse/NIP-30
> > > [2] https://apache.org/foundation/voting.html#LazyConsensus
> > > [3] https://github.com/apache/nifi-api/pull/89

Re: [lazy-consensus vote] NIP-30: Connector Observability

Posted by Kevin Doran <kd...@apache.org>.
Sounds good, thanks for the input, David!

The vote has been extended to 72 hours.

On Wed, May 27, 2026 at 9:11 PM David Handermann
<ex...@apache.org> wrote:
>
> Kevin,
>
> Thanks for proposing these improvements to support observability at
> the Connector level.
>
> Providing status tracking and attribution at the Connector level makes
> sense, and provides a very useful way of tracking behavior. On initial
> review, I'm concerned about introducing the generic Logging Attributes
> concept at the public API level on the Connector Initialization
> Context. As it exists now at the framework level, there are a
> different set of concerns around exposing it in the public API.
>
> With that background, I'm not ready to vote for or against the
> proposal right now. Given the concern, I believe this needs to become
> a 72 hour vote. I plan to review the proposal in more detail and
> follow up soon.
>
> Regards,
> David Handermann
>
> On Wed, May 27, 2026 at 2:35 PM Kevin Doran <kd...@apache.org> wrote:
> >
> > Team,
> >
> > I'd like to start a vote thread for NIP-30 [1] which aims to extend
> > existing observability features to the new Connectors feature. Mainly,
> > it will allow Connectors to have custom MDC logging attributes and
> > have their statuses included in the interfaces used by reporting
> > tasks.
> >
> > This is a small change, so using the updated NIP process recently
> > adopted, this will be a 24-hour lazy consensus vote [2].
> >
> > I already have a draft PR up for these changes [3] for those that want
> > a feel for what the changes will look like.
> >
> > Cheers,
> > Kevin
> >
> > [1] https://issues.apache.org/jira/browse/NIP-30
> > [2] https://apache.org/foundation/voting.html#LazyConsensus
> > [3] https://github.com/apache/nifi-api/pull/89

Re: [lazy-consensus vote] NIP-30: Connector Observability

Posted by David Handermann <ex...@apache.org>.
Kevin,

Thanks for proposing these improvements to support observability at
the Connector level.

Providing status tracking and attribution at the Connector level makes
sense, and provides a very useful way of tracking behavior. On initial
review, I'm concerned about introducing the generic Logging Attributes
concept at the public API level on the Connector Initialization
Context. As it exists now at the framework level, there are a
different set of concerns around exposing it in the public API.

With that background, I'm not ready to vote for or against the
proposal right now. Given the concern, I believe this needs to become
a 72 hour vote. I plan to review the proposal in more detail and
follow up soon.

Regards,
David Handermann

On Wed, May 27, 2026 at 2:35 PM Kevin Doran <kd...@apache.org> wrote:
>
> Team,
>
> I'd like to start a vote thread for NIP-30 [1] which aims to extend
> existing observability features to the new Connectors feature. Mainly,
> it will allow Connectors to have custom MDC logging attributes and
> have their statuses included in the interfaces used by reporting
> tasks.
>
> This is a small change, so using the updated NIP process recently
> adopted, this will be a 24-hour lazy consensus vote [2].
>
> I already have a draft PR up for these changes [3] for those that want
> a feel for what the changes will look like.
>
> Cheers,
> Kevin
>
> [1] https://issues.apache.org/jira/browse/NIP-30
> [2] https://apache.org/foundation/voting.html#LazyConsensus
> [3] https://github.com/apache/nifi-api/pull/89