You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Pierre Villard <pi...@gmail.com> on 2026/04/03 15:50:53 UTC
Re: [VOTE] NIP-4 - Extensions Registry Client
Hi all,
This definitely didn't get the love I was expecting :) Thanks Kevin
for your reply to this thread.
In order to, maybe, help a bit with this discussion, I submitted a
draft PR [1] with what would be the API changes for this new feature.
I guess I'll need to start a new vote thread in the future as I think
this feature would greatly benefit the community when using/deploying
custom NARs. It would also help the project as it would be easier to
no longer ship a lot of NARs but only focus on the most important ones
in the convenience binary and provide documentation on how to
configure an extension registry client to automatically get all of the
NARs not included by default.
[1] https://github.com/apache/nifi-api/pull/83
Thanks,
Pierre
Le jeu. 19 mars 2026 à 22:44, Kevin Doran <kd...@apache.org> a écrit :
>
> Hey Pierre,
>
> Thanks for the write up. I am generally an advocate for more modular
> nifi deployments, and this seems like a good approach.
>
> +1from me for a properties-configured extension registry client.
>
> Cheers,
> Kevin
>
> On Thu, Mar 19, 2026 at 7:11 AM Pierre Villard
> <pi...@gmail.com> wrote:
> >
> > Team,
> >
> > I'd like to start a formal vote process around NIP-4 [1] which aims to
> > introduce the concept of Extension Registry Clients that would also
> > deprecate the existing concept of ExternalResourceProvider for future
> > removal.
> >
> > The initial proposal was about introducing a completely new extension point
> > similar to the existing Flow Registry Clients but following discussion on
> > the NIP, we would go with a multi-steps approach where the first step would
> > be a nifi.properties level configuration of the extensions registry
> > clients. We would then evaluate if it is worth doing the work to add those
> > as true 1st class components in the NiFi UI - I have been running a POC for
> > both scenarios for quite some time now but doing so has the advantage of
> > making it easier to introduce breaking changes if needed.
> >
> > This change is also related to some additional work around the ability to
> > sign NARs [2] and the extension registry clients would support the ability
> > to only look for NARs that are signed by trusted entities.
> >
> > Finally, as we get closer to having the first release of NiFi with the new
> > concept of Connectors, the Extension Registry Clients would also be a nice
> > addition to externally retrieve connectors that are also packaged as NAR
> > files.
> >
> > Given that this is not a minor change, this vote is not following the
> > lazy-consensus approach and I'll leave this open for 3 days.
> >
> > The actual PR/code-change will still go through our normal RTC process.
> >
> > [1] https://issues.apache.org/jira/browse/NIP-4
> > [2] https://issues.apache.org/jira/browse/NIFI-15675
> >
> > Thanks,
> > Pierre
Re: [VOTE] NIP-4 - Extensions Registry Client
Posted by Michael Moser <mo...@gmail.com>.
Thanks for the explanation, Pierre, it makes sense.
With this email thread documenting the reasons to combine the
ExternalResourceProvider deprecation in the Extension Registry NIP, I think
it's fine to proceed with both at the same time.
-- Mike
On Fri, Apr 10, 2026 at 6:27 AM Pierre Villard <pi...@gmail.com>
wrote:
> Thanks for the feedback Michael. Resource providers were initially
> built with a broader intent in mind: being able to bring resources on
> all NiFi nodes from external locations (it could be for example a JDBC
> driver jar that is referenced by a SQL component and the driver needs
> to be available on the new NiFi node in case the cluster scales up).
> This is now better addressed by the concept of Asset and that should
> be the way to go for any resource that is being referenced in the
> flow. The concept of registry client would be specifically targeted at
> NARs and the intent is to go further than just bringing the NARs, we
> would bring the NARs only if actually needed by the user/flow, we
> would not blindly bring over everything that is available on the
> external location.
>
> All of that to say that the deprecation could definitely be its own
> JIRA and discuss thread, I agree.
> In my mind, doing both at the same time was to give clear guidance on
> what should be the replacement.
>
> Thanks,
> Pierre
>
> Le lun. 6 avr. 2026 à 17:10, Michael Moser <mo...@gmail.com> a écrit :
> >
> > I'm wondering why the ExternalResourceProvider deprecation is included in
> > this NIP. I have never used ExternalResourceProvider and only recently
> > tried to understand its use, but it seems to be unrelated in scope to the
> > ExtensionRegistryClient. I agree with its deprecation, but does it
> belong
> > in this NIP?
> >
> > -- Mike
> >
> >
> > On Fri, Apr 3, 2026 at 11:52 AM Pierre Villard <
> pierre.villard.fr@gmail.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > This definitely didn't get the love I was expecting :) Thanks Kevin
> > > for your reply to this thread.
> > >
> > > In order to, maybe, help a bit with this discussion, I submitted a
> > > draft PR [1] with what would be the API changes for this new feature.
> > >
> > > I guess I'll need to start a new vote thread in the future as I think
> > > this feature would greatly benefit the community when using/deploying
> > > custom NARs. It would also help the project as it would be easier to
> > > no longer ship a lot of NARs but only focus on the most important ones
> > > in the convenience binary and provide documentation on how to
> > > configure an extension registry client to automatically get all of the
> > > NARs not included by default.
> > >
> > > [1] https://github.com/apache/nifi-api/pull/83
> > >
> > > Thanks,
> > > Pierre
> > >
> > > Le jeu. 19 mars 2026 à 22:44, Kevin Doran <kd...@apache.org> a écrit
> :
> > > >
> > > > Hey Pierre,
> > > >
> > > > Thanks for the write up. I am generally an advocate for more modular
> > > > nifi deployments, and this seems like a good approach.
> > > >
> > > > +1from me for a properties-configured extension registry client.
> > > >
> > > > Cheers,
> > > > Kevin
> > > >
> > > > On Thu, Mar 19, 2026 at 7:11 AM Pierre Villard
> > > > <pi...@gmail.com> wrote:
> > > > >
> > > > > Team,
> > > > >
> > > > > I'd like to start a formal vote process around NIP-4 [1] which
> aims to
> > > > > introduce the concept of Extension Registry Clients that would also
> > > > > deprecate the existing concept of ExternalResourceProvider for
> future
> > > > > removal.
> > > > >
> > > > > The initial proposal was about introducing a completely new
> extension
> > > point
> > > > > similar to the existing Flow Registry Clients but following
> discussion
> > > on
> > > > > the NIP, we would go with a multi-steps approach where the first
> step
> > > would
> > > > > be a nifi.properties level configuration of the extensions registry
> > > > > clients. We would then evaluate if it is worth doing the work to
> add
> > > those
> > > > > as true 1st class components in the NiFi UI - I have been running a
> > > POC for
> > > > > both scenarios for quite some time now but doing so has the
> advantage
> > > of
> > > > > making it easier to introduce breaking changes if needed.
> > > > >
> > > > > This change is also related to some additional work around the
> ability
> > > to
> > > > > sign NARs [2] and the extension registry clients would support the
> > > ability
> > > > > to only look for NARs that are signed by trusted entities.
> > > > >
> > > > > Finally, as we get closer to having the first release of NiFi with
> the
> > > new
> > > > > concept of Connectors, the Extension Registry Clients would also
> be a
> > > nice
> > > > > addition to externally retrieve connectors that are also packaged
> as
> > > NAR
> > > > > files.
> > > > >
> > > > > Given that this is not a minor change, this vote is not following
> the
> > > > > lazy-consensus approach and I'll leave this open for 3 days.
> > > > >
> > > > > The actual PR/code-change will still go through our normal RTC
> process.
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/NIP-4
> > > > > [2] https://issues.apache.org/jira/browse/NIFI-15675
> > > > >
> > > > > Thanks,
> > > > > Pierre
> > >
>
Re: [VOTE] NIP-4 - Extensions Registry Client
Posted by Pierre Villard <pi...@gmail.com>.
Thanks for the feedback Michael. Resource providers were initially
built with a broader intent in mind: being able to bring resources on
all NiFi nodes from external locations (it could be for example a JDBC
driver jar that is referenced by a SQL component and the driver needs
to be available on the new NiFi node in case the cluster scales up).
This is now better addressed by the concept of Asset and that should
be the way to go for any resource that is being referenced in the
flow. The concept of registry client would be specifically targeted at
NARs and the intent is to go further than just bringing the NARs, we
would bring the NARs only if actually needed by the user/flow, we
would not blindly bring over everything that is available on the
external location.
All of that to say that the deprecation could definitely be its own
JIRA and discuss thread, I agree.
In my mind, doing both at the same time was to give clear guidance on
what should be the replacement.
Thanks,
Pierre
Le lun. 6 avr. 2026 à 17:10, Michael Moser <mo...@gmail.com> a écrit :
>
> I'm wondering why the ExternalResourceProvider deprecation is included in
> this NIP. I have never used ExternalResourceProvider and only recently
> tried to understand its use, but it seems to be unrelated in scope to the
> ExtensionRegistryClient. I agree with its deprecation, but does it belong
> in this NIP?
>
> -- Mike
>
>
> On Fri, Apr 3, 2026 at 11:52 AM Pierre Villard <pi...@gmail.com>
> wrote:
>
> > Hi all,
> >
> > This definitely didn't get the love I was expecting :) Thanks Kevin
> > for your reply to this thread.
> >
> > In order to, maybe, help a bit with this discussion, I submitted a
> > draft PR [1] with what would be the API changes for this new feature.
> >
> > I guess I'll need to start a new vote thread in the future as I think
> > this feature would greatly benefit the community when using/deploying
> > custom NARs. It would also help the project as it would be easier to
> > no longer ship a lot of NARs but only focus on the most important ones
> > in the convenience binary and provide documentation on how to
> > configure an extension registry client to automatically get all of the
> > NARs not included by default.
> >
> > [1] https://github.com/apache/nifi-api/pull/83
> >
> > Thanks,
> > Pierre
> >
> > Le jeu. 19 mars 2026 à 22:44, Kevin Doran <kd...@apache.org> a écrit :
> > >
> > > Hey Pierre,
> > >
> > > Thanks for the write up. I am generally an advocate for more modular
> > > nifi deployments, and this seems like a good approach.
> > >
> > > +1from me for a properties-configured extension registry client.
> > >
> > > Cheers,
> > > Kevin
> > >
> > > On Thu, Mar 19, 2026 at 7:11 AM Pierre Villard
> > > <pi...@gmail.com> wrote:
> > > >
> > > > Team,
> > > >
> > > > I'd like to start a formal vote process around NIP-4 [1] which aims to
> > > > introduce the concept of Extension Registry Clients that would also
> > > > deprecate the existing concept of ExternalResourceProvider for future
> > > > removal.
> > > >
> > > > The initial proposal was about introducing a completely new extension
> > point
> > > > similar to the existing Flow Registry Clients but following discussion
> > on
> > > > the NIP, we would go with a multi-steps approach where the first step
> > would
> > > > be a nifi.properties level configuration of the extensions registry
> > > > clients. We would then evaluate if it is worth doing the work to add
> > those
> > > > as true 1st class components in the NiFi UI - I have been running a
> > POC for
> > > > both scenarios for quite some time now but doing so has the advantage
> > of
> > > > making it easier to introduce breaking changes if needed.
> > > >
> > > > This change is also related to some additional work around the ability
> > to
> > > > sign NARs [2] and the extension registry clients would support the
> > ability
> > > > to only look for NARs that are signed by trusted entities.
> > > >
> > > > Finally, as we get closer to having the first release of NiFi with the
> > new
> > > > concept of Connectors, the Extension Registry Clients would also be a
> > nice
> > > > addition to externally retrieve connectors that are also packaged as
> > NAR
> > > > files.
> > > >
> > > > Given that this is not a minor change, this vote is not following the
> > > > lazy-consensus approach and I'll leave this open for 3 days.
> > > >
> > > > The actual PR/code-change will still go through our normal RTC process.
> > > >
> > > > [1] https://issues.apache.org/jira/browse/NIP-4
> > > > [2] https://issues.apache.org/jira/browse/NIFI-15675
> > > >
> > > > Thanks,
> > > > Pierre
> >
Re: [VOTE] NIP-4 - Extensions Registry Client
Posted by Michael Moser <mo...@gmail.com>.
I'm wondering why the ExternalResourceProvider deprecation is included in
this NIP. I have never used ExternalResourceProvider and only recently
tried to understand its use, but it seems to be unrelated in scope to the
ExtensionRegistryClient. I agree with its deprecation, but does it belong
in this NIP?
-- Mike
On Fri, Apr 3, 2026 at 11:52 AM Pierre Villard <pi...@gmail.com>
wrote:
> Hi all,
>
> This definitely didn't get the love I was expecting :) Thanks Kevin
> for your reply to this thread.
>
> In order to, maybe, help a bit with this discussion, I submitted a
> draft PR [1] with what would be the API changes for this new feature.
>
> I guess I'll need to start a new vote thread in the future as I think
> this feature would greatly benefit the community when using/deploying
> custom NARs. It would also help the project as it would be easier to
> no longer ship a lot of NARs but only focus on the most important ones
> in the convenience binary and provide documentation on how to
> configure an extension registry client to automatically get all of the
> NARs not included by default.
>
> [1] https://github.com/apache/nifi-api/pull/83
>
> Thanks,
> Pierre
>
> Le jeu. 19 mars 2026 à 22:44, Kevin Doran <kd...@apache.org> a écrit :
> >
> > Hey Pierre,
> >
> > Thanks for the write up. I am generally an advocate for more modular
> > nifi deployments, and this seems like a good approach.
> >
> > +1from me for a properties-configured extension registry client.
> >
> > Cheers,
> > Kevin
> >
> > On Thu, Mar 19, 2026 at 7:11 AM Pierre Villard
> > <pi...@gmail.com> wrote:
> > >
> > > Team,
> > >
> > > I'd like to start a formal vote process around NIP-4 [1] which aims to
> > > introduce the concept of Extension Registry Clients that would also
> > > deprecate the existing concept of ExternalResourceProvider for future
> > > removal.
> > >
> > > The initial proposal was about introducing a completely new extension
> point
> > > similar to the existing Flow Registry Clients but following discussion
> on
> > > the NIP, we would go with a multi-steps approach where the first step
> would
> > > be a nifi.properties level configuration of the extensions registry
> > > clients. We would then evaluate if it is worth doing the work to add
> those
> > > as true 1st class components in the NiFi UI - I have been running a
> POC for
> > > both scenarios for quite some time now but doing so has the advantage
> of
> > > making it easier to introduce breaking changes if needed.
> > >
> > > This change is also related to some additional work around the ability
> to
> > > sign NARs [2] and the extension registry clients would support the
> ability
> > > to only look for NARs that are signed by trusted entities.
> > >
> > > Finally, as we get closer to having the first release of NiFi with the
> new
> > > concept of Connectors, the Extension Registry Clients would also be a
> nice
> > > addition to externally retrieve connectors that are also packaged as
> NAR
> > > files.
> > >
> > > Given that this is not a minor change, this vote is not following the
> > > lazy-consensus approach and I'll leave this open for 3 days.
> > >
> > > The actual PR/code-change will still go through our normal RTC process.
> > >
> > > [1] https://issues.apache.org/jira/browse/NIP-4
> > > [2] https://issues.apache.org/jira/browse/NIFI-15675
> > >
> > > Thanks,
> > > Pierre
>