NiFi Extension Registry?

[Shalin Mehta <sh...@gmail.com>]

 Hey all,

I'm starting to cut my teeth on the extensibility of NiFi through Custom
Processors, and one of the big things on my mind is being able to manage a
catalog of them across an organization.

While watching the "BYOP" presentation given by Andy at DataWorks Summit
Barcelona earlier this year (awesome presentation, by the way!), there was
a really quick note of a future Extension Registry within NiFi Registry. I
dug around a bit and only came up with this Confluence page outlining ideas
for such a thing:

https://cwiki.apache.org/confluence/display/NIFI/Extension+Repositories+%28aka+Extension+Registry%29+for+Dynamically-loaded+Extensions

Is there any more information out there around this effort, such as updated
notes or specific Jira(s) about it? I'd love to get a better sense of what
the community has in mind, and what the state of the addition is like these
days. Rolling my own incarnation through process and supporting tools is
where my head is currently sitting, but something more "permanent" and in
line with the direction of the Project is way more appealing.

Thanks much in advance,

Shalin

Re: NiFi Extension Registry?

[Andy LoPresto <al...@apache.org>]

The best mechanism is the dev mailing list. The Slack workspace provides an opportunity for live conversation, but interaction is limited by timezones, and the system of record for Apache discussions is the mailing list. A thread here will allow people to contribute across time/location and document the discussion for future reference. 

As for beginning the code contribution journey, the best resources to start with are the Contributor Guide [1], which discusses the _process_ around contributing, and the Developer Guide [2], which discusses code concepts, behavioral patterns, testing, etc. If there is anything in here that isn’t clear, is incorrect, or needs updating, please let us know. A lot of this was written when the community was smaller and had a higher rate of experience/participation, in addition to the fact that obviously systems change over time. Improving these docs is a constant effort, and one of the places that new contributors actually provide the best feedback. We want to make the process accessible to everyone. Thanks, and looking forward to your involvement. 

[1] https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide <https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide>
[2] https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html


Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Oct 15, 2019, at 4:34 PM, Shalin Mehta <sh...@gmail.com> wrote:
> 
> You're very welcome, it's every bit deserved, Andy :)
> 
> I was actually talking to a good friend and colleague about this today, so
> if you wouldn't mind giving me a few more minutes of your time... We're
> early into our journey, but I suspect that we'll be coming up with cases in
> the near future. What's the best way to provide that feedback for NiFi
> and/or the Registry, the Slack workspace? And taking it a step further, any
> initial suggestions or guidance for jumping into contribution to the
> codebase? I believe I saw a thread about it on the Devs list in Sept or
> Aug, so I'll need to go back and look more closely, but always appreciate
> any thoughts you might have.
> 
> Thanks again for your time!
> 
> On Tue, Oct 15, 2019 at 3:09 PM Andy LoPresto <al...@apache.org> wrote:
> 
>> Thanks for the kind words Shalin. This is something that is in development
>> as Bryan mentioned. It’s a great time to contribute ideas and suggestions,
>> as well as get involved with contributing code if you’re interested. We
>> always welcome input from the community, especially if you have a real
>> world use case for this with specific requirements and scenarios. That will
>> help us build something that doesn’t just check off features, but solves
>> real problems.
>> 
>> 
>> Andy LoPresto
>> alopresto@apache.org
>> alopresto.apache@gmail.com
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>> 
>>> On Oct 14, 2019, at 2:24 PM, Shalin Mehta <sh...@gmail.com>
>> wrote:
>>> 
>>> Hey Bryan,
>>> 
>>> Thanks for the information, especially the 1.10.0 dependency. I know that
>>> there's been a lot of motion toward that release in the last couple
>> weeks,
>>> so I'll continue keeping an eye on that while looking into the
>>> functionality in NiFi Registry 0.5.0. If there are any further
>> suggestions
>>> for reading about the planned evolution, I'd love to know, but I'm sure
>>> this will keep me busy for a while.
>>> 
>>> Thanks again,
>>> 
>>> Shalin
>>> 
>>> On Mon, Oct 14, 2019 at 4:33 PM Bryan Bende <bb...@gmail.com> wrote:
>>> 
>>>> Hi Shalin,
>>>> 
>>>> There is a minimal version of an extension registry as part of NiFi
>>>> Registry 0.5.0. You can see an example of how to upload a NAR in the
>>>> user guide [1].
>>>> 
>>>> The issue is that in order to use those end-points, your NAR needs to
>>>> be built with NiFi 1.10.0 which hasn't been released yet. So we are
>>>> still in a little bit of a waiting period, but you could always build
>>>> NiFi master and set your NAR to use nifi-api 1.10.0-SNAPSHOT if you
>>>> wanted to try it out.
>>>> 
>>>> Thanks,
>>>> 
>>>> Bryan
>>>> 
>>>> [1]
>>>> 
>> https://nifi.apache.org/docs/nifi-registry-docs/html/user-guide.html#manage-bundles
>>>> 
>>>> On Mon, Oct 14, 2019 at 4:22 PM Shalin Mehta <sh...@gmail.com>
>>>> wrote:
>>>>> 
>>>>> Hey all,
>>>>> 
>>>>> I'm starting to cut my teeth on the extensibility of NiFi through
>> Custom
>>>>> Processors, and one of the big things on my mind is being able to
>> manage
>>>> a
>>>>> catalog of them across an organization.
>>>>> 
>>>>> While watching the "BYOP" presentation given by Andy at DataWorks
>> Summit
>>>>> Barcelona earlier this year (awesome presentation, by the way!), there
>>>> was
>>>>> a really quick note of a future Extension Registry within NiFi
>> Registry.
>>>> I
>>>>> dug around a bit and only came up with this Confluence page outlining
>>>> ideas
>>>>> for such a thing:
>>>>> 
>>>>> 
>>>> 
>> https://cwiki.apache.org/confluence/display/NIFI/Extension+Repositories+%28aka+Extension+Registry%29+for+Dynamically-loaded+Extensions
>>>>> 
>>>>> Is there any more information out there around this effort, such as
>>>> updated
>>>>> notes or specific Jira(s) about it? I'd love to get a better sense of
>>>> what
>>>>> the community has in mind, and what the state of the addition is like
>>>> these
>>>>> days. Rolling my own incarnation through process and supporting tools
>> is
>>>>> where my head is currently sitting, but something more "permanent" and
>> in
>>>>> line with the direction of the Project is way more appealing.
>>>>> 
>>>>> Thanks much in advance,
>>>>> 
>>>>> Shalin
>>>> 
>> 
>> 

Re: NiFi Extension Registry?

[Shalin Mehta <sh...@gmail.com>]

You're very welcome, it's every bit deserved, Andy :)

I was actually talking to a good friend and colleague about this today, so
if you wouldn't mind giving me a few more minutes of your time... We're
early into our journey, but I suspect that we'll be coming up with cases in
the near future. What's the best way to provide that feedback for NiFi
and/or the Registry, the Slack workspace? And taking it a step further, any
initial suggestions or guidance for jumping into contribution to the
codebase? I believe I saw a thread about it on the Devs list in Sept or
Aug, so I'll need to go back and look more closely, but always appreciate
any thoughts you might have.

Thanks again for your time!

On Tue, Oct 15, 2019 at 3:09 PM Andy LoPresto <al...@apache.org> wrote:

> Thanks for the kind words Shalin. This is something that is in development
> as Bryan mentioned. It’s a great time to contribute ideas and suggestions,
> as well as get involved with contributing code if you’re interested. We
> always welcome input from the community, especially if you have a real
> world use case for this with specific requirements and scenarios. That will
> help us build something that doesn’t just check off features, but solves
> real problems.
>
>
> Andy LoPresto
> alopresto@apache.org
> alopresto.apache@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> > On Oct 14, 2019, at 2:24 PM, Shalin Mehta <sh...@gmail.com>
> wrote:
> >
> > Hey Bryan,
> >
> > Thanks for the information, especially the 1.10.0 dependency. I know that
> > there's been a lot of motion toward that release in the last couple
> weeks,
> > so I'll continue keeping an eye on that while looking into the
> > functionality in NiFi Registry 0.5.0. If there are any further
> suggestions
> > for reading about the planned evolution, I'd love to know, but I'm sure
> > this will keep me busy for a while.
> >
> > Thanks again,
> >
> > Shalin
> >
> > On Mon, Oct 14, 2019 at 4:33 PM Bryan Bende <bb...@gmail.com> wrote:
> >
> >> Hi Shalin,
> >>
> >> There is a minimal version of an extension registry as part of NiFi
> >> Registry 0.5.0. You can see an example of how to upload a NAR in the
> >> user guide [1].
> >>
> >> The issue is that in order to use those end-points, your NAR needs to
> >> be built with NiFi 1.10.0 which hasn't been released yet. So we are
> >> still in a little bit of a waiting period, but you could always build
> >> NiFi master and set your NAR to use nifi-api 1.10.0-SNAPSHOT if you
> >> wanted to try it out.
> >>
> >> Thanks,
> >>
> >> Bryan
> >>
> >> [1]
> >>
> https://nifi.apache.org/docs/nifi-registry-docs/html/user-guide.html#manage-bundles
> >>
> >> On Mon, Oct 14, 2019 at 4:22 PM Shalin Mehta <sh...@gmail.com>
> >> wrote:
> >>>
> >>> Hey all,
> >>>
> >>> I'm starting to cut my teeth on the extensibility of NiFi through
> Custom
> >>> Processors, and one of the big things on my mind is being able to
> manage
> >> a
> >>> catalog of them across an organization.
> >>>
> >>> While watching the "BYOP" presentation given by Andy at DataWorks
> Summit
> >>> Barcelona earlier this year (awesome presentation, by the way!), there
> >> was
> >>> a really quick note of a future Extension Registry within NiFi
> Registry.
> >> I
> >>> dug around a bit and only came up with this Confluence page outlining
> >> ideas
> >>> for such a thing:
> >>>
> >>>
> >>
> https://cwiki.apache.org/confluence/display/NIFI/Extension+Repositories+%28aka+Extension+Registry%29+for+Dynamically-loaded+Extensions
> >>>
> >>> Is there any more information out there around this effort, such as
> >> updated
> >>> notes or specific Jira(s) about it? I'd love to get a better sense of
> >> what
> >>> the community has in mind, and what the state of the addition is like
> >> these
> >>> days. Rolling my own incarnation through process and supporting tools
> is
> >>> where my head is currently sitting, but something more "permanent" and
> in
> >>> line with the direction of the Project is way more appealing.
> >>>
> >>> Thanks much in advance,
> >>>
> >>> Shalin
> >>
>
>

Re: NiFi Extension Registry?

[Andy LoPresto <al...@apache.org>]

Thanks for the kind words Shalin. This is something that is in development as Bryan mentioned. It’s a great time to contribute ideas and suggestions, as well as get involved with contributing code if you’re interested. We always welcome input from the community, especially if you have a real world use case for this with specific requirements and scenarios. That will help us build something that doesn’t just check off features, but solves real problems. 


Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Oct 14, 2019, at 2:24 PM, Shalin Mehta <sh...@gmail.com> wrote:
> 
> Hey Bryan,
> 
> Thanks for the information, especially the 1.10.0 dependency. I know that
> there's been a lot of motion toward that release in the last couple weeks,
> so I'll continue keeping an eye on that while looking into the
> functionality in NiFi Registry 0.5.0. If there are any further suggestions
> for reading about the planned evolution, I'd love to know, but I'm sure
> this will keep me busy for a while.
> 
> Thanks again,
> 
> Shalin
> 
> On Mon, Oct 14, 2019 at 4:33 PM Bryan Bende <bb...@gmail.com> wrote:
> 
>> Hi Shalin,
>> 
>> There is a minimal version of an extension registry as part of NiFi
>> Registry 0.5.0. You can see an example of how to upload a NAR in the
>> user guide [1].
>> 
>> The issue is that in order to use those end-points, your NAR needs to
>> be built with NiFi 1.10.0 which hasn't been released yet. So we are
>> still in a little bit of a waiting period, but you could always build
>> NiFi master and set your NAR to use nifi-api 1.10.0-SNAPSHOT if you
>> wanted to try it out.
>> 
>> Thanks,
>> 
>> Bryan
>> 
>> [1]
>> https://nifi.apache.org/docs/nifi-registry-docs/html/user-guide.html#manage-bundles
>> 
>> On Mon, Oct 14, 2019 at 4:22 PM Shalin Mehta <sh...@gmail.com>
>> wrote:
>>> 
>>> Hey all,
>>> 
>>> I'm starting to cut my teeth on the extensibility of NiFi through Custom
>>> Processors, and one of the big things on my mind is being able to manage
>> a
>>> catalog of them across an organization.
>>> 
>>> While watching the "BYOP" presentation given by Andy at DataWorks Summit
>>> Barcelona earlier this year (awesome presentation, by the way!), there
>> was
>>> a really quick note of a future Extension Registry within NiFi Registry.
>> I
>>> dug around a bit and only came up with this Confluence page outlining
>> ideas
>>> for such a thing:
>>> 
>>> 
>> https://cwiki.apache.org/confluence/display/NIFI/Extension+Repositories+%28aka+Extension+Registry%29+for+Dynamically-loaded+Extensions
>>> 
>>> Is there any more information out there around this effort, such as
>> updated
>>> notes or specific Jira(s) about it? I'd love to get a better sense of
>> what
>>> the community has in mind, and what the state of the addition is like
>> these
>>> days. Rolling my own incarnation through process and supporting tools is
>>> where my head is currently sitting, but something more "permanent" and in
>>> line with the direction of the Project is way more appealing.
>>> 
>>> Thanks much in advance,
>>> 
>>> Shalin
>> 

Re: NiFi Extension Registry?

[Shalin Mehta <sh...@gmail.com>]

Hey Bryan,

Thanks for the information, especially the 1.10.0 dependency. I know that
there's been a lot of motion toward that release in the last couple weeks,
so I'll continue keeping an eye on that while looking into the
functionality in NiFi Registry 0.5.0. If there are any further suggestions
for reading about the planned evolution, I'd love to know, but I'm sure
this will keep me busy for a while.

Thanks again,

Shalin

On Mon, Oct 14, 2019 at 4:33 PM Bryan Bende <bb...@gmail.com> wrote:

> Hi Shalin,
>
> There is a minimal version of an extension registry as part of NiFi
> Registry 0.5.0. You can see an example of how to upload a NAR in the
> user guide [1].
>
> The issue is that in order to use those end-points, your NAR needs to
> be built with NiFi 1.10.0 which hasn't been released yet. So we are
> still in a little bit of a waiting period, but you could always build
> NiFi master and set your NAR to use nifi-api 1.10.0-SNAPSHOT if you
> wanted to try it out.
>
> Thanks,
>
> Bryan
>
> [1]
> https://nifi.apache.org/docs/nifi-registry-docs/html/user-guide.html#manage-bundles
>
> On Mon, Oct 14, 2019 at 4:22 PM Shalin Mehta <sh...@gmail.com>
> wrote:
> >
> >  Hey all,
> >
> > I'm starting to cut my teeth on the extensibility of NiFi through Custom
> > Processors, and one of the big things on my mind is being able to manage
> a
> > catalog of them across an organization.
> >
> > While watching the "BYOP" presentation given by Andy at DataWorks Summit
> > Barcelona earlier this year (awesome presentation, by the way!), there
> was
> > a really quick note of a future Extension Registry within NiFi Registry.
> I
> > dug around a bit and only came up with this Confluence page outlining
> ideas
> > for such a thing:
> >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Extension+Repositories+%28aka+Extension+Registry%29+for+Dynamically-loaded+Extensions
> >
> > Is there any more information out there around this effort, such as
> updated
> > notes or specific Jira(s) about it? I'd love to get a better sense of
> what
> > the community has in mind, and what the state of the addition is like
> these
> > days. Rolling my own incarnation through process and supporting tools is
> > where my head is currently sitting, but something more "permanent" and in
> > line with the direction of the Project is way more appealing.
> >
> > Thanks much in advance,
> >
> > Shalin
>

Re: NiFi Extension Registry?

[Bryan Bende <bb...@gmail.com>]

The work for that would be part of NiFi so I don't think it is tied to
any specific registry release.

The REST APIs are already there in NiFi Registry, that is what the CLI
command for 'download-bundle' is calling.

On Tue, Jan 21, 2020 at 10:02 AM Mike Thomsen <mi...@gmail.com> wrote:
>
> Is that slated for the 1.0 release?
>
> On Tue, Jan 21, 2020 at 9:33 AM Bryan Bende <bb...@gmail.com> wrote:
>
> > Hi Mike,
> >
> > There is no automatic retrieval from registry yet. Currently you would
> > have to use CLI commands, or curl/scripts, to download the desired
> > NARs from registry into the extensions folder on each node.
> >
> > Thanks,
> >
> > Bryan
> >
> > On Tue, Jan 21, 2020 at 7:34 AM Mike Thomsen <mi...@gmail.com>
> > wrote:
> > >
> > > Bryan,
> > >
> > > I tried this workflow:
> > >
> > > 1. Add NAR to extensions folder.
> > > 2. Add to canvas.
> > > 3. Stop NiFi and remove NAR and $NIFI/work to clear it out.
> > > 4. Restart NiFi
> > >
> > > The NAR was not pulled from the v0.5 Registry I had running. Is that
> > > expected behavior in 1.10 and 1.11?
> > >
> > > Thanks,
> > >
> > > Mike
> > >
> > > On Mon, Oct 14, 2019 at 4:33 PM Bryan Bende <bb...@gmail.com> wrote:
> > >
> > > > Hi Shalin,
> > > >
> > > > There is a minimal version of an extension registry as part of NiFi
> > > > Registry 0.5.0. You can see an example of how to upload a NAR in the
> > > > user guide [1].
> > > >
> > > > The issue is that in order to use those end-points, your NAR needs to
> > > > be built with NiFi 1.10.0 which hasn't been released yet. So we are
> > > > still in a little bit of a waiting period, but you could always build
> > > > NiFi master and set your NAR to use nifi-api 1.10.0-SNAPSHOT if you
> > > > wanted to try it out.
> > > >
> > > > Thanks,
> > > >
> > > > Bryan
> > > >
> > > > [1]
> > > >
> > https://nifi.apache.org/docs/nifi-registry-docs/html/user-guide.html#manage-bundles
> > > >
> > > > On Mon, Oct 14, 2019 at 4:22 PM Shalin Mehta <sh...@gmail.com>
> > > > wrote:
> > > > >
> > > > >  Hey all,
> > > > >
> > > > > I'm starting to cut my teeth on the extensibility of NiFi through
> > Custom
> > > > > Processors, and one of the big things on my mind is being able to
> > manage
> > > > a
> > > > > catalog of them across an organization.
> > > > >
> > > > > While watching the "BYOP" presentation given by Andy at DataWorks
> > Summit
> > > > > Barcelona earlier this year (awesome presentation, by the way!),
> > there
> > > > was
> > > > > a really quick note of a future Extension Registry within NiFi
> > Registry.
> > > > I
> > > > > dug around a bit and only came up with this Confluence page outlining
> > > > ideas
> > > > > for such a thing:
> > > > >
> > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/NIFI/Extension+Repositories+%28aka+Extension+Registry%29+for+Dynamically-loaded+Extensions
> > > > >
> > > > > Is there any more information out there around this effort, such as
> > > > updated
> > > > > notes or specific Jira(s) about it? I'd love to get a better sense of
> > > > what
> > > > > the community has in mind, and what the state of the addition is like
> > > > these
> > > > > days. Rolling my own incarnation through process and supporting
> > tools is
> > > > > where my head is currently sitting, but something more "permanent"
> > and in
> > > > > line with the direction of the Project is way more appealing.
> > > > >
> > > > > Thanks much in advance,
> > > > >
> > > > > Shalin
> > > >
> >

Re: NiFi Extension Registry?

[Mike Thomsen <mi...@gmail.com>]

Is that slated for the 1.0 release?

On Tue, Jan 21, 2020 at 9:33 AM Bryan Bende <bb...@gmail.com> wrote:

> Hi Mike,
>
> There is no automatic retrieval from registry yet. Currently you would
> have to use CLI commands, or curl/scripts, to download the desired
> NARs from registry into the extensions folder on each node.
>
> Thanks,
>
> Bryan
>
> On Tue, Jan 21, 2020 at 7:34 AM Mike Thomsen <mi...@gmail.com>
> wrote:
> >
> > Bryan,
> >
> > I tried this workflow:
> >
> > 1. Add NAR to extensions folder.
> > 2. Add to canvas.
> > 3. Stop NiFi and remove NAR and $NIFI/work to clear it out.
> > 4. Restart NiFi
> >
> > The NAR was not pulled from the v0.5 Registry I had running. Is that
> > expected behavior in 1.10 and 1.11?
> >
> > Thanks,
> >
> > Mike
> >
> > On Mon, Oct 14, 2019 at 4:33 PM Bryan Bende <bb...@gmail.com> wrote:
> >
> > > Hi Shalin,
> > >
> > > There is a minimal version of an extension registry as part of NiFi
> > > Registry 0.5.0. You can see an example of how to upload a NAR in the
> > > user guide [1].
> > >
> > > The issue is that in order to use those end-points, your NAR needs to
> > > be built with NiFi 1.10.0 which hasn't been released yet. So we are
> > > still in a little bit of a waiting period, but you could always build
> > > NiFi master and set your NAR to use nifi-api 1.10.0-SNAPSHOT if you
> > > wanted to try it out.
> > >
> > > Thanks,
> > >
> > > Bryan
> > >
> > > [1]
> > >
> https://nifi.apache.org/docs/nifi-registry-docs/html/user-guide.html#manage-bundles
> > >
> > > On Mon, Oct 14, 2019 at 4:22 PM Shalin Mehta <sh...@gmail.com>
> > > wrote:
> > > >
> > > >  Hey all,
> > > >
> > > > I'm starting to cut my teeth on the extensibility of NiFi through
> Custom
> > > > Processors, and one of the big things on my mind is being able to
> manage
> > > a
> > > > catalog of them across an organization.
> > > >
> > > > While watching the "BYOP" presentation given by Andy at DataWorks
> Summit
> > > > Barcelona earlier this year (awesome presentation, by the way!),
> there
> > > was
> > > > a really quick note of a future Extension Registry within NiFi
> Registry.
> > > I
> > > > dug around a bit and only came up with this Confluence page outlining
> > > ideas
> > > > for such a thing:
> > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/NIFI/Extension+Repositories+%28aka+Extension+Registry%29+for+Dynamically-loaded+Extensions
> > > >
> > > > Is there any more information out there around this effort, such as
> > > updated
> > > > notes or specific Jira(s) about it? I'd love to get a better sense of
> > > what
> > > > the community has in mind, and what the state of the addition is like
> > > these
> > > > days. Rolling my own incarnation through process and supporting
> tools is
> > > > where my head is currently sitting, but something more "permanent"
> and in
> > > > line with the direction of the Project is way more appealing.
> > > >
> > > > Thanks much in advance,
> > > >
> > > > Shalin
> > >
>

Re: NiFi Extension Registry?

[Bryan Bende <bb...@gmail.com>]

Hi Mike,

There is no automatic retrieval from registry yet. Currently you would
have to use CLI commands, or curl/scripts, to download the desired
NARs from registry into the extensions folder on each node.

Thanks,

Bryan

On Tue, Jan 21, 2020 at 7:34 AM Mike Thomsen <mi...@gmail.com> wrote:
>
> Bryan,
>
> I tried this workflow:
>
> 1. Add NAR to extensions folder.
> 2. Add to canvas.
> 3. Stop NiFi and remove NAR and $NIFI/work to clear it out.
> 4. Restart NiFi
>
> The NAR was not pulled from the v0.5 Registry I had running. Is that
> expected behavior in 1.10 and 1.11?
>
> Thanks,
>
> Mike
>
> On Mon, Oct 14, 2019 at 4:33 PM Bryan Bende <bb...@gmail.com> wrote:
>
> > Hi Shalin,
> >
> > There is a minimal version of an extension registry as part of NiFi
> > Registry 0.5.0. You can see an example of how to upload a NAR in the
> > user guide [1].
> >
> > The issue is that in order to use those end-points, your NAR needs to
> > be built with NiFi 1.10.0 which hasn't been released yet. So we are
> > still in a little bit of a waiting period, but you could always build
> > NiFi master and set your NAR to use nifi-api 1.10.0-SNAPSHOT if you
> > wanted to try it out.
> >
> > Thanks,
> >
> > Bryan
> >
> > [1]
> > https://nifi.apache.org/docs/nifi-registry-docs/html/user-guide.html#manage-bundles
> >
> > On Mon, Oct 14, 2019 at 4:22 PM Shalin Mehta <sh...@gmail.com>
> > wrote:
> > >
> > >  Hey all,
> > >
> > > I'm starting to cut my teeth on the extensibility of NiFi through Custom
> > > Processors, and one of the big things on my mind is being able to manage
> > a
> > > catalog of them across an organization.
> > >
> > > While watching the "BYOP" presentation given by Andy at DataWorks Summit
> > > Barcelona earlier this year (awesome presentation, by the way!), there
> > was
> > > a really quick note of a future Extension Registry within NiFi Registry.
> > I
> > > dug around a bit and only came up with this Confluence page outlining
> > ideas
> > > for such a thing:
> > >
> > >
> > https://cwiki.apache.org/confluence/display/NIFI/Extension+Repositories+%28aka+Extension+Registry%29+for+Dynamically-loaded+Extensions
> > >
> > > Is there any more information out there around this effort, such as
> > updated
> > > notes or specific Jira(s) about it? I'd love to get a better sense of
> > what
> > > the community has in mind, and what the state of the addition is like
> > these
> > > days. Rolling my own incarnation through process and supporting tools is
> > > where my head is currently sitting, but something more "permanent" and in
> > > line with the direction of the Project is way more appealing.
> > >
> > > Thanks much in advance,
> > >
> > > Shalin
> >

Re: NiFi Extension Registry?

[Mike Thomsen <mi...@gmail.com>]

Bryan,

I tried this workflow:

1. Add NAR to extensions folder.
2. Add to canvas.
3. Stop NiFi and remove NAR and $NIFI/work to clear it out.
4. Restart NiFi

The NAR was not pulled from the v0.5 Registry I had running. Is that
expected behavior in 1.10 and 1.11?

Thanks,

Mike

On Mon, Oct 14, 2019 at 4:33 PM Bryan Bende <bb...@gmail.com> wrote:

> Hi Shalin,
>
> There is a minimal version of an extension registry as part of NiFi
> Registry 0.5.0. You can see an example of how to upload a NAR in the
> user guide [1].
>
> The issue is that in order to use those end-points, your NAR needs to
> be built with NiFi 1.10.0 which hasn't been released yet. So we are
> still in a little bit of a waiting period, but you could always build
> NiFi master and set your NAR to use nifi-api 1.10.0-SNAPSHOT if you
> wanted to try it out.
>
> Thanks,
>
> Bryan
>
> [1]
> https://nifi.apache.org/docs/nifi-registry-docs/html/user-guide.html#manage-bundles
>
> On Mon, Oct 14, 2019 at 4:22 PM Shalin Mehta <sh...@gmail.com>
> wrote:
> >
> >  Hey all,
> >
> > I'm starting to cut my teeth on the extensibility of NiFi through Custom
> > Processors, and one of the big things on my mind is being able to manage
> a
> > catalog of them across an organization.
> >
> > While watching the "BYOP" presentation given by Andy at DataWorks Summit
> > Barcelona earlier this year (awesome presentation, by the way!), there
> was
> > a really quick note of a future Extension Registry within NiFi Registry.
> I
> > dug around a bit and only came up with this Confluence page outlining
> ideas
> > for such a thing:
> >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Extension+Repositories+%28aka+Extension+Registry%29+for+Dynamically-loaded+Extensions
> >
> > Is there any more information out there around this effort, such as
> updated
> > notes or specific Jira(s) about it? I'd love to get a better sense of
> what
> > the community has in mind, and what the state of the addition is like
> these
> > days. Rolling my own incarnation through process and supporting tools is
> > where my head is currently sitting, but something more "permanent" and in
> > line with the direction of the Project is way more appealing.
> >
> > Thanks much in advance,
> >
> > Shalin
>

Re: NiFi Extension Registry?

[Bryan Bende <bb...@gmail.com>]

Hi Shalin,

There is a minimal version of an extension registry as part of NiFi
Registry 0.5.0. You can see an example of how to upload a NAR in the
user guide [1].

The issue is that in order to use those end-points, your NAR needs to
be built with NiFi 1.10.0 which hasn't been released yet. So we are
still in a little bit of a waiting period, but you could always build
NiFi master and set your NAR to use nifi-api 1.10.0-SNAPSHOT if you
wanted to try it out.

Thanks,

Bryan

[1] https://nifi.apache.org/docs/nifi-registry-docs/html/user-guide.html#manage-bundles

On Mon, Oct 14, 2019 at 4:22 PM Shalin Mehta <sh...@gmail.com> wrote:
>
>  Hey all,
>
> I'm starting to cut my teeth on the extensibility of NiFi through Custom
> Processors, and one of the big things on my mind is being able to manage a
> catalog of them across an organization.
>
> While watching the "BYOP" presentation given by Andy at DataWorks Summit
> Barcelona earlier this year (awesome presentation, by the way!), there was
> a really quick note of a future Extension Registry within NiFi Registry. I
> dug around a bit and only came up with this Confluence page outlining ideas
> for such a thing:
>
> https://cwiki.apache.org/confluence/display/NIFI/Extension+Repositories+%28aka+Extension+Registry%29+for+Dynamically-loaded+Extensions
>
> Is there any more information out there around this effort, such as updated
> notes or specific Jira(s) about it? I'd love to get a better sense of what
> the community has in mind, and what the state of the addition is like these
> days. Rolling my own incarnation through process and supporting tools is
> where my head is currently sitting, but something more "permanent" and in
> line with the direction of the Project is way more appealing.
>
> Thanks much in advance,
>
> Shalin