You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Matthew Baine <ma...@airvantage.co.za> on 2023/04/04 14:54:13 UTC
Usage Documentation for Custom Processors
Hi Bryan,
Sorry, on a separate note, what would be the best way to set up Usage
Documentation for a custom processor?
[image: image.png]
We can't seem to get this right with the information online and on the Nifi
developer guide (
https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html). Our
custom processors seem to only publish documentation of the native
processors.
Kind Regards,
Matthew
On Tue, 4 Apr 2023 at 13:54, Matthew Baine <ma...@airvantage.co.za> wrote:
> Hi Bryan,
>
> Sorry for the delayed response, and thank you so much for the feedback!
>
> We will attempt the advised approach and revert if we run into any trouble.
>
> Thanks again!
>
> Regards,
>
> On Thu, 30 Mar 2023 at 16:49, Bryan Bende <bb...@gmail.com> wrote:
>
>> Hello,
>>
>> This might not give you exactly what you want, but the Minifi Toolkit
>> already has the ability to transform the JSON snapshot from registry,
>> there are actually two commands:
>>
>> "transform" - for XML templates
>> "transform-vfs" - for versioned flow snapshot (JSON from registry) [1]
>>
>> It doesn't pull the snapshot from registry directly, so you would have
>> to script something to download the snapshot and then run
>> transform-vfs.
>>
>> Thanks,
>>
>> Bryan
>>
>> [1]
>> https://github.com/apache/nifi/blob/main/minifi/minifi-toolkit/minifi-toolkit-configuration/src/main/java/org/apache/nifi/minifi/toolkit/configuration/ConfigMain.java#L62
>>
>> On Thu, Mar 30, 2023 at 10:22 AM Simeon Wentzel <si...@airvantage.co.za>
>> wrote:
>> >
>> > Dear Nifi dev team
>> >
>> > Can you add extended functionality to the MiNiFi toolkit to extract a
>> flow
>> > from the NiFi Registry software and convert it to the appropriate
>> conf.yml
>> > file?
>> >
>> > We have found a limitation regarding the conversion in the minifi
>> toolkit
>> > that it can only convert the .xml file template extracted from a Nifi
>> > canvas on Java version 8, it can not do the conversion on java 11 that
>> we
>> > have migrated to.
>> >
>> > Although extracting the flow as a template out of nifi and then
>> converting
>> > it to the conf.yaml file works we find it a bit cumbersome because we
>> can
>> > not implement it in our pipeline to automate the process.
>> >
>> > By allowing the minifi toolkit to pull a flow from the Nifi registry and
>> > then convert it will give us the functionality to add this in our
>> Jenkins
>> > pipeline to build individual docker containers for each of our flows.
>> >
>> > Regards
>> > Simeon
>> > DevOps Engineer
>>
>
>
> --
>
>
> *Matthew Baine | *DevOps Engineer
>
> *Johannesburg Head Office*
>
> E: matthew@airvantage.co.za | M: +27 (0) 71053 9012 <+27710539012>
>
> T: +27 (0) 11 100 1880 <+270111001880> | W: www.airvantage.co.za
>
> *Skype: matthew.baine57*
>
--
*Matthew Baine | *DevOps Engineer
*Johannesburg Head Office*
E: matthew@airvantage.co.za | M: +27 (0) 71053 9012 <+27710539012>
T: +27 (0) 11 100 1880 <+270111001880> | W: www.airvantage.co.za
*Skype: matthew.baine57*
Re: Usage Documentation for Custom Processors
Posted by Mark Bean <ma...@gmail.com>.
If the expected documentation is not showing up in the UI - particularly if
an older version of the documentation is present - you may have to clear
your browser cache and reload the documentation page.
-Mark
On Wed, Apr 5, 2023 at 11:52 AM Matthew Baine <ma...@airvantage.co.za>
wrote:
> Hi Russell and Bryan,
>
> Yes, we are currently making use of the @CapabilityDescription annotations
> etc, it's the additionalDetails.html file I am trying to implement.
>
> The feedback is much appreciated, I'll proceed as advised.
>
> Kind Regards,
> Matthew
>
> On Tue, 4 Apr 2023 at 17:54, Russell Bateman <ru...@windofkeltia.com>
> wrote:
>
> > Matthew,
> >
> > If you feel that the documentation generated from the annotations at the
> > top of your custom processor class (@CapabilityDescription, etc., of
> > which Bryan spoke) is insufficient, it's also possible to supplement it
> with
> >
> >
> >
> *src/main/resources/docs/<package-path>.CustomProcessorClass/additionalDetails.html*
> >
> > You write it in simple HTML with embedded CSS. By your user, it's reached
> > via a hyperlink on the (standard) processor usage page put there when the
> > framework notices that you've supplied it (directory name including
> package
> > path, filesystem location, etc. are crucial).
> >
> > I do this for almost every last custom processor I write as a favor to my
> > downstream flow writers.
> >
> > Cheers,
> >
> > Russ
> >
> >
> > On 4/4/23 08:54, Matthew Baine wrote:
> >
> > Hi Bryan,
> >
> > Sorry, on a separate note, what would be the best way to set up Usage
> > Documentation for a custom processor?
> >
> > [image: image.png]
> >
> > We can't seem to get this right with the information online and on the
> > Nifi developer guide (
> > https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html). Our
> > custom processors seem to only publish documentation of the native
> > processors.
> >
> >
> > Kind Regards,
> > Matthew
> >
> > On Tue, 4 Apr 2023 at 13:54, Matthew Baine <ma...@airvantage.co.za>
> > wrote:
> >
> >> Hi Bryan,
> >>
> >> Sorry for the delayed response, and thank you so much for the feedback!
> >>
> >> We will attempt the advised approach and revert if we run into any
> >> trouble.
> >>
> >> Thanks again!
> >>
> >> Regards,
> >>
> >> On Thu, 30 Mar 2023 at 16:49, Bryan Bende <bb...@gmail.com> wrote:
> >>
> >>> Hello,
> >>>
> >>> This might not give you exactly what you want, but the Minifi Toolkit
> >>> already has the ability to transform the JSON snapshot from registry,
> >>> there are actually two commands:
> >>>
> >>> "transform" - for XML templates
> >>> "transform-vfs" - for versioned flow snapshot (JSON from registry) [1]
> >>>
> >>> It doesn't pull the snapshot from registry directly, so you would have
> >>> to script something to download the snapshot and then run
> >>> transform-vfs.
> >>>
> >>> Thanks,
> >>>
> >>> Bryan
> >>>
> >>> [1]
> >>>
> https://github.com/apache/nifi/blob/main/minifi/minifi-toolkit/minifi-toolkit-configuration/src/main/java/org/apache/nifi/minifi/toolkit/configuration/ConfigMain.java#L62
> >>>
> >>> On Thu, Mar 30, 2023 at 10:22 AM Simeon Wentzel <
> simeon@airvantage.co.za>
> >>> wrote:
> >>> >
> >>> > Dear Nifi dev team
> >>> >
> >>> > Can you add extended functionality to the MiNiFi toolkit to extract a
> >>> flow
> >>> > from the NiFi Registry software and convert it to the appropriate
> >>> conf.yml
> >>> > file?
> >>> >
> >>> > We have found a limitation regarding the conversion in the minifi
> >>> toolkit
> >>> > that it can only convert the .xml file template extracted from a Nifi
> >>> > canvas on Java version 8, it can not do the conversion on java 11
> that
> >>> we
> >>> > have migrated to.
> >>> >
> >>> > Although extracting the flow as a template out of nifi and then
> >>> converting
> >>> > it to the conf.yaml file works we find it a bit cumbersome because we
> >>> can
> >>> > not implement it in our pipeline to automate the process.
> >>> >
> >>> > By allowing the minifi toolkit to pull a flow from the Nifi registry
> >>> and
> >>> > then convert it will give us the functionality to add this in our
> >>> Jenkins
> >>> > pipeline to build individual docker containers for each of our flows.
> >>> >
> >>> > Regards
> >>> > Simeon
> >>> > DevOps Engineer
> >>>
> >>
> >>
> >> --
> >>
> >>
> >> *Matthew Baine | *DevOps Engineer
> >>
> >> *Johannesburg Head Office*
> >>
> >> E: matthew@airvantage.co.za | M: +27 (0) 71053 9012 <+27710539012>
> >>
> >> T: +27 (0) 11 100 1880 <+270111001880> | W: www.airvantage.co.za
> >>
> >> *Skype: matthew.baine57*
> >>
> >
> >
> > --
> >
> >
> > *Matthew Baine | *DevOps Engineer
> >
> > *Johannesburg Head Office*
> >
> > E: matthew@airvantage.co.za | M: +27 (0) 71053 9012 <+27710539012>
> >
> > T: +27 (0) 11 100 1880 <+270111001880> | W: www.airvantage.co.za
> >
> > *Skype: matthew.baine57*
> >
> >
> >
>
> --
>
>
> *Matthew Baine | *DevOps Engineer
>
> *Johannesburg Head Office*
>
> E: matthew@airvantage.co.za | M: +27 (0) 71053 9012 <+27710539012>
>
> T: +27 (0) 11 100 1880 <+270111001880> | W: www.airvantage.co.za
>
> *Skype: matthew.baine57*
>
Re: Usage Documentation for Custom Processors
Posted by Matthew Baine <ma...@airvantage.co.za>.
Hi Russell and Bryan,
Yes, we are currently making use of the @CapabilityDescription annotations
etc, it's the additionalDetails.html file I am trying to implement.
The feedback is much appreciated, I'll proceed as advised.
Kind Regards,
Matthew
On Tue, 4 Apr 2023 at 17:54, Russell Bateman <ru...@windofkeltia.com> wrote:
> Matthew,
>
> If you feel that the documentation generated from the annotations at the
> top of your custom processor class (@CapabilityDescription, etc., of
> which Bryan spoke) is insufficient, it's also possible to supplement it with
>
>
> *src/main/resources/docs/<package-path>.CustomProcessorClass/additionalDetails.html*
>
> You write it in simple HTML with embedded CSS. By your user, it's reached
> via a hyperlink on the (standard) processor usage page put there when the
> framework notices that you've supplied it (directory name including package
> path, filesystem location, etc. are crucial).
>
> I do this for almost every last custom processor I write as a favor to my
> downstream flow writers.
>
> Cheers,
>
> Russ
>
>
> On 4/4/23 08:54, Matthew Baine wrote:
>
> Hi Bryan,
>
> Sorry, on a separate note, what would be the best way to set up Usage
> Documentation for a custom processor?
>
> [image: image.png]
>
> We can't seem to get this right with the information online and on the
> Nifi developer guide (
> https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html). Our
> custom processors seem to only publish documentation of the native
> processors.
>
>
> Kind Regards,
> Matthew
>
> On Tue, 4 Apr 2023 at 13:54, Matthew Baine <ma...@airvantage.co.za>
> wrote:
>
>> Hi Bryan,
>>
>> Sorry for the delayed response, and thank you so much for the feedback!
>>
>> We will attempt the advised approach and revert if we run into any
>> trouble.
>>
>> Thanks again!
>>
>> Regards,
>>
>> On Thu, 30 Mar 2023 at 16:49, Bryan Bende <bb...@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> This might not give you exactly what you want, but the Minifi Toolkit
>>> already has the ability to transform the JSON snapshot from registry,
>>> there are actually two commands:
>>>
>>> "transform" - for XML templates
>>> "transform-vfs" - for versioned flow snapshot (JSON from registry) [1]
>>>
>>> It doesn't pull the snapshot from registry directly, so you would have
>>> to script something to download the snapshot and then run
>>> transform-vfs.
>>>
>>> Thanks,
>>>
>>> Bryan
>>>
>>> [1]
>>> https://github.com/apache/nifi/blob/main/minifi/minifi-toolkit/minifi-toolkit-configuration/src/main/java/org/apache/nifi/minifi/toolkit/configuration/ConfigMain.java#L62
>>>
>>> On Thu, Mar 30, 2023 at 10:22 AM Simeon Wentzel <si...@airvantage.co.za>
>>> wrote:
>>> >
>>> > Dear Nifi dev team
>>> >
>>> > Can you add extended functionality to the MiNiFi toolkit to extract a
>>> flow
>>> > from the NiFi Registry software and convert it to the appropriate
>>> conf.yml
>>> > file?
>>> >
>>> > We have found a limitation regarding the conversion in the minifi
>>> toolkit
>>> > that it can only convert the .xml file template extracted from a Nifi
>>> > canvas on Java version 8, it can not do the conversion on java 11 that
>>> we
>>> > have migrated to.
>>> >
>>> > Although extracting the flow as a template out of nifi and then
>>> converting
>>> > it to the conf.yaml file works we find it a bit cumbersome because we
>>> can
>>> > not implement it in our pipeline to automate the process.
>>> >
>>> > By allowing the minifi toolkit to pull a flow from the Nifi registry
>>> and
>>> > then convert it will give us the functionality to add this in our
>>> Jenkins
>>> > pipeline to build individual docker containers for each of our flows.
>>> >
>>> > Regards
>>> > Simeon
>>> > DevOps Engineer
>>>
>>
>>
>> --
>>
>>
>> *Matthew Baine | *DevOps Engineer
>>
>> *Johannesburg Head Office*
>>
>> E: matthew@airvantage.co.za | M: +27 (0) 71053 9012 <+27710539012>
>>
>> T: +27 (0) 11 100 1880 <+270111001880> | W: www.airvantage.co.za
>>
>> *Skype: matthew.baine57*
>>
>
>
> --
>
>
> *Matthew Baine | *DevOps Engineer
>
> *Johannesburg Head Office*
>
> E: matthew@airvantage.co.za | M: +27 (0) 71053 9012 <+27710539012>
>
> T: +27 (0) 11 100 1880 <+270111001880> | W: www.airvantage.co.za
>
> *Skype: matthew.baine57*
>
>
>
--
*Matthew Baine | *DevOps Engineer
*Johannesburg Head Office*
E: matthew@airvantage.co.za | M: +27 (0) 71053 9012 <+27710539012>
T: +27 (0) 11 100 1880 <+270111001880> | W: www.airvantage.co.za
*Skype: matthew.baine57*
Re: Usage Documentation for Custom Processors
Posted by Russell Bateman <ru...@windofkeltia.com>.
Matthew,
If you feel that the documentation generated from the annotations at the
top of your custom processor class (@CapabilityDescription, etc., of
which Bryan spoke) is insufficient, it's also possible to supplement it with
/src/main/resources/docs/<package-path>.CustomProcessorClass/additionalDetails.html/
You write it in simple HTML with embedded CSS. By your user, it's
reached via a hyperlink on the (standard) processor usage page put there
when the framework notices that you've supplied it (directory name
including package path, filesystem location, etc. are crucial).
I do this for almost every last custom processor I write as a favor to
my downstream flow writers.
Cheers,
Russ
On 4/4/23 08:54, Matthew Baine wrote:
> Hi Bryan,
>
> Sorry, on a separate note, what would be the best way to set up Usage
> Documentation for a custom processor?
>
> image.png
>
> We can't seem to get this right with the information online and on the
> Nifi developer guide
> (https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html).
> Our custom processors seem to only publish documentation of the native
> processors.
>
>
> Kind Regards,
> Matthew
>
> On Tue, 4 Apr 2023 at 13:54, Matthew Baine <ma...@airvantage.co.za>
> wrote:
>
> Hi Bryan,
>
> Sorry for the delayed response, and thank you so much for the
> feedback!
>
> We will attempt the advised approach and revert if we run into any
> trouble.
>
> Thanks again!
>
> Regards,
>
> On Thu, 30 Mar 2023 at 16:49, Bryan Bende <bb...@gmail.com> wrote:
>
> Hello,
>
> This might not give you exactly what you want, but the Minifi
> Toolkit
> already has the ability to transform the JSON snapshot from
> registry,
> there are actually two commands:
>
> "transform" - for XML templates
> "transform-vfs" - for versioned flow snapshot (JSON from
> registry) [1]
>
> It doesn't pull the snapshot from registry directly, so you
> would have
> to script something to download the snapshot and then run
> transform-vfs.
>
> Thanks,
>
> Bryan
>
> [1]
> https://github.com/apache/nifi/blob/main/minifi/minifi-toolkit/minifi-toolkit-configuration/src/main/java/org/apache/nifi/minifi/toolkit/configuration/ConfigMain.java#L62
>
> On Thu, Mar 30, 2023 at 10:22 AM Simeon Wentzel
> <si...@airvantage.co.za> wrote:
> >
> > Dear Nifi dev team
> >
> > Can you add extended functionality to the MiNiFi toolkit to
> extract a flow
> > from the NiFi Registry software and convert it to the
> appropriate conf.yml
> > file?
> >
> > We have found a limitation regarding the conversion in the
> minifi toolkit
> > that it can only convert the .xml file template extracted
> from a Nifi
> > canvas on Java version 8, it can not do the conversion on
> java 11 that we
> > have migrated to.
> >
> > Although extracting the flow as a template out of nifi and
> then converting
> > it to the conf.yaml file works we find it a bit cumbersome
> because we can
> > not implement it in our pipeline to automate the process.
> >
> > By allowing the minifi toolkit to pull a flow from the Nifi
> registry and
> > then convert it will give us the functionality to add this
> in our Jenkins
> > pipeline to build individual docker containers for each of
> our flows.
> >
> > Regards
> > Simeon
> > DevOps Engineer
>
>
>
> --
>
>
>
>
> *Matthew Baine | *DevOps Engineer
>
> *Johannesburg Head Office*
>
> E: matthew@airvantage.co.za <ma...@airvantage.co.za> | M:
> +27 (0) 71053 9012 <tel:+27710539012>
>
> T: +27 (0) 11 100 1880 <tel:+270111001880> | W:
> www.airvantage.co.za <http://www.airvantage.co.za/>
>
> *Skype: matthew.baine57*
>
>
>
> --
>
>
>
>
> *Matthew Baine | *DevOps Engineer
>
> *Johannesburg Head Office*
>
> E: matthew@airvantage.co.za <ma...@airvantage.co.za> | M: +27
> (0) 71053 9012 <tel:+27710539012>
>
> T: +27 (0) 11 100 1880 <tel:+270111001880> | W: www.airvantage.co.za
> <http://www.airvantage.co.za/>
>
> *Skype: matthew.baine57*
>
Re: Usage Documentation for Custom Processors
Posted by Bryan Bende <bb...@gmail.com>.
Hello,
Documentation for a component is generated automatically from the component
itself by using available API methods and the annotations on the class [1].
For example, documentation on a processor will use the
getPropertyDescriptors and getRelationship methods of the component, as
well as annotations like @CapabilityDescription and @Tags, etc.
Thanks,
Bryan
[1]
https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#documenting-a-component
On Tue, Apr 4, 2023 at 10:54 AM Matthew Baine <ma...@airvantage.co.za>
wrote:
> Hi Bryan,
>
> Sorry, on a separate note, what would be the best way to set up Usage
> Documentation for a custom processor?
>
> [image: image.png]
>
> We can't seem to get this right with the information online and on the
> Nifi developer guide (
> https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html). Our
> custom processors seem to only publish documentation of the native
> processors.
>
>
> Kind Regards,
> Matthew
>
> On Tue, 4 Apr 2023 at 13:54, Matthew Baine <ma...@airvantage.co.za>
> wrote:
>
>> Hi Bryan,
>>
>> Sorry for the delayed response, and thank you so much for the feedback!
>>
>> We will attempt the advised approach and revert if we run into any
>> trouble.
>>
>> Thanks again!
>>
>> Regards,
>>
>> On Thu, 30 Mar 2023 at 16:49, Bryan Bende <bb...@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> This might not give you exactly what you want, but the Minifi Toolkit
>>> already has the ability to transform the JSON snapshot from registry,
>>> there are actually two commands:
>>>
>>> "transform" - for XML templates
>>> "transform-vfs" - for versioned flow snapshot (JSON from registry) [1]
>>>
>>> It doesn't pull the snapshot from registry directly, so you would have
>>> to script something to download the snapshot and then run
>>> transform-vfs.
>>>
>>> Thanks,
>>>
>>> Bryan
>>>
>>> [1]
>>> https://github.com/apache/nifi/blob/main/minifi/minifi-toolkit/minifi-toolkit-configuration/src/main/java/org/apache/nifi/minifi/toolkit/configuration/ConfigMain.java#L62
>>>
>>> On Thu, Mar 30, 2023 at 10:22 AM Simeon Wentzel <si...@airvantage.co.za>
>>> wrote:
>>> >
>>> > Dear Nifi dev team
>>> >
>>> > Can you add extended functionality to the MiNiFi toolkit to extract a
>>> flow
>>> > from the NiFi Registry software and convert it to the appropriate
>>> conf.yml
>>> > file?
>>> >
>>> > We have found a limitation regarding the conversion in the minifi
>>> toolkit
>>> > that it can only convert the .xml file template extracted from a Nifi
>>> > canvas on Java version 8, it can not do the conversion on java 11 that
>>> we
>>> > have migrated to.
>>> >
>>> > Although extracting the flow as a template out of nifi and then
>>> converting
>>> > it to the conf.yaml file works we find it a bit cumbersome because we
>>> can
>>> > not implement it in our pipeline to automate the process.
>>> >
>>> > By allowing the minifi toolkit to pull a flow from the Nifi registry
>>> and
>>> > then convert it will give us the functionality to add this in our
>>> Jenkins
>>> > pipeline to build individual docker containers for each of our flows.
>>> >
>>> > Regards
>>> > Simeon
>>> > DevOps Engineer
>>>
>>
>>
>> --
>>
>>
>> *Matthew Baine | *DevOps Engineer
>>
>> *Johannesburg Head Office*
>>
>> E: matthew@airvantage.co.za | M: +27 (0) 71053 9012 <+27710539012>
>>
>> T: +27 (0) 11 100 1880 <+270111001880> | W: www.airvantage.co.za
>>
>> *Skype: matthew.baine57*
>>
>
>
> --
>
>
> *Matthew Baine | *DevOps Engineer
>
> *Johannesburg Head Office*
>
> E: matthew@airvantage.co.za | M: +27 (0) 71053 9012 <+27710539012>
>
> T: +27 (0) 11 100 1880 <+270111001880> | W: www.airvantage.co.za
>
> *Skype: matthew.baine57*
>