Compiling custom processor

[John McGinn <am...@yahoo.com.INVALID>]

Ok, I am far from a maven expert, and am struggling on this problem.

I created a new project using the maven generate process, and compiled that sample processor and everything was fine, and I could see it in my local NiFi instance. I then copied over the Wait processor on top of the MyProcessor.java class, changed the package name, and the class name, and attempted to compile. Got errors due to dependency issues. This is where I get confused.

In the processor directory, I modify the pom.xml with a dependency line for nifi-api, nifi-utils, nifi-distribute-cache-client-service-api and nifi-standard-processors. This seemed to make things work, and I get a 33 meg NAR file, which contains lots of JARs. I load that up, and I get a problem with CalculateStats, or similar, because of RecordReaderFactory. I go back in and include nifi-record-serialization-services, and it compiles, and the JAR is included in the NAR file now, and I still get class not found.

I've also tried to change the dependencies to a status of provided, and therefore get a minimal 25k or so NAR file with no JARs included. I get the same issue with the class not found.

I had also attempted to copy Wait.java to AnotherWait.java within the actually NiFi source code, and it compiles, and shows up in the JARs and NARs as I'd expect, but I cannot get to it within my NiFi instance.

Can someone let me know what I'm doing wrong.

Thanks,
John McGinn

Re: Compiling custom processor

[Mike Thomsen <mi...@gmail.com>]

If you want to look at a large Maven project that builds several custom
NARs from one unified build system, take a look at my data generation
bundle here:

https://github.com/MikeThomsen/nifi-datageneration-bundle

On Tue, Oct 16, 2018 at 9:53 AM Bryan Bende <bb...@gmail.com> wrote:

> In general, if your processor uses a controller service then the
> processors pom file needs a provided dependency on the API of the CS,
> and your NAR pom needs a NAR dependency on the NAR where the CS API
> is.
>
> Example is shown here in the section linking processors and controller
> services -
> https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions
>
> Also, you do not want to include NiFI's standard processors in your
> own NAR, this will result in two copies of every standard processor.
> You will want to copy whatever code you need into your own NAR.
>
> In you last statement, if you made a new processor in
> nifi-standard-processors and it didn't show up, it is most likely
> because you didn't update META-INF/services file to include the new
> processor.
> On Tue, Oct 16, 2018 at 9:36 AM John McGinn
> <am...@yahoo.com.invalid> wrote:
> >
> > Ok, I am far from a maven expert, and am struggling on this problem.
> >
> > I created a new project using the maven generate process, and compiled
> that sample processor and everything was fine, and I could see it in my
> local NiFi instance. I then copied over the Wait processor on top of the
> MyProcessor.java class, changed the package name, and the class name, and
> attempted to compile. Got errors due to dependency issues. This is where I
> get confused.
> >
> > In the processor directory, I modify the pom.xml with a dependency line
> for nifi-api, nifi-utils, nifi-distribute-cache-client-service-api and
> nifi-standard-processors. This seemed to make things work, and I get a 33
> meg NAR file, which contains lots of JARs. I load that up, and I get a
> problem with CalculateStats, or similar, because of RecordReaderFactory. I
> go back in and include nifi-record-serialization-services, and it compiles,
> and the JAR is included in the NAR file now, and I still get class not
> found.
> >
> > I've also tried to change the dependencies to a status of provided, and
> therefore get a minimal 25k or so NAR file with no JARs included. I get the
> same issue with the class not found.
> >
> > I had also attempted to copy Wait.java to AnotherWait.java within the
> actually NiFi source code, and it compiles, and shows up in the JARs and
> NARs as I'd expect, but I cannot get to it within my NiFi instance.
> >
> > Can someone let me know what I'm doing wrong.
> >
> > Thanks,
> > John McGinn
>

Re: Compiling custom processor

[Bryan Bende <bb...@gmail.com>]

In general, if your processor uses a controller service then the
processors pom file needs a provided dependency on the API of the CS,
and your NAR pom needs a NAR dependency on the NAR where the CS API
is.

Example is shown here in the section linking processors and controller
services - https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions

Also, you do not want to include NiFI's standard processors in your
own NAR, this will result in two copies of every standard processor.
You will want to copy whatever code you need into your own NAR.

In you last statement, if you made a new processor in
nifi-standard-processors and it didn't show up, it is most likely
because you didn't update META-INF/services file to include the new
processor.
On Tue, Oct 16, 2018 at 9:36 AM John McGinn
<am...@yahoo.com.invalid> wrote:
>
> Ok, I am far from a maven expert, and am struggling on this problem.
>
> I created a new project using the maven generate process, and compiled that sample processor and everything was fine, and I could see it in my local NiFi instance. I then copied over the Wait processor on top of the MyProcessor.java class, changed the package name, and the class name, and attempted to compile. Got errors due to dependency issues. This is where I get confused.
>
> In the processor directory, I modify the pom.xml with a dependency line for nifi-api, nifi-utils, nifi-distribute-cache-client-service-api and nifi-standard-processors. This seemed to make things work, and I get a 33 meg NAR file, which contains lots of JARs. I load that up, and I get a problem with CalculateStats, or similar, because of RecordReaderFactory. I go back in and include nifi-record-serialization-services, and it compiles, and the JAR is included in the NAR file now, and I still get class not found.
>
> I've also tried to change the dependencies to a status of provided, and therefore get a minimal 25k or so NAR file with no JARs included. I get the same issue with the class not found.
>
> I had also attempted to copy Wait.java to AnotherWait.java within the actually NiFi source code, and it compiles, and shows up in the JARs and NARs as I'd expect, but I cannot get to it within my NiFi instance.
>
> Can someone let me know what I'm doing wrong.
>
> Thanks,
> John McGinn