You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Jeremy Taylor <jh...@everwatchsolutions.com> on 2019/05/15 20:26:30 UTC
issue w/ using MongoClientService w/ custom mongo processors
Greetings,
https://github.com/everwatchsolutions/aces-nifi-processors-bundle
Per the link above, there are some custom processors that I help support. Among those are two custom mongo processors called PartialUpdateMongo and PutMongoWithDuplicateCheck. I’m going to list a little background below and mention some highlights of changes in the pom that had to be made, so folks don’t respond with things we might already have. To cut to the chase, I am unable to select or create a new MongoClientService from the NiFi UI for these 2 custom processors. The error you get when you try to create a new Client service from this processors is: “No controller service types found that are applicable for this property.” We are not implementing a new controller service here at all, so I cannot imagine the need to place anything special in src/main/resources/META-INF. What makes this additionally interesting is I do seem to be able to create an SSLContext service if I want to from these custom procesors w/ the current state. Why I can create one thing and not the other is strange. Would there be any thoughts or tips on what I’m missing that I might need to adjust in my classes or poms to get rid of this error preventing me from creating a mongo client service for each processor?
Background:
We on average upgrade our own system to use a newer NiFi about once a year and upgraded our custom processors and NiFi flow near the end of last month to use NiFi 1.9.2 from NiFi 1.6.0. These two processors do static initialization like many other processors do, but due to a change in the AbstractMongoProcessor class, the “descriptors” member variable became default package level access, so I had to write an AbstractMongoBridgeProcessor class to extend AbstractMongoProcessor to be able to access that member variable while allowing our existing mongo custom processors to continue to do static initialization and pick all the necessary property descriptors up from AbstractMongoBridgeProcessor before adding more of our own to each custom processor during the static initialization phase. Thus, these custom processors don’t directly extend AbstractMongoProcessor like your PutMongo or GetMongo processors do.
Special POM Changes that were recently made:
1. Within the NAR project’s pom, I had to add the following upon upgrading our custom processor modules to support 1.9.2:
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-mongodb-client-service-api</artifactId>
<scope>runtime</scope>
</dependency>
1. In the Processors project’s pom, I changed the following to be “provided” in scope: nifi-mongodb-processors, mongo-java-driver, nifi-api, nifi-processor-utils
2. In the Processors project’s pom, I changed the following to be runtime in scope: nifi-standard-services-api-nar
3. As a bonus, I’ll mention that within the Processors project’s pom, I left nifi-ssl-context-service-api at scope provided.
Regards,
--
Jeremy H. Taylor
Software Developer
ACES Incorporated, an EverWatch Company
Email: jeremy.taylor@acesinc.net<ma...@acesinc.net>
Email: jhtaylor@everwatchsolutions.com<ma...@everwatchsolutions.com>
http://acesinc.net<http://acesinc.net/>
http://everwatchsolutions.com<http://everwatchsolutions.com/>
Re: issue w/ using MongoClientService w/ custom mongo processors
Posted by Mike Thomsen <mi...@gmail.com>.
See my response to your original post. (This looks like an accidental
double post to me)
On Wed, May 15, 2019 at 5:02 PM Jeremy Taylor <
jhtaylor@everwatchsolutions.com> wrote:
> Greetings,
> https://github.com/everwatchsolutions/aces-nifi-processors-bundle
> Per the link above, there are some custom processors that I help support.
> Among those are two custom mongo processors called PartialUpdateMongo and
> PutMongoWithDuplicateCheck. I’m going to list a little background below
> and mention some highlights of changes in the pom that had to be made, so
> folks don’t respond with things we might already have. To cut to the
> chase, I am unable to select or create a new MongoClientService from the
> NiFi UI for these 2 custom processors. The error you get when you try to
> create a new Client service from this processors is: “No controller
> service types found that are applicable for this property.” We are not
> implementing a new controller service here at all, so I cannot imagine the
> need to place anything special in src/main/resources/META-INF. What makes
> this additionally interesting is I do seem to be able to create an
> SSLContext service if I want to from these custom procesors w/ the current
> state. Why I can create one thing and not the other is strange. Would
> there be any thoughts or tips on what I’m missing that I might need to
> adjust in my classes or poms to get rid of this error preventing me from
> creating a mongo client service for each processor?
>
> Background:
> We on average upgrade our own system to use a newer NiFi about once a year
> and upgraded our custom processors and NiFi flow near the end of last month
> to use NiFi 1.9.2 from NiFi 1.6.0. These two processors do static
> initialization like many other processors do, but due to a change in the
> AbstractMongoProcessor class, the “descriptors” member variable became
> default package level access, so I had to write an
> AbstractMongoBridgeProcessor class to extend AbstractMongoProcessor to be
> able to access that member variable while allowing our existing mongo
> custom processors to continue to do static initialization and pick all the
> necessary property descriptors up from AbstractMongoBridgeProcessor before
> adding more of our own to each custom processor during the static
> initialization phase. Thus, these custom processors don’t directly extend
> AbstractMongoProcessor like your PutMongo or GetMongo processors do.
>
> Special POM Changes that were recently made:
>
> 1. Within the NAR project’s pom, I had to add the following upon
> upgrading our custom processor modules to support 1.9.2:
> <dependency>
> <groupId>org.apache.nifi</groupId>
> <artifactId>nifi-mongodb-client-service-api</artifactId>
> <scope>runtime</scope>
> </dependency>
>
> 1. In the Processors project’s pom, I changed the following to be
> “provided” in scope: nifi-mongodb-processors, mongo-java-driver, nifi-api,
> nifi-processor-utils
> 2. In the Processors project’s pom, I changed the following to be
> runtime in scope: nifi-standard-services-api-nar
> 3. As a bonus, I’ll mention that within the Processors project’s pom, I
> left nifi-ssl-context-service-api at scope provided.
> Regards,
>
> --
> Jeremy H. Taylor
> Software Developer
> ACES Incorporated, an EverWatch Company
> Email: jeremy.taylor@acesinc.net<ma...@acesinc.net>
> Email: jhtaylor@everwatchsolutions.com<mailto:
> jhtaylor@everwatchsolutions.com>
> http://acesinc.net<http://acesinc.net/>
> http://everwatchsolutions.com<http://everwatchsolutions.com/>
>