You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@nifi.apache.org by "Knapp, Michael" <Mi...@capitalone.com> on 2017/01/10 18:14:49 UTC

problems with custom controller services, and other comments

Devs,

For some reason NIFI is not working with some custom controller services I have written. I wrote new implementations of AWSCredentialsProviderService, that aim to work with session tokens. I am hoping to run NIFI from my local machine and to be able to connect with AWS using session tokens.

For both implementations I tried, it fails when I try to create them from the web UI. I created a PutS3Object processor, and configured the “AWS Credentials Provider Service” property. From that property I tried to create a new service and selected my custom implementation. When I click “create” the value for the credentials provider service is the ID of the controller service, not its name. While my controller services require several properties to be set, the web UI is not letting me set them. Usually I see an arrow next to the property, which allows me to configure a controller service, but I am not getting that now. I looked in the nifi-app logs, and I do not see any exception, I have even set the logging to TRACE for all things, and still don’t see any problem in the logs. The PutS3Object processor is not validating because the controller service is found to be invalid. I tried creating a unit test, it seems to work for me in tests, but I can’t use TestRunners because that is processor oriented, not meant for controller services. I have a suspicion that spring’s aspect oriented programming is somehow fuddling with my service.

Does anybody know what I am doing wrong here?

Other unrelated comments:

1. The first time you unpack NIFI it takes super long for it to start for me, like a half hour or more. I think you should make it easy for people to scale back their NIFI implementation. Really I would like to start it with just the minimum NAR files for it to start, and I can add others that I need. Maybe a sub-directory in lib for the essential nars could help people separate the essential stuff from the optional nars. The first time I tried installing it, I thought it was broken when really it just was taking forever (over 30 minutes). I think that new users will probably abandon NIFI if they can’t get it to start quickly out of the box. Maybe split the optional nars into an “extra-lib”, and people can move those into lib as necessary for their goals.

2. Building NIFI from source takes over an hour for me, really I just want to build the bare minimum things to get it to start. I tried creating maven profiles to have it build just the minimum pieces, but this proved to be non-trivial as maven does not seem to respect the “modules” tag in profiles, and the nifi-assembly project requires all of the optional nars to also be built. Creating this might be too complicated for me. Has anybody thought about supporting a quick/minimal build?

3. The “nifi-aws-processors” is challenging to use because in one project they have defined the interfaces for controller services (AWSCredentialsProviderService) and also included the services. I tried creating my own nar with an implementation of AWSCredentialsProviderService, but since it depended on “nifi-aws-processors”, my nar was also re-hosting all of the AWS processors. I was facing a lot of classpath issues because of this. I worked around this by using maven shade to shade in the “nifi-aws-processors” into my own jar, but excluding the services it provided. Then in my nar project I had to exclude the dependency on “nifi-aws-processors”. This was a lot of work on my part when all they needed to do was split that project into api, api-nar, impl, and impl-nar.

4. I think it is very confusing how there is a “Controller Services” for the entire NIFI canvas, and separate ones for individual processor groups. It seems that processors cannot use global controller services, and I am still uncertain about why I would ever create a global one. From Nifi settings, I would like to also see controller services in processor groups, and vice versa. From a processor, I would like to assign controller services that are global in scope, not limited to a processor group. I think this is something that will confuse and frustrate a lot of new developers, driving them to consider competing products.

5. I think the developer guide needs some clarification on what jars are provided and not. New developers will be unsure if they should include “nifi-api” as a provided or compile dependency, and same goes for nifi-framework-core.

6. Perhaps the maven-nar-plugin could let people tell NIFI to only use services listed under a certain set of bundled-dependencies. For example, my code depends on “nifi-aws-processors”, but I don’t want my nar to also host the services in that jar. From the MOJO you are able to exclude entire artifacts, but you can’t exclude the services within certain artifacts. This might be a problem to fix on the classloader side instead of from the maven plugin, I don’t know which is easier.

Please help,

Michael Knapp
Capital One
________________________________________________________

The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.

Re: problems with custom controller services, and other comments

Posted by Oleg Zhurakousky <oz...@hortonworks.com>.

Michael

Sorry took a while, but we just had one of our guys reporting the same symptoms and as Bryan suggested it was actually a problem in the POM, specifically the POM file for NAR modules of your custom processor.
Basically once your processor is referencing the service it’s NRA module has to declare dependency on such service.
I m pasting an example POM for a NAR module from one of our guys who is referencing DBCP service from his custom bundle

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-sql-bundle</artifactId>
<version>1.1.0</version>
</parent>

<dependencies>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-dbcp-service-nar</artifactId>
<type>nar</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-sql-processors</artifactId>
<version>1.1.0</version>
</dependency>
</dependencies>
</project>

Note the first dependency (wifi-dbcp-service-nar)

Hope that helps

Cheers
Oleg
On Jan 10, 2017, at 1:29 PM, Oleg Zhurakousky <oz...@hortonworks.com>> wrote:

Michael

That is indeed strange.
Quick question, is there any chance we can look at your code (the custom CS). Also, what version of NiFi you are using?
Will be looking into it, but any additional info is helpful

Cheers
Oleg

On Jan 10, 2017, at 1:14 PM, Knapp, Michael <Mi...@capitalone.com>> wrote:

Devs,

Does anybody know what I am doing wrong here?

Other unrelated comments:

Please help,

Michael Knapp
Capital One
________________________________________________________

Re: problems with custom controller services, and other comments

Posted by Andy LoPresto <al...@apache.org>.

Michael,

What platform were you running NiFi on the first time you deployed? We have seen occasional issues where a virtual machine environment on a headless server (i.e. no physical hardware input sources of entropy) can hang because /dev/random believes the “pool of entropy” is not sufficient to generate safe random values and blocks. This is reported as NIFI-3313 [1]. I believe I have a fix for this that should be in the next release. The normal “first start up time” on modern commodity hardware (i.e. 2015 MacBook Pro with 16 GB RAM) is 9 - 12 seconds (random sampling of my current various PRs and release verification logs) [2].

For building from source, the contrib-check can take some time because it cannot be parallelized, but I usually run the following commands:

* mvn clean install -Pcontrib-check -> This takes about 20 minutes on my machine (completely serial)
* mvn clean install -T2.0C && mvn checkstyle:check -> This can be down to 10 - 11 minutes (parallelizes the tests and install; then runs just the contrib-check)

If you don’t care about contrib-check (i.e. you are not performing review of a PR or verifying the release candidates), you can skip that entirely:

* mvn clean install -T2.0C -> Sometimes under 10 minutes even with other tasks running

I just realized building for you might take an hour the first time if your local M2 repository is empty and it has to download every artifact necessary over a bad network connection. Sadly I don’t think there is a way around this currently, but as Bryan mentioned, the various registries will help slim down the core codebase. Once the initial build has happened though, you should see much faster build times. In addition, for your development, you should only need to build your NAR and drop it into the lib directory in your running instance, so this should be fairly quick (1 - 2 minutes max).

Please let us know if any of this helps or if you have more information that we can use to improve your experiences.

[1] https://issues.apache.org/jira/browse/NIFI-3313 <https://issues.apache.org/jira/browse/NIFI-3313>
[2] https://gist.github.com/alopresto/9e269561330f0b65ffb19e243e16747d <https://gist.github.com/alopresto/9e269561330f0b65ffb19e243e16747d>

Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Jan 10, 2017, at 10:43 AM, Bryan Bende <bb...@gmail.com> wrote:
> 
> Hi Michael,
> 
> Regarding not being able to use the controller service and only seeing the
> id, this usually indicates a dependency problem. Take a look at these
> resources [1][2] to ensure you have the appropriate linkage between
> processors and controller service:
> 
> Some of the other questions...
> 
> For 1 & 2 they will likely be addressed by the extension registry [3]. The
> idea is that base NiFi would not include every NAR, most NARs would live in
> an extension repository and then users can pick and choose which NAR to
> include in their distribution. This would also alleviate the build time
> since all of the NARs source code wouldn't be part of the core NiFi build.
> 
> For 4, Controller services at the global level are only for use by
> reporting tasks, the only one right now is the
> SiteToSiteProvenanceReportingTask which can use a SSLContextService, but
> there could be others in the future. If you want to create a reporting task
> for use by any processor then you should create it at the root group level
> (the main canvas is a process group itself). The reason controller services
> are tied to process groups is because of security... you might not want
> someone outside that process group who doesn't permissions to it to be able
> to see the controller services with in the group.
> 
> For 5, there is a Maven archetype [4] that helps generate the recommended
> project structure and dependencies. In general though, the only JARs that
> will be automatically available are the JARs directly in the lib directory.
> 
> Hope that helps.
> 
> -Bryan
> 
> [1]
> https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions#MavenProjectsforExtensions-LinkingProcessorsandControllerServices
> [2] https://github.com/bbende/nifi-dependency-example
> [3]
> https://cwiki.apache.org/confluence/display/NIFI/Extension+Repositories+%28aka+Extension+Registry%29+for+Dynamically-loaded+Extensions
> [4]
> https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions
> 
> On Tue, Jan 10, 2017 at 1:29 PM, Oleg Zhurakousky <
> ozhurakousky@hortonworks.com> wrote:
> 
>> Michael
>> 
>> That is indeed strange.
>> Quick question, is there any chance we can look at your code (the custom
>> CS). Also, what version of NiFi you are using?
>> Will be looking into it, but any additional info is helpful
>> 
>> Cheers
>> Oleg
>> 
>> 
>>> On Jan 10, 2017, at 1:14 PM, Knapp, Michael <
>> Michael.Knapp@capitalone.com> wrote:
>>> 
>>> Devs,
>>> 
>>> For some reason NIFI is not working with some custom controller services
>> I have written.  I wrote new implementations of
>> AWSCredentialsProviderService, that aim to work with session tokens.  I am
>> hoping to run NIFI from my local machine and to be able to connect with AWS
>> using session tokens.
>>> 
>>> For both implementations I tried, it fails when I try to create them
>> from the web UI.  I created a PutS3Object processor, and configured the
>> “AWS Credentials Provider Service” property.  From that property I tried to
>> create a new service and selected my custom implementation.  When I click
>> “create” the value for the credentials provider service is the ID of the
>> controller service, not its name.  While my controller services require
>> several properties to be set, the web UI is not letting me set them.
>> Usually I see an arrow next to the property, which allows me to configure a
>> controller service, but I am not getting that now.  I looked in the
>> nifi-app logs, and I do not see any exception, I have even set the logging
>> to TRACE for all things, and still don’t see any problem in the logs.  The
>> PutS3Object processor is not validating because the controller service is
>> found to be invalid.  I tried creating a unit test, it seems to work for me
>> in tests, but I can’t use TestRunners because that is processor oriented,
>> not meant for controller services.  I have a suspicion that spring’s aspect
>> oriented programming is somehow fuddling with my service.
>>> 
>>> Does anybody know what I am doing wrong here?
>>> 
>>> Other unrelated comments:
>>> 
>>> 1.       The first time you unpack NIFI it takes super long for it to
>> start for me, like a half hour or more.  I think you should make it easy
>> for people to scale back their NIFI implementation.  Really I would like to
>> start it with just the minimum NAR files for it to start, and I can add
>> others that I need.  Maybe a sub-directory in lib for the essential nars
>> could help people separate the essential stuff from the optional nars.  The
>> first time I tried installing it, I thought it was broken when really it
>> just was taking forever (over 30 minutes).  I think that new users will
>> probably abandon NIFI if they can’t get it to start quickly out of the
>> box.  Maybe split the optional nars into an “extra-lib”, and people can
>> move those into lib as necessary for their goals.
>>> 
>>> 2.       Building NIFI from source takes over an hour for me, really I
>> just want to build the bare minimum things to get it to start.  I tried
>> creating maven profiles to have it build just the minimum pieces, but this
>> proved to be non-trivial as maven does not seem to respect the “modules”
>> tag in profiles, and the nifi-assembly project requires all of the optional
>> nars to also be built.  Creating this might be too complicated for me.  Has
>> anybody thought about supporting a quick/minimal build?
>>> 
>>> 3.       The “nifi-aws-processors” is challenging to use because in one
>> project they have defined the interfaces for controller services (AWSCredentialsProviderService)
>> and also included the services.  I tried creating my own nar with an
>> implementation of AWSCredentialsProviderService, but since it depended on
>> “nifi-aws-processors”, my nar was also re-hosting all of the AWS
>> processors.  I was facing a lot of classpath issues because of this.  I
>> worked around this by using maven shade to shade in the
>> “nifi-aws-processors” into my own jar, but excluding the services it
>> provided.  Then in my nar project I had to exclude the dependency on
>> “nifi-aws-processors”.  This was a lot of work on my part when all they
>> needed to do was split that project into api, api-nar, impl, and impl-nar.
>>> 
>>> 4.       I think it is very confusing how there is a “Controller
>> Services” for the entire NIFI canvas, and separate ones for individual
>> processor groups.  It seems that processors cannot use global controller
>> services, and I am still uncertain about why I would ever create a global
>> one.  From Nifi settings, I would like to also see controller services in
>> processor groups, and vice versa.  From a processor, I would like to assign
>> controller services that are global in scope, not limited to a processor
>> group.  I think this is something that will confuse and frustrate a lot of
>> new developers, driving them to consider competing products.
>>> 
>>> 5.       I think the developer guide needs some clarification on what
>> jars are provided and not.  New developers will be unsure if they should
>> include “nifi-api” as a provided or compile dependency, and same goes for
>> nifi-framework-core.
>>> 
>>> 6.       Perhaps the maven-nar-plugin could let people tell NIFI to only
>> use services listed under a certain set of bundled-dependencies.  For
>> example, my code depends on “nifi-aws-processors”, but I don’t want my nar
>> to also host the services in that jar.  From the MOJO you are able to
>> exclude entire artifacts, but you can’t exclude the services within certain
>> artifacts.  This might be a problem to fix on the classloader side instead
>> of from the maven plugin, I don’t know which is easier.
>>> 
>>> Please help,
>>> 
>>> Michael Knapp
>>> Capital One
>>> ________________________________________________________
>>> 
>>> The information contained in this e-mail is confidential and/or
>> proprietary to Capital One and/or its affiliates and may only be used
>> solely in performance of work or services for Capital One. The information
>> transmitted herewith is intended only for use by the individual or entity
>> to which it is addressed. If the reader of this message is not the intended
>> recipient, you are hereby notified that any review, retransmission,
>> dissemination, distribution, copying or other use of, or taking of any
>> action in reliance upon this information is strictly prohibited. If you
>> have received this communication in error, please contact the sender and
>> delete the material from your computer.
>> 
>>

Re: problems with custom controller services, and other comments

Posted by Bryan Bende <bb...@gmail.com>.

Hi Michael,

Regarding not being able to use the controller service and only seeing the
id, this usually indicates a dependency problem. Take a look at these
resources [1][2] to ensure you have the appropriate linkage between
processors and controller service:

Some of the other questions...

For 1 & 2 they will likely be addressed by the extension registry [3]. The
idea is that base NiFi would not include every NAR, most NARs would live in
an extension repository and then users can pick and choose which NAR to
include in their distribution. This would also alleviate the build time
since all of the NARs source code wouldn't be part of the core NiFi build.

For 4, Controller services at the global level are only for use by
reporting tasks, the only one right now is the
SiteToSiteProvenanceReportingTask which can use a SSLContextService, but
there could be others in the future. If you want to create a reporting task
for use by any processor then you should create it at the root group level
(the main canvas is a process group itself). The reason controller services
are tied to process groups is because of security... you might not want
someone outside that process group who doesn't permissions to it to be able
to see the controller services with in the group.

For 5, there is a Maven archetype [4] that helps generate the recommended
project structure and dependencies. In general though, the only JARs that
will be automatically available are the JARs directly in the lib directory.

Hope that helps.

-Bryan

[1]
https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions#MavenProjectsforExtensions-LinkingProcessorsandControllerServices
[2] https://github.com/bbende/nifi-dependency-example
[3]
https://cwiki.apache.org/confluence/display/NIFI/Extension+Repositories+%28aka+Extension+Registry%29+for+Dynamically-loaded+Extensions
[4]
https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions

On Tue, Jan 10, 2017 at 1:29 PM, Oleg Zhurakousky <
ozhurakousky@hortonworks.com> wrote:

> Michael
>
> That is indeed strange.
> Quick question, is there any chance we can look at your code (the custom
> CS). Also, what version of NiFi you are using?
> Will be looking into it, but any additional info is helpful
>
> Cheers
> Oleg
>
>
> > On Jan 10, 2017, at 1:14 PM, Knapp, Michael <
> Michael.Knapp@capitalone.com> wrote:
> >
> > Devs,
> >
> > For some reason NIFI is not working with some custom controller services
> I have written.  I wrote new implementations of
> AWSCredentialsProviderService, that aim to work with session tokens.  I am
> hoping to run NIFI from my local machine and to be able to connect with AWS
> using session tokens.
> >
> > For both implementations I tried, it fails when I try to create them
> from the web UI.  I created a PutS3Object processor, and configured the
> “AWS Credentials Provider Service” property.  From that property I tried to
> create a new service and selected my custom implementation.  When I click
> “create” the value for the credentials provider service is the ID of the
> controller service, not its name.  While my controller services require
> several properties to be set, the web UI is not letting me set them.
> Usually I see an arrow next to the property, which allows me to configure a
> controller service, but I am not getting that now.  I looked in the
> nifi-app logs, and I do not see any exception, I have even set the logging
> to TRACE for all things, and still don’t see any problem in the logs.  The
> PutS3Object processor is not validating because the controller service is
> found to be invalid.  I tried creating a unit test, it seems to work for me
> in tests, but I can’t use TestRunners because that is processor oriented,
> not meant for controller services.  I have a suspicion that spring’s aspect
> oriented programming is somehow fuddling with my service.
> >
> > Does anybody know what I am doing wrong here?
> >
> > Other unrelated comments:
> >
> > 1.       The first time you unpack NIFI it takes super long for it to
> start for me, like a half hour or more.  I think you should make it easy
> for people to scale back their NIFI implementation.  Really I would like to
> start it with just the minimum NAR files for it to start, and I can add
> others that I need.  Maybe a sub-directory in lib for the essential nars
> could help people separate the essential stuff from the optional nars.  The
> first time I tried installing it, I thought it was broken when really it
> just was taking forever (over 30 minutes).  I think that new users will
> probably abandon NIFI if they can’t get it to start quickly out of the
> box.  Maybe split the optional nars into an “extra-lib”, and people can
> move those into lib as necessary for their goals.
> >
> > 2.       Building NIFI from source takes over an hour for me, really I
> just want to build the bare minimum things to get it to start.  I tried
> creating maven profiles to have it build just the minimum pieces, but this
> proved to be non-trivial as maven does not seem to respect the “modules”
> tag in profiles, and the nifi-assembly project requires all of the optional
> nars to also be built.  Creating this might be too complicated for me.  Has
> anybody thought about supporting a quick/minimal build?
> >
> > 3.       The “nifi-aws-processors” is challenging to use because in one
> project they have defined the interfaces for controller services (AWSCredentialsProviderService)
> and also included the services.  I tried creating my own nar with an
> implementation of AWSCredentialsProviderService, but since it depended on
> “nifi-aws-processors”, my nar was also re-hosting all of the AWS
> processors.  I was facing a lot of classpath issues because of this.  I
> worked around this by using maven shade to shade in the
> “nifi-aws-processors” into my own jar, but excluding the services it
> provided.  Then in my nar project I had to exclude the dependency on
> “nifi-aws-processors”.  This was a lot of work on my part when all they
> needed to do was split that project into api, api-nar, impl, and impl-nar.
> >
> > 4.       I think it is very confusing how there is a “Controller
> Services” for the entire NIFI canvas, and separate ones for individual
> processor groups.  It seems that processors cannot use global controller
> services, and I am still uncertain about why I would ever create a global
> one.  From Nifi settings, I would like to also see controller services in
> processor groups, and vice versa.  From a processor, I would like to assign
> controller services that are global in scope, not limited to a processor
> group.  I think this is something that will confuse and frustrate a lot of
> new developers, driving them to consider competing products.
> >
> > 5.       I think the developer guide needs some clarification on what
> jars are provided and not.  New developers will be unsure if they should
> include “nifi-api” as a provided or compile dependency, and same goes for
> nifi-framework-core.
> >
> > 6.       Perhaps the maven-nar-plugin could let people tell NIFI to only
> use services listed under a certain set of bundled-dependencies.  For
> example, my code depends on “nifi-aws-processors”, but I don’t want my nar
> to also host the services in that jar.  From the MOJO you are able to
> exclude entire artifacts, but you can’t exclude the services within certain
> artifacts.  This might be a problem to fix on the classloader side instead
> of from the maven plugin, I don’t know which is easier.
> >
> > Please help,
> >
> > Michael Knapp
> > Capital One
> > ________________________________________________________
> >
> > The information contained in this e-mail is confidential and/or
> proprietary to Capital One and/or its affiliates and may only be used
> solely in performance of work or services for Capital One. The information
> transmitted herewith is intended only for use by the individual or entity
> to which it is addressed. If the reader of this message is not the intended
> recipient, you are hereby notified that any review, retransmission,
> dissemination, distribution, copying or other use of, or taking of any
> action in reliance upon this information is strictly prohibited. If you
> have received this communication in error, please contact the sender and
> delete the material from your computer.
>
>

Re: problems with custom controller services, and other comments

Posted by Oleg Zhurakousky <oz...@hortonworks.com>.

Michael

That is indeed strange. 
Quick question, is there any chance we can look at your code (the custom CS). Also, what version of NiFi you are using?
Will be looking into it, but any additional info is helpful 

Cheers
Oleg


> On Jan 10, 2017, at 1:14 PM, Knapp, Michael <Mi...@capitalone.com> wrote:
> 
> Devs,
> 
> For some reason NIFI is not working with some custom controller services I have written.  I wrote new implementations of AWSCredentialsProviderService, that aim to work with session tokens.  I am hoping to run NIFI from my local machine and to be able to connect with AWS using session tokens.
> 
> For both implementations I tried, it fails when I try to create them from the web UI.  I created a PutS3Object processor, and configured the “AWS Credentials Provider Service” property.  From that property I tried to create a new service and selected my custom implementation.  When I click “create” the value for the credentials provider service is the ID of the controller service, not its name.  While my controller services require several properties to be set, the web UI is not letting me set them.  Usually I see an arrow next to the property, which allows me to configure a controller service, but I am not getting that now.  I looked in the nifi-app logs, and I do not see any exception, I have even set the logging to TRACE for all things, and still don’t see any problem in the logs.  The PutS3Object processor is not validating because the controller service is found to be invalid.  I tried creating a unit test, it seems to work for me in tests, but I can’t use TestRunners because that is processor oriented, not meant for controller services.  I have a suspicion that spring’s aspect oriented programming is somehow fuddling with my service.
> 
> Does anybody know what I am doing wrong here?
> 
> Other unrelated comments:
> 
> 1.       The first time you unpack NIFI it takes super long for it to start for me, like a half hour or more.  I think you should make it easy for people to scale back their NIFI implementation.  Really I would like to start it with just the minimum NAR files for it to start, and I can add others that I need.  Maybe a sub-directory in lib for the essential nars could help people separate the essential stuff from the optional nars.  The first time I tried installing it, I thought it was broken when really it just was taking forever (over 30 minutes).  I think that new users will probably abandon NIFI if they can’t get it to start quickly out of the box.  Maybe split the optional nars into an “extra-lib”, and people can move those into lib as necessary for their goals.
> 
> 2.       Building NIFI from source takes over an hour for me, really I just want to build the bare minimum things to get it to start.  I tried creating maven profiles to have it build just the minimum pieces, but this proved to be non-trivial as maven does not seem to respect the “modules” tag in profiles, and the nifi-assembly project requires all of the optional nars to also be built.  Creating this might be too complicated for me.  Has anybody thought about supporting a quick/minimal build?
> 
> 3.       The “nifi-aws-processors” is challenging to use because in one project they have defined the interfaces for controller services (AWSCredentialsProviderService) and also included the services.  I tried creating my own nar with an implementation of AWSCredentialsProviderService, but since it depended on “nifi-aws-processors”, my nar was also re-hosting all of the AWS processors.  I was facing a lot of classpath issues because of this.  I worked around this by using maven shade to shade in the “nifi-aws-processors” into my own jar, but excluding the services it provided.  Then in my nar project I had to exclude the dependency on “nifi-aws-processors”.  This was a lot of work on my part when all they needed to do was split that project into api, api-nar, impl, and impl-nar.
> 
> 4.       I think it is very confusing how there is a “Controller Services” for the entire NIFI canvas, and separate ones for individual processor groups.  It seems that processors cannot use global controller services, and I am still uncertain about why I would ever create a global one.  From Nifi settings, I would like to also see controller services in processor groups, and vice versa.  From a processor, I would like to assign controller services that are global in scope, not limited to a processor group.  I think this is something that will confuse and frustrate a lot of new developers, driving them to consider competing products.
> 
> 5.       I think the developer guide needs some clarification on what jars are provided and not.  New developers will be unsure if they should include “nifi-api” as a provided or compile dependency, and same goes for nifi-framework-core.
> 
> 6.       Perhaps the maven-nar-plugin could let people tell NIFI to only use services listed under a certain set of bundled-dependencies.  For example, my code depends on “nifi-aws-processors”, but I don’t want my nar to also host the services in that jar.  From the MOJO you are able to exclude entire artifacts, but you can’t exclude the services within certain artifacts.  This might be a problem to fix on the classloader side instead of from the maven plugin, I don’t know which is easier.
> 
> Please help,
> 
> Michael Knapp
> Capital One
> ________________________________________________________
> 
> The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.