You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Russell Bateman <ru...@windofkeltia.com> on 2017/05/16 15:28:28 UTC
Service controller list NiFi 0.x to 1.x
It appears to me that that, unlike what happened in NiFi 0.x, in 1.x
when I look at controller services via the General menu -> Controller
Services, what I see is totally different from what I see when I
configure controller services for a processor.
If I use the General menu to set up my controller services, I do not see
nor am I given the option of using them in particular for processors I'm
configuring. Instead, I appear to get a "Process Group Configuration and
a list of controller services which are not the ones I'm looking for
(because when I set them up, I gave them "special" names or renamed
names I could recognize apart from any other use).
Note: I'm more of a processor and controller service author than an
experienced user of NiFi, so I may just be hopelessly confused.
My question is what's the point of being able to configure controller
services "globally" or "generally" if you can't reach them when you need
them?
Please confirm that I'm not just smoking funny weed and that this is
different, in fact, from how it worked in 0.7.1.
Thanks.
Re: Service controller list NiFi 0.x to 1.x
Posted by Russell Bateman <ru...@windofkeltia.com>.
It's because product planning can't be done overnight. It was a fight
(months ago) to get higher approval to move to1.x. 1.1.2was the current
release when approval was given. We'll plan to move ahead to 1.2.xin a
few months motivated by the next release. In development, I tend to keep
up, but product moves behind. As soon as my code for this release is
locked, I'll start a new project and move up--months ahead of everyone else.
We're not clustering yet. I think the make-over of the authentication
files and this issue were really the only ones that gave us pause in
making the move. I pushed hard because I'm tired of going back and forth
between the UI changes. However, as principally a write of custom
processors and a controller service or two, not much has really changed
for me.
NiFi's a great framework!
Russ
On 05/16/2017 03:37 PM, Andy LoPresto wrote:
> Russell,
>
> Is there a reason you are planning your migration to 1.1.2 and not
> 1.2.0? There are a number of significant feature and bug fixes that
> went into 1.2.0 that will likely make your experience much better.
>
>
> Andy LoPresto
> alopresto@apache.org <ma...@apache.org>
> /alopresto.apache@gmail.com <ma...@gmail.com>/
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
>> On May 16, 2017, at 2:30 PM, Russell Bateman <russ@windofkeltia.com
>> <ma...@windofkeltia.com>> wrote:
>>
>> We're still testing our migration from 0.7.2 to 1.1.2 which is
>> imminent. We still don't know how much of this is going to have to be
>> reconfigured. Our down-streamers tend to be on the grouchy side, so
>> we're testing to see what's going to irritate them. Were it not for
>> the magic ids, I'd feel safe writing a Python script to run through
>> their flow tarballs fixing the controller-setting impedance
>> mismatches. Who knows; maybe there will be nothing to change although
>> we know for certain of a /DBConnectionPool/ property change.
>>
>> I will come back here in a few hours or tomorrow with an update on
>> how it's going. I'll try to keep the noise down because what may
>> alarm us shouldn't be a cause for alarm for everyone.
>>
>> Thanks for the replies,
>>
>> Russ
>>
>> On 05/16/2017 02:47 PM, Michael Moser wrote:
>>> Hi Russ,
>>>
>>> In NiFi 1.2.0, there are 3 Reporting Tasks
>>> (SiteToSiteBulletionReportingTask, SiteToSiteProvenanceReportingTask,
>>> and SiteToSiteStatusReportingTask) that have the capability of using a
>>> Controller Service (StandardSSLContextService). So yes, the Global ->
>>> Controller Settings has a limited set of use cases, but they are very
>>> important. This becomes even more true considering the extensibility
>>> of NiFi components. Who can predict what new reporting tasks might be
>>> created to meet end user's needs, and what controller services those
>>> tasks might need?
>>>
>>> This is definitely high on the list of things that catch NiFi users
>>> off guard, as they transition from the 0.x versions into the 1.x
>>> versions. I can say that once I got used to using the Operate Palette
>>> (especially for defining controller services!) then it didn't seem as
>>> bad.
>>>
>>> I'll see what I can add to the 0.x to 1.0.0 Migration Guide [1] to
>>> call this out specifically.
>>>
>>> -- Mike
>>>
>>> [1] -https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance
>>>
>>>
>>>
>>> On Tue, May 16, 2017 at 3:53 PM, Russell Bateman<ru...@windofkeltia.com> wrote:
>>>> Thanks, Andrew!
>>>>
>>>> (From [1]:) "This means that the service will be available to all Processors
>>>> and Controller Services defined in that Process Group and below."
>>>>
>>>> In my experience, this isn't true. If I create a controller via the General
>>>> menu in the very root of my NiFi canvas, configure its name in Settings,
>>>> calling it Jack, then I create a new process group, then configure a new
>>>> processor in that group, when I try to configure to use the controller, Jack
>>>> is not among the options.
>>>>
>>>> In order for the statement above to be true, I have to create it via the
>>>> gear icon in the Operate menu/palette.
>>>>
>>>> Is this not what you see?
>>>>
>>>> So, beginning sometime in 1.x, the Controller Settings option in the General
>>>> menu became useless, even at the top level except for "all ReportingTasks
>>>> and services defined in the Controller Settings." But, when would any
>>>> reporting task or other service defined be able to benefit? Never if I'm any
>>>> judge.
>>>>
>>>> I think this is much more than a mere documentation issue. I wonder if
>>>> removing the Controller Services... option from the General menu would not
>>>> be the most important thing to do (even before documenting the gear icon in
>>>> the Operate menu).
>>>>
>>>> Russ
>>>>
>>>>
>>>> On 05/16/2017 10:05 AM, Andrew Lim wrote:
>>>>
>>>> Hi Russell,
>>>>
>>>> Thanks for your question.
>>>>
>>>> Yes, working with Controller Services has definitely changed in 1.x compared
>>>> to 0.x NiFi. Matt Gilman wrote a nice article about how Controller Service
>>>> scoping was updated in 1.x with the introduction of Multi-Tenant
>>>> Authorization and also discusses the recent improvements made in NiFi 1.2 to
>>>> alleviate some of the user confusion around scoping [1]. If you would like
>>>> to see further details, the parent Jira for the improvements can be found
>>>> here [2].
>>>>
>>>> I think there is opportunity to improve the Apache documentation we have
>>>> around this functionality, so I just filed a new Jira [3].
>>>>
>>>> Let us know if you have any more questions.
>>>>
>>>> Thanks,
>>>>
>>>> Drew
>>>>
>>>> [1]
>>>> https://community.hortonworks.com/articles/90259/understanding-controller-service-availability-in-a.html
>>>> [2]https://issues.apache.org/jira/browse/NIFI-3128
>>>> [3]https://issues.apache.org/jira/browse/NIFI-3911
>>>>
>>>>
>>>>
>>>> On May 16, 2017, at 11:28 AM, Russell Bateman<ru...@windofkeltia.com> wrote:
>>>>
>>>> It appears to me that that, unlike what happened in NiFi 0.x, in 1.x when I
>>>> look at controller services via the General menu -> Controller Services,
>>>> what I see is totally different from what I see when I configure controller
>>>> services for a processor.
>>>>
>>>> If I use the General menu to set up my controller services, I do not see nor
>>>> am I given the option of using them in particular for processors I'm
>>>> configuring. Instead, I appear to get a "Process Group Configuration and a
>>>> list of controller services which are not the ones I'm looking for (because
>>>> when I set them up, I gave them "special" names or renamed names I could
>>>> recognize apart from any other use).
>>>>
>>>> Note: I'm more of a processor and controller service author than an
>>>> experienced user of NiFi, so I may just be hopelessly confused.
>>>>
>>>> My question is what's the point of being able to configure controller
>>>> services "globally" or "generally" if you can't reach them when you need
>>>> them?
>>>>
>>>> Please confirm that I'm not just smoking funny weed and that this is
>>>> different, in fact, from how it worked in 0.7.1.
>>>>
>>>> Thanks.
>>>>
>>>>
>>
>
Re: Service controller list NiFi 0.x to 1.x
Posted by Andy LoPresto <al...@apache.org>.
Russell,
Is there a reason you are planning your migration to 1.1.2 and not 1.2.0? There are a number of significant feature and bug fixes that went into 1.2.0 that will likely make your experience much better.
Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> On May 16, 2017, at 2:30 PM, Russell Bateman <ru...@windofkeltia.com> wrote:
>
> We're still testing our migration from 0.7.2 to 1.1.2 which is imminent. We still don't know how much of this is going to have to be reconfigured. Our down-streamers tend to be on the grouchy side, so we're testing to see what's going to irritate them. Were it not for the magic ids, I'd feel safe writing a Python script to run through their flow tarballs fixing the controller-setting impedance mismatches. Who knows; maybe there will be nothing to change although we know for certain of a DBConnectionPool property change.
>
> I will come back here in a few hours or tomorrow with an update on how it's going. I'll try to keep the noise down because what may alarm us shouldn't be a cause for alarm for everyone.
>
> Thanks for the replies,
>
> Russ
>
> On 05/16/2017 02:47 PM, Michael Moser wrote:
>> Hi Russ,
>>
>> In NiFi 1.2.0, there are 3 Reporting Tasks
>> (SiteToSiteBulletionReportingTask, SiteToSiteProvenanceReportingTask,
>> and SiteToSiteStatusReportingTask) that have the capability of using a
>> Controller Service (StandardSSLContextService). So yes, the Global ->
>> Controller Settings has a limited set of use cases, but they are very
>> important. This becomes even more true considering the extensibility
>> of NiFi components. Who can predict what new reporting tasks might be
>> created to meet end user's needs, and what controller services those
>> tasks might need?
>>
>> This is definitely high on the list of things that catch NiFi users
>> off guard, as they transition from the 0.x versions into the 1.x
>> versions. I can say that once I got used to using the Operate Palette
>> (especially for defining controller services!) then it didn't seem as
>> bad.
>>
>> I'll see what I can add to the 0.x to 1.0.0 Migration Guide [1] to
>> call this out specifically.
>>
>> -- Mike
>>
>> [1] - https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance <https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance>
>>
>>
>>
>> On Tue, May 16, 2017 at 3:53 PM, Russell Bateman <ru...@windofkeltia.com> <ma...@windofkeltia.com> wrote:
>>> Thanks, Andrew!
>>>
>>> (From [1]:) "This means that the service will be available to all Processors
>>> and Controller Services defined in that Process Group and below."
>>>
>>> In my experience, this isn't true. If I create a controller via the General
>>> menu in the very root of my NiFi canvas, configure its name in Settings,
>>> calling it Jack, then I create a new process group, then configure a new
>>> processor in that group, when I try to configure to use the controller, Jack
>>> is not among the options.
>>>
>>> In order for the statement above to be true, I have to create it via the
>>> gear icon in the Operate menu/palette.
>>>
>>> Is this not what you see?
>>>
>>> So, beginning sometime in 1.x, the Controller Settings option in the General
>>> menu became useless, even at the top level except for "all ReportingTasks
>>> and services defined in the Controller Settings." But, when would any
>>> reporting task or other service defined be able to benefit? Never if I'm any
>>> judge.
>>>
>>> I think this is much more than a mere documentation issue. I wonder if
>>> removing the Controller Services... option from the General menu would not
>>> be the most important thing to do (even before documenting the gear icon in
>>> the Operate menu).
>>>
>>> Russ
>>>
>>>
>>> On 05/16/2017 10:05 AM, Andrew Lim wrote:
>>>
>>> Hi Russell,
>>>
>>> Thanks for your question.
>>>
>>> Yes, working with Controller Services has definitely changed in 1.x compared
>>> to 0.x NiFi. Matt Gilman wrote a nice article about how Controller Service
>>> scoping was updated in 1.x with the introduction of Multi-Tenant
>>> Authorization and also discusses the recent improvements made in NiFi 1.2 to
>>> alleviate some of the user confusion around scoping [1]. If you would like
>>> to see further details, the parent Jira for the improvements can be found
>>> here [2].
>>>
>>> I think there is opportunity to improve the Apache documentation we have
>>> around this functionality, so I just filed a new Jira [3].
>>>
>>> Let us know if you have any more questions.
>>>
>>> Thanks,
>>>
>>> Drew
>>>
>>> [1]
>>> https://community.hortonworks.com/articles/90259/understanding-controller-service-availability-in-a.html <https://community.hortonworks.com/articles/90259/understanding-controller-service-availability-in-a.html>
>>> [2] https://issues.apache.org/jira/browse/NIFI-3128 <https://issues.apache.org/jira/browse/NIFI-3128>
>>> [3] https://issues.apache.org/jira/browse/NIFI-3911 <https://issues.apache.org/jira/browse/NIFI-3911>
>>>
>>>
>>>
>>> On May 16, 2017, at 11:28 AM, Russell Bateman <ru...@windofkeltia.com> <ma...@windofkeltia.com> wrote:
>>>
>>> It appears to me that that, unlike what happened in NiFi 0.x, in 1.x when I
>>> look at controller services via the General menu -> Controller Services,
>>> what I see is totally different from what I see when I configure controller
>>> services for a processor.
>>>
>>> If I use the General menu to set up my controller services, I do not see nor
>>> am I given the option of using them in particular for processors I'm
>>> configuring. Instead, I appear to get a "Process Group Configuration and a
>>> list of controller services which are not the ones I'm looking for (because
>>> when I set them up, I gave them "special" names or renamed names I could
>>> recognize apart from any other use).
>>>
>>> Note: I'm more of a processor and controller service author than an
>>> experienced user of NiFi, so I may just be hopelessly confused.
>>>
>>> My question is what's the point of being able to configure controller
>>> services "globally" or "generally" if you can't reach them when you need
>>> them?
>>>
>>> Please confirm that I'm not just smoking funny weed and that this is
>>> different, in fact, from how it worked in 0.7.1.
>>>
>>> Thanks.
>>>
>>>
>
Re: Service controller list NiFi 0.x to 1.x
Posted by Russell Bateman <ru...@windofkeltia.com>.
As promised I'm posting back that all of our controller services are
migrated from any flow.xml.gx created by 0.7.2 and put into the
top-level process group so that we have no problem accessing them. Our
biggest problem is then education on how best to develop flows with
controller services beginning in NiFi 1.x. We're producing documentation
internally for that.
Thanks.
On 05/16/2017 03:30 PM, Russell Bateman wrote:
> We're still testing our migration from 0.7.2 to 1.1.2 which is
> imminent. We still don't know how much of this is going to have to be
> reconfigured. Our down-streamers tend to be on the grouchy side, so
> we're testing to see what's going to irritate them. Were it not for
> the magic ids, I'd feel safe writing a Python script to run through
> their flow tarballs fixing the controller-setting impedance
> mismatches. Who knows; maybe there will be nothing to change although
> we know for certain of a /DBConnectionPool/ property change.
>
> I will come back here in a few hours or tomorrow with an update on how
> it's going. I'll try to keep the noise down because what may alarm us
> shouldn't be a cause for alarm for everyone.
>
> Thanks for the replies,
>
> Russ
>
> On 05/16/2017 02:47 PM, Michael Moser wrote:
>> Hi Russ,
>>
>> In NiFi 1.2.0, there are 3 Reporting Tasks
>> (SiteToSiteBulletionReportingTask, SiteToSiteProvenanceReportingTask,
>> and SiteToSiteStatusReportingTask) that have the capability of using a
>> Controller Service (StandardSSLContextService). So yes, the Global ->
>> Controller Settings has a limited set of use cases, but they are very
>> important. This becomes even more true considering the extensibility
>> of NiFi components. Who can predict what new reporting tasks might be
>> created to meet end user's needs, and what controller services those
>> tasks might need?
>>
>> This is definitely high on the list of things that catch NiFi users
>> off guard, as they transition from the 0.x versions into the 1.x
>> versions. I can say that once I got used to using the Operate Palette
>> (especially for defining controller services!) then it didn't seem as
>> bad.
>>
>> I'll see what I can add to the 0.x to 1.0.0 Migration Guide [1] to
>> call this out specifically.
>>
>> -- Mike
>>
>> [1] -https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance
>>
>>
>>
>> On Tue, May 16, 2017 at 3:53 PM, Russell Bateman<ru...@windofkeltia.com> wrote:
>>> Thanks, Andrew!
>>>
>>> (From [1]:) "This means that the service will be available to all Processors
>>> and Controller Services defined in that Process Group and below."
>>>
>>> In my experience, this isn't true. If I create a controller via the General
>>> menu in the very root of my NiFi canvas, configure its name in Settings,
>>> calling it Jack, then I create a new process group, then configure a new
>>> processor in that group, when I try to configure to use the controller, Jack
>>> is not among the options.
>>>
>>> In order for the statement above to be true, I have to create it via the
>>> gear icon in the Operate menu/palette.
>>>
>>> Is this not what you see?
>>>
>>> So, beginning sometime in 1.x, the Controller Settings option in the General
>>> menu became useless, even at the top level except for "all ReportingTasks
>>> and services defined in the Controller Settings." But, when would any
>>> reporting task or other service defined be able to benefit? Never if I'm any
>>> judge.
>>>
>>> I think this is much more than a mere documentation issue. I wonder if
>>> removing the Controller Services... option from the General menu would not
>>> be the most important thing to do (even before documenting the gear icon in
>>> the Operate menu).
>>>
>>> Russ
>>>
>>>
>>> On 05/16/2017 10:05 AM, Andrew Lim wrote:
>>>
>>> Hi Russell,
>>>
>>> Thanks for your question.
>>>
>>> Yes, working with Controller Services has definitely changed in 1.x compared
>>> to 0.x NiFi. Matt Gilman wrote a nice article about how Controller Service
>>> scoping was updated in 1.x with the introduction of Multi-Tenant
>>> Authorization and also discusses the recent improvements made in NiFi 1.2 to
>>> alleviate some of the user confusion around scoping [1]. If you would like
>>> to see further details, the parent Jira for the improvements can be found
>>> here [2].
>>>
>>> I think there is opportunity to improve the Apache documentation we have
>>> around this functionality, so I just filed a new Jira [3].
>>>
>>> Let us know if you have any more questions.
>>>
>>> Thanks,
>>>
>>> Drew
>>>
>>> [1]
>>> https://community.hortonworks.com/articles/90259/understanding-controller-service-availability-in-a.html
>>> [2]https://issues.apache.org/jira/browse/NIFI-3128
>>> [3]https://issues.apache.org/jira/browse/NIFI-3911
>>>
>>>
>>>
>>> On May 16, 2017, at 11:28 AM, Russell Bateman<ru...@windofkeltia.com> wrote:
>>>
>>> It appears to me that that, unlike what happened in NiFi 0.x, in 1.x when I
>>> look at controller services via the General menu -> Controller Services,
>>> what I see is totally different from what I see when I configure controller
>>> services for a processor.
>>>
>>> If I use the General menu to set up my controller services, I do not see nor
>>> am I given the option of using them in particular for processors I'm
>>> configuring. Instead, I appear to get a "Process Group Configuration and a
>>> list of controller services which are not the ones I'm looking for (because
>>> when I set them up, I gave them "special" names or renamed names I could
>>> recognize apart from any other use).
>>>
>>> Note: I'm more of a processor and controller service author than an
>>> experienced user of NiFi, so I may just be hopelessly confused.
>>>
>>> My question is what's the point of being able to configure controller
>>> services "globally" or "generally" if you can't reach them when you need
>>> them?
>>>
>>> Please confirm that I'm not just smoking funny weed and that this is
>>> different, in fact, from how it worked in 0.7.1.
>>>
>>> Thanks.
>>>
>>>
>
Re: Service controller list NiFi 0.x to 1.x
Posted by Russell Bateman <ru...@windofkeltia.com>.
We're still testing our migration from 0.7.2 to 1.1.2 which is imminent.
We still don't know how much of this is going to have to be
reconfigured. Our down-streamers tend to be on the grouchy side, so
we're testing to see what's going to irritate them. Were it not for the
magic ids, I'd feel safe writing a Python script to run through their
flow tarballs fixing the controller-setting impedance mismatches. Who
knows; maybe there will be nothing to change although we know for
certain of a /DBConnectionPool/ property change.
I will come back here in a few hours or tomorrow with an update on how
it's going. I'll try to keep the noise down because what may alarm us
shouldn't be a cause for alarm for everyone.
Thanks for the replies,
Russ
On 05/16/2017 02:47 PM, Michael Moser wrote:
> Hi Russ,
>
> In NiFi 1.2.0, there are 3 Reporting Tasks
> (SiteToSiteBulletionReportingTask, SiteToSiteProvenanceReportingTask,
> and SiteToSiteStatusReportingTask) that have the capability of using a
> Controller Service (StandardSSLContextService). So yes, the Global ->
> Controller Settings has a limited set of use cases, but they are very
> important. This becomes even more true considering the extensibility
> of NiFi components. Who can predict what new reporting tasks might be
> created to meet end user's needs, and what controller services those
> tasks might need?
>
> This is definitely high on the list of things that catch NiFi users
> off guard, as they transition from the 0.x versions into the 1.x
> versions. I can say that once I got used to using the Operate Palette
> (especially for defining controller services!) then it didn't seem as
> bad.
>
> I'll see what I can add to the 0.x to 1.0.0 Migration Guide [1] to
> call this out specifically.
>
> -- Mike
>
> [1] - https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance
>
>
>
> On Tue, May 16, 2017 at 3:53 PM, Russell Bateman <ru...@windofkeltia.com> wrote:
>> Thanks, Andrew!
>>
>> (From [1]:) "This means that the service will be available to all Processors
>> and Controller Services defined in that Process Group and below."
>>
>> In my experience, this isn't true. If I create a controller via the General
>> menu in the very root of my NiFi canvas, configure its name in Settings,
>> calling it Jack, then I create a new process group, then configure a new
>> processor in that group, when I try to configure to use the controller, Jack
>> is not among the options.
>>
>> In order for the statement above to be true, I have to create it via the
>> gear icon in the Operate menu/palette.
>>
>> Is this not what you see?
>>
>> So, beginning sometime in 1.x, the Controller Settings option in the General
>> menu became useless, even at the top level except for "all ReportingTasks
>> and services defined in the Controller Settings." But, when would any
>> reporting task or other service defined be able to benefit? Never if I'm any
>> judge.
>>
>> I think this is much more than a mere documentation issue. I wonder if
>> removing the Controller Services... option from the General menu would not
>> be the most important thing to do (even before documenting the gear icon in
>> the Operate menu).
>>
>> Russ
>>
>>
>> On 05/16/2017 10:05 AM, Andrew Lim wrote:
>>
>> Hi Russell,
>>
>> Thanks for your question.
>>
>> Yes, working with Controller Services has definitely changed in 1.x compared
>> to 0.x NiFi. Matt Gilman wrote a nice article about how Controller Service
>> scoping was updated in 1.x with the introduction of Multi-Tenant
>> Authorization and also discusses the recent improvements made in NiFi 1.2 to
>> alleviate some of the user confusion around scoping [1]. If you would like
>> to see further details, the parent Jira for the improvements can be found
>> here [2].
>>
>> I think there is opportunity to improve the Apache documentation we have
>> around this functionality, so I just filed a new Jira [3].
>>
>> Let us know if you have any more questions.
>>
>> Thanks,
>>
>> Drew
>>
>> [1]
>> https://community.hortonworks.com/articles/90259/understanding-controller-service-availability-in-a.html
>> [2] https://issues.apache.org/jira/browse/NIFI-3128
>> [3] https://issues.apache.org/jira/browse/NIFI-3911
>>
>>
>>
>> On May 16, 2017, at 11:28 AM, Russell Bateman <ru...@windofkeltia.com> wrote:
>>
>> It appears to me that that, unlike what happened in NiFi 0.x, in 1.x when I
>> look at controller services via the General menu -> Controller Services,
>> what I see is totally different from what I see when I configure controller
>> services for a processor.
>>
>> If I use the General menu to set up my controller services, I do not see nor
>> am I given the option of using them in particular for processors I'm
>> configuring. Instead, I appear to get a "Process Group Configuration and a
>> list of controller services which are not the ones I'm looking for (because
>> when I set them up, I gave them "special" names or renamed names I could
>> recognize apart from any other use).
>>
>> Note: I'm more of a processor and controller service author than an
>> experienced user of NiFi, so I may just be hopelessly confused.
>>
>> My question is what's the point of being able to configure controller
>> services "globally" or "generally" if you can't reach them when you need
>> them?
>>
>> Please confirm that I'm not just smoking funny weed and that this is
>> different, in fact, from how it worked in 0.7.1.
>>
>> Thanks.
>>
>>
Re: Service controller list NiFi 0.x to 1.x
Posted by Michael Moser <mo...@gmail.com>.
Hi Russ,
In NiFi 1.2.0, there are 3 Reporting Tasks
(SiteToSiteBulletionReportingTask, SiteToSiteProvenanceReportingTask,
and SiteToSiteStatusReportingTask) that have the capability of using a
Controller Service (StandardSSLContextService). So yes, the Global ->
Controller Settings has a limited set of use cases, but they are very
important. This becomes even more true considering the extensibility
of NiFi components. Who can predict what new reporting tasks might be
created to meet end user's needs, and what controller services those
tasks might need?
This is definitely high on the list of things that catch NiFi users
off guard, as they transition from the 0.x versions into the 1.x
versions. I can say that once I got used to using the Operate Palette
(especially for defining controller services!) then it didn't seem as
bad.
I'll see what I can add to the 0.x to 1.0.0 Migration Guide [1] to
call this out specifically.
-- Mike
[1] - https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance
On Tue, May 16, 2017 at 3:53 PM, Russell Bateman <ru...@windofkeltia.com> wrote:
> Thanks, Andrew!
>
> (From [1]:) "This means that the service will be available to all Processors
> and Controller Services defined in that Process Group and below."
>
> In my experience, this isn't true. If I create a controller via the General
> menu in the very root of my NiFi canvas, configure its name in Settings,
> calling it Jack, then I create a new process group, then configure a new
> processor in that group, when I try to configure to use the controller, Jack
> is not among the options.
>
> In order for the statement above to be true, I have to create it via the
> gear icon in the Operate menu/palette.
>
> Is this not what you see?
>
> So, beginning sometime in 1.x, the Controller Settings option in the General
> menu became useless, even at the top level except for "all ReportingTasks
> and services defined in the Controller Settings." But, when would any
> reporting task or other service defined be able to benefit? Never if I'm any
> judge.
>
> I think this is much more than a mere documentation issue. I wonder if
> removing the Controller Services... option from the General menu would not
> be the most important thing to do (even before documenting the gear icon in
> the Operate menu).
>
> Russ
>
>
> On 05/16/2017 10:05 AM, Andrew Lim wrote:
>
> Hi Russell,
>
> Thanks for your question.
>
> Yes, working with Controller Services has definitely changed in 1.x compared
> to 0.x NiFi. Matt Gilman wrote a nice article about how Controller Service
> scoping was updated in 1.x with the introduction of Multi-Tenant
> Authorization and also discusses the recent improvements made in NiFi 1.2 to
> alleviate some of the user confusion around scoping [1]. If you would like
> to see further details, the parent Jira for the improvements can be found
> here [2].
>
> I think there is opportunity to improve the Apache documentation we have
> around this functionality, so I just filed a new Jira [3].
>
> Let us know if you have any more questions.
>
> Thanks,
>
> Drew
>
> [1]
> https://community.hortonworks.com/articles/90259/understanding-controller-service-availability-in-a.html
> [2] https://issues.apache.org/jira/browse/NIFI-3128
> [3] https://issues.apache.org/jira/browse/NIFI-3911
>
>
>
> On May 16, 2017, at 11:28 AM, Russell Bateman <ru...@windofkeltia.com> wrote:
>
> It appears to me that that, unlike what happened in NiFi 0.x, in 1.x when I
> look at controller services via the General menu -> Controller Services,
> what I see is totally different from what I see when I configure controller
> services for a processor.
>
> If I use the General menu to set up my controller services, I do not see nor
> am I given the option of using them in particular for processors I'm
> configuring. Instead, I appear to get a "Process Group Configuration and a
> list of controller services which are not the ones I'm looking for (because
> when I set them up, I gave them "special" names or renamed names I could
> recognize apart from any other use).
>
> Note: I'm more of a processor and controller service author than an
> experienced user of NiFi, so I may just be hopelessly confused.
>
> My question is what's the point of being able to configure controller
> services "globally" or "generally" if you can't reach them when you need
> them?
>
> Please confirm that I'm not just smoking funny weed and that this is
> different, in fact, from how it worked in 0.7.1.
>
> Thanks.
>
>
Re: Service controller list NiFi 0.x to 1.x
Posted by Russell Bateman <ru...@windofkeltia.com>.
Thanks, Andrew!
(From [1]:) "/This means that the service will be available to all
Processors and Controller Services defined in that Process Group _and
below_./"
In my experience, this isn't true. If I create a controller _via the
General menu_ in the very root of my NiFi canvas, configure its name in
Settings, calling it Jack, then I create a new process group, then
configure a new processor in that group, when I try to configure to use
the controller, Jack is not among the options.
In order for the statement above to be true, I have to create it _via
the gear icon_ in the Operate menu/palette.
Is this not what you see?
So, beginning sometime in 1.x, the Controller Settings option in the
General menu became useless, even at the top level except for "all
ReportingTasks and services defined in the Controller Settings." But,
when would any reporting task or other service defined be able to
benefit? Never if I'm any judge.
I think this is much more than a mere documentation issue. I wonder if
removing the Controller Services... option from the General menu would
not be the most important thing to do (even before documenting the gear
icon in the Operate menu).
Russ
On 05/16/2017 10:05 AM, Andrew Lim wrote:
> Hi Russell,
>
> Thanks for your question.
>
> Yes, working with Controller Services has definitely changed in 1.x compared to 0.x NiFi. Matt Gilman wrote a nice article about how Controller Service scoping was updated in 1.x with the introduction of Multi-Tenant Authorization and also discusses the recent improvements made in NiFi 1.2 to alleviate some of the user confusion around scoping [1]. If you would like to see further details, the parent Jira for the improvements can be found here [2].
>
> I think there is opportunity to improve the Apache documentation we have around this functionality, so I just filed a new Jira [3].
>
> Let us know if you have any more questions.
>
> Thanks,
>
> Drew
>
> [1] https://community.hortonworks.com/articles/90259/understanding-controller-service-availability-in-a.html
> [2] https://issues.apache.org/jira/browse/NIFI-3128
> [3] https://issues.apache.org/jira/browse/NIFI-3911
>
>
>
>> On May 16, 2017, at 11:28 AM, Russell Bateman <ru...@windofkeltia.com> wrote:
>>
>> It appears to me that that, unlike what happened in NiFi 0.x, in 1.x when I look at controller services via the General menu -> Controller Services, what I see is totally different from what I see when I configure controller services for a processor.
>>
>> If I use the General menu to set up my controller services, I do not see nor am I given the option of using them in particular for processors I'm configuring. Instead, I appear to get a "Process Group Configuration and a list of controller services which are not the ones I'm looking for (because when I set them up, I gave them "special" names or renamed names I could recognize apart from any other use).
>>
>> Note: I'm more of a processor and controller service author than an experienced user of NiFi, so I may just be hopelessly confused.
>>
>> My question is what's the point of being able to configure controller services "globally" or "generally" if you can't reach them when you need them?
>>
>> Please confirm that I'm not just smoking funny weed and that this is different, in fact, from how it worked in 0.7.1.
>>
>> Thanks.
Re: Service controller list NiFi 0.x to 1.x
Posted by Andrew Lim <an...@gmail.com>.
Hi Russell,
Thanks for your question.
Yes, working with Controller Services has definitely changed in 1.x compared to 0.x NiFi. Matt Gilman wrote a nice article about how Controller Service scoping was updated in 1.x with the introduction of Multi-Tenant Authorization and also discusses the recent improvements made in NiFi 1.2 to alleviate some of the user confusion around scoping [1]. If you would like to see further details, the parent Jira for the improvements can be found here [2].
I think there is opportunity to improve the Apache documentation we have around this functionality, so I just filed a new Jira [3].
Let us know if you have any more questions.
Thanks,
Drew
[1] https://community.hortonworks.com/articles/90259/understanding-controller-service-availability-in-a.html
[2] https://issues.apache.org/jira/browse/NIFI-3128
[3] https://issues.apache.org/jira/browse/NIFI-3911
> On May 16, 2017, at 11:28 AM, Russell Bateman <ru...@windofkeltia.com> wrote:
>
> It appears to me that that, unlike what happened in NiFi 0.x, in 1.x when I look at controller services via the General menu -> Controller Services, what I see is totally different from what I see when I configure controller services for a processor.
>
> If I use the General menu to set up my controller services, I do not see nor am I given the option of using them in particular for processors I'm configuring. Instead, I appear to get a "Process Group Configuration and a list of controller services which are not the ones I'm looking for (because when I set them up, I gave them "special" names or renamed names I could recognize apart from any other use).
>
> Note: I'm more of a processor and controller service author than an experienced user of NiFi, so I may just be hopelessly confused.
>
> My question is what's the point of being able to configure controller services "globally" or "generally" if you can't reach them when you need them?
>
> Please confirm that I'm not just smoking funny weed and that this is different, in fact, from how it worked in 0.7.1.
>
> Thanks.