You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Chad Woodhead <ch...@gmail.com> on 2019/02/13 14:32:49 UTC

AmbariReportingTask help

I was looking for some clarification on the AmbariReportingTask, and specifically the metrics FlowFilesReceivedLast5Minutes and BytesReceivedLast5Minutes. 

Looking at the configuration for AmbariReportingTask, it has the property ‘Process Group ID’ and explains that if left blank, the root process group is used and global metrics are sent. (I have ‘Process Group ID’ set to blank in my AmbariReportingTask)  So for global metrics on the root process group, what does it use as a metric for FlowFilesReceived? All processors or just the first processor in a flow considered for this metric? Is there a way to see these metrics on the root process group in NiFi similar to ‘Status History’ on a component?

Thanks,
Chad

Re: AmbariReportingTask help

Posted by Mark Payne <ma...@hotmail.com>.
Chad,

The custom Processor would just need to be emitting a Provenance RECEIVE event (if the processor receives data as a new FlowFile)
or a FETCH event (if the processor is changing the contents of an existing FlowFile by pulling data from an external source).

The framework will then handle the rest.

Thanks
-Mark


> On Feb 13, 2019, at 10:53 AM, Chad Woodhead <ch...@gmail.com> wrote:
> 
> Hey Mark,
> 
> I see. That makes sense why I didn’t see metrics for flowfiles coming from GenerateFlowFile processor.
> 
> When using custom NARs to consume from external sources, is there any code that needs to be added to the NAR so the processor’s metrics are reported in this category or does NiFi handle that automatically?
> 
> -Chad
> 
> 
>> On Feb 13, 2019, at 9:44 AM, Mark Payne <ma...@hotmail.com> wrote:
>> 
>> Chad,
>> 
>> It represents any FlowFile that was received from an external source. This could be via
>> Site-to-Site or could be from something like GetHTTP, FetchSFTP, etc. It correlates
>> to any RECEIVE or FETCH provenance events.
>> 
>> If you go to the Summary table (menu in the top right / Summary) and then go to the
>> Process Groups tab, you can see the Sent/Received metrics over the past 5 minutes there.
>> 
>> Thanks
>> -Mark
>> 
>> 
>>> On Feb 13, 2019, at 9:32 AM, Chad Woodhead <ch...@gmail.com> wrote:
>>> 
>>> I was looking for some clarification on the AmbariReportingTask, and specifically the metrics FlowFilesReceivedLast5Minutes and BytesReceivedLast5Minutes. 
>>> 
>>> Looking at the configuration for AmbariReportingTask, it has the property ‘Process Group ID’ and explains that if left blank, the root process group is used and global metrics are sent. (I have ‘Process Group ID’ set to blank in my AmbariReportingTask)  So for global metrics on the root process group, what does it use as a metric for FlowFilesReceived? All processors or just the first processor in a flow considered for this metric? Is there a way to see these metrics on the root process group in NiFi similar to ‘Status History’ on a component?
>>> 
>>> Thanks,
>>> Chad
>> 
>

Re: AmbariReportingTask help

Posted by Chad Woodhead <ch...@gmail.com>.
Hey Mark,

I see. That makes sense why I didn’t see metrics for flowfiles coming from GenerateFlowFile processor.

When using custom NARs to consume from external sources, is there any code that needs to be added to the NAR so the processor’s metrics are reported in this category or does NiFi handle that automatically?

-Chad


> On Feb 13, 2019, at 9:44 AM, Mark Payne <ma...@hotmail.com> wrote:
> 
> Chad,
> 
> It represents any FlowFile that was received from an external source. This could be via
> Site-to-Site or could be from something like GetHTTP, FetchSFTP, etc. It correlates
> to any RECEIVE or FETCH provenance events.
> 
> If you go to the Summary table (menu in the top right / Summary) and then go to the
> Process Groups tab, you can see the Sent/Received metrics over the past 5 minutes there.
> 
> Thanks
> -Mark
> 
> 
>> On Feb 13, 2019, at 9:32 AM, Chad Woodhead <ch...@gmail.com> wrote:
>> 
>> I was looking for some clarification on the AmbariReportingTask, and specifically the metrics FlowFilesReceivedLast5Minutes and BytesReceivedLast5Minutes. 
>> 
>> Looking at the configuration for AmbariReportingTask, it has the property ‘Process Group ID’ and explains that if left blank, the root process group is used and global metrics are sent. (I have ‘Process Group ID’ set to blank in my AmbariReportingTask)  So for global metrics on the root process group, what does it use as a metric for FlowFilesReceived? All processors or just the first processor in a flow considered for this metric? Is there a way to see these metrics on the root process group in NiFi similar to ‘Status History’ on a component?
>> 
>> Thanks,
>> Chad
>

Re: AmbariReportingTask help

Posted by Mark Payne <ma...@hotmail.com>.
Chad,

It represents any FlowFile that was received from an external source. This could be via
Site-to-Site or could be from something like GetHTTP, FetchSFTP, etc. It correlates
to any RECEIVE or FETCH provenance events.

If you go to the Summary table (menu in the top right / Summary) and then go to the
Process Groups tab, you can see the Sent/Received metrics over the past 5 minutes there.

Thanks
-Mark


> On Feb 13, 2019, at 9:32 AM, Chad Woodhead <ch...@gmail.com> wrote:
> 
> I was looking for some clarification on the AmbariReportingTask, and specifically the metrics FlowFilesReceivedLast5Minutes and BytesReceivedLast5Minutes. 
> 
> Looking at the configuration for AmbariReportingTask, it has the property ‘Process Group ID’ and explains that if left blank, the root process group is used and global metrics are sent. (I have ‘Process Group ID’ set to blank in my AmbariReportingTask)  So for global metrics on the root process group, what does it use as a metric for FlowFilesReceived? All processors or just the first processor in a flow considered for this metric? Is there a way to see these metrics on the root process group in NiFi similar to ‘Status History’ on a component?
> 
> Thanks,
> Chad