You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by GitBox <gi...@apache.org> on 2021/12/03 22:28:07 UTC
[GitHub] [nifi] exceptionfactory opened a new pull request #5571: NIFI-9435 Add filtering parameters to Flow Metrics
exceptionfactory opened a new pull request #5571:
URL: https://github.com/apache/nifi/pull/5571
#### Description of PR
NIFI-9435 Adds `includedRegistries` and `includedNames` as optional query parameters to the Flow Metrics REST API method to support filtering returned metrics. Both query parameters can be provided multiple times to include multiple selected elements. Specifying the `includedRegistries` parameter performs initial filtering and `includedNames` filters the metric family names from the selected registries.
The `includedRegistries` parameter supports the following four values based on the current Prometheus Collector Registries:
- NIFI
- JVM
- BULLETIN
- CONNECTION
The `includedNames` parameter supports any of the metric family names returned from any of the collector registries, such as `nifi_jvm_heap_usage` for the `JVM` registry or `nifi_total_bytes_written` from the `NIFI` registry.
These optional parameters preserve the current default behavior while also supporting a reduced response for NiFi deployments with a large number of configured components.
Additional improvements include removing redundant `required = false` annotation property settings in `FlowResource` and streamlining several inline function definitions.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [X] Does your PR title start with **NIFI-XXXX** where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [X] Has your PR been rebased against the latest commit within the target branch (typically `main`)?
- [X] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._
### For code changes:
- [X] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder?
- [X] Have you written or updated unit tests to verify your changes?
- [X] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties?
### For documentation related changes:
- [X] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #5571: NIFI-9435 Add filtering parameters to Flow Metrics
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on pull request #5571:
URL: https://github.com/apache/nifi/pull/5571#issuecomment-1010309784
@timeabarna I rebased and add a new commit that refactors the sample filtering approach to use a logical OR when processing both the sampleName and sampleLabelValue parameters.
I noticed that PR 5582 required both of those parameters to be specified, whereas this PR allows either one or both to be provided.
With the most recent change, the Sample will only be removed when it matches neither the sampleName nor the sampleLabelValue pattern. Describing it in positive terms, the Sample will be included if it matches either the sampleName or the sampleLabelValue pattern. I added an additional unit test for this scenario.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] timeabarna commented on pull request #5571: NIFI-9435 Add filtering parameters to Flow Metrics
Posted by GitBox <gi...@apache.org>.
timeabarna commented on pull request #5571:
URL: https://github.com/apache/nifi/pull/5571#issuecomment-1008811658
@exceptionfactory Unfortunately it is not working as expected for 5582. All the metrics required where sampleLabelValue is RootProcessGroup and all the jvm metrics. I am able to filter to these metrics separately but not at the same time as if I provide a sampleName not matching metrics will be removed from the iterator so won't be checked for sampleLabelValue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #5571: NIFI-9435 Add filtering parameters to Flow Metrics
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on pull request #5571:
URL: https://github.com/apache/nifi/pull/5571#issuecomment-1006621448
@timeabarna Just following up on this thread, let me know what you think of the recent additional changes to filtering parameters, hopefully it covers the additional fields in #5582. The approach of using regular expressions is different, but it should provide similar functionality.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] timeabarna commented on pull request #5571: NIFI-9435 Add filtering parameters to Flow Metrics
Posted by GitBox <gi...@apache.org>.
timeabarna commented on pull request #5571:
URL: https://github.com/apache/nifi/pull/5571#issuecomment-1010713927
LGTM +1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] timeabarna edited a comment on pull request #5571: NIFI-9435 Add filtering parameters to Flow Metrics
Posted by GitBox <gi...@apache.org>.
timeabarna edited a comment on pull request #5571:
URL: https://github.com/apache/nifi/pull/5571#issuecomment-1008811658
@exceptionfactory Unfortunately it is not working as expected for 5582. All the metrics required where sampleLabelValue is RootProcessGroup and all the jvm metrics. I am able to filter to these metrics separately but not at the same time as if I provide a sampleName not matching metrics will be removed from the iterator so won't be checked for sampleLabelValue. JVM metrics are generic so they don't have component type LabelValues.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #5571: NIFI-9435 Add filtering parameters to Flow Metrics
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on pull request #5571:
URL: https://github.com/apache/nifi/pull/5571#issuecomment-1008945850
Thanks for the feedback @timeabarna! It sounds like matching against provided parameters should be a logical OR instead of a logical AND, which is how it is working at the moment. I will evaluate a solution to cover that approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] timeabarna commented on pull request #5571: NIFI-9435 Add filtering parameters to Flow Metrics
Posted by GitBox <gi...@apache.org>.
timeabarna commented on pull request #5571:
URL: https://github.com/apache/nifi/pull/5571#issuecomment-1010713799
Thanks @exceptionfactory for your help, looks good now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #5571: NIFI-9435 Add filtering parameters to Flow Metrics
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on pull request #5571:
URL: https://github.com/apache/nifi/pull/5571#issuecomment-996751849
Thanks for following up on this @timeabarna, I will look at making the updates to this PR soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] asfgit closed pull request #5571: NIFI-9435 Add filtering parameters to Flow Metrics
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #5571:
URL: https://github.com/apache/nifi/pull/5571
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] timeabarna commented on pull request #5571: NIFI-9435 Add filtering parameters to Flow Metrics
Posted by GitBox <gi...@apache.org>.
timeabarna commented on pull request #5571:
URL: https://github.com/apache/nifi/pull/5571#issuecomment-996473194
Thanks for the improvements @exceptionfactory! As discussed separately can you please incorporate the filtering with https://github.com/apache/nifi/pull/5582? Please let me know when you are done so I can have another look and build my PR on yours. Thanks in advance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #5571: NIFI-9435 Add filtering parameters to Flow Metrics
Posted by GitBox <gi...@apache.org>.
exceptionfactory commented on pull request #5571:
URL: https://github.com/apache/nifi/pull/5571#issuecomment-997225825
@timeabarna I pushed an update with a refactored approach that supports filtering against the Sample Name and Sample Label Value fields using regular expression patterns. This required implementing a custom `Enumeration` to maintain integration with the Prometheus `TextFormat.write004()` method. The pattern matching approach provides a great degree of flexibility when it comes to filtering based on those fields. I also updated the `TestFlowResource` unit tests to provide some additional filtering examples.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org