You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by sbouchex <gi...@git.apache.org> on 2017/11/17 13:26:48 UTC
[GitHub] nifi pull request #2276: NIFI-4615 processor status is set to the ProcessorS...
GitHub user sbouchex opened a pull request:
https://github.com/apache/nifi/pull/2276
NIFI-4615 processor status is set to the ProcessorStatusDTO object
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [ x] Has your PR been rebased against the latest commit within the target branch (typically master)?
- [x ] Is your initial contribution a single, squashed commit?
### For code changes:
- [x ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
- [x ] Have you written or updated unit tests to verify your changes?
- [ x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ x] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
- [ x] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
- [x ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?
### For documentation related changes:
- [x ] Have you ensured that format looks appropriate for the output in which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/infovista/nifi NIFI-4615
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/2276.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2276
----
commit 329046a294118196c01c5d3c93d62f5479b23d55
Author: sbouchex <sb...@infovista.com>
Date: 2017-11-17T13:25:16Z
processor status is set to the ProcessorStatusDTO object
----
---
[GitHub] nifi pull request #2276: NIFI-4615 processor status is set to the ProcessorS...
Posted by mcgilman <gi...@git.apache.org>.
Github user mcgilman commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2276#discussion_r151693413
--- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/dto/DtoFactory.java ---
@@ -1083,13 +1083,15 @@ public ProcessorStatusDTO createProcessorStatusDto(final ProcessorStatus procSta
dto.setGroupId(procStatus.getGroupId());
dto.setName(procStatus.getName());
dto.setStatsLastRefreshed(new Date());
+ dto.setRunStatus(procStatus.getRunStatus().toString());
final ProcessorStatusSnapshotDTO snapshot = new ProcessorStatusSnapshotDTO();
dto.setAggregateSnapshot(snapshot);
snapshot.setId(procStatus.getId());
snapshot.setGroupId(procStatus.getGroupId());
snapshot.setName(procStatus.getName());
+ snapshot.setRunStatus(procStatus.getRunStatus().toString());
--- End diff --
This is already being set a few lines below.
---
[GitHub] nifi issue #2276: NIFI-4615 processor status is set to the ProcessorStatusDT...
Posted by mcgilman <gi...@git.apache.org>.
Github user mcgilman commented on the issue:
https://github.com/apache/nifi/pull/2276
We only use swagger for documentation generation. Changing the resulting value would break the current API that existing clients expect. I believe the best path forward would be to fix the documentation by adjusting the annotation value.
---
[GitHub] nifi pull request #2276: NIFI-4615 processor status is set to the ProcessorS...
Posted by mcgilman <gi...@git.apache.org>.
Github user mcgilman commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2276#discussion_r151711473
--- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/dto/DtoFactory.java ---
@@ -1083,13 +1083,15 @@ public ProcessorStatusDTO createProcessorStatusDto(final ProcessorStatus procSta
dto.setGroupId(procStatus.getGroupId());
dto.setName(procStatus.getName());
dto.setStatsLastRefreshed(new Date());
+ dto.setRunStatus(procStatus.getRunStatus().toString());
--- End diff --
I'm not sure I follow what "else" you're referring to. No change is needed when merging snapshots (line 384), that part is already correct. What we need to account for is the runStatus when merging status (line 354). We can account for the runStatus there using one of the two options I proposed above after merging the aggregate snapshot on line 363.
---
[GitHub] nifi issue #2276: NIFI-4615 processor status is set to the ProcessorStatusDT...
Posted by sbouchex <gi...@git.apache.org>.
Github user sbouchex commented on the issue:
https://github.com/apache/nifi/pull/2276
Well... Changing the API is not good, so I would change the RunStatus class instead by changing the constants to uppercase.
---
[GitHub] nifi issue #2276: NIFI-4615 processor status is set to the ProcessorStatusDT...
Posted by mcgilman <gi...@git.apache.org>.
Github user mcgilman commented on the issue:
https://github.com/apache/nifi/pull/2276
In some cases, human-readable labels are generated and other cases not. Often times we defer to the `toString` of the enum to drive the resulting value.
---
[GitHub] nifi issue #2276: NIFI-4615 processor status is set to the ProcessorStatusDT...
Posted by mcgilman <gi...@git.apache.org>.
Github user mcgilman commented on the issue:
https://github.com/apache/nifi/pull/2276
@sbouchex What version of NiFi are you running? When endpoint are you invoking that is not setting this value? Are you running standalone or clustered?
---
[GitHub] nifi pull request #2276: NIFI-4615 processor status is set to the ProcessorS...
Posted by sbouchex <gi...@git.apache.org>.
Github user sbouchex commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2276#discussion_r151703532
--- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/dto/DtoFactory.java ---
@@ -1083,13 +1083,15 @@ public ProcessorStatusDTO createProcessorStatusDto(final ProcessorStatus procSta
dto.setGroupId(procStatus.getGroupId());
dto.setName(procStatus.getName());
dto.setStatsLastRefreshed(new Date());
+ dto.setRunStatus(procStatus.getRunStatus().toString());
--- End diff --
So, something like target.setRunStatus(toMerge.getRunStatus()); ?
---
[GitHub] nifi issue #2276: NIFI-4615 processor status is set to the ProcessorStatusDT...
Posted by sbouchex <gi...@git.apache.org>.
Github user sbouchex commented on the issue:
https://github.com/apache/nifi/pull/2276
I open the issue as the issue is still there... the processor status is reported as null instead of RUNNING
---
[GitHub] nifi issue #2276: NIFI-4615 processor status is set to the ProcessorStatusDT...
Posted by sbouchex <gi...@git.apache.org>.
Github user sbouchex commented on the issue:
https://github.com/apache/nifi/pull/2276
Yes, but the problem is the toString() which is not good :
ProcessorStatusSnapshotDTO is defining RUNNING but the call in DtoFactory is setting "Running"
So the value is *not* consistent with the API definition (as it defines RUNNING)
Do you want a separate ticket or do you reopen this one ?
---
[GitHub] nifi issue #2276: NIFI-4615 processor status is set to the ProcessorStatusDT...
Posted by mcgilman <gi...@git.apache.org>.
Github user mcgilman commented on the issue:
https://github.com/apache/nifi/pull/2276
I see. So the issue is with this annotation?
@ApiModelProperty(
value = "The state of the processor.",
allowableValues = "RUNNING, STOPPED, DISABLED, INVALID"
)
Please open a new ticket. This ticket has already been released and cannot be re-opened now.
---
[GitHub] nifi issue #2276: NIFI-4615 processor status is set to the ProcessorStatusDT...
Posted by sbouchex <gi...@git.apache.org>.
Github user sbouchex commented on the issue:
https://github.com/apache/nifi/pull/2276
Ok fine for me. I'll do thé ticket + PR soon
---
[GitHub] nifi pull request #2276: NIFI-4615 processor status is set to the ProcessorS...
Posted by sbouchex <gi...@git.apache.org>.
Github user sbouchex commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2276#discussion_r151708941
--- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/dto/DtoFactory.java ---
@@ -1083,13 +1083,15 @@ public ProcessorStatusDTO createProcessorStatusDto(final ProcessorStatus procSta
dto.setGroupId(procStatus.getGroupId());
dto.setName(procStatus.getName());
dto.setStatsLastRefreshed(new Date());
+ dto.setRunStatus(procStatus.getRunStatus().toString());
--- End diff --
I will add a "else" to the already existing check and set the status in that case. I believe, it will be find.
---
[GitHub] nifi pull request #2276: NIFI-4615 processor status is set to the ProcessorS...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/2276
---
[GitHub] nifi pull request #2276: NIFI-4615 processor status is set to the ProcessorS...
Posted by mcgilman <gi...@git.apache.org>.
Github user mcgilman commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2276#discussion_r151706697
--- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/dto/DtoFactory.java ---
@@ -1083,13 +1083,15 @@ public ProcessorStatusDTO createProcessorStatusDto(final ProcessorStatus procSta
dto.setGroupId(procStatus.getGroupId());
dto.setName(procStatus.getName());
dto.setStatsLastRefreshed(new Date());
+ dto.setRunStatus(procStatus.getRunStatus().toString());
--- End diff --
You cannot just set target.setRunStatus(toMerge.getRunStatus) in case target is invalid. I believe you have two options...
1) Duplicate the isValid check
if (RunStatus.Invalid.name().equals(toMerge.getRunStatus())) {
target.setRunStatus(RunStatus.Invalid.name());
}
2) Copy the calculated runStatus from the aggregate snapshot
target.setRunStatus(target.getAggregateSnapshot().getRunStatus());
---
[GitHub] nifi issue #2276: NIFI-4615 processor status is set to the ProcessorStatusDT...
Posted by mcgilman <gi...@git.apache.org>.
Github user mcgilman commented on the issue:
https://github.com/apache/nifi/pull/2276
Thanks @sbouchex! This has been merged to master.
---
[GitHub] nifi issue #2276: NIFI-4615 processor status is set to the ProcessorStatusDT...
Posted by sbouchex <gi...@git.apache.org>.
Github user sbouchex commented on the issue:
https://github.com/apache/nifi/pull/2276
I'm running a build from the latest sources in standalone mode.
I have a process group with 4 processors all running (from the web ui pow) and when I do retrieve the ProcessGroupStatusDTO of my process group, iterate on each entry entry of the ProcessGroupStatusDTO.AggregateSnapshot, the RunStatus is always null.
FYI, I'm using a java API (https://github.com/hermannpencole/nifi-swagger-client)
I'm going to trace if I'm finding something.
---
[GitHub] nifi issue #2276: NIFI-4615 processor status is set to the ProcessorStatusDT...
Posted by mcgilman <gi...@git.apache.org>.
Github user mcgilman commented on the issue:
https://github.com/apache/nifi/pull/2276
@sbouchex This is a great catch and thanks for the contribution! Reviewing the code it appears that the runStatus was moved into the status snapshot likely during the 0.x -> 1.x transtion. This value is being set currently and is what the UI uses to render the status. However, with the runStatus still being available in the status it makes sense to populate it as well. I'll continue the review by commenting on the code...
---
[GitHub] nifi issue #2276: NIFI-4615 processor status is set to the ProcessorStatusDT...
Posted by sbouchex <gi...@git.apache.org>.
Github user sbouchex commented on the issue:
https://github.com/apache/nifi/pull/2276
org.apache.nifi.controller.status.RunStatus contains lowercase characters but org.apache.nifi.web.api.dto.status.ProcessorStatusSnapshotDTO (and probably other classes) are defining UPPERCASE statuses... Looks strange... no ?
---
[GitHub] nifi pull request #2276: NIFI-4615 processor status is set to the ProcessorS...
Posted by mcgilman <gi...@git.apache.org>.
Github user mcgilman commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2276#discussion_r151695374
--- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/dto/DtoFactory.java ---
@@ -1083,13 +1083,15 @@ public ProcessorStatusDTO createProcessorStatusDto(final ProcessorStatus procSta
dto.setGroupId(procStatus.getGroupId());
dto.setName(procStatus.getName());
dto.setStatsLastRefreshed(new Date());
+ dto.setRunStatus(procStatus.getRunStatus().toString());
--- End diff --
In addition to setting this here, we need to update it when merging responses in a clustered scenario. This merging happens to account for cases when a processor is invalid based on an environmental condition. A good example of this is a processor configured with a directory where that directory does not exist on one node of a cluster. On that node, the processor is invalid, but on the others that processor would be considered stopped. In this case, we need to ensure the invalid run status takes precedence. You can see an example of this here [1].
I think the easiest way to handle this would be to update the runStatus in the ProcessorStatusDTO when the method referenced in [1] returns when appropriate. Let me know if you have any questions about this.
[1] https://github.com/apache/nifi/blob/2da72f15243ef62d47369e8a87cc202a82dbe004/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/src/main/java/org/apache/nifi/cluster/manager/StatusMerger.java#L403
---
[GitHub] nifi issue #2276: NIFI-4615 processor status is set to the ProcessorStatusDT...
Posted by sbouchex <gi...@git.apache.org>.
Github user sbouchex commented on the issue:
https://github.com/apache/nifi/pull/2276
OK, the problem is not in the REST API which does contain the right value :
`"runStatus":"Running"`
However, the problem is in the generated code from swagger.io (or in Nifi) : In ProcessorStatusSnapshotDTO, the RunStatusEnum values are defined uppercase (RUNNING, STOPPED...) but the value sent returned by Nifi REST API are not : Running, Stopped...
---
[GitHub] nifi issue #2276: NIFI-4615 processor status is set to the ProcessorStatusDT...
Posted by sbouchex <gi...@git.apache.org>.
Github user sbouchex commented on the issue:
https://github.com/apache/nifi/pull/2276
`.toUpperCase()` is probably missing...
---
[GitHub] nifi issue #2276: NIFI-4615 processor status is set to the ProcessorStatusDT...
Posted by mcgilman <gi...@git.apache.org>.
Github user mcgilman commented on the issue:
https://github.com/apache/nifi/pull/2276
I'd suggest invoking the endpoints using curl or looking at Dev Tools in your browser to verify that the issue is with NiFi.
---