[jira] [Commented] (NIFI-14712) Performance issue with flow-analysis for a process group

["Tamas Palfy (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-14712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18004467#comment-18004467 ] 

Tamas Palfy commented on NIFI-14712:
------------------------------------

There might be an option 4. (probable together with option 1.):

The algorithm is recursively collects the violations for process groups and it's children. We might collect the ids first and pass the list to the method instead.
That way the violation collection from the 2-layered map would no longer be recursive. The collection of the ids would sill be though, but I would guess that is much quicker.
  

> Performance issue with flow-analysis for a process group
> --------------------------------------------------------
>
>                 Key: NIFI-14712
>                 URL: https://issues.apache.org/jira/browse/NIFI-14712
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Core Framework
>    Affects Versions: 2.4.0
>            Reporter: Eric Secules
>            Priority: Major
>         Attachments: Screenshot 2025-07-03 at 7.04.37 PM.png, Screenshot 2025-07-03 at 7.05.42 PM-1.png
>
>
> Background: I have a very large flow with 50000 or so processors and the flow-analysis endpoint is taking 20 or more seconds to complete. This is likely because the endpoint is traversing all the process groups recursively each time it is called. While User A had the flow canvas open to the root node, User B was unable to make configuration changes to the flow and also had a very hard time making updates to the flow.
> This endpoint also holds a read lock on the flow for the entire time, so it will prevent any writes. 
> Suggestions:
>  # Short circuit this endpoint and return a default success value if there are no violations at all.
>  # Only show detailed validation errors for the current process group and not descendants in the UI. For process groups within the current process group, show a cumulative count of violations within each process group. Counts would be updated lazily.
>  # The UI only checks whether there exist violations at the current process group and descendants, only load the detailed validation errors if the user clicks on the first aid kit icon. A map of process groupID to boolean would be updated at the same time violations are updated and a violation on a process group would set the boolean to true for its ancestors up to the root.
>  ## This can be tricky because process groups don't know their parent already. Also I think changing the meaning of the "first aid kit" to only correspond to the root PG, while it would help this performance situation, it would be less intuitive for the user if they're on any other process group than the root.
> Screenshots from browser dev tools attached
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)