[jira] [Commented] (NIFI-950) Perform component validation asynchronously

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467771#comment-16467771 ] 

ASF GitHub Bot commented on NIFI-950:
-------------------------------------

GitHub user markap14 opened a pull request:

    https://github.com/apache/nifi/pull/2689

    NIFI-950: Perform component validation asynchronously, other cluster-related performance enhancements

    This PR addresses NIFI-950 and a handful of other JIRA's. They were all done in a single PR because the modifications tend to build upon one another, and also because this PR really addresses the issue of making nifi clustering more responsive (in terms of UI) and more stable.
    
    
    Thank you for submitting a contribution to Apache NiFi.
    
    In order to streamline the review of the contribution we ask you
    to ensure the following steps have been taken:
    
    ### For all changes:
    - [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
         in the commit message?
    
    - [ ] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
    
    - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)?
    
    - [ ] Is your initial contribution a single, squashed commit?
    
    ### For code changes:
    - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
    - [ ] Have you written or updated unit tests to verify your changes?
    - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? 
    - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
    - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
    - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?
    
    ### For documentation related changes:
    - [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
    
    ### Note:
    Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/markap14/nifi NIFI-950

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/nifi/pull/2689.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2689
    
----
commit 05f77401395b4274f2b09bf43b812af325e81690
Author: Mark Payne <ma...@...>
Date:   2018-04-11T19:36:54Z

    NIFI-950: Make component validation asynchronous
    NIFI-950: Still seeing some slow response times when instantiating a large template in cluster mode so making some minor tweaks based on the results of CPU profiling
    NIFI-5112: Refactored FlowSerializer so that it creates the desired intermediate data model that can be serialized, separate from serializing. This allows us to hold the FlowController's Read Lock only while creating the data model, not while actually serializing the data. Configured Jersey Client in ThreadPoolRequestReplicator not to look for features using the Service Loader for every request. Updated Template object to hold a DOM Node that represents the template contents instead of having to serialize the DTO, then parse the serialized form as a DOM object each time that it needs to be serialized.
    NIFI-5112: Change ThreadPoolRequestReplicator to use OkHttp client instead of Jersey Client
    NIFI-5111: Ensure that if a node is no longer cluster coordinator, that it clears any stale heartbeats.
    NIFI-5110: Notify StandardProcessScheduler when a component is removed so that it will clean up any resource related to component lifecycle.

commit 3ef86dc0c4e2c4855972cd0c200b7b9982e0c040
Author: Mark Payne <ma...@...>
Date:   2018-04-24T19:52:36Z

    NIFI-950: Avoid gathering the Status objects for entire flow when we don't need them; removed unnecessary code

commit a7c7fcb7c3fc72f8fc0a32e6ebdc4795bc12a8b4
Author: Mark Payne <ma...@...>
Date:   2018-05-03T20:07:10Z

    NIFI-950: Bug fixes

commit 434aa1dd85c53a7cc58497aad42246692a19a739
Author: Mark Payne <ma...@...>
Date:   2018-05-08T17:35:44Z

    NIFI-950: Bug fix; added validation status to ProcessorDTO, ControllerServiceDTO, ReportingTaskDTO; updated DebugFlow to allow for pause time to be set in the customValidate method for testing functionality

----


> Perform component validation asynchronously
> -------------------------------------------
>
>                 Key: NIFI-950
>                 URL: https://issues.apache.org/jira/browse/NIFI-950
>             Project: Apache NiFi
>          Issue Type: Improvement
>            Reporter: Joseph Percivall
>            Priority: Major
>         Attachments: self_reference_flow_fix.xml
>
>
> I created a flow that is a self referencing http loop. The flow was working fine but I wanted to save the template for later testing. I downloaded the the flow as a template. Then I tried testing a thread.sleep in the beginning of onConfigured, createSSLContext, and validate methods of StandardSSLContextService. I did a mvn clean install in the nifi-nar-bundles/nifi-standard-services/nifi-ssl-context-bundle/nifi-ssl-context-service directory. Then a mvn clean install in the nifi-assembly directory. After I imported the template the UI became very slow when clicking to different windows of the UI such as configuring a processor and the controller services window.
> I then stashed my changes and rebuilt the files. Once again I imported my template, and attempting to configure a processor or accessing the controller services window became very slow.
> The flow xml is attached. 
> -------------------------------------------
> The description and attachment showed an issue where long running validation caused the UI to become unresponsive. This validation should be done asynchronously so that the UI always remains responsive. Initial thoughts...
> - new state to indicate that validation is in progress
> - a mechanism for refreshing validation results
> - time out for waiting for validation to complete? or need to always be validating all components in case their validity is based on something environmental (like a configuration file that is modified outside of the application)?
> - provide better support for components that are running and become invalid
> -- related to this we need to provide guidance regarding the difference between become invalid and when we should use features like bulletins and yielding to rely runtime issues



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)