[jira] [Commented] (NIFI-12693) When Processor is removed, Python Process should be notified asynchronously

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-12693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17813248#comment-17813248 ] 

ASF subversion and git services commented on NIFI-12693:
--------------------------------------------------------

Commit 7edf0d4e2aee21949bc3ef755ca4b27fa324d20f in nifi's branch refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=7edf0d4e2a ]

NIFI-12693: Moved notification of python process that a Processor was removed to a background (virtual) thread. Also noted in testing that in one instance a Python Processor never became
valid because it had cached property descriptors before the processor was fully initialized, so updated code to ensure that we do not cache values before initialization is completed.

This closes #8315

Signed-off-by: David Handermann <ex...@apache.org>


> When Processor is removed, Python Process should be notified asynchronously
> ---------------------------------------------------------------------------
>
>                 Key: NIFI-12693
>                 URL: https://issues.apache.org/jira/browse/NIFI-12693
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Core Framework
>            Reporter: Mark Payne
>            Assignee: Mark Payne
>            Priority: Major
>             Fix For: 2.0.0
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> When a Processor is removed, the PythonBridge is notified of the removal, and it then notifies any relevant Python process. This is done synchronously during the removal. I encountered two occurrences in which notifying the Python process failed.
> While the failure itself is not a huge concern, the handling of those failures resulted in very bad outcomes. In the first instance, the communication with the Python process was blocked on a socket read or write. As a result, the Service Facade's lock was never released, and no web requests could be made; they all blocked on the read lock. This resulted in requiring a restart of NiFi.
> In the other scenario, the call did not block indefinitely but threw an Exception. In this case, the associated Connections were never removed. As a result, I could no longer navigate to that Process Group in the UI, or the UI would have errors because there were Connections whose source or destination didn't exist. This required manually removing those connections from the flow.json file to recover.
> Since the intention of this action is simply a notification so that the Python process can cleanup after itself, this notification should be moved to a background thread, so that any failures are simply logged without causing problematic side effects.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)