[jira] [Commented] (NIFI-6895) PutKudu Processor Warnings - Applying an operation in a closed session; this is unsafe

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-6895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16986561#comment-16986561 ] 

ASF subversion and git services commented on NIFI-6895:
-------------------------------------------------------

Commit 1898ad44be3186410e01baef60325a6fa5e81187 in nifi's branch refs/heads/master from Grant Henke
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=1898ad4 ]

NIFI-6895: Fix PutKudu processor concurrency issues

Calls to `trigger()` may be called concurrently from different threads,
however the PutKudu processor is storing the `kuduSession`
in a class level field. This can result in the logging issue reported in
NIFI-6895 and likely other unusual anomolies including performace
issues depending on the processor configuration.

Additionally the `operationType` was also stored in a class level field
and could be set concurrently resulting in the incorrect operation type
used.

This patch fixes the issue by moving both kuduSession and operationType
to be local. Additionaly some minor code cleanup was included.

An integration test, ITPutKudu, was added and used to manual verify the
logging issue existed and is fixed by this patch. I ran the test using
`mvn -Pintegration-tests verify -Dtest=ITPutKudu`

Signed-off-by: Pierre Villard <pi...@gmail.com>

This closes #3910.


> PutKudu Processor Warnings - Applying an operation in a closed session; this is unsafe
> --------------------------------------------------------------------------------------
>
>                 Key: NIFI-6895
>                 URL: https://issues.apache.org/jira/browse/NIFI-6895
>             Project: Apache NiFi
>          Issue Type: Bug
>    Affects Versions: 1.10.0
>         Environment: Kudu 1.10.0; NiFi 1.10.0, OpenJDK 8 (232); 8 Node Cluster
>            Reporter: Josef Zahner
>            Assignee: Grant Henke
>            Priority: Major
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> We have just upgraded from NiFi 1.9.2 to NiFi 1.10.0. We have seen that we got 1’000 times more logs ({{nifi-app.log}}) since the upgrade, caused mainly by the PutKudu processor.
> The logmessages we are always getting are: 
> {code:java}
> 2019-11-21 08:42:27,627 WARN [Timer-Driven Process Thread-2] org.apache.kudu.client.AsyncKuduSession Applying an operation in a closed session; this is unsafe{code}
> The PutKudu processor itself seems to work fine.
>  
> The line of code which from Apache Kudu Client which reflects the message is here: (line 547):
>  [https://github.com/apache/kudu/blob/master/java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduSession.java]
>  
> Since we are getting a huge amount of loglines we had to insert the following *workaround* in {{logback.xml}}:
> {code:java}
> <logger name="org.apache.kudu.client.AsyncKuduSession" level="ERROR"/>
> {code}
> Without suppressing the AsyncKuduSession messages we are getting multiple gigabytes of data per hour, but sadly with the workaround we don't see any PutKudu warnings anymore. 
>  
> Can the dev's please check why we are getting the warning and fix the root cause? Thanks in advance.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)