[jira] [Commented] (PHOENIX-5802) Connection leaks in UPSERT SELECT/DELETE paths due to MutatingParallelIteratorFactory iterator not being closed

["Hadoop QA (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-5802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17068360#comment-17068360 ] 

Hadoop QA commented on PHOENIX-5802:
------------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12997962/PHOENIX-5802-4.x-v1.patch
  against 4.x branch at commit d04ba4eb945ba502652888220097ebca44f4a4dd.
  ATTACHMENT ID: 12997962

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 0 new or modified tests.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of javac compiler warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number of release audit warnings.

    {color:red}-1 lineLengths{color}.  The patch introduces the following lines longer than 100:
    +                    "SELECT current_date(), x_integer+2, entity_id FROM " + aTable + " WHERE a_integer >= ?";
+                    + "'澴粖蟤य褻酃岤豦팑薰鄩脼ժ끦碉碉碉碉碉碉' from " + t1 + " WHERE id = 1");
+            assertEquals("澴粖蟤य褻酃岤豦팑薰鄩脼ժ끦碉碉碉碉碉碉", rs.getString(2));
+                    + "'澴粖蟤य褻酃岤豦팑薰鄩脼ժ끦碉碉碉碉碉碉碉' from " + t1 + " WHERE id = 1");

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
                       org.apache.phoenix.mapreduce.index.IndexScrutinyTableOutputTest

Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3664//testReport/
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/3664//console

This message is automatically generated.

> Connection leaks in UPSERT SELECT/DELETE paths due to MutatingParallelIteratorFactory iterator not being closed
> ---------------------------------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-5802
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5802
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 5.0.0, 4.15.0
>            Reporter: Chinmay Kulkarni
>            Assignee: Chinmay Kulkarni
>            Priority: Major
>             Fix For: 5.1.0, 4.16.0
>
>         Attachments: PHOENIX-5802-4.x-v1.patch
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> During an UPSERT SELECT query or client-side DELETE, we clone the existing Phoenix connection and call [mutate|https://github.com/apache/phoenix/blob/73a86be9b588353457cd2c9de41239211330e7a7/phoenix-core/src/main/java/org/apache/phoenix/compile/MutatingParallelIteratorFactory.java#L59] which eventually calls either [upsertSelect|https://github.com/apache/phoenix/blob/73a86be9b588353457cd2c9de41239211330e7a7/phoenix-core/src/main/java/org/apache/phoenix/compile/UpsertCompiler.java#L178] or [deleteRows|https://github.com/apache/phoenix/blob/73a86be9b588353457cd2c9de41239211330e7a7/phoenix-core/src/main/java/org/apache/phoenix/compile/DeleteCompiler.java#L126].
> When getting iterators, if we encounter any exception, then on calling [close|https://github.com/apache/phoenix/blob/73a86be9b588353457cd2c9de41239211330e7a7/phoenix-core/src/main/java/org/apache/phoenix/iterate/BaseResultIterators.java#L1377], we attempt to cancel any queued work, while [accumulating already-started task futures|https://github.com/apache/phoenix/blob/73a86be9b588353457cd2c9de41239211330e7a7/phoenix-core/src/main/java/org/apache/phoenix/iterate/BaseResultIterators.java#L1453]. 
>  Later we attempt to block on getting the result (iterator) of already-started tasks and [close each iterator|https://github.com/apache/phoenix/blob/73a86be9b588353457cd2c9de41239211330e7a7/phoenix-core/src/main/java/org/apache/phoenix/iterate/BaseResultIterators.java#L1464-L1465].
> Here however, if any of the upsert-select or client-side delete tasks had thrown an exception _E_, we wouldn't be able to close the underlying iterator (which in-turn would've [closed the cloned connection|https://github.com/apache/phoenix/blob/73a86be9b588353457cd2c9de41239211330e7a7/phoenix-core/src/main/java/org/apache/phoenix/compile/MutatingParallelIteratorFactory.java#L101]). This manifests as [these logs|https://github.com/apache/phoenix/blob/73a86be9b588353457cd2c9de41239211330e7a7/phoenix-core/src/main/java/org/apache/phoenix/iterate/BaseResultIterators.java#L1470]:
> {code:java}
> "Failed to execute task during cancel", <exception E stack trace>
> {code}
> This is leading to a connection leak. Because we use CQSI to create the Phoenix Connection here, it also gets accounted for during open Phoenix connection throttling, thus leaving the client application with less connections that can be opened.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)