[jira] [Created] (ARROW-11569) Out-of-heap memory leaks in FlightClient.getStream

["lulijun (Jira)" <ji...@apache.org>]

lulijun created ARROW-11569:
-------------------------------

             Summary: Out-of-heap memory leaks in FlightClient.getStream
                 Key: ARROW-11569
                 URL: https://issues.apache.org/jira/browse/ARROW-11569
             Project: Apache Arrow
          Issue Type: Bug
          Components: FlightRPC
    Affects Versions: 3.0.0
         Environment: Java version: 8
Flight&Arrow version: 3.0.0
            Reporter: lulijun


I'm trying to use Arrow and Flight in my Java project, I used FlightClient.getStream to request data from a remote node, but I found the allocator's allocated memory size grows with the request.

Java version: 8

Flight&Arrow version: 3.0.0

 

Here's a snippet of my code, I found the allocated memory of flithStream still exist after root.close:

try (FlightStream flightStream = client.getStream(ticket, CallOptions.timeout(1, TimeUnit.MILLISECONDS))) {
    VectorSchemaRoot root = flightStream.getRoot();
    root.clear();
    while (flightStream.next()) {

        // transfer root to ArrowRecordBatch and execute
        dispatchNext(shuffleChunk, root);
        root.clear();
    }
    root.close();
} catch (FlightRuntimeException fe) {
    String errorMsg = "IndexReadOperator read index fail ={}  "+indexChunkDesc.toString();
    LOGGER.error(errorMsg, fe);
    throw new NetworkException(errorMsg, fe);
} catch (Exception e) {
    LOGGER.error("IndexReadOperator read index fail!", e);
 throw new NetworkException("IndexReadOperator getStream fail! ", e);
}

 

Is this a bug or am I using it incorrectly?

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)