[jira] [Resolved] (ARROW-9587) [FlightRPC][Java] Clean up DoPut/FlightStream memory handling

["David Li (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-9587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

David Li resolved ARROW-9587.
-----------------------------
    Fix Version/s: 2.0.0
       Resolution: Fixed

Issue resolved by pull request 8010
[https://github.com/apache/arrow/pull/8010]

> [FlightRPC][Java] Clean up DoPut/FlightStream memory handling
> -------------------------------------------------------------
>
>                 Key: ARROW-9587
>                 URL: https://issues.apache.org/jira/browse/ARROW-9587
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: FlightRPC, Java
>    Affects Versions: 1.0.0
>            Reporter: David Li
>            Assignee: David Li
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 2.0.0
>
>          Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> We've been running into issues with DoPut in Java. In particular:
>  * Closing a FlightStream without draining it should not send a cancellation to the other side (or should send the cancellation, but also drain the queue). A server will have sent an explicit error message, or will simply just not want to read the entire stream. A client should explicitly cancel/gRPC will cancel for you anyways when  you end the call. Also, the gRPC call may already have ended and cancelling the call may result in a runtime exception.
>  * Cancelling a FlightStream explicitly should not immediately mark the stream as completed - it should wait for gRPC to acknowledge the cancellation as there may be undelivered messages.
>  * Make sure there is no race between the gRPC observer in the FlightStream and the consumer. (Ideally the only way for a FlightStream to end is for the observer to end the stream; that does open us up to the possibility of a FlightStream being stuck forever for servers that do not respect cancellation.)
>  * The server should close/clean up things properly in DoPut (it should act like DoExchange and tie closing of the stream to the onCompleted/onError callbacks). Otherwise trying to use it with ARROW-9586 becomes impossible (you need to close the FlightStream before ending the call, or you'll close the per-call allocator before you close the FlightStream)
> I think this also ties into flakiness in unit tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)