[jira] [Commented] (IMPALA-4249) Clarify lifetime of DiskIoMgr::ScanRange objects

["Tim Armstrong (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-4249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16749057#comment-16749057 ] 

Tim Armstrong commented on IMPALA-4249:
---------------------------------------

We could consider using std::shared_ptr here since it does seem to be a genuine case of shared ownership.

> Clarify lifetime of DiskIoMgr::ScanRange objects
> ------------------------------------------------
>
>                 Key: IMPALA-4249
>                 URL: https://issues.apache.org/jira/browse/IMPALA-4249
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Backend
>    Affects Versions: Impala 2.8.0
>            Reporter: Tim Armstrong
>            Priority: Minor
>              Labels: resource-management
>
> When working on the DiskIoMgr code I noticed that the lifetime of the ScanRange objects isn't very clear. In particular, it's not clear when it is safe for a client to free the ScanRange object. We should at least document the expectation in the code.
> It seems like in principle that the client should be able to free the scan range after it hits eosr() or cancels the scan range. However, that's not true because a) BufferDescriptors hold internal pointers to the ScanRange and b) the DiskIoMgr touches the ScanRange in the below code after enqueueing the last buffer:
> {code}
>   bool queue_full = scan_range->EnqueueBuffer(buffer);
>   if (eosr) {
>     // For cached buffers, we can't close the range until the cached buffer is returned.
>     // Close() is called from DiskIoMgr::ReturnBuffer().
>     if (!is_cached) scan_range->Close();
>   } else {
> ...
> {code}
> It seems like we (mostly?) avoid issues by sticking the ScanRanges in object pools where they will outlive the reader contexts. This seems suboptimal - we shouldn't really leave lots of small unused objects lying around.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org