You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Tim Armstrong (JIRA)" <ji...@apache.org> on 2017/06/16 16:25:00 UTC
[jira] [Resolved] (IMPALA-5389) Clarify lifetime of
DiskIoMgr::BufferDescriptor objects
[ https://issues.apache.org/jira/browse/IMPALA-5389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong resolved IMPALA-5389.
-----------------------------------
Resolution: Fixed
Fix Version/s: Impala 2.10.0
IMPALA-5389: simplify BufferDescriptor lifetime
This is cleanup to make the code easier to understand in anticipation of
some trickier changes to I/O buffer management for IMPALA-4835. I do
not expect any changes in behaviour as a result of this patch.
* Use unique_ptr to make BufferDescriptor ownership transfer more
explicit and allow enforcing that the buffers are not leaked via
a DCHECK in ~BufferDescriptor.
* Remove 'free_buffer_descs_' cache. TCMalloc is good at caching small
objects and there will likely be a lot less lock contention compared
with a single global lock. The cache did not avoid calls to
malloc() anyway because appending to std::list<> requires allocating
a list node.
* Use std::deque instead of std::list in a couple of places - it offers
O(1) push_*()/pop_*() at both ends and requires fewer calls into
malloc()/free().
Testing:
Ran ASAN and exhaustive builds.
Change-Id: I007d098e9a1abb1f684be37b7f1ee6c03d3879b2
Reviewed-on: http://gerrit.cloudera.org:8080/7182
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Tested-by: Impala Public Jenkins
---
> Clarify lifetime of DiskIoMgr::BufferDescriptor objects
> -------------------------------------------------------
>
> Key: IMPALA-5389
> URL: https://issues.apache.org/jira/browse/IMPALA-5389
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.9.0
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Minor
> Labels: resource-management
> Fix For: Impala 2.10.0
>
>
> The transfer of DiskIoMgr::BufferDescriptor objects between different parts of the code is somewhat subtle. It would be good to enhance the code so that the ownership transfer was more explicit and it was easier to spot errors via inspection or assertions.
> Some ideas:
> * We could use unique_ptr instead of raw pointers - there is always a single owner of a BufferDescriptor
> * We could add a DCHECK to the destructor to ensure it is returned.
> * We don't really need to cache and recycle the objects (it's probably a pessimisation compared to TCMalloc's small object caching).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)