You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Tim Armstrong (JIRA)" <ji...@apache.org> on 2018/09/17 21:49:00 UTC
[jira] [Assigned] (IMPALA-5833) Add automatic validation of memory
ownership and transfer
[ https://issues.apache.org/jira/browse/IMPALA-5833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong reassigned IMPALA-5833:
-------------------------------------
Assignee: (was: Tim Armstrong)
> Add automatic validation of memory ownership and transfer
> ---------------------------------------------------------
>
> Key: IMPALA-5833
> URL: https://issues.apache.org/jira/browse/IMPALA-5833
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Tim Armstrong
> Priority: Major
>
> Currently we have no reliable way to directly detect violations of our memory ownership or transfer, e.g. if something is attached too early or is freed or unpinned too early. We can catch many bugs indirectly because they cause crashes or incorrect results, and some others using tools like AddressSanitizer, but it would be much better to have a direct way of detecting the violations that can provide better diagnostics.
> The idea is to come up with a way to validate that a RowBatch is only referencing memory that is valid and owned by the right entity. Currently it's valid for a RowBatch to reference the following things:
> * Any memory directly attached to the batch
> * Any memory attached to a batch that is returned from a subsequent GetNext() call
> * Any memory owned by an ExecNode at or below the current node in the plan in one of the following forms:
> * * Buffers/pages
> * * MemPools
> When validating a RowBatch we could potentially create a map of all valid memory ranges that the RowBatch is allowed to reference, then traverse the whole RowBatch and check that all pointers are contained in one of those ranges. That would be expensive so we would probably have to treat it in the same way as a tool like AddressSanitizer and run a separate build with it enabled.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org