You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Tim Armstrong (JIRA)" <ji...@apache.org> on 2018/07/27 22:47:00 UTC

[jira] [Created] (IMPALA-7366) Remove MarkNeedsDeepCopy() from UnionNode

Tim Armstrong created IMPALA-7366:
-------------------------------------

             Summary: Remove MarkNeedsDeepCopy() from UnionNode
                 Key: IMPALA-7366
                 URL: https://issues.apache.org/jira/browse/IMPALA-7366
             Project: IMPALA
          Issue Type: Sub-task
          Components: Backed
            Reporter: Tim Armstrong


This case is a bit trickier than the others. The issue is that some plan nodes, like HdfsScanNode, hold onto memory that is referenced by returned batches and don't necessarily attach the memory before returning *eos.

E.g. if you have a scan with a limit feeding into a UnionNode, the scan may terminate earlier before it attaches all memory to its output batches, then HdfsScanNode::Close() will free that memory. Thus it's not safe for the UnionNode to close its child until all referenced data has been copied out.

The fix is either to attach the memory to the final batch in Close() or to have a new method like FlushResources() that is called before Close().



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)