You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Sahil Takiar (JIRA)" <ji...@apache.org> on 2019/08/01 20:22:00 UTC
[jira] [Resolved] (IMPALA-8780) Implementation of
BufferedPlanRootSink where FlushFinal blocks until all rows are fetched
[ https://issues.apache.org/jira/browse/IMPALA-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sahil Takiar resolved IMPALA-8780.
----------------------------------
Resolution: Fixed
Fix Version/s: Impala 3.3.0
Closing as fixed. We ended up referring the complete re-factoring of IMPALA-8779 to a later patch.
> Implementation of BufferedPlanRootSink where FlushFinal blocks until all rows are fetched
> -----------------------------------------------------------------------------------------
>
> Key: IMPALA-8780
> URL: https://issues.apache.org/jira/browse/IMPALA-8780
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Reporter: Sahil Takiar
> Assignee: Sahil Takiar
> Priority: Major
> Fix For: Impala 3.3.0
>
>
> Implement {{BufferedPlanRootSink}} so that {{FlushFinal}} blocks until all rows are fetched. The implementation should use the {{RowBatchQueue}} introduced by IMPALA-8779. By blocking in {{FlushFinal}} all non-coordinator fragments will be closed if all results fit in the {{RowBatchQueue}}. {{BufferedPlanRootSink::Send}} should enqueue each given {{RowBatch}} onto the queue and then return. If the queue is full, it should block until there is more space left in the queue. {{BufferedPlanRootSink::GetNext}} reads from the queue and then fills in the given {{QueryResultSet}} by using the {{DataSink}} {{ScalarExprEvaluator}}-s. Since the producer thread can call {{BufferedPlanRootSink::Close}} while the consumer is calling {{BufferedPlanRootSink::GetNext}} the two methods need to be synchronized so that the {{DataSink}} {{MemTracker}}-s are not closed while {{GetNext}} is running.
> The implementation of {{BufferedPlanRootSink}} should remain the same regardless of whether a {{std::queue}} backed {{RowBatchQueue}} or a {{BufferedTupleStream}} backed {{RowBatchQueue}} is used.
> {{BufferedPlanRootSink}} and {{BlockingPlanRootSink}} are similar in the sense that {{BlockingPlanRootSink}} buffers one {{RowBatch}}, so for queries that return under 1024 rows, all non-coordinator fragments are closed immediately as well. The advantage of {{BufferedPlanRootSink}} is that allows buffering for 1+ {{RowBatch}}-es.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org