[jira] [Assigned] (IMPALA-5705) Parallelise read I/O by prefetching pages when iterating over unpinned BufferedTupleStream

["Yida Wu (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-5705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yida Wu reassigned IMPALA-5705:
-------------------------------

    Assignee: Yida Wu

> Parallelise read I/O by prefetching pages when iterating over unpinned BufferedTupleStream
> ------------------------------------------------------------------------------------------
>
>                 Key: IMPALA-5705
>                 URL: https://issues.apache.org/jira/browse/IMPALA-5705
>             Project: IMPALA
>          Issue Type: Sub-task
>          Components: Backend
>    Affects Versions: Impala 2.10.0
>            Reporter: Tim Armstrong
>            Assignee: Yida Wu
>            Priority: Major
>
> We could improve read I/O performance when iterating over unpinned streams in the hash join and hash aggregation by using additional memory to prefetch pages ahead of the current read position. Currently iterating over the unpinned stream only uses a single buffer, and only issues a read I/O when it has finished processing the previous page.
> This slows down processing of spilled probe rows in the hash join and spilled unaggregated rows in the hash aggregation.
> We'd need to figure out how to expose this in the BufferedTupleStream interface, but probably when preparing to read a stream, the client could specify a number of bytes to read ahead in the stream, which would require additional memory but increase performance.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org