You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues-all@impala.apache.org by "Michael Smith (Jira)" <ji...@apache.org> on 2022/06/15 23:27:00 UTC

[jira] [Commented] (IMPALA-10545) Tune data_cache_write_concurrency based on the type of IO device

    [ https://issues.apache.org/jira/browse/IMPALA-10545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17554819#comment-17554819 ] 

Michael Smith commented on IMPALA-10545:
----------------------------------------

Evaluate effectiveness by looking at improved cache hit rate of concurrent queries.

> Tune data_cache_write_concurrency based on the type of IO device
> ----------------------------------------------------------------
>
>                 Key: IMPALA-10545
>                 URL: https://issues.apache.org/jira/browse/IMPALA-10545
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Backend
>    Affects Versions: Impala 4.0.0
>            Reporter: Joe McDonnell
>            Assignee: Michael Smith
>            Priority: Major
>              Labels: ramp-up
>
> The data cache limits concurrency writes to the cache to avoid overwhelming the underlying IO device. This is controlled by the data_cache_write_concurrency flags and defaults to 1. For SSDs, we should be able to increase this to allow more concurrent writes to the data cache. This would allow the data cache to warm up faster and stay more up to date.
> One option is to detect the underlying IO device (similar to how we do this for other parts of Disk IO Mgr) and tune this parameter higher for SSDs.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org