[jira] [Assigned] (IMPALA-8005) Randomize partitioning exchanges destinations

["Michael Ho (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-8005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Ho reassigned IMPALA-8005:
----------------------------------

    Assignee: Anurag Mantripragada  (was: Michael Ho)

> Randomize partitioning exchanges destinations
> ---------------------------------------------
>
>                 Key: IMPALA-8005
>                 URL: https://issues.apache.org/jira/browse/IMPALA-8005
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Distributed Exec
>    Affects Versions: Impala 3.1.0
>            Reporter: Michael Ho
>            Assignee: Anurag Mantripragada
>            Priority: Major
>              Labels: ramp-up
>
> Currently, we use the same hash seed for partitioning exchanges at the sender. For a table with skew in distribution in the shuffling keys, multiple queries using the same shuffling keys for exchanges will end up hashing to the same destination fragments running on particular host and potentially overloading that host.
> We should consider using the query id or other query specific information to seed the hashing function to randomize the destinations for different queries. Thanks to [~tlipcon] for pointing this problem out.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org