You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Tim Armstrong (JIRA)" <ji...@apache.org> on 2019/06/20 00:27:00 UTC
[jira] [Updated] (IMPALA-8687) --rpc_use_loopback may not work for
runtime filter RPCs
[ https://issues.apache.org/jira/browse/IMPALA-8687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong updated IMPALA-8687:
----------------------------------
Description:
Following on from IMPALA-8659, we may have some cases where impalads do self-RPCs via the thrift internal service IMPALA-7984. This JIRA is to investigate if this is a problem, and to fix it (either by intercepting self-RPCs in Thrift or by making code changes to avoid it).
Basic join where global runtime filters should apply:
{code}
select straight_join count(*)
from alltypes t1 join /*+ shuffle */ alltypes t2 on t1.id = t2.id
where t2.string_col = '1';
{code}
Interesting cases
* Dedicated coordinator with distributed plan ==> expect that all joins run on executors and all filter aggregation happens on coordinator
* Single node plan (num_nodes=1) ==> expect that all filters are local ==> no RPCs required
* Combined coordinator/executor with distributed plan ==> may do self-RPC
was:Following on from IMPALA-8659, we may have some cases where impalads do self-RPCs via the thrift internal service IMPALA-7984. This JIRA is to investigate if this is a problem, and to fix it (either by intercepting self-RPCs in Thrift or by making code changes to avoid it).
> --rpc_use_loopback may not work for runtime filter RPCs
> -------------------------------------------------------
>
> Key: IMPALA-8687
> URL: https://issues.apache.org/jira/browse/IMPALA-8687
> Project: IMPALA
> Issue Type: Sub-task
> Components: Distributed Exec
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
>
> Following on from IMPALA-8659, we may have some cases where impalads do self-RPCs via the thrift internal service IMPALA-7984. This JIRA is to investigate if this is a problem, and to fix it (either by intercepting self-RPCs in Thrift or by making code changes to avoid it).
> Basic join where global runtime filters should apply:
> {code}
> select straight_join count(*)
> from alltypes t1 join /*+ shuffle */ alltypes t2 on t1.id = t2.id
> where t2.string_col = '1';
> {code}
> Interesting cases
> * Dedicated coordinator with distributed plan ==> expect that all joins run on executors and all filter aggregation happens on coordinator
> * Single node plan (num_nodes=1) ==> expect that all filters are local ==> no RPCs required
> * Combined coordinator/executor with distributed plan ==> may do self-RPC
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org