You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Csaba Ringhofer (Jira)" <ji...@apache.org> on 2020/04/25 12:13:00 UTC
[jira] [Resolved] (IMPALA-9609) Minimize Frontend activity in
executor only Impalas
[ https://issues.apache.org/jira/browse/IMPALA-9609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Csaba Ringhofer resolved IMPALA-9609.
-------------------------------------
Fix Version/s: Impala 4.0
Resolution: Implemented
> Minimize Frontend activity in executor only Impalas
> ---------------------------------------------------
>
> Key: IMPALA-9609
> URL: https://issues.apache.org/jira/browse/IMPALA-9609
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend, Frontend
> Reporter: Csaba Ringhofer
> Assignee: Csaba Ringhofer
> Priority: Critical
> Fix For: Impala 4.0
>
>
> Currently Impala both starts a JVM and creates a Frontend in executor mode impalads:
> JVM:
> https://github.com/apache/impala/blob/04fd9ae268d89b07e2a692a916bf2ddcfb2e351b/be/src/service/impalad-main.cc#L61
> The 3rd argument of is init_jvm. This is actually needed for some executor features, e.g. Hive UDFs and HBase.
> Frontend:
> https://github.com/apache/impala/blob/d877dbc572e95f086b37142e1d41231d5b6b7f9f/be/src/runtime/exec-env.cc#L250
> The frontend code itself doesn't really seem to care about whether it serves as a coordinator.
> It would be great to remove the Frontend object from executors completely, but this may be tricky as Java features used by the executor can have widespread dependencies.
> The critical part is to avoid starting services that are both not needed and are problematic, e.g. connecting to HMS and Auth services. These can consume resource and cause errors if not configured well.
> The executor only setting is only tested by custom_cluster test, so probably we'll need to extend those to ensure that HBase/Hive UDFs are not broken.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org