[jira] [Created] (IMPALA-9609) Minimize Frontend activity in executor only Impalas

["Csaba Ringhofer (Jira)" <ji...@apache.org>]

Csaba Ringhofer created IMPALA-9609:
---------------------------------------

             Summary: Minimize Frontend activity in executor only Impalas
                 Key: IMPALA-9609
                 URL: https://issues.apache.org/jira/browse/IMPALA-9609
             Project: IMPALA
          Issue Type: Improvement
          Components: Backend, Frontend
            Reporter: Csaba Ringhofer


Currently Impala both   starts a JVM and creates a Frontend in executor mode impalads:

JVM:
https://github.com/apache/impala/blob/04fd9ae268d89b07e2a692a916bf2ddcfb2e351b/be/src/service/impalad-main.cc#L61
The 3rd argument of is init_jvm. This is actually needed for some executor features, e.g. Hive UDFs and HBase.

Frontend:
https://github.com/apache/impala/blob/d877dbc572e95f086b37142e1d41231d5b6b7f9f/be/src/runtime/exec-env.cc#L250

The frontend code itself doesn't really seem to care about whether it serves as a coordinator.

It would be great to remove the Frontend object from executors completely, but this may be tricky as Java features by the executor can have widespread dependencies.

The critical part is to avoid starting services that are both not needed and are problematic, e.g. connecting to HMS and Auth services. These can consume resource and cause errors if not configured well.

The executor only setting is only tested by custom_cluster test, so probably we'll need to extend those to ensure that HBase/Hive UDFs are not broken.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org