[jira] [Commented] (IMPALA-10522) Support external use of frontend libraries

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-10522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17300685#comment-17300685 ] 

ASF subversion and git services commented on IMPALA-10522:
----------------------------------------------------------

Commit b3d95d8f32c33c60657d6f518fe87a624509a618 in impala's branch refs/heads/master from Kurt Deschler
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b3d95d8 ]

IMPALA-10522: Support external use of frontend libraries

This patch enables the Impala frontend jar and dependent library
libfesupport.so to be used by an external Java frontend.

Calling FeSupport.setExternalFE() will cause external frontend
initialization mode to be used during FeSupport.loadLibrary(). This
mode builds upon logic that is used to initialize the frontend jar for
unit tests.

Initialization in external frontend mode differs as follows:

- Skip instantiating Frontend object and it's dependents
- Skip loading libhdfs
- Skip starting JVM Pause monitor
- Disable Minidumper
- Initialize TimezoneDatabase for external frontends
- Disable redirect of stderr/stdout to libfesupport.so glog
- Log messages from libfesupport.so to stderr
- Use libfesupport.so for JNI symbol look up

Null check were added in places where objects were assumed to be
instantiated but are now skipped during initialization.

Additional change:
1) Add libfesupport.lib path to JAVA_LIBRARY_PATH in test driver

Testing: - Initialized frontend jar from external frontend
 - Verified that frontend Java objects can be used externally without
   issues
 - Verified that exceptions thrown from Impala Java or libfesupport
   can be caught or propagated correctly by the external frontend
 - Manual verification of minicluster logs
 - Ran queries with external frontend

Co-authored-by: John Sherman <jf...@cloudera.com>
Co-authored-by: Aman Sinha <am...@cloudera.com>

Change-Id: I4e3a84721ba196ec00773ce2923b19610b90edd9
Reviewed-on: http://gerrit.cloudera.org:8080/17115
Reviewed-by: Thomas Tauber-Marshall <tm...@cloudera.com>
Tested-by: Thomas Tauber-Marshall <tm...@cloudera.com>


> Support external use of frontend libraries
> ------------------------------------------
>
>                 Key: IMPALA-10522
>                 URL: https://issues.apache.org/jira/browse/IMPALA-10522
>             Project: IMPALA
>          Issue Type: Sub-task
>          Components: Backend, Frontend
>            Reporter: Aman Sinha
>            Assignee: Aman Sinha
>            Priority: Major
>
> Existing impala frontend library initialization is largely driven by and complements backend initialization. There is a 2nd initialization path that is designed to support standalone test applications. These paths need to be reconciled so that the frontend libraries can also used by an external frontend that has different initialization requirements.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org