[jira] [Commented] (AIRFLOW-3246) Make hmsclient import optional

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16724841#comment-16724841 ] 

ASF GitHub Bot commented on AIRFLOW-3246:
-----------------------------------------

ashb closed pull request #4080: [AIRFLOW-3246] Make hmsclient import optional
URL: https://github.com/apache/incubator-airflow/pull/4080
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/airflow/hooks/hive_hooks.py b/airflow/hooks/hive_hooks.py
index 178606aa4e..42a42318c6 100644
--- a/airflow/hooks/hive_hooks.py
+++ b/airflow/hooks/hive_hooks.py
@@ -27,7 +27,6 @@
 from collections import OrderedDict
 from tempfile import NamedTemporaryFile
 
-import hmsclient
 import six
 import unicodecsv as csv
 from past.builtins import basestring
@@ -496,6 +495,7 @@ def get_metastore_client(self):
         """
         Returns a Hive thrift client.
         """
+        import hmsclient
         from thrift.transport import TSocket, TTransport
         from thrift.protocol import TBinaryProtocol
         ms = self.metastore_conn


 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Make hmsclient import optional
> ------------------------------
>
>                 Key: AIRFLOW-3246
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3246
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: hive_hooks
>    Affects Versions: 1.10.0
>            Reporter: Gavrilov Seva
>            Priority: Minor
>
> Currently hmsclient is imported globally in hive_hooks.py, which is inconsistent with the general style in this file: hive dependencies are imported during the runtime. For example thrift components are imported inside the {{get_metastore_client}} method, but hmsclient also imports thrift components, so it forces you to have them installed.
> I moved the import in this PR: https://github.com/apache/incubator-airflow/pull/4080
> To give you a bit more information on why i even bother to do such a change, we are having problems with the new hive dependencies of airflow 1.10, particularly new version of pyhive. I described the problem [here|https://github.com/dropbox/PyHive/issues/240], seems like a combination of docker environment with newest versions of these libraries. We opted to rollback HiveServer2 hook to use the old dependencies, among them {{thrift==0.9.3}}, and hmsclient requires newer version of thrift. If you by chance have any clue on how we can diagnose our problem, please let me know.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)