[GitHub] [airflow] potiuk commented on issue #8933: pyhive is installed without Hive dependencies

[GitBox <gi...@apache.org>]

potiuk commented on issue #8933:
URL: https://github.com/apache/airflow/issues/8933#issuecomment-636322845


   > Wait, pyhive has an extra called hive....? 
   > 
   > That would be amazing, yes, but would make our test suite take days to run to completion.
   > 
   > Still, might be something to think about.
   
   This is already planned for AIP-4 - system tests -  combined with AIP-8  (likely) - split Airflow 2.0 into separate providers (follow up after AIP-21)  and with AIP-26 Production image
   
   That's one of the reasons why we refactored and moved everything to separate backport providers and have clear dependencies for all of them. This will allow to turn them into "airflow-providers" and once we setup automated system tests, we will be able to run those system tests separately for each provider - for hive for example by installing only hive provider (together with its dependencies) and running tests in a production image that will have only dependencies needed by Hive and run them in that image.
   
   This is a mid-term goal I want to achieve (Still this year if possible) 
   
   
   BTW. @snazzyfox @eladkal - > do you think we need to add pyhive to the current set of dependencies for hive ? should we add it?  


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org