You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/01/22 21:37:44 UTC
[GitHub] [airflow] mik-laj opened a new issue #21037: Supports SQL in DatabricksHook
mik-laj opened a new issue #21037:
URL: https://github.com/apache/airflow/issues/21037
### Description
Currently, we support submitting spark jobs for the Databricks service. On December 15, 2021, they announced [General Availability of Databricks SQL](https://databricks.com/blog/2021/12/15/announcing-general-availability-of-databricks-sql.html) and we should provide support for it also.
To do this, we can use one of the existing libraries that provide integration with SQLAlchemy/DBApi:
- [https://pypi.org/project/databricks-sql-connector/](`databricks-sql-connector`) - Apache 2 Licensed (?), officially maintained by Databricks, code source is not available publicly (yet?)
- [https://github.com/crflynn/databricks-dbapi](`databricks-dbapi`) - MIT Licensed, unmaintained, open-source public available
CC: @eskarimov @alexott @christophergrant @Mokubyow
### Use case/motivation
Support a new kind of task.
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #21037: Supports SQL in DatabricksHook
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #21037:
URL: https://github.com/apache/airflow/issues/21037#issuecomment-1019363012
Ahh yes. So I close this PR. I opened this PR because I saw that Apache Superset had some changes related to Databricks.
https://github.com/apache/superset/issues/18139
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj edited a comment on issue #21037: Supports SQL in DatabricksHook
Posted by GitBox <gi...@apache.org>.
mik-laj edited a comment on issue #21037:
URL: https://github.com/apache/airflow/issues/21037#issuecomment-1019363012
Ahh yes. So I close this PR. I opened this PR because I saw that Apache Superset has a new issue related to Databricks.
https://github.com/apache/superset/issues/18139
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj closed issue #21037: Supports SQL in DatabricksHook
Posted by GitBox <gi...@apache.org>.
mik-laj closed issue #21037:
URL: https://github.com/apache/airflow/issues/21037
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] alexott commented on issue #21037: Supports SQL in DatabricksHook
Posted by GitBox <gi...@apache.org>.
alexott commented on issue #21037:
URL: https://github.com/apache/airflow/issues/21037#issuecomment-1019362611
It’s basically #21030 that I’ve filed today
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org