You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/10/22 12:57:22 UTC
[GitHub] [airflow] robertsaxby opened a new pull request #19156: Updated Databricks operator
robertsaxby opened a new pull request #19156:
URL: https://github.com/apache/airflow/pull/19156
Added support for pipeline task, corrected documentation urls
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] lennartkats-db commented on pull request #19156: Updated Databricks operator, example dags and docs
Posted by GitBox <gi...@apache.org>.
lennartkats-db commented on pull request #19156:
URL: https://github.com/apache/airflow/pull/19156#issuecomment-951258633
@robertsaxby Static checks appear to be failing, please review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #19156: Updated Databricks operator, example dags and docs
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #19156:
URL: https://github.com/apache/airflow/pull/19156#discussion_r736235664
##########
File path: airflow/providers/databricks/provider.yaml
##########
@@ -22,6 +22,7 @@ description: |
`Databricks <https://databricks.com/>`__
versions:
+ - 2.0.3
Review comment:
We have separate release cycle for airflow and providers. Version bump will be done by release manager.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] closed pull request #19156: Databricks operators: added support for pipeline task, jar_params, example DAG for RunNowOperator, and updated docs
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #19156:
URL: https://github.com/apache/airflow/pull/19156
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] robertsaxby commented on pull request #19156: Updated Databricks operator
Posted by GitBox <gi...@apache.org>.
robertsaxby commented on pull request #19156:
URL: https://github.com/apache/airflow/pull/19156#issuecomment-950795955
> Thx, I was confused that it's **not** the latest version (2.1), but v 2.0 it's missing the `access_controll_list` property for job submission for example.
>
> This makes the docs inline with code.
yes this is still v2.0 (as before, even though some of the doc links incorrectly pointed to latest-which is v2.1) of the Databricks Jobs API. I will submit another update later today to include the jar_params in the RunNow operators plus some examples and doc changes.
The work to upgrade to v2.1 of the Databricks Jobs API is still oustanding.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] abij commented on pull request #19156: Updated Databricks operator
Posted by GitBox <gi...@apache.org>.
abij commented on pull request #19156:
URL: https://github.com/apache/airflow/pull/19156#issuecomment-950731513
Thx, I was confused that it's **not** the latest version (2.1), but v 2.0 it's missing the `access_controll_list` property for job submission for example.
This makes the docs inline with code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] lennartkats-db commented on pull request #19156: Updated Databricks operator, example dags and docs
Posted by GitBox <gi...@apache.org>.
lennartkats-db commented on pull request #19156:
URL: https://github.com/apache/airflow/pull/19156#issuecomment-950957358
Good point about the 2.0 API, @abij. The goal here is to just improve the current operator documentation and fill some minor gaps. The same feature set would be supported on 2.1 but that migration can be covered separately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #19156: Updated Databricks operator, example dags and docs
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #19156:
URL: https://github.com/apache/airflow/pull/19156#discussion_r736235924
##########
File path: docs/apache-airflow-providers-databricks/commits.rst
##########
@@ -27,6 +27,16 @@ This is detailed commit list of changes for versions provider package: ``databri
For high-level changelog, see :doc:`package information including changelog <index>`.
+2.0.3
+.....
+
Review comment:
Please remove this change. This file will be updated by release manager.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #19156: Databricks operators: added support for pipeline task, jar_params, example DAG for RunNowOperator, and updated docs
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #19156:
URL: https://github.com/apache/airflow/pull/19156#issuecomment-1016984487
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] lennartkats-db commented on a change in pull request #19156: Updated Databricks operator, example dags and docs
Posted by GitBox <gi...@apache.org>.
lennartkats-db commented on a change in pull request #19156:
URL: https://github.com/apache/airflow/pull/19156#discussion_r736214841
##########
File path: airflow/providers/databricks/provider.yaml
##########
@@ -22,6 +22,7 @@ description: |
`Databricks <https://databricks.com/>`__
versions:
+ - 2.0.3
Review comment:
Should this version number follow that from the core Airflow package, i.e. 2.1 or even that of the upcoming 2.2 series? I'm seeing e.g. https://github.com/apache/airflow/pull/17116/files bump another provider to 2.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #19156: Updated Databricks operator, example dags and docs
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #19156:
URL: https://github.com/apache/airflow/pull/19156#discussion_r736234790
##########
File path: airflow/providers/databricks/provider.yaml
##########
@@ -22,6 +22,7 @@ description: |
`Databricks <https://databricks.com/>`__
versions:
Review comment:
Please remove this change. This file will be updated by release manager.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #19156: Databricks operators: added support for pipeline task, jar_params, example DAG for RunNowOperator, and updated docs
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #19156:
URL: https://github.com/apache/airflow/pull/19156#issuecomment-986307003
Needs rebase!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] robertsaxby commented on a change in pull request #19156: Databricks operators: added support for pipeline task, jar_params, example DAG for RunNowOperator, and updated docs
Posted by GitBox <gi...@apache.org>.
robertsaxby commented on a change in pull request #19156:
URL: https://github.com/apache/airflow/pull/19156#discussion_r736321002
##########
File path: airflow/providers/databricks/provider.yaml
##########
@@ -22,6 +22,7 @@ description: |
`Databricks <https://databricks.com/>`__
versions:
+ - 2.0.3
Review comment:
do you have some details around the provider release cycle and frequency that I could look at? Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #19156: Updated Databricks operator, example dags and docs
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #19156:
URL: https://github.com/apache/airflow/pull/19156#discussion_r736234529
##########
File path: airflow/providers/databricks/CHANGELOG.rst
##########
@@ -19,6 +19,13 @@
Changelog
---------
+2.0.3
+.....
+
+Misc
+~~~~
+* ``Updated operator to add support for DLT pipeline, example dags and docs (#19156)``
Review comment:
Please remove this change. This file will be updated by release manager.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] alexott commented on pull request #19156: Databricks operators: added support for pipeline task, jar_params, example DAG for RunNowOperator, and updated docs
Posted by GitBox <gi...@apache.org>.
alexott commented on pull request #19156:
URL: https://github.com/apache/airflow/pull/19156#issuecomment-986200756
`pipeline_task` and `jar_params` where added in #19443.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org