You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/07/24 13:08:17 UTC
[GitHub] [airflow] alexott opened a new pull request, #25260: More improvements in the Databricks operators
alexott opened a new pull request, #25260:
URL: https://github.com/apache/airflow/pull/25260
* make `databricks_conn_id` templated in all operators (follow up from #24945)
* rename `python_named_parameters` to `python_named_params` in `DatabricksRunNowOperator`
to match changes in the Databricks jobs API (it shouldn't break existing DAGs as noone
complained yet after the changes were made)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] alexott commented on a diff in pull request #25260: More improvements in the Databricks operators
Posted by GitBox <gi...@apache.org>.
alexott commented on code in PR #25260:
URL: https://github.com/apache/airflow/pull/25260#discussion_r928625968
##########
airflow/providers/databricks/operators/databricks.py:
##########
@@ -590,7 +590,7 @@ def __init__(
python_params: Optional[List[str]] = None,
jar_params: Optional[List[str]] = None,
spark_submit_params: Optional[List[str]] = None,
- python_named_parameters: Optional[Dict[str, str]] = None,
+ python_named_params: Optional[Dict[str, str]] = None,
Review Comment:
This is to match Databricks Jobs API changes (really was a problem with published spec). This parameter most probably wasn't used yet, otherwise we would get an error report because API. I can keep it compatible with previous versions by adding a new parameters, and marking old one as deprecated...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #25260: More improvements in the Databricks operators
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #25260:
URL: https://github.com/apache/airflow/pull/25260#discussion_r929802608
##########
airflow/providers/databricks/operators/databricks.py:
##########
@@ -590,7 +590,7 @@ def __init__(
python_params: Optional[List[str]] = None,
jar_params: Optional[List[str]] = None,
spark_submit_params: Optional[List[str]] = None,
- python_named_parameters: Optional[Dict[str, str]] = None,
+ python_named_params: Optional[Dict[str, str]] = None,
Review Comment:
If it was not matching the expected value, this is a bugfix, not a breaking change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #25260: More improvements in the Databricks operators
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #25260:
URL: https://github.com/apache/airflow/pull/25260#issuecomment-1195309582
Also rebased it to account for Werkzeug drama
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr merged pull request #25260: More improvements in the Databricks operators
Posted by GitBox <gi...@apache.org>.
uranusjr merged PR #25260:
URL: https://github.com/apache/airflow/pull/25260
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a diff in pull request #25260: More improvements in the Databricks operators
Posted by GitBox <gi...@apache.org>.
uranusjr commented on code in PR #25260:
URL: https://github.com/apache/airflow/pull/25260#discussion_r928616540
##########
airflow/providers/databricks/operators/databricks.py:
##########
@@ -590,7 +590,7 @@ def __init__(
python_params: Optional[List[str]] = None,
jar_params: Optional[List[str]] = None,
spark_submit_params: Optional[List[str]] = None,
- python_named_parameters: Optional[Dict[str, str]] = None,
+ python_named_params: Optional[Dict[str, str]] = None,
Review Comment:
Why change this? The old argument name sounds perfectly fine to me and it’s not worthwhile to break backward compatibility IMO.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org