You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "adam133 (via GitHub)" <gi...@apache.org> on 2023/07/26 15:33:06 UTC
[GitHub] [airflow] adam133 opened a new issue, #32866: DatabricksSubmitRunOperator should accept a pipeline name for a pipeline_task
adam133 opened a new issue, #32866:
URL: https://github.com/apache/airflow/issues/32866
### Description
It would be nice if we could give the DatabricksSubmitRunOperator a pipeline name instead of a pipeline_id for cases when you do not already know the pipeline_id but do know the name. I'm not sure if there's an easy way to fetch a pipeline_id.
### Use case/motivation
Avoid hardcoding pipeline ID's, storing the ID's elsewhere, or fetching the pipeline list and filtering it manually if the pipeline name is known, but ID is not.
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] adam133 commented on issue #32866: DatabricksSubmitRunOperator should accept a pipeline name for a pipeline_task
Posted by "adam133 (via GitHub)" <gi...@apache.org>.
adam133 commented on issue #32866:
URL: https://github.com/apache/airflow/issues/32866#issuecomment-1690028489
PR should be ready for review now if anyone minds taking a look. If I get a chance I'll try adding a more specific test case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] adam133 commented on issue #32866: DatabricksSubmitRunOperator should accept a pipeline name for a pipeline_task
Posted by "adam133 (via GitHub)" <gi...@apache.org>.
adam133 commented on issue #32866:
URL: https://github.com/apache/airflow/issues/32866#issuecomment-1652080748
Should be pretty similar function and use to the find_job_id_by_name function [here](https://github.com/apache/airflow/blob/9e3104b5effc56f77833772d86e7201bcc50b8c9/airflow/providers/databricks/hooks/databricks.py#L193 ).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] adam133 commented on issue #32866: DatabricksSubmitRunOperator should accept a pipeline name for a pipeline_task
Posted by "adam133 (via GitHub)" <gi...@apache.org>.
adam133 commented on issue #32866:
URL: https://github.com/apache/airflow/issues/32866#issuecomment-1654073698
If I get time, yes - problem is lack of time 😄
For now I've just hardcoded pipeline IDs to keep moving on my project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #32866: DatabricksSubmitRunOperator should accept a pipeline name for a pipeline_task
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #32866:
URL: https://github.com/apache/airflow/issues/32866#issuecomment-1652057566
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] adam133 commented on issue #32866: DatabricksSubmitRunOperator should accept a pipeline name for a pipeline_task
Posted by "adam133 (via GitHub)" <gi...@apache.org>.
adam133 commented on issue #32866:
URL: https://github.com/apache/airflow/issues/32866#issuecomment-1654855146
Got a draft linked above, needs testing, tests, and documentation updates I think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hussein-awala commented on issue #32866: DatabricksSubmitRunOperator should accept a pipeline name for a pipeline_task
Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala commented on issue #32866:
URL: https://github.com/apache/airflow/issues/32866#issuecomment-1652365509
@adam133 Sounds like a good feature, would you like to implement it and become a contributor on Airflow?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #32866: DatabricksSubmitRunOperator should accept a pipeline name for a pipeline_task
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #32866: DatabricksSubmitRunOperator should accept a pipeline name for a pipeline_task
URL: https://github.com/apache/airflow/issues/32866
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org