You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "AlefRP (via GitHub)" <gi...@apache.org> on 2024/11/21 23:23:40 UTC

[PR] Add EmptyOperator and decorator to decorators/empty.py [airflow]

AlefRP opened a new pull request, #44272:
URL: https://github.com/apache/airflow/pull/44272

   # Add EmptyOperator with @task.empty decorator
   
   ## Objective
   This PR introduces a new operator called `EmptyOperator` with support for the `@task.empty` decorator. This operator serves as a placeholder for tasks in DAGs that do not have any execution logic but are useful for defining the structure of the workflow.
   
   ## Key Changes
   - Created `airflow/decorators/empty.py` with the implementation of `EmptyOperator` and the `@task.empty` decorator.
   - Modified `airflow/decorators/__init__.pyi` to expose the new `empty` decorator.
   - Added functionality to capture `args/kwargs` and log custom messages when executing tasks decorated with `@task.empty`.
   
   ## Example Usage
   ```python
   from airflow.decorators import dag, task
   from airflow.utils.dates import days_ago
   
   @dag(schedule_interval=None, start_date=days_ago(1), catchup=False)
   def my_empty_dag():
       @task.empty
       def start_task():
           print("This is a placeholder start task.")
   
       @task.empty
       def end_task():
           print("This is a placeholder end task.")
   
       start_task() >> end_task()
   
   dag = my_empty_dag()
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add EmptyOperator and decorator to decorators/empty.py [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.

potiuk commented on PR #44272:
URL: https://github.com/apache/airflow/pull/44272#issuecomment-2494977035

   I think it's also not a good idea. The whole idea of an EmptyOperator is that it is empty. so decorating a method with it makes no sense - because it will never have a callable. The whole idea of an empty operator is that it is there, serviing as a placeholder but it is also skipped when scheduling. Enttirely skipped in scheduler without even attempting to run anything - without even considering running anything.
   
   As far as I understand it now - when it decorates a funtion, it's essentially the same as `@task` - the only difference is that it wraps around EmptyOperator, but - unless I understand it wrongly and please correct me if I am wrong - it will actually attempt to execute the "execute()" method (unlike when you use EmptyOperator in your DAG directly). This is also why you have logs at execution time.
   
   So basically it's no different to write:
   
   ```
   @task.empty()
   def empty_task_a():
        pass
   ```
   
   than
   
   ```
   @task
   def empty_task_a():
        pass
   ```
   
   IMHO - they will behave exactly the same.
   
   Or am I wrong @AlefRP ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add EmptyOperator and decorator to decorators/empty.py [airflow]

Posted by "AlefRP (via GitHub)" <gi...@apache.org>.

AlefRP closed pull request #44272: Add EmptyOperator and decorator to decorators/empty.py
URL: https://github.com/apache/airflow/pull/44272


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add EmptyOperator and decorator to decorators/empty.py [airflow]

Posted by "eladkal (via GitHub)" <gi...@apache.org>.

eladkal commented on PR #44272:
URL: https://github.com/apache/airflow/pull/44272#issuecomment-2492925682

   Given that we plan to move decorators to standard provider https://github.com/apache/airflow/pull/44027 it's best to add new decorators to the standard provider directly rather than to the old path


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add EmptyOperator and decorator to decorators/empty.py [airflow]

Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.

boring-cyborg[bot] commented on PR #44272:
URL: https://github.com/apache/airflow/pull/44272#issuecomment-2492554569

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:
- Pay attention to the quality of your code (ruff, mypy and type annotations). Our [pre-commits]( https://github.com/apache/airflow/blob/main/contributing-docs/08_static_code_checks.rst#prerequisites-for-pre-commit-hooks) will help you with that.
- In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/main/docs/apache-airflow/howto/custom-operator.rst) Consider adding an example DAG that shows how users should use it.
- Consider using [Breeze environment](https://github.com/apache/airflow/blob/main/dev/breeze/doc/README.rst) for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
- Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
- Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
- Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#coding-style-and-best-practices).
- Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org