You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/07/26 18:56:11 UTC
[GitHub] [airflow] josh-fell opened a new issue, #25319: Update code examples from "classic" operators to TaskFlow decorators
josh-fell opened a new issue, #25319:
URL: https://github.com/apache/airflow/issues/25319
### Body
Related to #9415, which updates the code example DAGs to use the TaskFlow API, the documentation throughout should also reflect the same functional style for overall documentation alignment as well as encourage users and bring more visibility to these TaskFlow decorators.
Currently there are 3 core TaskFlow decorators can be used instead of their "classic" operator counterparts.
| Classic Operator | TaskFlow Decorator |
|------------------ | --------------------|
| `PythonOperator`| `@task`|
| `PythonVirtualenvOperator`| `@task.virtualenv`|
| `BranchPythonOperator`| `@task.branch`|
There are a number of doc files that could be updated to the TaskFlow equivalents of the "classic" operators:
- [ ] _docs/apache-airflow/best-practices.rst_
- [ ] _docs/apache-airflow/faq.rst_
- [ ] _docs/apache-airflow/index.rst_
- [ ] _docs/apache-airflow/tutorial.rst_
- I'll be bold and say _maybe_ there shouldn't be a "classic" tutorial and a TaskFlow API tutorial especially since the desire is to encourage TaskFlow and it's been 1.5 years since its release. Open to discuss this of course and what the community thinks here.
- [ ] _docs/apache-airflow/concepts/dags.rst_
- [ ] _docs/apache-airflow/concepts/operators.rst_
- [ ] _docs/apache-airflow/concepts/params.rst_
- [ ] _docs/apache-airflow/howto/operator/python.rst_
- Seems worthy to update these how-to guide as well since the underlying example DAG snippets have been changed to use TaskFlow API. Any other thoughts here?
- [ ] _docs/apache-airflow/upgrading-from-1-10/index.rst_
### Committer
- [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on issue #25319: Update code examples from "classic" operators to TaskFlow decorators in docs
Posted by GitBox <gi...@apache.org>.
josh-fell commented on issue #25319:
URL: https://github.com/apache/airflow/issues/25319#issuecomment-1238550468
> > I'll be bold and say maybe there shouldn't be a "classic" tutorial and a TaskFlow API tutorial especially since the desire is to encourage TaskFlow and it's been 1.5 years since its release. Open to discuss this of course and what the community thinks here.
>
> I agree with this. How about combining the two? Right now the TaskFlow API tutorial "builds on" the existing tutorial. I say we..
>
> 1. combine "Tutorial" and "TaskFlow API tutorial" into one page (named "Tutorial")
> 2. add a code example of `PythonOperator` etc.. to (1) but describe it as "discouraged"
>
> If others are ok with this I'll go ahead and create a draft.
Yeah go ahead and create a draft. That's going to be the best place for discussion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on issue #25319: Update code examples from "classic" operators to TaskFlow decorators in docs
Posted by GitBox <gi...@apache.org>.
hankehly commented on issue #25319:
URL: https://github.com/apache/airflow/issues/25319#issuecomment-1211561527
#25657 updates the following
* docs/apache-airflow/best-practices.rst
* docs/apache-airflow/faq.rst
* docs/apache-airflow/index.rst
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on issue #25319: Update code examples from "classic" operators to TaskFlow decorators in docs
Posted by GitBox <gi...@apache.org>.
hankehly commented on issue #25319:
URL: https://github.com/apache/airflow/issues/25319#issuecomment-1221203289
I will finish up the remaining files
* docs/apache-airflow/tutorial.rst
* docs/apache-airflow/howto/operator/python.rst
* docs/apache-airflow/upgrading-from-1-10/index.rst
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on issue #25319: Update code examples from "classic" operators to TaskFlow decorators in docs
Posted by GitBox <gi...@apache.org>.
hankehly commented on issue #25319:
URL: https://github.com/apache/airflow/issues/25319#issuecomment-1196640744
Please assign this to me!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Bowrna commented on issue #25319: Update code examples from "classic" operators to TaskFlow decorators in docs
Posted by GitBox <gi...@apache.org>.
Bowrna commented on issue #25319:
URL: https://github.com/apache/airflow/issues/25319#issuecomment-1200358520
yeah sure @hankehly Please choose the files that you are taking and share it here. I will choose the ones that i will pick and share it too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ferruzzi commented on issue #25319: Update code examples from "classic" operators to TaskFlow decorators in docs
Posted by "ferruzzi (via GitHub)" <gi...@apache.org>.
ferruzzi commented on issue #25319:
URL: https://github.com/apache/airflow/issues/25319#issuecomment-1609013686
If this project is still of interest, it looks like https://github.com/apache/airflow/pull/25711 got `docs/apache-airflow/concepts/dags.rst` and `docs/apache-airflow/concepts/operators.rst`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Bowrna commented on issue #25319: Update code examples from "classic" operators to TaskFlow decorators in docs
Posted by GitBox <gi...@apache.org>.
Bowrna commented on issue #25319:
URL: https://github.com/apache/airflow/issues/25319#issuecomment-1211714441
@hankehly I am picking up all the .rst files under folder `docs/apache-airflow/concepts/` and making changes in the classic operators
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #25319: Update code examples from "classic" operators to TaskFlow decorators in docs
Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #25319:
URL: https://github.com/apache/airflow/issues/25319#issuecomment-1211746451
Copying my comment from the PR for discussion:
Hmm, the static check failures raise a point for discussion. XComArg has an (unfortunate) API design `xcom_arg[key]` that can be used to pull an XCom under a different key, but I’m not sure I like it being used with `output`. `task.output["foo"]` feel to me more like accessing a dict, not pulling another XCom. The API design can be discussed/changed later, but here I think we should decide whether that is a usage we want to encourage or not (personally I’m against).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #25319: Update code examples from "classic" operators to TaskFlow decorators in docs
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #25319:
URL: https://github.com/apache/airflow/issues/25319#issuecomment-1269433512
We need to be carful with `upgrading-from-1-10/index.rst`
This doc initial goal was to upgrade to 2.0 (to assist with breaking changes) so we need to make sure not to introduce syntax that can't work on 2.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on issue #25319: Update code examples from "classic" operators to TaskFlow decorators in docs
Posted by GitBox <gi...@apache.org>.
hankehly commented on issue #25319:
URL: https://github.com/apache/airflow/issues/25319#issuecomment-1221206525
> I'll be bold and say maybe there shouldn't be a "classic" tutorial and a TaskFlow API tutorial especially since the desire is to encourage TaskFlow and it's been 1.5 years since its release. Open to discuss this of course and what the community thinks here.
I agree with this. How about combining the two? Right now the TaskFlow API tutorial "builds on" the existing tutorial. I say we..
1. combine "Tutorial" and "TaskFlow API tutorial" into one page (named "Tutorial")
2. add a code example of `PythonOperator` etc.. to (1) but describe it as "discouraged"
If others are ok with this I'll go ahead and create a draft.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on issue #25319: Update code examples from "classic" operators to TaskFlow decorators in docs
Posted by GitBox <gi...@apache.org>.
hankehly commented on issue #25319:
URL: https://github.com/apache/airflow/issues/25319#issuecomment-1200349880
Absolutely. Let's create a PR per file so we don't overlap
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Bowrna commented on issue #25319: Update code examples from "classic" operators to TaskFlow decorators in docs
Posted by GitBox <gi...@apache.org>.
Bowrna commented on issue #25319:
URL: https://github.com/apache/airflow/issues/25319#issuecomment-1200247263
@hankehly Could i take part of the issue if you don't mind? :) I am happy to help with this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on issue #25319: Update code examples from "classic" operators to TaskFlow decorators in docs
Posted by GitBox <gi...@apache.org>.
josh-fell commented on issue #25319:
URL: https://github.com/apache/airflow/issues/25319#issuecomment-1196675865
All yours! FYI, if you feel that these updates would be best done in multiple PRs that's totally fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on issue #25319: Update code examples from "classic" operators to TaskFlow decorators in docs
Posted by GitBox <gi...@apache.org>.
josh-fell commented on issue #25319:
URL: https://github.com/apache/airflow/issues/25319#issuecomment-1238541403
>docs/apache-airflow/howto/operator/python.rst
Seems worthy to update these how-to guide as well since the underlying example DAG snippets have been changed to use TaskFlow API. Any other thoughts here?
Thinking about this item again, the how-to should be refactored to `_docs/apache-airflow/howto/decorators/python.rst` now that based on #25752 being the last decorator to be added for Python operators. I updated the list in the description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] josh-fell commented on issue #25319: Update code examples from "classic" operators to TaskFlow decorators in docs
Posted by GitBox <gi...@apache.org>.
josh-fell commented on issue #25319:
URL: https://github.com/apache/airflow/issues/25319#issuecomment-1238547471
> changing to task decorator in docs from classic operator use #25711
Yeah go ahead and create a draft. That's going to be the best place for discussion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hankehly commented on issue #25319: Update code examples from "classic" operators to TaskFlow decorators in docs
Posted by GitBox <gi...@apache.org>.
hankehly commented on issue #25319:
URL: https://github.com/apache/airflow/issues/25319#issuecomment-1240424793
Great, I'll get started on that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal closed issue #25319: Update code examples from "classic" operators to TaskFlow decorators in docs
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal closed issue #25319: Update code examples from "classic" operators to TaskFlow decorators in docs
URL: https://github.com/apache/airflow/issues/25319
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org