You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Kaxil Naik <ka...@apache.org> on 2020/09/18 12:44:03 UTC
[DISCUSS] Documentation Improvements for Airflow 2.0
Hi all,
One of the thing that we skipped in last Airflow 2.0 dev was around
"documentation". The main reason for skipping it in the meeting was to
gather good data-points and feedbacks from the community.
So my question to everyone reading this email is "What improvements do you
want in the docs on http://airflow.apache.org/docs/stable/ "?:
- What is it that we are missing there?
- What is that is "outdated"
- Better organization? Any ideas?
- Is there something that can be simplified?
- Where would you prefer docs for "Docker Image" and "Helm Chart" (yet
to be officially released)?
Feedback from someone who has just recently started looking at Airflow
would be very much appreciated since you would know what challenges you
faced while using Airflow for the first time.
Regards,
Kaxil
Re: [DISCUSS] Documentation Improvements for Airflow 2.0
Posted by Patrick Cando <p....@gmail.com>.
Hi all,
Airflow newbie here!
Here were some ideas I had:
I saw that the FAQs were at the bottom of the page and the project resources at the top. Perhaps this could be inverted with the FAQs being moved closer to the top and project resources move toward the bottom? My assumption here is the higher likelihood of users in the community reading the documentation for technical guidance.
With Airflow 2.0 being worked on, perhaps we could also think about refreshing the FAQs given the questions we see on Slack/other channels and perhaps anticipate some questions around 2.0 that might help the community?
There is also a high level link in Home/Executor which includes information on the Kubernetes Executor. We also then have a high level Home/Kubernetes link. Could we move this to Home/Executor?
We also have Home/DAG Runs and Home/DAG Serialization. Could this be moved to Home/DAGs > [DAG Runs, DAG Serialization,…] In Home/Best-Practises we also have a few resources for writing and testing DAGs. Could this be moved under DAGs too? Or are users searching for specific best practice resources?
Just some ideas! Thanks for reaching out @kaxil :)
Cheers,
Patrick
On 2020/09/18 12:44:03, Kaxil Naik <ka...@apache.org> wrote:
> Hi all,
>
> One of the thing that we skipped in last Airflow 2.0 dev was around
> "documentation". The main reason for skipping it in the meeting was to
> gather good data-points and feedbacks from the community.
>
> So my question to everyone reading this email is "What improvements do you
> want in the docs on http://airflow.apache.org/docs/stable/ "?:
>
> - What is it that we are missing there?
> - What is that is "outdated"
> - Better organization? Any ideas?
> - Is there something that can be simplified?
> - Where would you prefer docs for "Docker Image" and "Helm Chart" (yet
> to be officially released)?
>
> Feedback from someone who has just recently started looking at Airflow
> would be very much appreciated since you would know what challenges you
> faced while using Airflow for the first time.
>
> Regards,
> Kaxil
>
Re: [DISCUSS] Documentation Improvements for Airflow 2.0
Posted by Kaxil Naik <ka...@gmail.com>.
Thanks all, I am going to go through all the feedback and get something
actionable soon.
On Fri, Oct 23, 2020 at 11:54 PM Leah Cole <co...@google.com.invalid>
wrote:
> Hey! Sorry I'm so late to this party. My email management in 2020 is not
> quite what I want it to be. If you're still looking for opinions I know I
> have some about operator documentation, and if they're not relevant now,
> I'm sending them in case they're useful in the future! I'm looking at the
> documentation right now pretending I'm a user who doesn't know my way
> around the repo - I went to the Airflow website, then to Documentation, and
> Content. In the "Using Operators" section
> <https://airflow.apache.org/docs/stable/#content>, and saw that we do
> list many operators. Now, putting on my "experienced-Airflow-user hat"
> (primarily in GCP), I know that there are there are more operators
> <https://github.com/apache/airflow/tree/master/airflow/providers/google/cloud/operators>
> than just on this page, but can say with a lot of confidence that many GCP
> customers do have trouble finding out more info about the operators. I know
> there are also operators from other providers that aren't listed here at
> all. Even if we're just linking to source code and not a proper how-to or
> example (though having those is always ideal!), I think it would be
> beneficial to many users, not just GCP ones, to see a list of the many
> wonderful operators available. :)
>
> On Sun, Oct 4, 2020 at 4:27 AM Fabien BEURET <fb...@gmail.com> wrote:
>
>> Hi,
>>
>> Definitely some examples on how to use operators in context would be
>> helpful
>> It's sometimes kind of hard to figure out how to use them
>>
>> So, just examples... (not to copy/paste, but to get inspired)
>>
>> Thx,
>> Fabien
>>
>> On 2020/09/18 12:44:03, Kaxil Naik <ka...@apache.org> wrote:
>> > Hi all,
>> >
>> > One of the thing that we skipped in last Airflow 2.0 dev was around
>> > "documentation". The main reason for skipping it in the meeting was to
>> > gather good data-points and feedbacks from the community.
>> >
>> > So my question to everyone reading this email is "What improvements do
>> you
>> > want in the docs on http://airflow.apache.org/docs/stable/ "?:
>> >
>> > - What is it that we are missing there?
>> > - What is that is "outdated"
>> > - Better organization? Any ideas?
>> > - Is there something that can be simplified?
>> > - Where would you prefer docs for "Docker Image" and "Helm Chart"
>> (yet
>> > to be officially released)?
>> >
>> > Feedback from someone who has just recently started looking at Airflow
>> > would be very much appreciated since you would know what challenges you
>> > faced while using Airflow for the first time.
>> >
>> > Regards,
>> > Kaxil
>> >
>>
>
>
> --
>
> Leah Cole (she/her) | Developer Programs Engineer | coleleah@google.com | (925)
> 257-2112
> *I'm working weird hours during this pandemic and am sometimes a bit
> slower to respond to PRs/CLs than normal. Please feel free to send me a
> gentle ping for a status update if my slowness is blocking you and I'll do
> my best to give you an ETA. *
>
>
Re: [DISCUSS] Documentation Improvements for Airflow 2.0
Posted by Leah Cole <co...@google.com.INVALID>.
Hey! Sorry I'm so late to this party. My email management in 2020 is not
quite what I want it to be. If you're still looking for opinions I know I
have some about operator documentation, and if they're not relevant now,
I'm sending them in case they're useful in the future! I'm looking at the
documentation right now pretending I'm a user who doesn't know my way
around the repo - I went to the Airflow website, then to Documentation, and
Content. In the "Using Operators" section
<https://airflow.apache.org/docs/stable/#content>, and saw that we do list
many operators. Now, putting on my "experienced-Airflow-user hat"
(primarily in GCP), I know that there are there are more operators
<https://github.com/apache/airflow/tree/master/airflow/providers/google/cloud/operators>
than just on this page, but can say with a lot of confidence that many GCP
customers do have trouble finding out more info about the operators. I know
there are also operators from other providers that aren't listed here at
all. Even if we're just linking to source code and not a proper how-to or
example (though having those is always ideal!), I think it would be
beneficial to many users, not just GCP ones, to see a list of the many
wonderful operators available. :)
On Sun, Oct 4, 2020 at 4:27 AM Fabien BEURET <fb...@gmail.com> wrote:
> Hi,
>
> Definitely some examples on how to use operators in context would be
> helpful
> It's sometimes kind of hard to figure out how to use them
>
> So, just examples... (not to copy/paste, but to get inspired)
>
> Thx,
> Fabien
>
> On 2020/09/18 12:44:03, Kaxil Naik <ka...@apache.org> wrote:
> > Hi all,
> >
> > One of the thing that we skipped in last Airflow 2.0 dev was around
> > "documentation". The main reason for skipping it in the meeting was to
> > gather good data-points and feedbacks from the community.
> >
> > So my question to everyone reading this email is "What improvements do
> you
> > want in the docs on http://airflow.apache.org/docs/stable/ "?:
> >
> > - What is it that we are missing there?
> > - What is that is "outdated"
> > - Better organization? Any ideas?
> > - Is there something that can be simplified?
> > - Where would you prefer docs for "Docker Image" and "Helm Chart" (yet
> > to be officially released)?
> >
> > Feedback from someone who has just recently started looking at Airflow
> > would be very much appreciated since you would know what challenges you
> > faced while using Airflow for the first time.
> >
> > Regards,
> > Kaxil
> >
>
--
Leah Cole (she/her) | Developer Programs Engineer | coleleah@google.com | (925)
257-2112
*I'm working weird hours during this pandemic and am sometimes a bit slower
to respond to PRs/CLs than normal. Please feel free to send me a gentle
ping for a status update if my slowness is blocking you and I'll do my best
to give you an ETA. *
Re: [DISCUSS] Documentation Improvements for Airflow 2.0
Posted by Fabien BEURET <fb...@gmail.com>.
Hi,
Definitely some examples on how to use operators in context would be helpful
It's sometimes kind of hard to figure out how to use them
So, just examples... (not to copy/paste, but to get inspired)
Thx,
Fabien
On 2020/09/18 12:44:03, Kaxil Naik <ka...@apache.org> wrote:
> Hi all,
>
> One of the thing that we skipped in last Airflow 2.0 dev was around
> "documentation". The main reason for skipping it in the meeting was to
> gather good data-points and feedbacks from the community.
>
> So my question to everyone reading this email is "What improvements do you
> want in the docs on http://airflow.apache.org/docs/stable/ "?:
>
> - What is it that we are missing there?
> - What is that is "outdated"
> - Better organization? Any ideas?
> - Is there something that can be simplified?
> - Where would you prefer docs for "Docker Image" and "Helm Chart" (yet
> to be officially released)?
>
> Feedback from someone who has just recently started looking at Airflow
> would be very much appreciated since you would know what challenges you
> faced while using Airflow for the first time.
>
> Regards,
> Kaxil
>
Re: [DISCUSS] Documentation Improvements for Airflow 2.0
Posted by Kaxil Naik <ka...@gmail.com>.
Bumping this thread again to get more responses.
On Sat, Sep 19, 2020 at 5:13 PM Tomasz Urbaszek <tu...@apache.org>
wrote:
> Hi all,
>
> From my perspective the following sections could be better:
> - how to on cross DAG dependencies, best practises, watchouts etc (I
> should be able to contribute something over next weekend)
> - configuration settings - it would be lovely to have not only
> technical description of a setting but also suggestion on how to
> adjust it, how it will impact the deployment etc.
>
> And if I'm not mistaken we are missing a nice "best practices for your
> Airflow" that would help use and manage their Airflows in "best" way.
> For example section on monitoring Airflow deployments (not a list of
> metrics) would be appreciated by users.
>
> Cheers,
> Tomek
>
> On Fri, Sep 18, 2020 at 2:44 PM Kaxil Naik <ka...@apache.org> wrote:
> >
> > Hi all,
> >
> > One of the thing that we skipped in last Airflow 2.0 dev was around
> > "documentation". The main reason for skipping it in the meeting was to
> > gather good data-points and feedbacks from the community.
> >
> > So my question to everyone reading this email is "What improvements do
> you
> > want in the docs on http://airflow.apache.org/docs/stable/ "?:
> >
> > - What is it that we are missing there?
> > - What is that is "outdated"
> > - Better organization? Any ideas?
> > - Is there something that can be simplified?
> > - Where would you prefer docs for "Docker Image" and "Helm Chart" (yet
> > to be officially released)?
> >
> > Feedback from someone who has just recently started looking at Airflow
> > would be very much appreciated since you would know what challenges you
> > faced while using Airflow for the first time.
> >
> > Regards,
> > Kaxil
>
>
>
> --
>
> Tomasz Urbaszek
> Polidea | Software Engineer
>
> M: +48 505 628 493
> E: tomasz.urbaszek@polidea.com
>
> Unique Tech
> Check out our projects!
>
Re: [DISCUSS] Documentation Improvements for Airflow 2.0
Posted by Tomasz Urbaszek <tu...@apache.org>.
Hi all,
From my perspective the following sections could be better:
- how to on cross DAG dependencies, best practises, watchouts etc (I
should be able to contribute something over next weekend)
- configuration settings - it would be lovely to have not only
technical description of a setting but also suggestion on how to
adjust it, how it will impact the deployment etc.
And if I'm not mistaken we are missing a nice "best practices for your
Airflow" that would help use and manage their Airflows in "best" way.
For example section on monitoring Airflow deployments (not a list of
metrics) would be appreciated by users.
Cheers,
Tomek
On Fri, Sep 18, 2020 at 2:44 PM Kaxil Naik <ka...@apache.org> wrote:
>
> Hi all,
>
> One of the thing that we skipped in last Airflow 2.0 dev was around
> "documentation". The main reason for skipping it in the meeting was to
> gather good data-points and feedbacks from the community.
>
> So my question to everyone reading this email is "What improvements do you
> want in the docs on http://airflow.apache.org/docs/stable/ "?:
>
> - What is it that we are missing there?
> - What is that is "outdated"
> - Better organization? Any ideas?
> - Is there something that can be simplified?
> - Where would you prefer docs for "Docker Image" and "Helm Chart" (yet
> to be officially released)?
>
> Feedback from someone who has just recently started looking at Airflow
> would be very much appreciated since you would know what challenges you
> faced while using Airflow for the first time.
>
> Regards,
> Kaxil
--
Tomasz Urbaszek
Polidea | Software Engineer
M: +48 505 628 493
E: tomasz.urbaszek@polidea.com
Unique Tech
Check out our projects!