[PROPOSAL] Refactor installation page for Airflow

[Jarek Potiuk <ja...@potiuk.com>]

Hello everyone,

TL;DR; I have a proposal ready on how we can refactor the installation page
of Airflow (based on listening and answering to a big number of user
issues/discussions/questions recently).
The PR is  https://github.com/apache/airflow/pull/18282 (ready for
comments/discussions/critique), but I thought this is an important
"entrypoint" for new users and I wanted to make sure to send it to devlist
and provide more context and get feedback.

I have recently answered a number of questions and tried to help a number
of Airflow users (especially new users) who tried to get their experience
with installing Airflow. There were numerous issues, discussions, stack
overflow questions I looked at and there was an important observation I
made - I think the "installation" with Airflow was not really geared
towards new users.

Yeah, we have a great "quick -start" spear-headed by Kamil, and it's great
for someone who just wants to try Airflow, but I think if someone wanted to
decide how to approach their first "real" installation of Airflow, the
"installation" page of Airflow was pretty confusing and lacked answer to
many questions the users had. One very recent example from a discussion I
had - nowhere in the installation part it is mentioned that you should run
"airflow db upgrade" at installation and each upgrade (there was a mention
in Production Deployment but not in Installation). This sounds obvious to
us, but it might be a surprise for a new user that the upgrade should be
run after even a minor upgrade of Airflow.

Those are small things, but there is one - more important (and I think this
is the most important observation I got from listening to the users). I had
to explain over and over, what each installation means, what the user
should expect, that things will not "magically work" in case of local
installation and explain that some parts like maintenance, upgrades,
cleanup and linking the components together are in the hands of users when
they choose "simpler" installation methods.

The installation page did not explain to new users the
different installation methods the user can take (local installation,
docker, helm, managed services). Also it did not explain what expectations
the users should have when using each of those methods. Some users lack
understanding on what they are responsible for when installing Airflow,
they did not know that when they choose docker-compose path, they must know
how to build their own image etc. and more importantly they did not know
what they can expect from Airflow Community in terms of resources and help.

I think having such an overview would be extremely helpful for our users,
so that they could make a decision which route they are going to take. I
attempted to turn the "installation" page into separate, interlinked pages
with exactly this kind overview of installation methods:

* when this option works best
* intended users
* what you are expected to handle
* what Apache Airflow Community provides for that method

I am open to change wording, or make a different split, or restructure it,
but I think a change to the installation page is needed - especially that
we are past the time when we focused on migrating users from 1.10 but now
we are addressing new users who did not install Airflow before.

The PR contains some screenshots, and you can also build the docs for
yourselves from my branch. Looking forward to improving the installation
docs.

J.