Diagramming approach

[Jarek Potiuk <Ja...@polidea.com>]

I'd like to start a discussion about the approach for diagrams that we use
in our doc. I think my comments (where I have again fallen into the trap of
cc-ing all committers :( ) could be misunderstood so maybe I will explain
it here :).

TL;DR; My proposal is that we just agree to simple rules, that whenever we
add a diagram - either of the  two methods are used:

1) we have sources and link to those sources which anyone can use and
modify the image using free, ideally OSS tools.
2) we automatically generate those images from text "logical description"
sources


More details:

I think we should agree on some basic principles (and possibly set of
tools) that we use for the diagrams we add to the documentation. We have
more of those being added  (which is fantastic) and more are coming in
various parts. But the more of them we have the more liability it becomes
because those diagrams will have to be maintained as the architecture
changes for example.

In my opinion, for every diagram we have, there should be an obvious way
how anyone can modify the diagram easily (and for free), Having just a
diagram without a way to modify it is in my opinion an equivalent to a
"closed-source" code.

I believe it might mean two things:

1) we have sources that we can use in a free tool (draw.io seems to be a
popular choice). We store the sources in Cwiki that are under community
control and we always refer to where the sources are in the documentation.
The sources might be added as a file in CWiki
<https://cwiki.apache.org/confluence/display/AIRFLOW/Drawio+Diagrams> or
embedded into the page
<https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-23+Migrate+out+of+Travis+CI>.
This is a perfect solution in case of hand-drawn diagrams where WYSIWYG
drawing is important.

2) we have sources in the code where we can automatically generate such
diagrams from a markdown-like description in case of "logical" diagrams
that can be easily described as a "code". Perfect example of that are
sequence diagrams that are very easy to generate out of logical description
of what's going on. The example in my CI documentation PR
<https://github.com/PolideaInternal/airflow/blob/future-ci-documentation/CI.rst#ci-sequence-diagrams>.
with the ".mermaid" sources"
<https://github.com/PolideaInternal/airflow/blob/future-ci-documentation/images/ci/pull_request_ci_flow.mermaid>.
I used open-source mermaid tool which is rather popular, has an online
editor <https://mermaid-js.github.io/mermaid-live-editor/> (which you can
also install locally from sources). This has also benefit that you can
add code to the github repo and automatically generate the code from
sources in pre-commit - so you can basically rename something with source
tools (which I did in my CI PR).

I believe both solutions are fina and can be even combined (as I did in my
PR) - architecture diagrams are usually better if drawn by hand, sequence
diagrams and the like are better when generated from sources, because then
you can do the usual rename/refactor automatically and it's easier to keep
it in-sync with the code.

J.









-- 

Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer

M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>