You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/07 19:10:48 UTC
[GitHub] [airflow] mik-laj opened a new pull request #12892: Simplify publishing of documentation
mik-laj opened a new pull request #12892:
URL: https://github.com/apache/airflow/pull/12892
Close: https://github.com/apache/airflow/issues/11423
Close: https://github.com/apache/airflow/issues/11152
This is a small automation to make publishing documentation for multiple packages easier.
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #12892: Simplify publishing of documentation
Posted by GitBox <gi...@apache.org>.
kaxil commented on pull request #12892:
URL: https://github.com/apache/airflow/pull/12892#issuecomment-741225171
Should we merge https://github.com/apache/airflow-site/pull/318 and this one @mik-laj
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #12892: Simplify publishing of documentation
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #12892:
URL: https://github.com/apache/airflow/pull/12892#discussion_r538886908
##########
File path: dev/README_RELEASE_AIRFLOW.md
##########
@@ -583,7 +621,7 @@ https://pypi.python.org/pypi/apache-airflow
The documentation is available on:
https://airflow.apache.org/
-https://airflow.apache.org/docs/${VERSION}/
+https://airflow.apache.org/docs/apache-airflow/${VERSION}/
Review comment:
Something about having `airflow` appear in the URL twice bugs me unduly, but I don't really have an alternate suggestion (and I agree that having providers separately is right)
Oh well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #12892: Simplify publishing of documentation
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #12892:
URL: https://github.com/apache/airflow/pull/12892#discussion_r538326783
##########
File path: dev/README_RELEASE_AIRFLOW.md
##########
@@ -551,6 +552,43 @@ At this point we release an official package:
- Update CHANGELOG.md with the details, and commit it.
+## Publish documentation
+
+Documentation is an essential part of the product and should be made available to users.
+In our cases, documentation for the released versions is published in a separate repository - [`apache/airflow-site`](https://github.com/apache/airflow-site), but the documentation source code and build tools are available in the `apache/airflow` repository, so you have to coordinate between the two repositories to be able to build the documentation.
+
+Documentation for providers can be found in the ``/docs/apache-airflow`` directory.
+
+- First, copy the airflow-site repository and set the environment variable ``AIRFLOW_SITE_DIRECTORY``.
+
+ ```shell script
+ git clone https://github.com/apache/airflow-site.git airflow-site
+ cd airflow-site
+ export AIRFLOW_SITE_DIRECTORY="$(pwd)"
+ ```
+
+- Then you can go to the directory and build the necessary documentation packages
+
+ ```shell script
+ cd "${AIRFLOW_REPO_ROOT}"
+ ./breeze build-docs -- --package apache-airflow --for-production
+ ```
+
+- Now you can preview the documentation.
+
+ ```shell script
+ ./docs/start_doc_server.sh
+ ``
Review comment:
```suggestion
```
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #12892: Simplify publishing of documentation
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #12892:
URL: https://github.com/apache/airflow/pull/12892#discussion_r538325775
##########
File path: dev/README_RELEASE_AIRFLOW.md
##########
@@ -551,6 +552,43 @@ At this point we release an official package:
- Update CHANGELOG.md with the details, and commit it.
+## Publish documentation
+
+Documentation is an essential part of the product and should be made available to users.
+In our cases, documentation for the released versions is published in a separate repository - [`apache/airflow-site`](https://github.com/apache/airflow-site), but the documentation source code and build tools are available in the `apache/airflow` repository, so you have to coordinate between the two repositories to be able to build the documentation.
Review comment:
```suggestion
Documentation is an essential part of the product and should be made available to users.
In our cases, documentation for the released versions is published in a separate repository - [`apache/airflow-site`](https://github.com/apache/airflow-site), but the documentation source code and build tools are available in the `apache/airflow` repository, so you have to coordinate between the two repositories to be able to build the documentation.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil merged pull request #12892: Simplify publishing of documentation
Posted by GitBox <gi...@apache.org>.
kaxil merged pull request #12892:
URL: https://github.com/apache/airflow/pull/12892
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on pull request #12892: Simplify publishing of documentation
Posted by GitBox <gi...@apache.org>.
mik-laj commented on pull request #12892:
URL: https://github.com/apache/airflow/pull/12892#issuecomment-741256784
@kaxil We can merge this PR now. https://github.com/apache/airflow-site/pull/318 should be merged with the first release of the documentation, as there is a new layour for the /docs/ page.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #12892: Simplify publishing of documentation
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #12892:
URL: https://github.com/apache/airflow/pull/12892#discussion_r538897860
##########
File path: dev/README_RELEASE_AIRFLOW.md
##########
@@ -583,7 +621,7 @@ https://pypi.python.org/pypi/apache-airflow
The documentation is available on:
https://airflow.apache.org/
-https://airflow.apache.org/docs/${VERSION}/
+https://airflow.apache.org/docs/apache-airflow/${VERSION}/
Review comment:
I also have no idea for a better directory structure. Now we will be able to create a documentation link easily based on the package name in the pypi repository.
https://airflow.apache.org/docs/<PYPII-NAME>/${VERSION}/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #12892: Simplify publishing of documentation
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #12892:
URL: https://github.com/apache/airflow/pull/12892#discussion_r538897860
##########
File path: dev/README_RELEASE_AIRFLOW.md
##########
@@ -583,7 +621,7 @@ https://pypi.python.org/pypi/apache-airflow
The documentation is available on:
https://airflow.apache.org/
-https://airflow.apache.org/docs/${VERSION}/
+https://airflow.apache.org/docs/apache-airflow/${VERSION}/
Review comment:
I also have no idea for a better directory structure. Now we will be able to create a documentation link easily based on the package name in the pypi repository.
```
https://airflow.apache.org/docs/${PYPI-NAME}/${VERSION}/
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #12892: Simplify publishing of documentation
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #12892:
URL: https://github.com/apache/airflow/pull/12892#discussion_r538897860
##########
File path: dev/README_RELEASE_AIRFLOW.md
##########
@@ -583,7 +621,7 @@ https://pypi.python.org/pypi/apache-airflow
The documentation is available on:
https://airflow.apache.org/
-https://airflow.apache.org/docs/${VERSION}/
+https://airflow.apache.org/docs/apache-airflow/${VERSION}/
Review comment:
I also have no idea for a better directory structure. Now we will be able to create a documentation link easily based on the package name in the pypi repository.
https://airflow.apache.org/docs/${PYPI-NAME}/${VERSION}/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #12892: Simplify publishing of documentation
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #12892:
URL: https://github.com/apache/airflow/pull/12892#discussion_r538327995
##########
File path: dev/README_RELEASE_PROVIDER_PACKAGES.md
##########
@@ -884,6 +885,58 @@ twine upload -r pypi dist/*
* Again, confirm that the packages are available under the links printed.
+## Publish documentation
+
+Documentation is an essential part of the product and should be made available to users.
+In our cases, documentation for the released versions is published in a separate repository - [`apache/airflow-site`](https://github.com/apache/airflow-site), but the documentation source code and build tools are available in the `apache/airflow` repository, so you have to coordinate between the two repositories to be able to build the documentation.
+
+Documentation for providers can be found in the `/docs/apache-airflow-providers` directory and the `/docs/apache-airflow-providers-*/` directory. The first directory contains the package contents lists and should be updated every time a new version of provider packages is released.
+
+- First, copy the airflow-site repository and set the environment variable ``AIRFLOW_SITE_DIRECTORY``.
+
+ ```shell script
+ git clone https://github.com/apache/airflow-site.git airflow-site
+ cd airflow-site
+ export AIRFLOW_SITE_DIRECTORY="$(pwd)"
+ ```
+
+- Then you can go to the directory and build the necessary documentation packages
+
+ ```shell script
+ cd "${AIRFLOW_REPO_ROOT}"
+ ./breeze build-docs -- \
+ --package apache-airflow-providers \
+ --package apache-airflow-providers-apache-airflow \
+ --package apache-airflow-providers-telegram \
+ --for-production
+ ```
+
+- Now you can preview the documentation.
+
+ ```shell script
+ ./docs/start_doc_server.sh
+ ``
Review comment:
```suggestion
```
```
##########
File path: dev/README_RELEASE_PROVIDER_PACKAGES.md
##########
@@ -884,6 +885,58 @@ twine upload -r pypi dist/*
* Again, confirm that the packages are available under the links printed.
+## Publish documentation
+
+Documentation is an essential part of the product and should be made available to users.
+In our cases, documentation for the released versions is published in a separate repository - [`apache/airflow-site`](https://github.com/apache/airflow-site), but the documentation source code and build tools are available in the `apache/airflow` repository, so you have to coordinate between the two repositories to be able to build the documentation.
+
+Documentation for providers can be found in the `/docs/apache-airflow-providers` directory and the `/docs/apache-airflow-providers-*/` directory. The first directory contains the package contents lists and should be updated every time a new version of provider packages is released.
+
+- First, copy the airflow-site repository and set the environment variable ``AIRFLOW_SITE_DIRECTORY``.
+
+ ```shell script
+ git clone https://github.com/apache/airflow-site.git airflow-site
+ cd airflow-site
+ export AIRFLOW_SITE_DIRECTORY="$(pwd)"
+ ```
+
+- Then you can go to the directory and build the necessary documentation packages
+
+ ```shell script
+ cd "${AIRFLOW_REPO_ROOT}"
+ ./breeze build-docs -- \
+ --package apache-airflow-providers \
+ --package apache-airflow-providers-apache-airflow \
+ --package apache-airflow-providers-telegram \
+ --for-production
+ ```
+
+- Now you can preview the documentation.
+
+ ```shell script
+ ./docs/start_doc_server.sh
+ ``
+
+- Copy the documentation to the ``airflow-site`` repository
+
+ ```shell script
+ ./docs/publish_docs.py \
+ --package apache-airflow-providers \
+ --package apache-airflow-providers-apache-airflow \
+ --package apache-airflow-providers-telegram \
+
+ cd "${AIRFLOW_SITE_DIRECTORY}"
+ ````
Review comment:
```suggestion
```
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #12892: Simplify publishing of documentation
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #12892:
URL: https://github.com/apache/airflow/pull/12892#discussion_r538326970
##########
File path: dev/README_RELEASE_AIRFLOW.md
##########
@@ -551,6 +552,43 @@ At this point we release an official package:
- Update CHANGELOG.md with the details, and commit it.
+## Publish documentation
+
+Documentation is an essential part of the product and should be made available to users.
+In our cases, documentation for the released versions is published in a separate repository - [`apache/airflow-site`](https://github.com/apache/airflow-site), but the documentation source code and build tools are available in the `apache/airflow` repository, so you have to coordinate between the two repositories to be able to build the documentation.
+
+Documentation for providers can be found in the ``/docs/apache-airflow`` directory.
+
+- First, copy the airflow-site repository and set the environment variable ``AIRFLOW_SITE_DIRECTORY``.
+
+ ```shell script
+ git clone https://github.com/apache/airflow-site.git airflow-site
+ cd airflow-site
+ export AIRFLOW_SITE_DIRECTORY="$(pwd)"
+ ```
+
+- Then you can go to the directory and build the necessary documentation packages
+
+ ```shell script
+ cd "${AIRFLOW_REPO_ROOT}"
+ ./breeze build-docs -- --package apache-airflow --for-production
+ ```
+
+- Now you can preview the documentation.
+
+ ```shell script
+ ./docs/start_doc_server.sh
+ ``
+
+- Copy the documentation to the ``airflow-site`` repository, create commit and push changes.
+
+ ```shell script
+ ./docs/publish_docs.py --package apache-airflow
+ cd "${AIRFLOW_SITE_DIRECTORY}"
+ git commit -m "Add documentation for Apache Airflow ${VERSION}"
+ git push
+ ````
Review comment:
```suggestion
```
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #12892: Simplify publishing of documentation
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #12892:
URL: https://github.com/apache/airflow/pull/12892#issuecomment-740604207
The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide that full test matrix is needed and add the 'full tests needed' label. Then you should rebase it to the latest master or amend the last commit of the PR, and push it with --force-with-lease.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #12892: Simplify publishing of documentation
Posted by GitBox <gi...@apache.org>.
kaxil commented on pull request #12892:
URL: https://github.com/apache/airflow/pull/12892#issuecomment-740901393
ping @mik-laj This PR is good to go -- just few minor suggestions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org