You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/04/03 15:18:24 UTC
[GitHub] [airflow] mik-laj opened a new pull request #15176: Less docker magic in docs building
mik-laj opened a new pull request #15176:
URL: https://github.com/apache/airflow/pull/15176
Recently @potiuk worked on building docs in parallel and managed to do so, but used a lot of Docker tricks. https://github.com/apache/airflow/blob/master/docs/build_docs.py
I would like to present another workaround that patches only one function from `sphinx-autoapi` and does not change the rest of the process. This simplifies the build process and makes it independent of Docker. You can use Docker to run a documentation build if you want, but it's not required.
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-813667266
[The Workflow run](https://github.com/apache/airflow/actions/runs/720475959) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#discussion_r607317115
##########
File path: scripts/ci/docs/ci_docs.sh
##########
@@ -22,16 +22,4 @@ build_images::prepare_ci_build
build_images::rebuild_ci_image_if_needed_with_group
-start_end::group_start "Preparing venv for doc building"
-
-python3 -m venv .docs-venv
-source .docs-venv/bin/activate
-export PYTHONPATH=${AIRFLOW_SOURCES}
-
-pip install --upgrade pip==20.2.4
-
-pip install .[doc] --upgrade --constraint "${AIRFLOW_SOURCES}/constraints.txt"
-
-start_end::group_end
-
-"${AIRFLOW_SOURCES}/docs/build_docs.py" -j 0 "${@}"
+runs::run_docs "${@}"
Review comment:
Yes @mik-laj - I think -j 0 is missing. I think you should set the jobs parameter to be 0 by default.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#discussion_r606676122
##########
File path: docs/exts/docs_build/run-patched-sphinx.py
##########
@@ -0,0 +1,105 @@
+#!/usr/bin/env python
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import os
+import sys
+
+import autoapi
+from autoapi.extension import (
+ LOGGER,
+ ExtensionError,
+ bold,
+ darkgreen,
+ default_backend_mapping,
+ default_file_mapping,
+ default_ignore_patterns,
+)
+from sphinx.cmd.build import main
+
+
+def run_autoapi(app):
+ """Load AutoAPI data from the filesystem."""
+ if not app.config.autoapi_dirs:
+ raise ExtensionError("You must configure an autoapi_dirs setting")
+
+ # Make sure the paths are full
+ normalized_dirs = []
+ autoapi_dirs = app.config.autoapi_dirs
+ if isinstance(autoapi_dirs, str):
+ autoapi_dirs = [autoapi_dirs]
+ for path in autoapi_dirs:
+ if os.path.isabs(path):
+ normalized_dirs.append(path)
+ else:
+ normalized_dirs.append(os.path.normpath(os.path.join(app.confdir, path)))
+
+ for _dir in normalized_dirs:
+ if not os.path.exists(_dir):
+ raise ExtensionError(
+ "AutoAPI Directory `{dir}` not found. "
+ "Please check your `autoapi_dirs` setting.".format(dir=_dir)
+ )
+
+ # Change from app.confdir to app.srcdir.
+ # Before:
+ # - normalized_root = os.path.normpath(
+ # - os.path.join(app.confdir, app.config.autoapi_root)
+ # -)
+ normalized_root = os.path.normpath(os.path.join(app.srcdir, app.config.autoapi_root))
Review comment:
Here is the root of the problems. `app.confdir` > `app.srcdir`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-812932421
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest master at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-813634515
I think it's better to change the --jobs default value to be 0 actually
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-813971530
@mik-laj Can you please report the AutoAPI src dir issue upstream?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-813734939
I pushed rebase for master that likely caused the kubernetes tests failures
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-813071520
[The Workflow run](https://github.com/apache/airflow/actions/runs/717110282) is cancelling this PR. Building images for the PR has failed. Follow the workflow link to check the reason.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
potiuk closed pull request #15176:
URL: https://github.com/apache/airflow/pull/15176
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-813734939
I pushed rebase for master that likely casued the kubernetes tests failures
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-812881777
Assuming it all works, I think we should make one more change - I think the --jobs args should default to 0. this will significantly speed up docs building for an average user who has more than 2 CPUS without any special action from the user side.
we could also remove the "sequentially" part altogether. Running wth -j 1 would be equivalent of it (with different logging, but I think for an average user it is still perfectly ok to see the output until error occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-813632832
It looks like the build is sequential
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-813088267
[The Workflow run](https://github.com/apache/airflow/actions/runs/717227521) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-812881777
Assuming it all works, I think we should make one more change - I think the --jobs args should default to 0. this will significantly speed up docs building for an average user who have more than 2 CPUS without any special action from the user side.
we could also remove the "sequentially" part altogether. Running wth -j 1 would be equivalent of it (with different logging, but I think for an average user it is still perfectly ok to see the output until error occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-813752048
It's a different problem - merging this one now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-813983418
I believe @mik-laj works on moving to newer version of autoapi #15206 and I think it's either solved there already or will be done as part of this work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-813079869
Hey @mik-laj - FYI. I rebased / force-pushed this one, as it was based on a commit which was failing (lack of `blinker`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-813671971
Yeah :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-812881777
Assuming it all works, I think we should make one more change - I think the --jobs args should default to 0. this will significantly speed up docs building for an average user who has more than 2 CPUS without any special action from the user side.
we could also remove the "sequentially" functions altogether. Running wth -j 1 would be equivalent of it (with different logging, but I think for an average user it is still perfectly ok to see the output until error occurs.
This is just one process more, but it will simplify the code quite a bit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #15176:
URL: https://github.com/apache/airflow/pull/15176
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#discussion_r606676122
##########
File path: docs/exts/docs_build/run-patched-sphinx.py
##########
@@ -0,0 +1,105 @@
+#!/usr/bin/env python
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import os
+import sys
+
+import autoapi
+from autoapi.extension import (
+ LOGGER,
+ ExtensionError,
+ bold,
+ darkgreen,
+ default_backend_mapping,
+ default_file_mapping,
+ default_ignore_patterns,
+)
+from sphinx.cmd.build import main
+
+
+def run_autoapi(app):
+ """Load AutoAPI data from the filesystem."""
+ if not app.config.autoapi_dirs:
+ raise ExtensionError("You must configure an autoapi_dirs setting")
+
+ # Make sure the paths are full
+ normalized_dirs = []
+ autoapi_dirs = app.config.autoapi_dirs
+ if isinstance(autoapi_dirs, str):
+ autoapi_dirs = [autoapi_dirs]
+ for path in autoapi_dirs:
+ if os.path.isabs(path):
+ normalized_dirs.append(path)
+ else:
+ normalized_dirs.append(os.path.normpath(os.path.join(app.confdir, path)))
+
+ for _dir in normalized_dirs:
+ if not os.path.exists(_dir):
+ raise ExtensionError(
+ "AutoAPI Directory `{dir}` not found. "
+ "Please check your `autoapi_dirs` setting.".format(dir=_dir)
+ )
+
+ # Change from app.confdir to app.srcdir.
+ # Before:
+ # - normalized_root = os.path.normpath(
+ # - os.path.join(app.confdir, app.config.autoapi_root)
+ # -)
+ normalized_root = os.path.normpath(os.path.join(app.srcdir, app.config.autoapi_root))
Review comment:
Here is the root of the problems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
ashb commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-813989524
Ah cool.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-813666946
[The Workflow run](https://github.com/apache/airflow/actions/runs/720460399) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-813071240
Closed and re-opened to trigger the build again. @mik-laj - If you want, I can remove the parallel/sequential dualism and set -j 0 as default in a follow-up PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#discussion_r606677094
##########
File path: docs/exts/docs_build/docs_builder.py
##########
@@ -255,69 +190,44 @@ def check_spelling(self, verbose: bool, dockerized: bool) -> List[SpellingError]
with open(filepath) as spelling_file:
warning_text += spelling_file.read()
- spelling_errors.extend(parse_spelling_warnings(warning_text, self._src_dir, dockerized))
+ spelling_errors.extend(parse_spelling_warnings(warning_text, self._src_dir))
console.print(f"[blue]{self.package_name:60}:[/] [red]Finished spell-checking with errors[/]")
else:
if spelling_errors:
console.print(
- f"[blue]{self.package_name:60}:[/] [yellow]Finished spell-checking " f"with warnings[/]"
+ f"[blue]{self.package_name:60}:[/] [yellow]Finished spell-checkingwith warnings[/]"
Review comment:
```suggestion
f"[blue]{self.package_name:60}:[/] [yellow]Finished spell-checking with warnings[/]"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
mik-laj commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-813664215
There seems to be some problem. I am looking at it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
mik-laj commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-813701693
"Build docs" step is greeen.
Before:
> Elapsed time spent in the script: 589 seconds
After:
> Elapsed time spent in the script: 510 seconds
~15% faster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-813640328
❤️
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-812885053
[The Workflow run](https://github.com/apache/airflow/actions/runs/714621006) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-812880405
Sounds promising :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
mik-laj commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-813683155
I pushed the fix. In addition, I noticed that multiprocessing does not work on Mac OS and I fixed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#discussion_r607317249
##########
File path: scripts/ci/docs/ci_docs.sh
##########
@@ -22,16 +22,4 @@ build_images::prepare_ci_build
build_images::rebuild_ci_image_if_needed_with_group
-start_end::group_start "Preparing venv for doc building"
-
-python3 -m venv .docs-venv
-source .docs-venv/bin/activate
-export PYTHONPATH=${AIRFLOW_SOURCES}
-
-pip install --upgrade pip==20.2.4
-
-pip install .[doc] --upgrade --constraint "${AIRFLOW_SOURCES}/constraints.txt"
-
-start_end::group_end
-
-"${AIRFLOW_SOURCES}/docs/build_docs.py" -j 0 "${@}"
+runs::run_docs "${@}"
Review comment:
```suggestion
runs::run_docs -j 0 "${@}"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on pull request #15176: Less docker magic in docs building
Posted by GitBox <gi...@apache.org>.
mik-laj commented on pull request #15176:
URL: https://github.com/apache/airflow/pull/15176#issuecomment-813608588
I fixed the static checks errors and rebased to latest master. I hope I can merge this soon and then we can make other improvements. I would like to improve the stability a bit, because by introducing parallel building now building is less stable, e.g. docs packakes without inventories should be built first (see: https://github.com/apache/airflow/blob/e49722859b81cfcdd7e4bb8e8aba4efb049a8590/docs/build_docs.py#L516)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org