You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "amoghrajesh (via GitHub)" <gi...@apache.org> on 2023/09/02 15:46:01 UTC
[GitHub] [airflow] amoghrajesh commented on pull request #34004: Allowing short package names in build-docs breeze command
amoghrajesh commented on PR #34004:
URL: https://github.com/apache/airflow/pull/34004#issuecomment-1703870529
@eladkal @potiuk with the current changes, we have not retired the older `package-filter` at all.
One test I tried was to try a hybrid of it, since some users might not find it easy to migrate suddenly:
```
(new-env) ➜ airflow git:(shortenPackageFilter) ✗ breeze build-docs cncf.kubernetes sftp --package-filter apache-airflow-providers-airbyte
Good version of Docker: 24.0.2.
Good version of docker-compose: 2.18.1
The following important files are modified in /Users/adesai/Documents/OSS/airflow since last time image was built:
* setup.py
* setup.cfg
* Dockerfile.ci
* .dockerignore
* generated/provider_dependencies.json
* scripts/docker/common.sh
* scripts/docker/install_additional_dependencies.sh
* scripts/docker/install_airflow.sh
* scripts/docker/install_airflow_dependencies_from_branch_tip.sh
* scripts/docker/install_from_docker_context_files.sh
* scripts/docker/install_mysql.sh
Likely CI image needs rebuild
Do you want to build the image (this works best when you have good connection and can take usually from 20 seconds to few minutes depending how old your image is)?
Press y/N/q. Auto-select n in 10 seconds (add `--answer n` to avoid delay next time): n
The CI image for Python version 3.8 may be outdated
Please run at the earliest convenience:
breeze ci-image build --python 3.8
Running Initialization. Your basic configuration is:
* Airflow home: /root/airflow
* Airflow sources: /opt/airflow
* Airflow core SQL connection:
Using airflow version from current sources
Checking backend and integrations.
Skip fixing ownership of generated files as Host OS is darwin
#################### Available packages ####################
- apache-airflow
- apache-airflow-providers
- apache-airflow-providers-airbyte
- apache-airflow-providers-alibaba
- apache-airflow-providers-amazon
- apache-airflow-providers-apache-beam
- apache-airflow-providers-apache-cassandra
- apache-airflow-providers-apache-drill
- apache-airflow-providers-apache-druid
- apache-airflow-providers-apache-flink
- apache-airflow-providers-apache-hdfs
- apache-airflow-providers-apache-hive
- apache-airflow-providers-apache-impala
- apache-airflow-providers-apache-kafka
- apache-airflow-providers-apache-kylin
- apache-airflow-providers-apache-livy
- apache-airflow-providers-apache-pig
- apache-airflow-providers-apache-pinot
- apache-airflow-providers-apache-spark
- apache-airflow-providers-apache-sqoop
- apache-airflow-providers-apprise
- apache-airflow-providers-arangodb
- apache-airflow-providers-asana
- apache-airflow-providers-atlassian-jira
- apache-airflow-providers-celery
- apache-airflow-providers-cloudant
- apache-airflow-providers-cncf-kubernetes
- apache-airflow-providers-common-sql
- apache-airflow-providers-daskexecutor
- apache-airflow-providers-databricks
- apache-airflow-providers-datadog
- apache-airflow-providers-dbt-cloud
- apache-airflow-providers-dingding
- apache-airflow-providers-discord
- apache-airflow-providers-docker
- apache-airflow-providers-elasticsearch
- apache-airflow-providers-exasol
- apache-airflow-providers-facebook
- apache-airflow-providers-ftp
- apache-airflow-providers-github
- apache-airflow-providers-google
- apache-airflow-providers-grpc
- apache-airflow-providers-hashicorp
- apache-airflow-providers-http
- apache-airflow-providers-imap
- apache-airflow-providers-influxdb
- apache-airflow-providers-jdbc
- apache-airflow-providers-jenkins
- apache-airflow-providers-microsoft-azure
- apache-airflow-providers-microsoft-mssql
- apache-airflow-providers-microsoft-psrp
- apache-airflow-providers-microsoft-winrm
- apache-airflow-providers-mongo
- apache-airflow-providers-mysql
- apache-airflow-providers-neo4j
- apache-airflow-providers-odbc
- apache-airflow-providers-openfaas
- apache-airflow-providers-openlineage
- apache-airflow-providers-opsgenie
- apache-airflow-providers-oracle
- apache-airflow-providers-pagerduty
- apache-airflow-providers-papermill
- apache-airflow-providers-plexus
- apache-airflow-providers-postgres
- apache-airflow-providers-presto
- apache-airflow-providers-redis
- apache-airflow-providers-salesforce
- apache-airflow-providers-samba
- apache-airflow-providers-segment
- apache-airflow-providers-sendgrid
- apache-airflow-providers-sftp
- apache-airflow-providers-singularity
- apache-airflow-providers-slack
- apache-airflow-providers-smtp
- apache-airflow-providers-snowflake
- apache-airflow-providers-sqlite
- apache-airflow-providers-ssh
- apache-airflow-providers-tableau
- apache-airflow-providers-tabular
- apache-airflow-providers-telegram
- apache-airflow-providers-trino
- apache-airflow-providers-vertica
- apache-airflow-providers-yandex
- apache-airflow-providers-zendesk
- docker-stack
- helm-chart
Current package filters:
['apache-airflow-providers-cncf-kubernetes', 'apache-airflow-providers-sftp', 'apache-airflow-providers-airbyte']
#################### Fetching inventories ####################
Nothing to do
#################### Documentation will be built for 3 package(s) with 5 parallel jobs ####################
1. apache-airflow-providers-airbyte
2. apache-airflow-providers-cncf-kubernetes
3. apache-airflow-providers-sftp
#################### Cleaning documentation files ####################
apache-airflow-providers-airbyte : Cleaning files
apache-airflow-providers-cncf-kubernetes : Cleaning files
apache-airflow-providers-sftp : Cleaning files
#################### Scheduling documentation to build ####################
apache-airflow-providers-airbyte : Scheduling documentation to build
apache-airflow-providers-cncf-kubernetes : Scheduling documentation to build
apache-airflow-providers-sftp : Scheduling documentation to build
#################### Running docs building ####################
apache-airflow-providers-airbyte : Building documentation
apache-airflow-providers-cncf-kubernetes : Building documentation
apache-airflow-providers-sftp : Building documentation
apache-airflow-providers-sftp : Running sphinx. The output is hidden until an error occurs.
apache-airflow-providers-cncf-kubernetes : Running sphinx. The output is hidden until an error occurs.
apache-airflow-providers-airbyte : Running sphinx. The output is hidden until an error occurs.
^CProcess ForkPoolWorker-5:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/local/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 114, in worker
task = get()
File "/usr/local/lib/python3.8/multiprocessing/queues.py", line 355, in get
with self._rlock:
File "/usr/local/lib/python3.8/multiprocessing/synchronize.py", line 95, in __enter__
return self._semlock.__enter__()
KeyboardInterrupt
Process ForkPoolWorker-4:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/local/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 114, in worker
task = get()
File "/usr/local/lib/python3.8/multiprocessing/queues.py", line 356, in get
res = self._reader.recv_bytes()
File "/usr/local/lib/python3.8/multiprocessing/connection.py", line 216, in recv_bytes
buf = self._recv_bytes(maxlength)
File "/usr/local/lib/python3.8/multiprocessing/connection.py", line 414, in _recv_bytes
buf = self._recv(4)
File "/usr/local/lib/python3.8/multiprocessing/connection.py", line 379, in _recv
chunk = read(handle, remaining)
KeyboardInterrupt
Traceback (most recent call last):
File "/usr/local/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/opt/airflow/docs/build_docs.py", line 572, in <module>
main()
File "/opt/airflow/docs/build_docs.py", line 478, in main
package_build_errors, package_spelling_errors = build_docs_for_packages(
File "/opt/airflow/docs/build_docs.py", line 234, in build_docs_for_packages
run_in_parallel(
File "/opt/airflow/docs/build_docs.py", line 300, in run_in_parallel
run_docs_build_in_parallel(
File "/opt/airflow/docs/build_docs.py", line 345, in run_docs_build_in_parallel
result_list = pool.map(perform_docs_build_for_single_package, doc_build_specifications)
File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 364, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 765, in get
self.wait(timeout)
File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 762, in wait
self._event.wait(timeout)
File "/usr/local/lib/python3.8/threading.py", line 558, in wait
signaled = self._cond.wait(timeout)
File "/usr/local/lib/python3.8/threading.py", line 302, in wait
Aborted.
```
It takes combination of args + option from package-filter together!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org