You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "potiuk (via GitHub)" <gi...@apache.org> on 2023/08/29 10:23:26 UTC
[GitHub] [airflow] potiuk opened a new issue, #33876: Replace `--package-filter` usage for docs breeze command with short package names
potiuk opened a new issue, #33876:
URL: https://github.com/apache/airflow/issues/33876
### Body
The `--package-filter` while nice in theory to specify which packages to build, has quite bad UX (lots of repetitions when specifying multiple packages, long package names. We practically (except `--package-filter apache-airflow-providers-*` never use the functionality of the filter with glob patterns.
It's much more practical to use "short" package names ("apache.hdfs" rather that `--package-filter apache-airflow-providers-apache-hdfs` and we already use it in a few places in Breeze.
We should likely replace all the places when we use `--package-filter` with those short names, add a special alias for `all-providers` and this should help our users who build documentation and release manager to do their work faster and nicer.
This would also allow to remove the separate ./dev/provider_packages/publish_provider_documentation.sh bash script that is aimed to do somethign similar in a "hacky way".
### Committer
- [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #33876: Replace `--package-filter` usage for docs breeze command with short package names
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on issue #33876:
URL: https://github.com/apache/airflow/issues/33876#issuecomment-1697221869
I would even prefer listing providers as: `microsoft.azure cncf.kubernetes`
rather than `apache-airflow-providers-microsoft-azure apache-airflow-providers-cncf-kubernetes`
mainly because` breeze release-management prepare-provider-documentation` gives them as summary
```
Success:
alibaba amazon apache.beam apache.flink apache.hive apache.livy apache.pinot apache.spark arangodb
celery cncf.kubernetes common.sql daskexecutor databricks datadog dbt.cloud docker elasticsearch
exasol ftp github google grpc http imap influxdb microsoft.azure microsoft.psrp microsoft.winrm
mysql openlineage oracle plexus presto salesforce sendgrid sftp singularity slack smtp snowflake
ssh trino yandex
Skipped:
apache.impala jdbc microsoft.mssql neo4j postgres qubole sqlite vertica
Marked as doc-only (please commit those!):
airbyte apache.cassandra apache.drill apache.druid apache.hdfs apache.kafka apache.kylin apache.pig
apache.sqoop apprise asana atlassian.jira cloudant dingding discord facebook hashicorp jenkins
mongo odbc openfaas opsgenie pagerduty papermill redis samba segment tableau tabular telegram
zendesk
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #33876: Replace `--package-filter` usage for docs breeze command with short package names
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #33876:
URL: https://github.com/apache/airflow/issues/33876#issuecomment-1697168433
cc: @amoghrajesh -> maybe another task for you :) ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #33876: Replace `--package-filter` usage for docs breeze command with short package names
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #33876:
URL: https://github.com/apache/airflow/issues/33876#issuecomment-1697354235
BTW. The way I'd do it @amoghrajesh - I'd still keep the --package-filter option as is but then I'd also add the `argument` above - and replace the suggestion, documentation etc. to use the argument instead of `--package-filter` for regular tasks (there are a few places where we print help and suggestion about `--package-filter`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #33876: Replace `--package-filter` usage for docs breeze command with short package names
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #33876: Replace `--package-filter` usage for docs breeze command with short package names
URL: https://github.com/apache/airflow/issues/33876
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] amoghrajesh commented on issue #33876: Replace `--package-filter` usage for docs breeze command with short package names
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on issue #33876:
URL: https://github.com/apache/airflow/issues/33876#issuecomment-1697170775
Nice nice. Can you assign this to me?
I'd love to approach this!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #33876: Replace `--package-filter` usage for docs breeze command with short package names
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #33876:
URL: https://github.com/apache/airflow/issues/33876#issuecomment-1697336214
> I would even prefer listing providers as: microsoft.azure cncf.kubernetes
rather than apache-airflow-providers-microsoft-azure apache-airflow-providers-cncf-kubernetes
Absolutely - this is what we do in a few other places already and there is even a dedicated option we have:
```
argument_packages = click.argument(
"packages",
nargs=-1,
required=False,
type=BetterChoice(get_available_documentation_packages(short_version=True)),
)
```
The `short_version` is doing this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org