You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "potiuk (via GitHub)" <gi...@apache.org> on 2023/08/29 10:23:26 UTC

[GitHub] [airflow] potiuk opened a new issue, #33876: Replace `--package-filter` usage for docs breeze command with short package names

potiuk opened a new issue, #33876:
URL: https://github.com/apache/airflow/issues/33876

   ### Body
   
   The `--package-filter` while nice in theory to specify which packages to build, has quite bad UX (lots of repetitions when specifying multiple packages, long package names. We practically (except `--package-filter apache-airflow-providers-*` never use the functionality of the filter with glob patterns.
   
   It's much more practical to use "short" package names ("apache.hdfs" rather that `--package-filter apache-airflow-providers-apache-hdfs` and we already use it in a few places in Breeze. 
   
   We should likely replace all the places when we use `--package-filter` with those short names, add a special alias for `all-providers` and this should help our users who build documentation and release manager to do their work faster and nicer. 
   
   This would also allow to remove the separate ./dev/provider_packages/publish_provider_documentation.sh  bash script that is aimed to do somethign similar in a "hacky way".
   
   ### Committer
   
   - [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] eladkal commented on issue #33876: Replace `--package-filter` usage for docs breeze command with short package names

Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on issue #33876:
URL: https://github.com/apache/airflow/issues/33876#issuecomment-1697221869

   I would even prefer listing providers as: `microsoft.azure cncf.kubernetes`
   rather than `apache-airflow-providers-microsoft-azure apache-airflow-providers-cncf-kubernetes`
   
   mainly because` breeze release-management prepare-provider-documentation` gives them as summary
   
   
   ```
      Success:
   alibaba amazon apache.beam apache.flink apache.hive apache.livy apache.pinot apache.spark arangodb 
   celery cncf.kubernetes common.sql daskexecutor databricks datadog dbt.cloud docker elasticsearch 
   exasol ftp github google grpc http imap influxdb microsoft.azure microsoft.psrp microsoft.winrm 
   mysql openlineage oracle plexus presto salesforce sendgrid sftp singularity slack smtp snowflake 
   ssh trino yandex
      Skipped:
   apache.impala jdbc microsoft.mssql neo4j postgres qubole sqlite vertica
      Marked as doc-only (please commit those!):
   airbyte apache.cassandra apache.drill apache.druid apache.hdfs apache.kafka apache.kylin apache.pig 
   apache.sqoop apprise asana atlassian.jira cloudant dingding discord facebook hashicorp jenkins 
   mongo odbc openfaas opsgenie pagerduty papermill redis samba segment tableau tabular telegram 
   zendesk
   
   ```
    


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] potiuk commented on issue #33876: Replace `--package-filter` usage for docs breeze command with short package names

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #33876:
URL: https://github.com/apache/airflow/issues/33876#issuecomment-1697168433

   cc: @amoghrajesh -> maybe another task for you :) ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] potiuk commented on issue #33876: Replace `--package-filter` usage for docs breeze command with short package names

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #33876:
URL: https://github.com/apache/airflow/issues/33876#issuecomment-1697354235

   BTW. The way I'd do it @amoghrajesh - I'd still keep the --package-filter option as is but then I'd also add the `argument` above  - and replace the suggestion, documentation etc. to use the argument instead of `--package-filter` for regular tasks (there are a few places where we print help and suggestion about `--package-filter`. 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] potiuk closed issue #33876: Replace `--package-filter` usage for docs breeze command with short package names

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #33876: Replace `--package-filter` usage for docs breeze command with short package names
URL: https://github.com/apache/airflow/issues/33876


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] amoghrajesh commented on issue #33876: Replace `--package-filter` usage for docs breeze command with short package names

Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on issue #33876:
URL: https://github.com/apache/airflow/issues/33876#issuecomment-1697170775

   Nice nice. Can you assign this to me? 
   
   I'd love to approach this! 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [airflow] potiuk commented on issue #33876: Replace `--package-filter` usage for docs breeze command with short package names

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #33876:
URL: https://github.com/apache/airflow/issues/33876#issuecomment-1697336214

   > I would even prefer listing providers as: microsoft.azure cncf.kubernetes
   rather than apache-airflow-providers-microsoft-azure apache-airflow-providers-cncf-kubernetes
   
   Absolutely - this is what we do in a few other places already and there is even a dedicated option we have:
   
   ```
   argument_packages = click.argument(
       "packages",
       nargs=-1,
       required=False,
       type=BetterChoice(get_available_documentation_packages(short_version=True)),
   )
   ```
   
   The `short_version` is doing this


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org