You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "timonviola (via GitHub)" <gi...@apache.org> on 2023/12/30 15:51:17 UTC
[PR] Add pre-commit check to check integrations table. [airflow]
timonviola opened a new pull request, #36497:
URL: https://github.com/apache/airflow/pull/36497
<!-- Please keep an empty line above the dashes. -->
Add pre-commit check to check if integrations and TESTING documentation are synchronized.
---
The new pre-commit test:
- reads `integration-*.yml` files
- reads the table of integrations in `TESTING.rst`
- compares the two lists and fails if they are not synced.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add pre-commit check to check integrations table. [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #36497:
URL: https://github.com/apache/airflow/pull/36497#issuecomment-1874607563
Some other pre-commits need to be run and fixed :).
I highly recomment `pre-commit install` - and then you do not have to even think about it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add pre-commit check to check integrations table. [airflow]
Posted by "timonviola (via GitHub)" <gi...@apache.org>.
timonviola commented on PR #36497:
URL: https://github.com/apache/airflow/pull/36497#issuecomment-1874370217
@potiuk Thank's for the feedback, pushed 2 commits, which should cover it.
> As a follow-up we could update "version" and "labels" for other files (for airflow service we only need to set label for "base.yml".
I did not change the base.yml in this PR as it is not part of `integrations-*.yml`
Maybe this is what you meant, but would it make sense to create another PR for standardizing `breeze.description`?
- Add a hook, which checks whether `breeze.description` exists and
- add `breeze.description` to other files as well e.g. `base.yml`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add pre-commit check to check integrations table. [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #36497:
URL: https://github.com/apache/airflow/pull/36497#issuecomment-1875783224
Cool !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add pre-commit check to check integrations table. [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #36497:
URL: https://github.com/apache/airflow/pull/36497#issuecomment-1872574142
> Particularly, I am not sure about auto generating the table, as the description part is missing. Could the description be pulled from the yml files?
Generally looks cool. We do not have descriptions yet. But we can possibly also already improve the `integration-*.yml" files and add description there (and read it from there)
See: https://docs.docker.com/compose/compose-file/compose-file-v3/#reference-and-guidelines - you could add "labels" and we could standardize to use `breeze.description` label for it (you can also add version as well to all the integration compose files - I tihnk "3.8" seems to make most sense as it is "19.03.0+" and we are already "24.0.0+".
```yaml
version: "3.8"
services:
kafka:
image: ....
labels:
breeze.description: "This label will appear on all containers for the web service"
```
As a follow-up we could update "version" and "labels" for other files (for airflow service we only need to set label for "base.yml".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add pre-commit check to check integrations table. [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #36497:
URL: https://github.com/apache/airflow/pull/36497
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add pre-commit check to check integrations table. [airflow]
Posted by "timonviola (via GitHub)" <gi...@apache.org>.
timonviola commented on PR #36497:
URL: https://github.com/apache/airflow/pull/36497#issuecomment-1872553584
@potiuk Could you give some first feedback on this?
Particularly, I am not sure about **auto generating** the table, as the description part is missing. Could the description be pulled from the `yml` files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add pre-commit check to check integrations table. [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #36497:
URL: https://github.com/apache/airflow/pull/36497#issuecomment-1874515361
> Maybe this is what you meant, but would it make sense to create another PR for standardizing breeze.description?
> Add a hook, which checks whether breeze.description exists and
add breeze.description to other files as well e.g. base.yml?
Yep. Absolutely. Separate PR, And having pre-commit to check it is a good idea :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add pre-commit check to check integrations table. [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on PR #36497:
URL: https://github.com/apache/airflow/pull/36497#issuecomment-1875783169
Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add pre-commit check to check integrations table. [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on PR #36497:
URL: https://github.com/apache/airflow/pull/36497#issuecomment-1872552766
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:
- Pay attention to the quality of your code (ruff, mypy and type annotations). Our [pre-commits]( https://github.com/apache/airflow/blob/main/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks) will help you with that.
- In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/main/docs/apache-airflow/howto/custom-operator.rst) Consider adding an example DAG that shows how users should use it.
- Consider using [Breeze environment](https://github.com/apache/airflow/blob/main/BREEZE.rst) for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
- Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
- Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
- Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#coding-style-and-best-practices).
- Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org