You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "gopidesupavan (via GitHub)" <gi...@apache.org> on 2026/07/13 04:11:28 UTC
[PR] Add Common Data Quality provider skeleton [airflow]
gopidesupavan opened a new pull request, #69795:
URL: https://github.com/apache/airflow/pull/69795
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
Adds the initial skeleton for the `apache-airflow-providers-common-dataquality` provider.
This PR only introduces the provider package structure and repository wiring. It does not add data quality operators, hooks, rules, UI, or runtime behavior yet. The goal is to land the provider skeleton first so follow-up PRs can add functionality in smaller reviewable steps.
<!--
Thank you for contributing!
Please provide above a brief description of the changes made in this pull request.
Write a good git commit message following this guide: http://chris.beams.io/posts/git-commit/
Please make sure that your code changes are covered with tests.
And in case of new features or big changes remember to adjust the documentation.
For user-facing UI changes, please attach before/after screenshots (or a short
screen recording) so reviewers can assess the visual impact.
Feel free to ping (in general) for the review if you do not see reaction for a few days
(72 Hours is the minimum reaction time you can expect from volunteers) - we sometimes miss notifications.
In case of an existing issue, reference it using one of the following:
* closes: #ISSUE
* related: #ISSUE
-->
---
##### Was generative AI tooling used to co-author this PR?
<!--
If generative AI tooling has been used in the process of authoring this PR, please
change below checkbox to `[X]` followed by the name of the tool, uncomment the "Generated-by".
-->
- [ ] Yes (please specify the tool below)
<!--
Generated-by: [Tool Name] following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
-->
---
* Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Common Data Quality provider skeleton [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on PR #69795:
URL: https://github.com/apache/airflow/pull/69795#issuecomment-4964489835
> > Are we expected to have something really basic, or is it okay to keep it mostly empty for the first phase?
>
> yeah the reason is provider with functionality is getting more changes into single PR and it feels heavy .. have here with functionality #69575. after the skelton merge will rebase immediately. so would like to merge provider skelton first :)
Thank! Will also take a look at that PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Common Data Quality provider skeleton [airflow]
Posted by "gopidesupavan (via GitHub)" <gi...@apache.org>.
gopidesupavan merged PR #69795:
URL: https://github.com/apache/airflow/pull/69795
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Common Data Quality provider skeleton [airflow]
Posted by "gopidesupavan (via GitHub)" <gi...@apache.org>.
gopidesupavan commented on code in PR #69795:
URL: https://github.com/apache/airflow/pull/69795#discussion_r3570259491
##########
providers/common/dataquality/.gitignore:
##########
@@ -0,0 +1,3 @@
+# Do not commit hash files, this is just to speed-up local builds
Review Comment:
removed
##########
providers/common/dataquality/.gitignore:
##########
@@ -0,0 +1,3 @@
+# Do not commit hash files, this is just to speed-up local builds
+www-hash.txt
Review Comment:
yeah it was wrongly copied from the main PR https://github.com/apache/airflow/pull/69413 :) , removed it now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Common Data Quality provider skeleton [airflow]
Posted by "gopidesupavan (via GitHub)" <gi...@apache.org>.
gopidesupavan commented on PR #69795:
URL: https://github.com/apache/airflow/pull/69795#issuecomment-4959618478
> Are we expected to have something really basic, or is it okay to keep it mostly empty for the first phase?
yeah the reason is provider with functionality is getting more changes into single PR and it feels heavy .. have here with functionality https://github.com/apache/airflow/pull/69575. after the skelton merge will rebase immediately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Common Data Quality provider skeleton [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on code in PR #69795:
URL: https://github.com/apache/airflow/pull/69795#discussion_r3574203563
##########
.github/boring-cyborg.yml:
##########
@@ -141,6 +141,9 @@ labelPRBasedOnFilePath:
provider:docker:
- providers/docker/**
+ provider:common-dataquality:
Review Comment:
The other common-* labels sit alphabetically further up (common-ai through common-sql). This one landed between docker and edge, worth moving next to common-compat/common-io.
##########
airflow-core/tests/unit/always/test_project_structure.py:
##########
@@ -107,6 +107,7 @@ def test_providers_modules_should_have_tests(self):
"providers/common/compat/tests/unit/common/compat/standard/test_utils.py",
"providers/common/messaging/tests/unit/common/messaging/providers/test_base_provider.py",
"providers/common/messaging/tests/unit/common/messaging/providers/test_sqs.py",
+ "providers/common/dataquality/tests/unit/common/dataquality/test_exceptions.py",
Review Comment:
This entry points at a test for a module this PR doesn't add, there's no `exceptions.py` in the skeleton, so it's a no-op here. Looks like another carry-over from #69413, like the .gitignore entries. Since the comment above says this list should shrink to zero, I'd drop it here and add a real `test_exceptions.py` when the follow-up brings in `exceptions.py`.
##########
providers/common/dataquality/docs/commits.rst:
##########
@@ -0,0 +1,35 @@
+
+ .. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ .. http://www.apache.org/licenses/LICENSE-2.0
+
+ .. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+ .. NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE OVERWRITTEN!
+
+ .. IF YOU WANT TO MODIFY THIS FILE, YOU SHOULD MODIFY THE TEMPLATE
+ `PROVIDER_COMMITS_TEMPLATE.rst.jinja2` IN the `dev/breeze/src/airflow_breeze/templates` DIRECTORY
+
+ .. THE REMAINDER OF THE FILE IS AUTOMATICALLY GENERATED. IT WILL BE OVERWRITTEN!
+
+Package apache-airflow-providers-common-dataquality
+------------------------------------------------------
+
+``Data Quality Provider``
+
+
+This is detailed commit list of changes for versions provider package: ``dataquality``.
Review Comment:
The template renders the provider id here, so regeneration would produce ``common.dataquality`` (compare common/ai and common/io), and the description line above would be "Common Data Quality Provider" from provider.yaml. Worth matching the generated output so the first release prep doesn't produce a diff.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Common Data Quality provider skeleton [airflow]
Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #69795:
URL: https://github.com/apache/airflow/pull/69795#discussion_r3570010466
##########
providers/common/dataquality/.gitignore:
##########
@@ -0,0 +1,3 @@
+# Do not commit hash files, this is just to speed-up local builds
Review Comment:
why do we need this
##########
providers/common/dataquality/.gitignore:
##########
@@ -0,0 +1,3 @@
+# Do not commit hash files, this is just to speed-up local builds
+www-hash.txt
Review Comment:
same for this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Common Data Quality provider skeleton [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #69795:
URL: https://github.com/apache/airflow/pull/69795#issuecomment-4963465925
### Backport failed to create: v3-3-test. View the failure log <a href='https://github.com/apache/airflow/actions/runs/29290480532'> Run details </a>
Note: As of [Merging PRs targeted for Airflow 3.X](https://github.com/apache/airflow/blob/main/dev/README_AIRFLOW3_DEV.md#merging-prs-targeted-for-airflow-3x)
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.
In matter of doubt please ask in [#release-management](https://apache-airflow.slack.com/archives/C03G9H97MM2) Slack channel.
<table>
<tr>
<th>Status</th>
<th>Branch</th>
<th>Result</th>
</tr>
<tr>
<td>❌</td>
<td>v3-3-test</td>
<td><a href="https://github.com/apache/airflow/commit/2e511e6b7eea3865cd91d639ce49e795cb8082ce"><img src='https://img.shields.io/badge/Commit-2e511e6-red' alt='Commit Link'></a></td>
</tr>
</table>
You can attempt to backport this manually by running:
```bash
cherry_picker 2e511e6 v3-3-test
```
This should apply the commit to the v3-3-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.
After you have resolved the conflicts, you can continue the backport process by running:
```bash
cherry_picker --continue
```
If you don't have cherry-picker installed, see the [installation guide](https://github.com/apache/airflow/blob/main/dev/README.md#how-to-backport-pr-with-cherry-picker-cli).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Common Data Quality provider skeleton [airflow]
Posted by "gopidesupavan (via GitHub)" <gi...@apache.org>.
gopidesupavan commented on code in PR #69795:
URL: https://github.com/apache/airflow/pull/69795#discussion_r3574258545
##########
providers/common/dataquality/docs/commits.rst:
##########
@@ -0,0 +1,35 @@
+
+ .. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ .. http://www.apache.org/licenses/LICENSE-2.0
+
+ .. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+ .. NOTE! THIS FILE IS AUTOMATICALLY GENERATED AND WILL BE OVERWRITTEN!
+
+ .. IF YOU WANT TO MODIFY THIS FILE, YOU SHOULD MODIFY THE TEMPLATE
+ `PROVIDER_COMMITS_TEMPLATE.rst.jinja2` IN the `dev/breeze/src/airflow_breeze/templates` DIRECTORY
+
+ .. THE REMAINDER OF THE FILE IS AUTOMATICALLY GENERATED. IT WILL BE OVERWRITTEN!
+
+Package apache-airflow-providers-common-dataquality
+------------------------------------------------------
+
+``Data Quality Provider``
+
+
+This is detailed commit list of changes for versions provider package: ``dataquality``.
Review Comment:
updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add Common Data Quality provider skeleton [airflow]
Posted by "gopidesupavan (via GitHub)" <gi...@apache.org>.
gopidesupavan commented on code in PR #69795:
URL: https://github.com/apache/airflow/pull/69795#discussion_r3574256188
##########
airflow-core/tests/unit/always/test_project_structure.py:
##########
@@ -107,6 +107,7 @@ def test_providers_modules_should_have_tests(self):
"providers/common/compat/tests/unit/common/compat/standard/test_utils.py",
"providers/common/messaging/tests/unit/common/messaging/providers/test_base_provider.py",
"providers/common/messaging/tests/unit/common/messaging/providers/test_sqs.py",
+ "providers/common/dataquality/tests/unit/common/dataquality/test_exceptions.py",
Review Comment:
yeah bad commit ..removed now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org