You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Don-Burns (via GitHub)" <gi...@apache.org> on 2024/01/30 11:07:18 UTC
[I] common-sql SQLThresholdCheckOperator always returns error if query yields value of 0 [airflow]
Don-Burns opened a new issue, #37089:
URL: https://github.com/apache/airflow/issues/37089
### Apache Airflow Provider(s)
common-sql
### Versions of Apache Airflow Providers
apache-airflow-providers-common-sql==1.5.2 (from constraints file)
### Apache Airflow version
2.6.3
### Operating System
WSL running Ubuntu 20.04.6 LTS
### Deployment
Docker-Compose
### Deployment details
Am running the [aws mwaa-local-runner](https://github.com/aws/aws-mwaa-local-runner/tree/v2.6.3) on v2.6.3
### What happened
Threshold operator fails with an error when SQL query returns a value of 0.
The error
```
[2024-01-30, 10:43:18 UTC] {{taskinstance.py:1824}} ERROR - Task failed with exception
Traceback (most recent call last):
File "/usr/local/airflow/.local/lib/python3.10/site-packages/airflow/providers/common/sql/operators/sql.py", line 986, in execute
self._raise_exception(f"The following query returned zero rows: {self.sql}")
File "/usr/local/airflow/.local/lib/python3.10/site-packages/airflow/providers/common/sql/operators/sql.py", line 185, in _raise_exception
raise AirflowException(exception_string)
airflow.exceptions.AirflowException: The following query returned zero rows: SELECT 0
```
### What you think should happen instead
The check should pass so long as 0 is within the threshold define on the min and max
### How to reproduce
Minimal Example with same check passing and failing with only difference being that returned value is either 0 or 1. Which are both within the threshold:
```python
from datetime import datetime
from airflow import DAG
from airflow.providers.common.sql.operators.sql import (
SQLThresholdCheckOperator,
)
with DAG(
dag_id="test_dag",
start_date=datetime(2024, 1, 1),
catchup=False,
max_active_runs=1,
is_paused_upon_creation=False,
) as dag:
result_is_non_0 = SQLThresholdCheckOperator(
task_id="ThresholdCheckOperatorFailOn0",
conn_id= "db_connection",
sql="SELECT 0",
min_threshold=0,
max_threshold=1,
)
result_is_non_0 = SQLThresholdCheckOperator(
task_id="ThresholdCheckOperatorPassOver0",
conn_id= "db_connection",
sql="SELECT 1",
min_threshold=0,
max_threshold=1,
)
```
Result:
### Anything else
I believe the issue is due to this "falsey" check in the operator:
From v1.5.2 of the operator
https://github.com/apache/airflow/blob/providers-common-sql/1.5.2/airflow/providers/common/sql/operators/sql.py#L985-L987
From v1.10.1 of the operator (latest from what I see) https://github.com/apache/airflow/blob/cead3da4a6f483fa626b81efd27a24dcb5a36ab0/airflow/providers/common/sql/operators/sql.py#L1061-L1064
Since 0 will trigger the if condition here, perhaps it should be `if not result or result not == 0` or something similar?
But happy to get feedback there for a PR if something else would be preferred.
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] common-sql SQLThresholdCheckOperator always returns error if query yields value of 0 [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #37089: common-sql SQLThresholdCheckOperator always returns error if query yields value of 0
URL: https://github.com/apache/airflow/issues/37089
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] common-sql SQLThresholdCheckOperator always returns error if query yields value of 0 [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #37089:
URL: https://github.com/apache/airflow/issues/37089#issuecomment-1916606770
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org