You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/01/03 17:10:45 UTC
[GitHub] [airflow] AmarEL opened a new issue #13451: Modify HttpSensor to continue poking if the response is not 404
AmarEL opened a new issue #13451:
URL: https://github.com/apache/airflow/issues/13451
**Description**
As documented in the [HttpSensor](https://airflow.apache.org/docs/apache-airflow-providers-http/stable/_modules/airflow/providers/http/sensors/http.html) if the response for the HTTP call is an error different from "404" the task will Fail.
>HTTP Error codes other than 404 (like 403) or Connection Refused Error
> would fail the sensor itself directly (no more poking).
The code block that apply this behavior:
```
except AirflowException as exc:
if str(exc).startswith("404"):
return False
raise exc
```
**Use case / motivation**
I am working with an API that returns 500 for any error that happens internally (unauthorized, Not Acceptable, etc) and need the sensor be able to continue poking even the response is different from 404.
Another case's an API that sometimes returns 429 and makes the task fail. (Could solve with a large interval)
The first API has a bad design, but since we need to consume some services like this, I would like to have more flexibility when working with HttpSensor
**What do you want to happen**
When creating a HttpSensor task, I would like to be able to pass a list of status codes that will make the Sensor return "False" if the HTTP status code in the response match one code of the list to make the Sensor continue poking.
If no status code is set, the default to return False and continue poking will be 404 like is now.
**Are you willing to submit a PR?**
Yep!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #13451: Modify HttpSensor to continue poking if the response is not 404
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #13451:
URL: https://github.com/apache/airflow/issues/13451#issuecomment-753807394
@AmarEL assigned to you
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil closed issue #13451: Modify HttpSensor to continue poking if the response is not 404
Posted by GitBox <gi...@apache.org>.
kaxil closed issue #13451:
URL: https://github.com/apache/airflow/issues/13451
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org