[I] HttpSensor deferrable mode ignores response_error_codes_allowlist [airflow]

["Tomeshwari-02 (via GitHub)" <gi...@apache.org>]

Tomeshwari-02 opened a new issue, #69903:
URL: https://github.com/apache/airflow/issues/69903

   ### What happened and how to reproduce it?
   
   `HttpSensor` supports `response_error_codes_allowlist` so users can treat selected HTTP errors as "keep poking" instead of failing the task. In the synchronous path this is honored by checking `self.response_error_codes_allowlist` in `HttpSensor.poke()`.
   
   However, in deferrable mode the behavior appears to diverge after the task is deferred:
   
   - `providers/http/src/airflow/providers/http/sensors/http.py` stores the configured allowlist and uses it in `poke()`.
   - `HttpSensor.execute()` creates an `HttpSensorTrigger`, but does not pass `response_error_codes_allowlist` to the trigger.
   - `providers/http/src/airflow/providers/http/triggers/http.py` has `HttpSensorTrigger.run()` retry only when the exception string starts with `"404"`.
   
   A DAG using something like this can behave differently depending on whether deferrable mode is enabled:
   
   ```python
   HttpSensor(
       task_id="wait_for_api",
       endpoint="/health",
       http_conn_id="my_api",
       response_error_codes_allowlist=["404", "503"],
       deferrable=True,
   )
   ```
   
   Expected behavior: `503` should be treated the same way as in non-deferrable mode, i.e. the sensor should keep waiting according to `poke_interval`.
   
   Current behavior: the initial worker-side `poke()` can honor `503`, but after deferral the trigger only handles `404`. For other allowlisted statuses such as `503`, the trigger does not sleep via the allowlist path and does not preserve the documented sensor behavior.
   
   ### What you think should happen instead?
   
   `HttpSensorTrigger` should accept and serialize the configured `response_error_codes_allowlist`, and `HttpSensor.execute()` should pass the value when creating the trigger.
   
   The trigger should use that allowlist instead of hard-coding `404`, so deferrable and non-deferrable `HttpSensor` behavior are consistent.
   
   A unit test should cover a deferrable `HttpSensor` with a custom allowlist such as `["404", "503"]`.
   
   ### Operating System
   
   N/A
   
   ### Versions of Apache Airflow Providers
   
   Current `main` branch, HTTP provider code under `providers/http`.
   
   ### Deployment
   
   N/A
   
   ### Deployment details
   
   N/A
   
   ### Anything else?
   
   The existing sync-path tests already cover custom `response_error_codes_allowlist`; the missing coverage appears to be the deferrable trigger path.
   
   ### Are you willing to submit PR?
   
   Yes, I am willing to submit a PR.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org