Re: [PR] Add SRV DNS record support to HTTP Operator [airflow]

["aaron-y-chen (via GitHub)" <gi...@apache.org>]

aaron-y-chen commented on code in PR #70960:
URL: https://github.com/apache/airflow/pull/70960#discussion_r4014172660


##########
providers/http/src/airflow/providers/http/hooks/http.py:
##########
@@ -52,6 +55,19 @@ def _url_from_endpoint(base_url: str | None, endpoint: str | None) -> str:
     return (base_url or "") + (endpoint or "")
 
 
+def _select_srv_target(answers: Iterable[SRV]) -> tuple[str, int]:
+    """Select a target host and port from resolved DNS SRV records."""
+    candidates_by_priority: dict[int, list[SRV]] = {}
+    for record in answers:
+        candidates_by_priority.setdefault(record.priority, []).append(record)
+    # RFC 2782 priority failover; weight is not honored, ties broken uniformly at random.
+    candidates = candidates_by_priority[min(candidates_by_priority)]

Review Comment:
   With `{10: [primary], 20: [backup]}`, this selects only `[primary]`. If `primary` is unreachable, all three `run_with_advanced_retry()` attempts still target it; the healthy `backup` is never tried, so the advertised SRV failover does not work.



##########
providers/http/docs/connections/http.rst:
##########
@@ -75,3 +86,10 @@ For example:
 .. code-block:: bash
 
    export AIRFLOW_CONN_HTTP_DEFAULT='http://username:password@service.com:80/https?headers=header'
+
+To enable SRV lookup via an environment variable, set ``srv_lookup`` in the Extra query
+parameter:
+
+.. code-block:: bash
+
+   export AIRFLOW_CONN_HTTP_DEFAULT='https://_http._tcp.example.com/https?srv_lookup=true'

Review Comment:
   Copying this URI sets `conn_type` to `https`. `HttpOperator` then fails when resolving the hook with `Unknown hook type "https"`, before making any HTTP request.
   
   
   ```suggestion
      export AIRFLOW_CONN_HTTP_DEFAULT='http://_http._tcp.example.com/https?srv_lookup=true'
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org