[I] Java SDK E2E test fails [airflow]

["jscheffl (via GitHub)" <gi...@apache.org>]

jscheffl opened a new issue, #68160:
URL: https://github.com/apache/airflow/issues/68160

   ### Under which category would you file this issue?
   
   Task SDK
   
   ### Apache Airflow version
   
   main branch
   
   ### What happened and how to reproduce it?
   
   See: https://apache-airflow.slack.com/archives/C06K9Q5G2UA/p1780822897380349
   Example run failing: https://github.com/apache/airflow/actions/runs/27089151287/job/79951483489?pr=67551
   
   ### What you think should happen instead?
   
   Test should pass
   
   ### Operating System
   
   _No response_
   
   ### Deployment
   
   None
   
   ### Apache Airflow Provider(s)
   
   _No response_
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Official Helm Chart version
   
   Not Applicable
   
   ### Kubernetes Version
   
   _No response_
   
   ### Helm Chart configuration
   
   _No response_
   
   ### Docker Image customizations
   
   _No response_
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Java SDK E2E test fails [airflow]

["jscheffl (via GitHub)" <gi...@apache.org>]

jscheffl closed issue #68160: Java SDK E2E test fails
URL: https://github.com/apache/airflow/issues/68160


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Java SDK E2E test fails [airflow]

["Dev-iL (via GitHub)" <gi...@apache.org>]

Dev-iL commented on issue #68160:
URL: https://github.com/apache/airflow/issues/68160#issuecomment-4642761210

   Below is the AI's analysis, generated when I was trying to understand if my PR caused it. Might be useful to someone.
   
   ----
   
   ## Diagnosis: Java SDK e2e failure
   
   This is a regression introduced by #67781 ("Verify TCP connection ownership…"), partially addressed by #68147, and the failing CI commit `5610f9536b` **already contains #68147** (verified: it's an ancestor). No newer fix exists on `main`.
   
   **Root cause — IPv4-mapped IPv6 address mismatch:**
   
   1. The supervisor's listening socket is `AF_INET` (`socket.socket()` bound to `127.0.0.1`, `_subprocess.py:62`). When it accepts the child's connection, `conn.getpeername()` returns plain IPv4 → `peer = ('127.0.0.1', <port>)`.
   2. The Java task command is `[java, -classpath, …, main_class]`, so **`proc.pid` is the JVM itself** — it connects back directly (no descendant). On an IPv6-enabled host the JVM uses a **dual-stack `AF_INET6`** client socket, so the kernel records the connection in `/proc/net/tcp6` with the **IPv4-mapped** local address `::ffff:127.0.0.1`.
   3. `_connection_owned_by_process_tree` (`_subprocess.py:102`) compares `_socket_address(connection.laddr) == peer`, i.e. `('::ffff:127.0.0.1', port)` vs `('127.0.0.1', port)` → **never equal** → both `comm` and `logs` connections rejected → child gets its sockets closed → exits 1 → `RuntimeError: process exited with 1 before connecting`.
   
   **I verified the two checkable halves locally:** an `AF_INET` server's `getpeername()` yields plain `('127.0.0.1', …)`, while a dual-stack `AF_INET6` client's own socket address is `('::ffff:127.0.0.1', …)`. And `ipaddress.ip_address("::ffff:127.0.0.1").ipv4_mapped == 127.0.0.1`.
   
   **Why #68147's tests pass but the real JVM fails:** every test in `test_subprocess.py` simulates the peer with a plain-IPv4 Python `socket.socket()` (including the "descendant" test). None exercise a dual-stack/`AF_INET6` client, so none reproduce the v4-mapped form. The descendant-tree work in #68147 was the wrong axis — the JVM *is* `proc.pid`; the failure is address representation, not process topology.
   
   **The fix** is to normalize IPv4-mapped IPv6 addresses before comparison — cleanest in `_socket_address`, which canonicalizes both sides:
   
   ```python
   try:
       mapped = ipaddress.ip_address(host).ipv4_mapped
       if mapped is not None:
           host = str(mapped)
   except ValueError:
       pass
   ```
   
   That makes `::ffff:127.0.0.1` compare equal to `127.0.0.1`, preserving #67781's security property (still loopback, still the right port pair, still the child's process tree).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org