You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "TomB-123 (via GitHub)" <gi...@apache.org> on 2023/10/27 07:58:38 UTC
[I] DNS Error (gaierror) within DAG using OracleHook [airflow]
TomB-123 opened a new issue, #35211:
URL: https://github.com/apache/airflow/issues/35211
### Apache Airflow version
Other Airflow 2 version (please specify below)
### What happened
I am running DAGs in a local k8s cluster using Airflow deployed from the official Helm chart. One DAG is using an Oracle Connection, but during instantiation of the OracleHook, the DNS name resolution of the host object in the supplied host string (eg. HOST=my-database-host) fails with a getaddrinfo error (see below).
However, both `curl` and `socket.getaddrinfo("my-database-host")` correctly resolve the hostname to the k8s pod IP. The k8s name server is in /etc/resolv.conf at DAG runtime, so I am convinced this is a bug.
### What you think should happen instead
Since name resolution works in other commands within the pod, and the DAG works when run locally (pointed at HOST=localhost and port-forwarded into the cluster), I would expect the DAG to run normally.
### How to reproduce
Connections file:
```
oracle_conn:
conn_type: oracle
description: Ora Conn
host: "(DESCRIPTION=(ADDRESS=(host=oracle-db)(port=1521)(protocol=tcp)) (CONNECT_DATA=(SERVICE_NAME=XE)))"
login: system
password: password
```
Simple DAG:
```
from datetime import datetime
from airflow.decorators import dag, task
from airflow.models import TaskInstance
from airflow.providers.oracle.hooks.oracle import OracleHook
@task
def connect_to_oracle(**context: TaskInstance) -> bool:
conn = OracleHook("oracle_conn")
csr = conn.get_conn().cursor()
if csr:
return True
return False
@dag(
dag_id="test",
description="Attempt to connect to oracle XE database",
start_date=datetime.now(),
)
def taskflow() -> None:
connect_to_oracle()
dag_object = taskflow()
```
Database deployment/svc spec:
```
apiVersion: apps/v1
kind: Deployment
metadata:
name: oracle-db
spec:
replicas: 1
selector:
matchLabels:
name: oracle-db
template:
metadata:
labels:
name: oracle-db
spec:
containers:
- name: oracle-db
image: gvenzl/oracle-xe:slim
imagePullPolicy: IfNotPresent
ports:
- name: ora-tcp
containerPort: 1521
protocol: TCP
- name: ora-udp
containerPort: 1521
protocol: UDP
env:
- name: ORACLE_PASSWORD
value: password
---
apiVersion: v1
kind: Service
metadata:
name: oracle-db
spec:
selector:
name: oracle-db
ports:
- port: 1521
targetPort: 1521
protocol: TCP
name: ora-tcp
- port: 1521
targetPort: 1521
protocol: UDP
name: ora-udp
```
1. kubectl apply -f <deployment+svc_spec>.yaml
2. deploy airflow container from helm (v2, chart version 0.2.0) with the above connections.yaml file mounted as connections volume
3. attempt to run the example DAG, get GAIError
4. optionally, add `import time; time.sleep(9999)` to the start of the DAG task, open a shell on the pod (using k9s) and:
1. verify that `curl oracle-db:1521` successfully opens a connection to the Oracle XE container.
2. verify that `python3 -c 'import socket; socket.getaddrinfo("oracle-db", 1521, 0, socket.SOCK_STREAM)'` successfully resolves to the Oracle pod IP.
### Operating System
macOS Sonoma 14.0
### Versions of Apache Airflow Providers
apache-airflow-providers-common-sql==1.5.1
apache-airflow-providers-oracle==3.7.2
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
K8s Rev: v1.27.3+k3s1
helm: stable 3.13.1 (bottled)
### Anything else
Occurs every time. The error is the following, occurring during the call to `OracleHook()`:
```
Traceback (most recent call last):
File "src/oracledb/impl/thin/connection.pyx", line 107, in oracledb.thin_impl.ThinConnImpl._connect_with_address
File "src/oracledb/impl/thin/protocol.pyx", line 150, in oracledb.thin_impl.Protocol._connect_phase_one
File "src/oracledb/impl/thin/protocol.pyx", line 296, in oracledb.thin_impl.Protocol._connect_tcp
File "/usr/local/lib/python3.11/socket.py", line 827, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/socket.py", line 962, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -2] Name or service not known
```
Unfortunately I could not determine what the actual issue is caused by, and so don't know how to fix it.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] DNS Error (gaierror) within DAG using OracleHook [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #35211:
URL: https://github.com/apache/airflow/issues/35211#issuecomment-1782471574
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] DNS Error (gaierror) within DAG using OracleHook [airflow]
Posted by "Taragolis (via GitHub)" <gi...@apache.org>.
Taragolis closed issue #35211: DNS Error (gaierror) within DAG using OracleHook
URL: https://github.com/apache/airflow/issues/35211
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org