You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Alexis Rolland <al...@ubisoft.com> on 2018/01/16 03:37:53 UTC

Close SqlSensor Connection

Hello everyone,

I'm reaching out to discuss / suggest a small improvement in the class SqlSensor:
https://pythonhosted.org/airflow/_modules/airflow/operators/sensors.html

We are currently using SqlSensors on top of Teradata in several DAGs. When the DAGs execute we receive the following error message from Teradata engine: Error 8024 All virtual circuits are currently in use.
This error message would typically appear when we reach the maximum number of simultaneous connections to the database.

I am suspecting the SqlSensor task to create a lot of connections - basically every time it (re)tries, and these connections would end up in idle state.
Does closing the connection at the end of the SqlSensor poke method sounds feasible?

I'd like to take this opportunity as well to thank you for the awesome work you've been doing with Airflow.
Keep it up!

Best,

[far-cry-signature]