[jira] [Commented] (AIRFLOW-387) Dropped postgres connections

["Cole Howard (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173833#comment-16173833 ] 

Cole Howard commented on AIRFLOW-387:
-------------------------------------

We moved to MySQL and there's no more dropped DB connections

> Dropped postgres connections
> ----------------------------
>
>                 Key: AIRFLOW-387
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-387
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: core
>    Affects Versions: Airflow 1.7.1.3
>         Environment: Running aiflow with postgresql+psycopg2, Postgres 9.1.19 backend.
>            Reporter: Andreas Merkel
>
> With the {{postgresql+psycopg2}} driver, we observe dropped connections in the database logs on the postgres server:
> {code}
> 2016-08-02 14:49:26 UTC LOG:  could not receive data from client: Connection reset by peer
> 2016-08-02 14:49:26 UTC LOG:  unexpected EOF on client connection
> {code}
> Three of these messages appear even if Airflow is started without any DAGs. If there are DAGs, the messages appear whenever the DAG is executed (even for simple DAGs like a Python operator that does nothing).
> On the Airflow side, everything works fine; it's just the messages in the log indicating that Airflow is somehow dropping the connections to Postgres without closing them properly.
> h2. Environment
> {{airflow.cfg}}:
> {code}
> [core]
> airflow_home = $PWD
> dags_folder = $PWD/dags
> base_log_folder = $PWD/airflow_logs
> plugins_folder = $PWD/plugins
> executor = LocalExecutor
> sql_alchemy_conn = postgresql+psycopg2://postgres:postgres@127.0.0.1/airflow
> parallelism = 32
> dag_concurrency = 16
> max_active_runs_per_dag = 16
> load_examples = False
> donot_pickle = False
> fernet_key =  ; provided via environment
> dags_are_paused_at_creation = False
> [webserver]
> expose_config = true
> authenticate = False
> filter_by_owner = False
> workers = 4
> worker_class = sync
> [scheduler]
> job_heartbeat_sec = 5
> scheduler_heartbeat_sec = 5
> [celery]
> celeryd_concurrency = 1  ; Oddly enough, this is needed though we don't use celery.
> {code}
> Installed packages:
> {code}
> airflow==1.7.1.3
> alembic==0.8.6
> Babel==1.3
> cffi==1.6.0
> chartkick==0.4.2
> croniter==0.3.12
> cryptography==1.3.2
> dill==0.2.5
> docutils==0.12
> enum34==1.1.6
> Flask==0.10.1
> Flask-Admin==1.4.0
> Flask-Cache==0.13.1
> Flask-Login==0.2.11
> Flask-WTF==0.12
> funcsigs==0.4
> future==0.15.2
> gunicorn==19.3.0
> idna==2.1
> ipaddress==1.0.16
> itsdangerous==0.24
> Jinja2==2.8
> lockfile==0.12.2
> Mako==1.0.4
> Markdown==2.6.6
> MarkupSafe==0.23
> numpy==1.11.0
> pandas==0.18.1
> psycopg2==2.6.1
> pyasn1==0.1.9
> pycparser==2.14
> Pygments==2.1.3
> python-daemon==2.1.1
> python-dateutil==2.5.3
> python-editor==1.0.1
> pytz==2016.4
> requests==2.10.0
> setproctitle==1.1.10
> six==1.10.0
> smoketest-dag==0.7
> SQLAlchemy==1.0.13
> thrift==0.9.3
> Werkzeug==0.11.10
> WTForms==2.1
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)