[jira] [Commented] (AIRFLOW-2495) Celery should depend on kombu >= 4.1.0 for sqla transport

["Apache Spark (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-2495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16601468#comment-16601468 ] 

Apache Spark commented on AIRFLOW-2495:
---------------------------------------

User 'rodrigc' has created a pull request for this issue:
https://github.com/apache/incubator-airflow/pull/3388

> Celery should depend on kombu >= 4.1.0 for sqla transport
> ---------------------------------------------------------
>
>                 Key: AIRFLOW-2495
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2495
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: celery
>    Affects Versions: 1.10
>            Reporter: Craig Rodrigues
>            Priority: Major
>             Fix For: 2.0.0
>
>
> I created a small requirements.txt that has one line:
>  
> {{git+https://github.com/apache/incubator-airflow@v1-10-test#egg=apache-airflow[celery,emr,hive,hdfs,ldap,mysql,postgres,redis,slack,s3]}}
>  
> I created a virtualenv, and did:
>  
> {{pip install -r requrements.txt}}
>  
> During the install I saw this:
>  
> {{Collecting kombu<5.0,>=4.0.2 (from celery>=4.0.2->apache-airflow[celery,emr,hdfs,hive,ldap,mysql,postgres,redis,s3,slack]->-r requirements.txt (line 1))}}
>  
> I had a system where kombu 4.0.2 was being installed.
> This is bad because I also had in my airflow.cfg:
> {{broker_url = sqla+mysql://airflow:password@hostname:3306/database}}
>  
> My celery transport was not working.
>  
>  
> kombu 4.0.2 does not support sqlalchemy urls:
> [https://github.com/celery/kombu/blob/v4.0.2/kombu/transport/__init__.py#L20]
> but kombu 4.1.0 does:
> [https://github.com/celery/kombu/blob/v4.1.0/kombu/transport/__init__.py#L32]
>  
> This is also mentioned in:
> https://github.com/celery/kombu/issues/686
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)