[jira] [Comment Edited] (AIRFLOW-1645) librabbitmq 1.6.1 is incompatibe with celery 4.1

["Badger (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-1645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16335191#comment-16335191 ] 

Badger edited comment on AIRFLOW-1645 at 1/23/18 12:11 AM:
-----------------------------------------------------------

I'm also seeing this with the install of airflow 1.9. It appears that amqp is been installed somewhere else as I all I had to do was remove librabbitmq. I also install the following extra packages when I install airflow (via ansible):
 - jdbc
 - hdfs
 - gcp_api
 - crypto
 - celery
 - async
 - hive
 - password
 - postgres
 - rabbitmq

The setup.py in master ([https://github.com/apache/incubator-airflow/blob/master/setup.py]) pulls in Celery 4+ and librabbitmq as part of this install. I'm not really a python developer but this looks to be the case. 

 


was (Author: badger):
I've also seeing this with the install of airflow 1.9. It appears that amqp is been installed somewhere else as I all I had to do was remove librabbitmq. I install the following extra packages when I install airflow (via ansible):

- jdbc
 - hdfs
 - gcp_api
 - crypto
 - celery
 - async
 - hive
 - password
 - postgres
 *- rabbitmq*

The setup.py in master ([https://github.com/apache/incubator-airflow/blob/master/setup.py]) pulls in Celery 4+ and librabbitmq. I'm not really a python developer but this looks to be the case. 

 

> librabbitmq 1.6.1 is incompatibe with celery 4.1
> ------------------------------------------------
>
>                 Key: AIRFLOW-1645
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1645
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: celery
>    Affects Versions: 1.8.2
>            Reporter: Christoph Hösler
>            Priority: Minor
>
> After installing airflow in a bitnami/minideb docker container with Python 2.7.13 using
> {code}pip install "apache-airflow[celery, mysql, rabbitmq, crypto, s3, hdfs, druid] == 1.8.2"{code}, I ran it in distributed mode on kubernetes with the celery executor backed by rabbitmq. I was unable to execute any task and found the following warning in the worker log:
> {code}
> Received and deleted unknown message. Wrong destinaton?
> {code}
> I seems to be the issue reported [here|https://github.com/celery/librabbitmq/issues/93]. Uninstalling librabbitmq and installing "amqp == 2.2.2" resolves it as suggested [here|https://github.com/celery/celery/issues/3675].
> Maybe you want to adjust the default dependencies of airflow accordingly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)