Re: [I] Celery Executor is not working with redis-py 5.0.0 [airflow]

["rdjouder (via GitHub)" <gi...@apache.org>]

rdjouder commented on issue #33744:
URL: https://github.com/apache/airflow/issues/33744#issuecomment-1784769872

   Hello,
   I think the subject has been closed too early, redis 4.6.0 has this issue too
   
   Python 3.8.6
   
   Name: kombu
   Version: 5.3.2
   
   Name: apache-airflow
   Version: 2.7.2
   
   Name: redis
   Version: 4.6.0
   
   Traceback (most recent call last):
   File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/celery/executors/celery_executor_utils.py", line 199, in send_task_to_executor
   result = task_to_run.apply_async(args=[command], queue=queue)
   File "/home/airflow/.local/lib/python3.8/site-packages/celery/app/task.py", line 594, in apply_async
   return app.send_task(
   File "/home/airflow/.local/lib/python3.8/site-packages/celery/app/base.py", line 794, in send_task
   with self.producer_or_acquire(producer) as P:
   File "/home/airflow/.local/lib/python3.8/site-packages/celery/app/base.py", line 929, in producer_or_acquire
   producer, self.producer_pool.acquire, block=True,
   File "/home/airflow/.local/lib/python3.8/site-packages/celery/app/base.py", line 1344, in producer_pool
   return self.amqp.producer_pool
   File "/home/airflow/.local/lib/python3.8/site-packages/celery/app/amqp.py", line 590, in producer_pool
   self.app.connection_for_write()]
   File "/home/airflow/.local/lib/python3.8/site-packages/celery/app/base.py", line 826, in connection_for_write
   return self._connection(url or self.conf.broker_write_url, **kwargs)
   File "/home/airflow/.local/lib/python3.8/site-packages/celery/app/base.py", line 877, in _connection
   return self.amqp.Connection(
   File "/home/airflow/.local/lib/python3.8/site-packages/kombu/connection.py", line 201, in init
   if not get_transport_cls(transport).can_parse_url:
   File "/home/airflow/.local/lib/python3.8/site-packages/kombu/transport/init.py", line 90, in get_transport_cls
   _transport_cache[transport] = resolve_transport(transport)
   File "/home/airflow/.local/lib/python3.8/site-packages/kombu/transport/init.py", line 75, in resolve_transport
   return symbol_by_name(transport)
   File "/home/airflow/.local/lib/python3.8/site-packages/kombu/utils/imports.py", line 59, in symbol_by_name
   module = imp(module_name, package=package, **kwargs)
   File "/usr/local/lib/python3.8/importlib/init.py", line 127, in import_module
   return _bootstrap._gcd_import(name[level:], package, level)
   File "", line 1014, in _gcd_import
   File "", line 991, in _find_and_load
   File "", line 975, in _find_and_load_unlocked
   File "", line 671, in _load_unlocked
   File "", line 783, in exec_module
   File "", line 219, in _call_with_frames_removed
   File "/home/airflow/.local/lib/python3.8/site-packages/kombu/transport/redis.py", line 281, in
   class PrefixedRedisPipeline(GlobalKeyPrefixMixin, redis.client.Pipeline):
   AttributeError: module 'redis' has no attribute 'client'
   
   [airflow@xxxxx-scheduler-1 airflow]$ ls /home/airflow/.local/lib/python3.8/site-packages/redis
   init.py pycache asyncio backoff.py client.py cluster.py commands compat.py connection.py crc.py credentials.py exceptions.py lock.py ocsp.py retry.py sentinel.py typing.py utils.py
   [airflow@xxxx-scheduler-1 airflow]$ vi /home/airflow/.local/lib/python3.8/site-packages/kombu/transport/redis.py
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org