You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/02/05 13:20:16 UTC
[GitHub] [airflow] Rudasek86 opened a new issue #14100: Airflow 1.10.14 - celery worker doesn't want to connect to redis.
Rudasek86 opened a new issue #14100:
URL: https://github.com/apache/airflow/issues/14100
**Apache Airflow version**: 1.10.14
**Kubernetes version (if you are using kubernetes)** (use `kubectl version`):
**Environment**: Red Hat Enterprise Linux 8.2 (VMware)
- **Cloud provider or hardware configuration**:
- **OS** (e.g. from /etc/os-release): Red Hat Enterprise Linux 8.2
- **Kernel** (e.g. `uname -a`): 4.18.0-193.19.1.el8_2.x86_64 #1 SMP Wed Aug 26 15:29:02 EDT 2020 x86_64 x86_64 x86_64 GNU/Linux
- **Install tools**: Airflow 1.10.14, redis.x86_64 (version 5.0.3.2), redis client installed with airflow (redis 3.5.3)
- **Others**:
**What happened**:
We have upgraded version of airflow from 1.10.2 to 1.10.14 and there is a problem to connect to redis by celery worker.
This is really wierd problem because when we run a airflow celery worker (without -D parameter) it's connecting without problem. In Flower tool we can see redis connection is online, but when we try do it with -D, airflow celery worker -D there are errors like below.
[2021-02-04 17:33:00,442: ERROR/MainProcess] consumer: Cannot connect to redis://localhost:6379/0: Error 107 connecting to redis:6379. Operation on closed file..
Trying again in 12.00 seconds... (6/100)
Before upgrade on version 1.10.2, we doesnt have a problems like that. Our configuration has been working with settings
broker_url = redis://localhost:6379/0
Of course our redis is working on default port on this same machine
netstat-ntlp
tcp 0 0 127.0.0.1:6379 0.0.0.0:* LISTEN -
Now we have a settings IN airflow cfg we have setting like below:
broker_url = redis://localhost/0
We have tried also settings like below:
broker_url = redis://localhost:6379/0
broker_url = redis://127.0.0.1/0
broker_url = redis://127.0.0.1:6379/0
broker_url = redis://redis:6379/0
broker_url = redis://redis/0
**What you expected to happen**:
We would like to connect to redis on airflow celery deamon functionality.
<!-- What do you think went wrong? -->
We do not really know, redis server version is this same, when airflow 1.10.2 was connecting sucessfully.
Later Version of airflow and redis client has been upgraded.
**How to reproduce it**:
<!---
Just install this same of software and try to run this same operation
If this is a UI bug, please provide a screenshot of the bug or a link to a youtube video of the bug in action
Screeenshot is not needed. I have copied a error from log. It's not a gui problem
--->
**Anything else we need to know**:
<!--
How often does this problem occur? Once? Every time etc?
Every Time for airflow celery worker -D command
Any relevant logs to include? Put them here in side a detail tag:
<details><summary>x.log</summary> lots of stuff </details>
-->
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #14100: Airflow 1.10.14 - celery worker doesn't want to connect to redis.
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #14100:
URL: https://github.com/apache/airflow/issues/14100#issuecomment-774028702
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal closed issue #14100: Airflow 1.10.14 - celery worker doesn't want to connect to redis.
Posted by GitBox <gi...@apache.org>.
eladkal closed issue #14100:
URL: https://github.com/apache/airflow/issues/14100
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org