Re: [PR] Add Redis task handler [airflow]

["thiagogds (via GitHub)" <gi...@apache.org>]

thiagogds commented on PR #31855:
URL: https://github.com/apache/airflow/pull/31855#issuecomment-3089857250

   Hi @michalc, thank you for this! I really need an easy distributed log solution.
   
   Sorry for asking here, but I couldn't find more documentation and I tried debugging it locally to find what's happening but I couldn't find what's going on. I'm new to Airflow.
   
   I can't get the RedisTaskHandler to write logs, only to read them. I've added a few extra debugging logs to the [`_read`](https://github.com/apache/airflow/blob/4ecebc2973587ebaa2cb12482de82e93d15c092f/providers/redis/src/airflow/providers/redis/log/redis_task_handler.py#L74) and I can see it looking for the correct file names.
   
   I then tried to add something to the [`set_context`](https://github.com/apache/airflow/blob/4ecebc2973587ebaa2cb12482de82e93d15c092f/providers/redis/src/airflow/providers/redis/log/redis_task_handler.py#L87), but the code there is never executed. I'm out of ideas, maybe I'm missing something obvious.
   
   I was also confused with this [comment](https://github.com/apache/airflow/issues/48721#issuecomment-2774116022). 
   
   I'm using Airflow==3.0.1 and apache-airflow-providers-redis==4.1.1
   
   This is my logging config
   ```
   [logging]
   colored_console_log = False
   remote_logging = False
   logging_config_class = log_config.LOGGING_CONFIG
   ```
   
   And this is the log_config.py
   ```python
   from copy import deepcopy
   from airflow.config_templates.airflow_local_settings import DEFAULT_LOGGING_CONFIG
   
   LOGGING_CONFIG = deepcopy(DEFAULT_LOGGING_CONFIG)
   LOGGING_CONFIG["handlers"]["task"] = {
       "class": "airflow.providers.redis.log.redis_task_handler.RedisTaskHandler",
       "conn_id": "redis_default",
       "base_log_folder": "/opt/airflow/logs",
   }
   
   REMOTE_TASK_LOG = None
   ```
   
   I've tried running it using Docker, local Python, and in my staging Kubernetes cluster, all with the same problem.
   
   I understand if you can't help, but it's worth a shot hehe 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org