[GitHub] [airflow] potiuk commented on issue #20394: Airflow 2.2.2 - Celery worker ContentDisallowed('Refusing to deserialize untrusted content of type pickle (application/x-python-serialize)')

[GitBox <gi...@apache.org>]

potiuk commented on issue #20394:
URL: https://github.com/apache/airflow/issues/20394#issuecomment-997181063


   In your logs you have some deprecation configuration messages, so I'd advice you to review the [configuration options](https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html)  of Airflow 2 (as a general note).
   
   If you look at Airlfow 2 configuration, celery config should be configured by providing an import path for the configuration dictionary, rather than through environment variables:
   Specifically https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#celery-config-options
   
   Here is the default config:
   
   ```
   DEFAULT_CELERY_CONFIG = {
       'accept_content': ['json'],
       'event_serializer': 'json',
       'worker_prefetch_multiplier': conf.getint('celery', 'worker_prefetch_multiplier', fallback=1),
       'task_acks_late': True,
       'task_default_queue': conf.get('operators', 'DEFAULT_QUEUE'),
       'task_default_exchange': conf.get('operators', 'DEFAULT_QUEUE'),
       'task_track_started': conf.get('celery', 'task_track_started', fallback=True),
       'broker_url': broker_url,
       'broker_transport_options': broker_transport_options,
       'result_backend': conf.get('celery', 'RESULT_BACKEND'),
       'worker_concurrency': conf.getint('celery', 'WORKER_CONCURRENCY'),
   }
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org