You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Kamil Breguła <ka...@polidea.com> on 2020/04/01 16:27:45 UTC
Re: how to set CELERY_TASK_RESULT_EXPIRES in Airflow
Hello,
I am not sure, but please look at this configuration option. This is
probably the solution to your problem
https://airflow.readthedocs.io/en/latest/configurations-ref.html#celery-config-options
Best regards,
Kamil
On Wed, Apr 1, 2020 at 5:43 PM heng gu <he...@yahoo.com.invalid> wrote:
>
> I am using Celery Executor with RabbitMQ and PostgreSQL database backend. The celery_taskmeta table never got automatically cleaned by Airflow. In Celery configuration, you can set CELERY_TASK_RESULT_EXPIRES, but I cannot find it in airflow.cfg or in other config templates. Does anybody know how to set it?
> Thanks,
Re: how to set CELERY_TASK_RESULT_EXPIRES in Airflow
Posted by Kamil Breguła <ka...@polidea.com>.
it's a very old ticket. We have introduced mechanisms that allow for
much more flexible configurations. if you need to add only one option
in the Celery configuration then you can do it using the option
mentioned earlier.
On Wed, Apr 1, 2020 at 8:32 PM heng gu <he...@yahoo.com.invalid> wrote:
>
> thanks for your quick response.
> found it on [AIRFLOW-661] Celery Task Result Expiry - ASF JIRA
>
> |
> |
> | |
> [AIRFLOW-661] Celery Task Result Expiry - ASF JIRA
>
>
> |
>
> |
>
> |
>
>
> It looks like the fix has not been released.
>
>
>
> On Wednesday, April 1, 2020, 12:28:00 PM EDT, Kamil Breguła <ka...@polidea.com> wrote:
>
> Hello,
>
> I am not sure, but please look at this configuration option. This is
> probably the solution to your problem
> https://airflow.readthedocs.io/en/latest/configurations-ref.html#celery-config-options
>
> Best regards,
> Kamil
>
> On Wed, Apr 1, 2020 at 5:43 PM heng gu <he...@yahoo.com.invalid> wrote:
> >
> > I am using Celery Executor with RabbitMQ and PostgreSQL database backend. The celery_taskmeta table never got automatically cleaned by Airflow. In Celery configuration, you can set CELERY_TASK_RESULT_EXPIRES, but I cannot find it in airflow.cfg or in other config templates. Does anybody know how to set it?
> > Thanks,
Re: how to set CELERY_TASK_RESULT_EXPIRES in Airflow
Posted by heng gu <he...@yahoo.com.INVALID>.
thanks for your quick response.
found it on [AIRFLOW-661] Celery Task Result Expiry - ASF JIRA
|
|
| |
[AIRFLOW-661] Celery Task Result Expiry - ASF JIRA
|
|
|
It looks like the fix has not been released.
On Wednesday, April 1, 2020, 12:28:00 PM EDT, Kamil Breguła <ka...@polidea.com> wrote:
Hello,
I am not sure, but please look at this configuration option. This is
probably the solution to your problem
https://airflow.readthedocs.io/en/latest/configurations-ref.html#celery-config-options
Best regards,
Kamil
On Wed, Apr 1, 2020 at 5:43 PM heng gu <he...@yahoo.com.invalid> wrote:
>
> I am using Celery Executor with RabbitMQ and PostgreSQL database backend. The celery_taskmeta table never got automatically cleaned by Airflow. In Celery configuration, you can set CELERY_TASK_RESULT_EXPIRES, but I cannot find it in airflow.cfg or in other config templates. Does anybody know how to set it?
> Thanks,