You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Chris (Jira)" <ji...@apache.org> on 2020/04/01 00:59:00 UTC
[jira] [Commented] (AIRFLOW-1135) Clearing XCom data from previous
executions
[ https://issues.apache.org/jira/browse/AIRFLOW-1135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072277#comment-17072277 ]
Chris commented on AIRFLOW-1135:
--------------------------------
Yes, [~ultrabug] stated precisely a common use case.
I found this Stackoverflow which could do this manually, but having a flag in the xcom pull to delete the xcom once accessed would be very useful.
> Clearing XCom data from previous executions
> -------------------------------------------
>
> Key: AIRFLOW-1135
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1135
> Project: Apache Airflow
> Issue Type: New Feature
> Components: database
> Reporter: Ultrabug
> Priority: Major
>
> It looks like XCom data from previous executions of tasks are never cleared from database and that there's no easy way to do so.
> Since XComs are created by execution_date of a task, when a task heavily uses XComs (like 20 000+) we can end up with a large xcom table on the database.
> Is there something I'm missing about cleaning up those previous XComs from database (since they're not used anymore) ?
> Would you accept the idea of having a new PR providing a helper to remove them ?
> Thanks
--
This message was sent by Atlassian Jira
(v8.3.4#803005)