[jira] [Resolved] (IMPALA-8082) Save intermediate state and data if applicable

["Tim Armstrong (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-8082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tim Armstrong resolved IMPALA-8082.
-----------------------------------
    Resolution: Later

> Save intermediate state and data if applicable
> ----------------------------------------------
>
>                 Key: IMPALA-8082
>                 URL: https://issues.apache.org/jira/browse/IMPALA-8082
>             Project: IMPALA
>          Issue Type: Improvement
>            Reporter: Janaki Lahorani
>            Priority: Major
>
> When a query is stalled, it will be beneficial to flush the state and if needed data as well, to disk (temporary space) so that the query can be suspended and resources can be freed.  The query can resume execution at a later point when it becomes un-stalled.  The amount of space that can be used should probably be configurable.  There should be life cycle management to clean up this space and abort stalled queries.  In reality, this space will be quite big.  If it is getting filled up then there is a problem that needs to be analyzed and addressed - may be in code, or in terms of management and logistics at deployment.  Consequently necessary tools, logging and diagnostics should be built in tandem.
> When a query crashes, it could potentially affect many queries that are running in that process.  It looks like end user is required to manually restart all these queries.  If there is an infrastructure that saved stages, then the non-crashed queries could be restarted from a saved point and these could be used to finish running the queries without requiring a user intervention.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)