[jira] [Updated] (PHOENIX-5943) Consider splitting up SYSTEM.TASK into 2 tables - one for queued jobs and one for task history

["Chinmay Kulkarni (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PHOENIX-5943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chinmay Kulkarni updated PHOENIX-5943:
--------------------------------------
    Fix Version/s:     (was: 4.16.0)
                   4.16.1

> Consider splitting up SYSTEM.TASK into 2 tables - one for queued jobs and one for task history
> ----------------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-5943
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5943
>             Project: Phoenix
>          Issue Type: Improvement
>    Affects Versions: 4.15.0
>            Reporter: Chinmay Kulkarni
>            Priority: Major
>             Fix For: 5.1.0, 4.16.1
>
>
> We can split SYSTEM.TASK into 2 tables:
>  # SYSTEM.TASK_QUEUE: used as an ephemeral task queue with potentially a smaller TTL which contains tasks that are new and yet to be picked up.
>  # SYSTEM.TASK_HISTORY: used to store tasks that have already been picked up and may or may not have succeeded, along with more data about the task (stack trace, job IDs, etc.). This table can have a longer TTL and can be used for auditing/debugging purposes.
> The advantages of doing this are:
>  * Makes SYSTEM.TASK_QUEUE more lightweight.
>  * Less data scanned when picking up new tasks.
>  * Combined with a PK change, we can introduce a natural priority queue using the TASK_PRIORITY field.
>  * Different TTL value on the TASK_HISTORY table so we can maintain past tasks for a longer period of time.
>  * Potentially makes this more in-line with other use-cases that want to maintain a similar kind of task and task history table (for ex: View TTL's async delete functionality).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)