[jira] [Assigned] (HDDS-7951) [Snapshot] Clean SnapDiff job and report table

["Arpit Agarwal (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDDS-7951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Arpit Agarwal reassigned HDDS-7951:
-----------------------------------

    Assignee: Hemant Kumar

> [Snapshot] Clean SnapDiff job and report table
> ----------------------------------------------
>
>                 Key: HDDS-7951
>                 URL: https://issues.apache.org/jira/browse/HDDS-7951
>             Project: Apache Ozone
>          Issue Type: Sub-task
>            Reporter: Hemant Kumar
>            Assignee: Hemant Kumar
>            Priority: Major
>              Labels: pull-request-available
>
> Problem: For snapshot diff report persistence, we use RocksDB. Since RocksDB doesn't expose any way to get *ColumnFamilyHandle* for existing table which is needed to support pagination and we decided to have a global table *snap-diff-report-table* to store diff report where key will be prefixed with *ReportId* or *JobId*. Over the time this table will grow and will needed to clean.
> Task: Expectation from this is to remove entries from *snap-diff-report-table* for a job once persistence time is over (like 7 or 30 days which is configurable) or job failed due OM crash or any other reason.
> Approach: One approach could be following.
> We have another table *snap-diff-job-table* which can be used to determine which job and thier report can be removed. It would be based on time instance (creation time) and the job status (FAILED). Iterator over this table and move entries from table: *snap-diff-job-table* to some new table (may be something *delete-snap-diff-report-table*). Later deleted all the reports related to entries in table *delete-snap-diff-report-table*.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org