[jira] [Updated] (KUDU-3453) Fine-grained anchoring for WAL segments for tablet copy

["Alexey Serbin (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KUDU-3453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alexey Serbin updated KUDU-3453:
--------------------------------
    Summary: Fine-grained anchoring for WAL segments for tablet copy  (was: Fine-grained anchoring for WAL segments during tablet copying)

> Fine-grained anchoring for WAL segments for tablet copy
> -------------------------------------------------------
>
>                 Key: KUDU-3453
>                 URL: https://issues.apache.org/jira/browse/KUDU-3453
>             Project: Kudu
>          Issue Type: Improvement
>          Components: tablet, tserver
>            Reporter: Alexey Serbin
>            Priority: Major
>
> Tablet copying is a provision to implement the process of automatic tablet re-replication in Kudu.  When the system catalog (Kudu master) detects that a tablet replica is no longer available, it automatically re-replicates a tablet to a destination tablet server using another healthy tablet replica in the cluster as the source.
> When copying a tablet from one tablet server to another, the source tablet copying session "anchors" WAL segments to be transfered to the destination server, so they are not GC-ed by the tablet maintenance operation when they are no longer needed locally, but the tablet copy session is still in progress.
> The anchored WAL segments are releases all at once when the tablet copying session completes with success of failure.  However, there might be long running tablet copying sessions, and with high data ingest rate, the source tablet replica might accumulate huge amount of WAL data which isn't relevant at both the source and the destination server.
> To prevent accumulation of WAL data for long-running tablet copying sessions, it's necessary to update the WAL anchors in a more granular manner, e.g. un-anchor a segment once it has been successfully copied and persisted by the client tablet copying session.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)