You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "Christopher Tubbs (Jira)" <ji...@apache.org> on 2020/10/28 22:50:00 UTC
[jira] [Resolved] (ACCUMULO-3232) Improve consumption of WAL header
in partial replication case
[ https://issues.apache.org/jira/browse/ACCUMULO-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christopher Tubbs resolved ACCUMULO-3232.
-----------------------------------------
Resolution: Abandoned
Closing this stale issue. If this is still a problem, please open a new issue or PR at https://github.com/apache/accumulo
> Improve consumption of WAL header in partial replication case
> -------------------------------------------------------------
>
> Key: ACCUMULO-3232
> URL: https://issues.apache.org/jira/browse/ACCUMULO-3232
> Project: Accumulo
> Issue Type: Improvement
> Components: replication
> Reporter: Josh Elser
> Priority: Major
>
> Consider a system that is actively replicating from one instance to another. Specifically, assume there is one WAL that is currently being replicated to the destination and the source instance is shutdown.
> When the source instance is restarted, it will notice that the WAL has read through N {{LogFileKey}}/{{LogFileValue}} pairs (from before it was shutdown) and while proceed past these records to get to the data in the file which it needs to read.
> We have to re-read each of these pairs from the file because the WAL is an append-only structure, and we can't efficiently seek to some point in the file, as we wouldn't know how to correlate the byte offset to entries.
> As we read the WAL, in addition (or perhaps instead of) tracking the offset in the WAL, it would be good to track the correlation of N bytes read to M records consumed which would help us better resume replication.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)