You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Andor Molnár <an...@apache.org> on 2026/08/06 18:16:08 UTC
Re: [DISCUSS] Merge HBASE-28957 (Add support for Continuous Backup and Point-in-Time Recovery)
I haven’t looked at this PR for a while, so just did a rebase to the master branch.
Now it seems that GH Actions are timing out waiting for a runner.
“
Waiting for a runner to pick up this job…
“
Anyone has a clue what’s going on?
Regards,
Andor
> On Jul 9, 2026, at 11:48, Andor Molnár <an...@apache.org> wrote:
>
> Hi all,
>
> We’ve recently rebased the feature branch to reflect the changes committed to master.
>
> Please review the updated pull request: https://github.com/apache/hbase/pull/7445
>
> @Nick,
> I’m not sure what’s the problem with some of Yetus test.
> Would you please take a look and help me out again?
>
> Thanks,
>
> Andor
>
>
>
>
>> On Apr 8, 2026, at 09:22, Andor Molnár <an...@apache.org> wrote:
>>
>> Hi all,
>>
>> Just a gentle reminder on the PITR feature that we’d like to merge in the codebase and which I briefly talked about at the meetup.
>>
>> We’re looking for reviews on the following PR which is a prerequisite of the feature and based on the change requested in a review feedback by Duo:
>>
>> https://github.com/apache/hbase/pull/7617
>>
>> Please take a look when you get a chance. Patch is rather small, 3 files changed.
>>
>> Thanks,
>> Andor
>>
>>
>>
>>
>>> On Nov 6, 2025, at 10:47, Andor Molnár <an...@apache.org> wrote:
>>>
>>> Thanks Vinayak!
>>>
>>> I rebased the feature branch and created the pull request here:
>>>
>>> https://github.com/apache/hbase/pull/7445
>>>
>>> Regards,
>>> Andor
>>>
>>>
>>>
>>>
>>>> On Nov 4, 2025, at 21:56, Vinayak Hegde <vi...@gmail.com> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> We would like to propose merging the feature “Continuous Backup and
>>>> Point-in-Time Recovery (PITR)” into the main branch.
>>>> BackgroundExisting mechanisms such as replication and snapshots provide
>>>> data redundancy but are insufficient for effective point-in-time recovery.
>>>>
>>>> -
>>>>
>>>> *Replication* requires maintaining a live mirror cluster, which
>>>> significantly increases operational costs.
>>>> -
>>>>
>>>> *Snapshots* and *incremental snapshots* only capture data at discrete
>>>> points in time, resulting in possible data loss between snapshots.
>>>>
>>>> Limitations of the Current Incremental Backup Solution
>>>>
>>>> The existing incremental backup framework in HBase exhibits several
>>>> limitations:
>>>>
>>>> -
>>>>
>>>> *Risk of data loss:* Incremental backups are batch-based, leading to
>>>> potential data loss between backup intervals.
>>>> -
>>>>
>>>> *Limited restore flexibility:* Recovery is restricted to specific backup
>>>> timestamps rather than any desired point in time.
>>>> -
>>>>
>>>> *WAL management overhead:* Write-Ahead Logs (WALs) cannot be archived
>>>> until the backup operation completes, increasing storage overhead and
>>>> complexity.
>>>> -
>>>>
>>>> *Complex tracking:* Manual tracking of backup IDs, job history, and logs
>>>> introduces operational challenges.
>>>>
>>>> Summary of the Proposed Feature
>>>>
>>>> The *Continuous Backup and PITR* feature introduces a continuous and
>>>> fine-grained backup mechanism that addresses the above limitations. It
>>>> enables:
>>>>
>>>> -
>>>>
>>>> Continuous archival of WALs to support near real-time backup.
>>>> -
>>>>
>>>> Restoration of data to any desired point in time (PITR) for improved
>>>> data protection and flexibility.
>>>> -
>>>>
>>>> Simplified backup lifecycle and WAL management.
>>>>
>>>> A detailed description of the design and implementation can be found in the
>>>> following document:
>>>> Design Document: Continuous Backup and Point-in-Time Recovery
>>>> <https://docs.google.com/document/d/1csQBMyM1mwpe4QpWkCbyqvsC9F5nUBr4ierOo8IuGpE/edit?pli=1&tab=t.0>
>>>>
>>>> Please review and share your feedback or comments.
>>>>
>>>> Best regards,
>>>> Vinayak Hegde
>>>
>>
>
Re: [DISCUSS] Merge HBASE-28957 (Add support for Continuous Backup and Point-in-Time Recovery)
Posted by Andor Molnár <an...@apache.org>.
Hi HBase community,
Please be aware that the feature has now been accepted and merged to the master branch.
Thanks to everybody for the contribution.
Regards,
Andor
> On Aug 8, 2026, at 10:24, Andor Molnár <an...@apache.org> wrote:
>
> Build is now green.
> Please review when you have a chance.
>
> https://github.com/apache/hbase/pull/7445
>
> Regards,
> Andor
>
>
> On Thu, 2026-08-06 at 13:16 -0500, Andor Molnár wrote:
>> I haven’t looked at this PR for a while, so just did a rebase to the
>> master branch.
>> Now it seems that GH Actions are timing out waiting for a runner.
>>
>> “
>> Waiting for a runner to pick up this job…
>> “
>>
>> Anyone has a clue what’s going on?
>>
>> Regards,
>> Andor
>>
>>
>>
>>> On Jul 9, 2026, at 11:48, Andor Molnár <an...@apache.org> wrote:
>>>
>>> Hi all,
>>>
>>> We’ve recently rebased the feature branch to reflect the changes
>>> committed to master.
>>>
>>> Please review the updated pull request:
>>> https://github.com/apache/hbase/pull/7445
>>>
>>> @Nick,
>>> I’m not sure what’s the problem with some of Yetus test.
>>> Would you please take a look and help me out again?
>>>
>>> Thanks,
>>>
>>> Andor
>>>
>>>
>>>
>>>
>>>> On Apr 8, 2026, at 09:22, Andor Molnár <an...@apache.org> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> Just a gentle reminder on the PITR feature that we’d like to
>>>> merge in the codebase and which I briefly talked about at the
>>>> meetup.
>>>>
>>>> We’re looking for reviews on the following PR which is a
>>>> prerequisite of the feature and based on the change requested in
>>>> a review feedback by Duo:
>>>>
>>>> https://github.com/apache/hbase/pull/7617
>>>>
>>>> Please take a look when you get a chance. Patch is rather small,
>>>> 3 files changed.
>>>>
>>>> Thanks,
>>>> Andor
>>>>
>>>>
>>>>
>>>>
>>>>> On Nov 6, 2025, at 10:47, Andor Molnár <an...@apache.org>
>>>>> wrote:
>>>>>
>>>>> Thanks Vinayak!
>>>>>
>>>>> I rebased the feature branch and created the pull request here:
>>>>>
>>>>> https://github.com/apache/hbase/pull/7445
>>>>>
>>>>> Regards,
>>>>> Andor
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> On Nov 4, 2025, at 21:56, Vinayak Hegde
>>>>>> <vi...@gmail.com> wrote:
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> We would like to propose merging the feature “Continuous
>>>>>> Backup and
>>>>>> Point-in-Time Recovery (PITR)” into the main branch.
>>>>>> BackgroundExisting mechanisms such as replication and
>>>>>> snapshots provide
>>>>>> data redundancy but are insufficient for effective point-in-
>>>>>> time recovery.
>>>>>>
>>>>>> -
>>>>>>
>>>>>> *Replication* requires maintaining a live mirror cluster,
>>>>>> which
>>>>>> significantly increases operational costs.
>>>>>> -
>>>>>>
>>>>>> *Snapshots* and *incremental snapshots* only capture data at
>>>>>> discrete
>>>>>> points in time, resulting in possible data loss between
>>>>>> snapshots.
>>>>>>
>>>>>> Limitations of the Current Incremental Backup Solution
>>>>>>
>>>>>> The existing incremental backup framework in HBase exhibits
>>>>>> several
>>>>>> limitations:
>>>>>>
>>>>>> -
>>>>>>
>>>>>> *Risk of data loss:* Incremental backups are batch-based,
>>>>>> leading to
>>>>>> potential data loss between backup intervals.
>>>>>> -
>>>>>>
>>>>>> *Limited restore flexibility:* Recovery is restricted to
>>>>>> specific backup
>>>>>> timestamps rather than any desired point in time.
>>>>>> -
>>>>>>
>>>>>> *WAL management overhead:* Write-Ahead Logs (WALs) cannot be
>>>>>> archived
>>>>>> until the backup operation completes, increasing storage
>>>>>> overhead and
>>>>>> complexity.
>>>>>> -
>>>>>>
>>>>>> *Complex tracking:* Manual tracking of backup IDs, job
>>>>>> history, and logs
>>>>>> introduces operational challenges.
>>>>>>
>>>>>> Summary of the Proposed Feature
>>>>>>
>>>>>> The *Continuous Backup and PITR* feature introduces a
>>>>>> continuous and
>>>>>> fine-grained backup mechanism that addresses the above
>>>>>> limitations. It
>>>>>> enables:
>>>>>>
>>>>>> -
>>>>>>
>>>>>> Continuous archival of WALs to support near real-time backup.
>>>>>> -
>>>>>>
>>>>>> Restoration of data to any desired point in time (PITR) for
>>>>>> improved
>>>>>> data protection and flexibility.
>>>>>> -
>>>>>>
>>>>>> Simplified backup lifecycle and WAL management.
>>>>>>
>>>>>> A detailed description of the design and implementation can
>>>>>> be found in the
>>>>>> following document:
>>>>>> Design Document: Continuous Backup and Point-in-Time Recovery
>>>>>> <
>>>>>> https://docs.google.com/document/d/1csQBMyM1mwpe4QpWkCbyqvsC9F
>>>>>> 5nUBr4ierOo8IuGpE/edit?pli=1&tab=t.0>
>>>>>>
>>>>>> Please review and share your feedback or comments.
>>>>>>
>>>>>> Best regards,
>>>>>> Vinayak Hegde
>>>>>
>>>>
>>>
Re: [DISCUSS] Merge HBASE-28957 (Add support for Continuous Backup and Point-in-Time Recovery)
Posted by Andor Molnár <an...@apache.org>.
Build is now green.
Please review when you have a chance.
https://github.com/apache/hbase/pull/7445
Regards,
Andor
On Thu, 2026-08-06 at 13:16 -0500, Andor Molnár wrote:
> I haven’t looked at this PR for a while, so just did a rebase to the
> master branch.
> Now it seems that GH Actions are timing out waiting for a runner.
>
> “
> Waiting for a runner to pick up this job…
> “
>
> Anyone has a clue what’s going on?
>
> Regards,
> Andor
>
>
>
> > On Jul 9, 2026, at 11:48, Andor Molnár <an...@apache.org> wrote:
> >
> > Hi all,
> >
> > We’ve recently rebased the feature branch to reflect the changes
> > committed to master.
> >
> > Please review the updated pull request:
> > https://github.com/apache/hbase/pull/7445
> >
> > @Nick,
> > I’m not sure what’s the problem with some of Yetus test.
> > Would you please take a look and help me out again?
> >
> > Thanks,
> >
> > Andor
> >
> >
> >
> >
> > > On Apr 8, 2026, at 09:22, Andor Molnár <an...@apache.org> wrote:
> > >
> > > Hi all,
> > >
> > > Just a gentle reminder on the PITR feature that we’d like to
> > > merge in the codebase and which I briefly talked about at the
> > > meetup.
> > >
> > > We’re looking for reviews on the following PR which is a
> > > prerequisite of the feature and based on the change requested in
> > > a review feedback by Duo:
> > >
> > > https://github.com/apache/hbase/pull/7617
> > >
> > > Please take a look when you get a chance. Patch is rather small,
> > > 3 files changed.
> > >
> > > Thanks,
> > > Andor
> > >
> > >
> > >
> > >
> > > > On Nov 6, 2025, at 10:47, Andor Molnár <an...@apache.org>
> > > > wrote:
> > > >
> > > > Thanks Vinayak!
> > > >
> > > > I rebased the feature branch and created the pull request here:
> > > >
> > > > https://github.com/apache/hbase/pull/7445
> > > >
> > > > Regards,
> > > > Andor
> > > >
> > > >
> > > >
> > > >
> > > > > On Nov 4, 2025, at 21:56, Vinayak Hegde
> > > > > <vi...@gmail.com> wrote:
> > > > >
> > > > > Hi all,
> > > > >
> > > > > We would like to propose merging the feature “Continuous
> > > > > Backup and
> > > > > Point-in-Time Recovery (PITR)” into the main branch.
> > > > > BackgroundExisting mechanisms such as replication and
> > > > > snapshots provide
> > > > > data redundancy but are insufficient for effective point-in-
> > > > > time recovery.
> > > > >
> > > > > -
> > > > >
> > > > > *Replication* requires maintaining a live mirror cluster,
> > > > > which
> > > > > significantly increases operational costs.
> > > > > -
> > > > >
> > > > > *Snapshots* and *incremental snapshots* only capture data at
> > > > > discrete
> > > > > points in time, resulting in possible data loss between
> > > > > snapshots.
> > > > >
> > > > > Limitations of the Current Incremental Backup Solution
> > > > >
> > > > > The existing incremental backup framework in HBase exhibits
> > > > > several
> > > > > limitations:
> > > > >
> > > > > -
> > > > >
> > > > > *Risk of data loss:* Incremental backups are batch-based,
> > > > > leading to
> > > > > potential data loss between backup intervals.
> > > > > -
> > > > >
> > > > > *Limited restore flexibility:* Recovery is restricted to
> > > > > specific backup
> > > > > timestamps rather than any desired point in time.
> > > > > -
> > > > >
> > > > > *WAL management overhead:* Write-Ahead Logs (WALs) cannot be
> > > > > archived
> > > > > until the backup operation completes, increasing storage
> > > > > overhead and
> > > > > complexity.
> > > > > -
> > > > >
> > > > > *Complex tracking:* Manual tracking of backup IDs, job
> > > > > history, and logs
> > > > > introduces operational challenges.
> > > > >
> > > > > Summary of the Proposed Feature
> > > > >
> > > > > The *Continuous Backup and PITR* feature introduces a
> > > > > continuous and
> > > > > fine-grained backup mechanism that addresses the above
> > > > > limitations. It
> > > > > enables:
> > > > >
> > > > > -
> > > > >
> > > > > Continuous archival of WALs to support near real-time backup.
> > > > > -
> > > > >
> > > > > Restoration of data to any desired point in time (PITR) for
> > > > > improved
> > > > > data protection and flexibility.
> > > > > -
> > > > >
> > > > > Simplified backup lifecycle and WAL management.
> > > > >
> > > > > A detailed description of the design and implementation can
> > > > > be found in the
> > > > > following document:
> > > > > Design Document: Continuous Backup and Point-in-Time Recovery
> > > > > <
> > > > > https://docs.google.com/document/d/1csQBMyM1mwpe4QpWkCbyqvsC9F
> > > > > 5nUBr4ierOo8IuGpE/edit?pli=1&tab=t.0>
> > > > >
> > > > > Please review and share your feedback or comments.
> > > > >
> > > > > Best regards,
> > > > > Vinayak Hegde
> > > >
> > >
> >