You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Bryan Beaudreault <bb...@apache.org> on 2024/06/05 19:34:57 UTC
[DISCUSS] Blockers for 2.6.1
Hey all,
It's been 2 weeks since 2.6.0 was released. As discussed in the vote
thread, there were a few outstanding backup-related issues. I believe we've
made some progress on some of those.
I'd like to start compiling a list of important backup-related fixes to
target for the 2.6.1 release so that we can track progress. Can those of
you who are involved (Ray Mattingly, Nick Dimiduck, Dieter De Paepe & team,
and any others) please list any important jiras here?
With a list of jiras in hand, we can check to make sure blockers &
fixVersions are set and use that to track what we need to drill down before
releasing.
Here's what I know of so far, let me know if I'm missing anything:
Not yet started:
- HBASE-28084: incremental backups should be forbidden after deleting
backups
- HBASE-28602: Incremental backup fails when WALs move
- HBASE-28462: (similar to ^, but in a different part of the code)
- HBASE-28538: BackupHFileCleaner is very expensive
Patch available:
- HBASE-28539: backup merging does not work when using cloud storage as
filesystem
- HBASE-28562: another possible failure cause for incremental backups +
possibly cause of overly big backup metadata
Resolved:
- HBASE-28502: backed up tables are not listed correctly in backup
metadata, which causes unreliable backup validation
- HBASE-28568: the set of tables included in incremental backups might be
too big
Re: [DISCUSS] Blockers for 2.6.1
Posted by Nick Dimiduk <nd...@apache.org>.
Heya team,
I'm landing the last patch flagged for 2.6.1rc0 in the next couple of
hours. Git and Jira are in alignment. Please pause your backports to
branch-2.6 until you see the VOTE thread.
Thanks,
Nick
On Thu, Sep 19, 2024 at 5:25 PM Nick Dimiduk <nd...@apache.org> wrote:
> I've trimmed down the remaining Jiras for 2.6.1 to this list of 5. Several
> of them have already merged to master and are in the process of backports.
> The only one I'm considering a blocker is the backup issue. I believe that
> we could use some test support on 28803 and 28850.
>
> If you have any other fixes that are close and you'd like to see included,
> please @-mention me or reply on this thread.
>
> Next up is to review the nightlies and isolate any egregious failures.
>
> I intend to ITBLL the tip of branch-2.6 in the next couple of days.
>
> With luck, we'll have an RC next week.
>
> Thanks for your support,
> Nick
>
>
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20HBASE%20AND%20resolution%20%3D%20Unresolved%20AND%20fixVersion%20%3D%202.6.1%20ORDER%20BY%20key%20ASC
>
> On Wed, Sep 18, 2024 at 10:50 AM Nick Dimiduk <nd...@apache.org> wrote:
>
>> I've filed a tracking ticket for burning down towards 2.6.1. Anyone who
>> would like to participate, please assign a subtask to yourself. If i've
>> forgotten any steps, please add them as well.
>>
>> https://issues.apache.org/jira/browse/HBASE-28847
>>
>> Thanks,
>> Nick
>>
>> On Sun, Sep 15, 2024 at 4:21 AM 张铎(Duo Zhang) <pa...@gmail.com>
>> wrote:
>>
>>> Thanks Bryan and Nick.
>>>
>>> I could also help if needed.
>>>
>>> Thanks.
>>>
>>> Nick Dimiduk <nd...@apache.org> 于2024年9月15日周日 01:14写道：
>>> >
>>> > I agree — we’re due.
>>> >
>>> > On Sat, 14 Sep 2024 at 17:05, Bryan Beaudreault <
>>> bbeaudreault@apache.org>
>>> > wrote:
>>> >
>>> > > Sorry for the delay, I've been too busy with other things and expect
>>> that
>>> > > to continue for a while. I’ve spoken to Nick Dimiduk and he’s going
>>> to take
>>> > > over as RM for 2.6. I expect next week he’ll be able to get a sense
>>> of any
>>> > > outstanding blockers and set a date for the release.
>>> > >
>>> > > On Sat, Sep 14, 2024 at 10:47 AM 张铎(Duo Zhang) <
>>> palomino219@gmail.com>
>>> > > wrote:
>>> > >
>>> > > > Bump.
>>> > > >
>>> > > > Are we ready for the 2.6.1 release now? We have delayed too much
>>> till
>>> > > > now...
>>> > > >
>>> > > > 张铎(Duo Zhang) <pa...@gmail.com> 于2024年8月5日周一 22:20写道：
>>> > > > >
>>> > > > > Thanks Bryan!
>>> > > > >
>>> > > > > Bryan Beaudreault <bb...@apache.org> 于2024年8月5日周一
>>> 20:59写道：
>>> > > > > >
>>> > > > > > Ray is out on holiday right now, and should be returning next
>>> week.
>>> > > I'm
>>> > > > > > waiting for him and the team to give an update on their
>>> > > backups-related
>>> > > > > > fixes and reviews of Dieter's work. I've been a bit busy with
>>> an
>>> > > > important
>>> > > > > > internal project, so haven't had as much bandwidth myself.
>>> Once those
>>> > > > final
>>> > > > > > issues are merged, I'm happy to do another release
>>> > > > > >
>>> > > > > > On Sat, Jul 27, 2024 at 10:26 AM 张铎(Duo Zhang) <
>>> > > palomino219@gmail.com>
>>> > > > > > wrote:
>>> > > > > >
>>> > > > > > > Are we ready to make the 2.6.1 release now?
>>> > > > > > >
>>> > > > > > > Are there any remaining issues that need to be addressed?
>>> > > > > > >
>>> > > > > > > Thanks.
>>> > > > > > >
>>> > > > > > > Dieter De Paepe <di...@ngdata.com.invalid> 于2024年7月2日周二
>>> > > 22:53写道：
>>> > > > > > > >
>>> > > > > > > > I found 2 more possible causes for data loss when using
>>> multiple
>>> > > > backup
>>> > > > > > > roots. These should also be included in the 2.6.1 in my
>>> opinion.
>>> > > > > > > Unfortunately, I only had time to fix one of them:
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > * HBASE-28705 (PR available)
>>> > > > > > > >
>>> > > > > > > > * HBASE-28706
>>> > > > > > > >
>>> > > > > > > > Regards,
>>> > > > > > > > Dieter
>>> > > > > > > > ________________________________
>>> > > > > > > > From: Dieter De Paepe <di...@ngdata.com.INVALID>
>>> > > > > > > > Sent: 10 June 2024 15:29
>>> > > > > > > > To: dev@hbase.apache.org <de...@hbase.apache.org>
>>> > > > > > > > Subject: Re: [DISCUSS] Blockers for 2.6.1
>>> > > > > > > >
>>> > > > > > > > Hi Bryan,
>>> > > > > > > >
>>> > > > > > > > I suggest to add HBASE-28389, and possibly HBASE-28103 to
>>> the
>>> > > list
>>> > > > as
>>> > > > > > > well, as they both cause some kind of broken functionality.
>>> > > > > > > >
>>> > > > > > > > Regards,
>>> > > > > > > > Dieter
>>> > > > > > > > ________________________________
>>> > > > > > > > From: Bryan Beaudreault <bb...@apache.org>
>>> > > > > > > > Sent: 05 June 2024 21:34
>>> > > > > > > > To: HBase Dev List <de...@hbase.apache.org>
>>> > > > > > > > Subject: [DISCUSS] Blockers for 2.6.1
>>> > > > > > > >
>>> > > > > > > > Hey all,
>>> > > > > > > >
>>> > > > > > > > It's been 2 weeks since 2.6.0 was released. As discussed
>>> in the
>>> > > > vote
>>> > > > > > > > thread, there were a few outstanding backup-related
>>> issues. I
>>> > > > believe
>>> > > > > > > we've
>>> > > > > > > > made some progress on some of those.
>>> > > > > > > >
>>> > > > > > > > I'd like to start compiling a list of important
>>> backup-related
>>> > > > fixes to
>>> > > > > > > > target for the 2.6.1 release so that we can track
>>> progress. Can
>>> > > > those of
>>> > > > > > > > you who are involved (Ray Mattingly, Nick Dimiduck, Dieter
>>> De
>>> > > > Paepe &
>>> > > > > > > team,
>>> > > > > > > > and any others) please list any important jiras here?
>>> > > > > > > >
>>> > > > > > > > With a list of jiras in hand, we can check to make sure
>>> blockers
>>> > > &
>>> > > > > > > > fixVersions are set and use that to track what we need to
>>> drill
>>> > > > down
>>> > > > > > > before
>>> > > > > > > > releasing.
>>> > > > > > > >
>>> > > > > > > > Here's what I know of so far, let me know if I'm missing
>>> > > anything:
>>> > > > > > > >
>>> > > > > > > > Not yet started:
>>> > > > > > > > - HBASE-28084: incremental backups should be forbidden
>>> after
>>> > > > deleting
>>> > > > > > > > backups
>>> > > > > > > > - HBASE-28602: Incremental backup fails when WALs move
>>> > > > > > > > - HBASE-28462: (similar to ^, but in a different part of
>>> the
>>> > > code)
>>> > > > > > > > - HBASE-28538: BackupHFileCleaner is very expensive
>>> > > > > > > >
>>> > > > > > > > Patch available:
>>> > > > > > > > - HBASE-28539: backup merging does not work when using
>>> cloud
>>> > > > storage as
>>> > > > > > > > filesystem
>>> > > > > > > > - HBASE-28562: another possible failure cause for
>>> incremental
>>> > > > backups +
>>> > > > > > > > possibly cause of overly big backup metadata
>>> > > > > > > >
>>> > > > > > > > Resolved:
>>> > > > > > > > - HBASE-28502: backed up tables are not listed correctly in
>>> > > backup
>>> > > > > > > > metadata, which causes unreliable backup validation
>>> > > > > > > > - HBASE-28568: the set of tables included in incremental
>>> backups
>>> > > > might be
>>> > > > > > > > too big
>>> > > > > > >
>>> > > >
>>> > >
>>>
>>
Re: [DISCUSS] Blockers for 2.6.1
Posted by Nick Dimiduk <nd...@apache.org>.
I've trimmed down the remaining Jiras for 2.6.1 to this list of 5. Several
of them have already merged to master and are in the process of backports.
The only one I'm considering a blocker is the backup issue. I believe that
we could use some test support on 28803 and 28850.
If you have any other fixes that are close and you'd like to see included,
please @-mention me or reply on this thread.
Next up is to review the nightlies and isolate any egregious failures.
I intend to ITBLL the tip of branch-2.6 in the next couple of days.
With luck, we'll have an RC next week.
Thanks for your support,
Nick
https://issues.apache.org/jira/issues/?jql=project%20%3D%20HBASE%20AND%20resolution%20%3D%20Unresolved%20AND%20fixVersion%20%3D%202.6.1%20ORDER%20BY%20key%20ASC
On Wed, Sep 18, 2024 at 10:50 AM Nick Dimiduk <nd...@apache.org> wrote:
> I've filed a tracking ticket for burning down towards 2.6.1. Anyone who
> would like to participate, please assign a subtask to yourself. If i've
> forgotten any steps, please add them as well.
>
> https://issues.apache.org/jira/browse/HBASE-28847
>
> Thanks,
> Nick
>
> On Sun, Sep 15, 2024 at 4:21 AM 张铎(Duo Zhang) <pa...@gmail.com>
> wrote:
>
>> Thanks Bryan and Nick.
>>
>> I could also help if needed.
>>
>> Thanks.
>>
>> Nick Dimiduk <nd...@apache.org> 于2024年9月15日周日 01:14写道：
>> >
>> > I agree — we’re due.
>> >
>> > On Sat, 14 Sep 2024 at 17:05, Bryan Beaudreault <
>> bbeaudreault@apache.org>
>> > wrote:
>> >
>> > > Sorry for the delay, I've been too busy with other things and expect
>> that
>> > > to continue for a while. I’ve spoken to Nick Dimiduk and he’s going
>> to take
>> > > over as RM for 2.6. I expect next week he’ll be able to get a sense
>> of any
>> > > outstanding blockers and set a date for the release.
>> > >
>> > > On Sat, Sep 14, 2024 at 10:47 AM 张铎(Duo Zhang) <palomino219@gmail.com
>> >
>> > > wrote:
>> > >
>> > > > Bump.
>> > > >
>> > > > Are we ready for the 2.6.1 release now? We have delayed too much
>> till
>> > > > now...
>> > > >
>> > > > 张铎(Duo Zhang) <pa...@gmail.com> 于2024年8月5日周一 22:20写道：
>> > > > >
>> > > > > Thanks Bryan!
>> > > > >
>> > > > > Bryan Beaudreault <bb...@apache.org> 于2024年8月5日周一 20:59写道：
>> > > > > >
>> > > > > > Ray is out on holiday right now, and should be returning next
>> week.
>> > > I'm
>> > > > > > waiting for him and the team to give an update on their
>> > > backups-related
>> > > > > > fixes and reviews of Dieter's work. I've been a bit busy with an
>> > > > important
>> > > > > > internal project, so haven't had as much bandwidth myself. Once
>> those
>> > > > final
>> > > > > > issues are merged, I'm happy to do another release
>> > > > > >
>> > > > > > On Sat, Jul 27, 2024 at 10:26 AM 张铎(Duo Zhang) <
>> > > palomino219@gmail.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Are we ready to make the 2.6.1 release now?
>> > > > > > >
>> > > > > > > Are there any remaining issues that need to be addressed?
>> > > > > > >
>> > > > > > > Thanks.
>> > > > > > >
>> > > > > > > Dieter De Paepe <di...@ngdata.com.invalid> 于2024年7月2日周二
>> > > 22:53写道：
>> > > > > > > >
>> > > > > > > > I found 2 more possible causes for data loss when using
>> multiple
>> > > > backup
>> > > > > > > roots. These should also be included in the 2.6.1 in my
>> opinion.
>> > > > > > > Unfortunately, I only had time to fix one of them:
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > * HBASE-28705 (PR available)
>> > > > > > > >
>> > > > > > > > * HBASE-28706
>> > > > > > > >
>> > > > > > > > Regards,
>> > > > > > > > Dieter
>> > > > > > > > ________________________________
>> > > > > > > > From: Dieter De Paepe <di...@ngdata.com.INVALID>
>> > > > > > > > Sent: 10 June 2024 15:29
>> > > > > > > > To: dev@hbase.apache.org <de...@hbase.apache.org>
>> > > > > > > > Subject: Re: [DISCUSS] Blockers for 2.6.1
>> > > > > > > >
>> > > > > > > > Hi Bryan,
>> > > > > > > >
>> > > > > > > > I suggest to add HBASE-28389, and possibly HBASE-28103 to
>> the
>> > > list
>> > > > as
>> > > > > > > well, as they both cause some kind of broken functionality.
>> > > > > > > >
>> > > > > > > > Regards,
>> > > > > > > > Dieter
>> > > > > > > > ________________________________
>> > > > > > > > From: Bryan Beaudreault <bb...@apache.org>
>> > > > > > > > Sent: 05 June 2024 21:34
>> > > > > > > > To: HBase Dev List <de...@hbase.apache.org>
>> > > > > > > > Subject: [DISCUSS] Blockers for 2.6.1
>> > > > > > > >
>> > > > > > > > Hey all,
>> > > > > > > >
>> > > > > > > > It's been 2 weeks since 2.6.0 was released. As discussed in
>> the
>> > > > vote
>> > > > > > > > thread, there were a few outstanding backup-related issues.
>> I
>> > > > believe
>> > > > > > > we've
>> > > > > > > > made some progress on some of those.
>> > > > > > > >
>> > > > > > > > I'd like to start compiling a list of important
>> backup-related
>> > > > fixes to
>> > > > > > > > target for the 2.6.1 release so that we can track progress.
>> Can
>> > > > those of
>> > > > > > > > you who are involved (Ray Mattingly, Nick Dimiduck, Dieter
>> De
>> > > > Paepe &
>> > > > > > > team,
>> > > > > > > > and any others) please list any important jiras here?
>> > > > > > > >
>> > > > > > > > With a list of jiras in hand, we can check to make sure
>> blockers
>> > > &
>> > > > > > > > fixVersions are set and use that to track what we need to
>> drill
>> > > > down
>> > > > > > > before
>> > > > > > > > releasing.
>> > > > > > > >
>> > > > > > > > Here's what I know of so far, let me know if I'm missing
>> > > anything:
>> > > > > > > >
>> > > > > > > > Not yet started:
>> > > > > > > > - HBASE-28084: incremental backups should be forbidden after
>> > > > deleting
>> > > > > > > > backups
>> > > > > > > > - HBASE-28602: Incremental backup fails when WALs move
>> > > > > > > > - HBASE-28462: (similar to ^, but in a different part of the
>> > > code)
>> > > > > > > > - HBASE-28538: BackupHFileCleaner is very expensive
>> > > > > > > >
>> > > > > > > > Patch available:
>> > > > > > > > - HBASE-28539: backup merging does not work when using cloud
>> > > > storage as
>> > > > > > > > filesystem
>> > > > > > > > - HBASE-28562: another possible failure cause for
>> incremental
>> > > > backups +
>> > > > > > > > possibly cause of overly big backup metadata
>> > > > > > > >
>> > > > > > > > Resolved:
>> > > > > > > > - HBASE-28502: backed up tables are not listed correctly in
>> > > backup
>> > > > > > > > metadata, which causes unreliable backup validation
>> > > > > > > > - HBASE-28568: the set of tables included in incremental
>> backups
>> > > > might be
>> > > > > > > > too big
>> > > > > > >
>> > > >
>> > >
>>
>
Re: [DISCUSS] Blockers for 2.6.1
Posted by Nick Dimiduk <nd...@apache.org>.
I've filed a tracking ticket for burning down towards 2.6.1. Anyone who
would like to participate, please assign a subtask to yourself. If i've
forgotten any steps, please add them as well.
https://issues.apache.org/jira/browse/HBASE-28847
Thanks,
Nick
On Sun, Sep 15, 2024 at 4:21 AM 张铎(Duo Zhang) <pa...@gmail.com> wrote:
> Thanks Bryan and Nick.
>
> I could also help if needed.
>
> Thanks.
>
> Nick Dimiduk <nd...@apache.org> 于2024年9月15日周日 01:14写道：
> >
> > I agree — we’re due.
> >
> > On Sat, 14 Sep 2024 at 17:05, Bryan Beaudreault <bbeaudreault@apache.org
> >
> > wrote:
> >
> > > Sorry for the delay, I've been too busy with other things and expect
> that
> > > to continue for a while. I’ve spoken to Nick Dimiduk and he’s going to
> take
> > > over as RM for 2.6. I expect next week he’ll be able to get a sense of
> any
> > > outstanding blockers and set a date for the release.
> > >
> > > On Sat, Sep 14, 2024 at 10:47 AM 张铎(Duo Zhang) <pa...@gmail.com>
> > > wrote:
> > >
> > > > Bump.
> > > >
> > > > Are we ready for the 2.6.1 release now? We have delayed too much till
> > > > now...
> > > >
> > > > 张铎(Duo Zhang) <pa...@gmail.com> 于2024年8月5日周一 22:20写道：
> > > > >
> > > > > Thanks Bryan!
> > > > >
> > > > > Bryan Beaudreault <bb...@apache.org> 于2024年8月5日周一 20:59写道：
> > > > > >
> > > > > > Ray is out on holiday right now, and should be returning next
> week.
> > > I'm
> > > > > > waiting for him and the team to give an update on their
> > > backups-related
> > > > > > fixes and reviews of Dieter's work. I've been a bit busy with an
> > > > important
> > > > > > internal project, so haven't had as much bandwidth myself. Once
> those
> > > > final
> > > > > > issues are merged, I'm happy to do another release
> > > > > >
> > > > > > On Sat, Jul 27, 2024 at 10:26 AM 张铎(Duo Zhang) <
> > > palomino219@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Are we ready to make the 2.6.1 release now?
> > > > > > >
> > > > > > > Are there any remaining issues that need to be addressed?
> > > > > > >
> > > > > > > Thanks.
> > > > > > >
> > > > > > > Dieter De Paepe <di...@ngdata.com.invalid> 于2024年7月2日周二
> > > 22:53写道：
> > > > > > > >
> > > > > > > > I found 2 more possible causes for data loss when using
> multiple
> > > > backup
> > > > > > > roots. These should also be included in the 2.6.1 in my
> opinion.
> > > > > > > Unfortunately, I only had time to fix one of them:
> > > > > > > >
> > > > > > > >
> > > > > > > > * HBASE-28705 (PR available)
> > > > > > > >
> > > > > > > > * HBASE-28706
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > Dieter
> > > > > > > > ________________________________
> > > > > > > > From: Dieter De Paepe <di...@ngdata.com.INVALID>
> > > > > > > > Sent: 10 June 2024 15:29
> > > > > > > > To: dev@hbase.apache.org <de...@hbase.apache.org>
> > > > > > > > Subject: Re: [DISCUSS] Blockers for 2.6.1
> > > > > > > >
> > > > > > > > Hi Bryan,
> > > > > > > >
> > > > > > > > I suggest to add HBASE-28389, and possibly HBASE-28103 to the
> > > list
> > > > as
> > > > > > > well, as they both cause some kind of broken functionality.
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > Dieter
> > > > > > > > ________________________________
> > > > > > > > From: Bryan Beaudreault <bb...@apache.org>
> > > > > > > > Sent: 05 June 2024 21:34
> > > > > > > > To: HBase Dev List <de...@hbase.apache.org>
> > > > > > > > Subject: [DISCUSS] Blockers for 2.6.1
> > > > > > > >
> > > > > > > > Hey all,
> > > > > > > >
> > > > > > > > It's been 2 weeks since 2.6.0 was released. As discussed in
> the
> > > > vote
> > > > > > > > thread, there were a few outstanding backup-related issues. I
> > > > believe
> > > > > > > we've
> > > > > > > > made some progress on some of those.
> > > > > > > >
> > > > > > > > I'd like to start compiling a list of important
> backup-related
> > > > fixes to
> > > > > > > > target for the 2.6.1 release so that we can track progress.
> Can
> > > > those of
> > > > > > > > you who are involved (Ray Mattingly, Nick Dimiduck, Dieter De
> > > > Paepe &
> > > > > > > team,
> > > > > > > > and any others) please list any important jiras here?
> > > > > > > >
> > > > > > > > With a list of jiras in hand, we can check to make sure
> blockers
> > > &
> > > > > > > > fixVersions are set and use that to track what we need to
> drill
> > > > down
> > > > > > > before
> > > > > > > > releasing.
> > > > > > > >
> > > > > > > > Here's what I know of so far, let me know if I'm missing
> > > anything:
> > > > > > > >
> > > > > > > > Not yet started:
> > > > > > > > - HBASE-28084: incremental backups should be forbidden after
> > > > deleting
> > > > > > > > backups
> > > > > > > > - HBASE-28602: Incremental backup fails when WALs move
> > > > > > > > - HBASE-28462: (similar to ^, but in a different part of the
> > > code)
> > > > > > > > - HBASE-28538: BackupHFileCleaner is very expensive
> > > > > > > >
> > > > > > > > Patch available:
> > > > > > > > - HBASE-28539: backup merging does not work when using cloud
> > > > storage as
> > > > > > > > filesystem
> > > > > > > > - HBASE-28562: another possible failure cause for incremental
> > > > backups +
> > > > > > > > possibly cause of overly big backup metadata
> > > > > > > >
> > > > > > > > Resolved:
> > > > > > > > - HBASE-28502: backed up tables are not listed correctly in
> > > backup
> > > > > > > > metadata, which causes unreliable backup validation
> > > > > > > > - HBASE-28568: the set of tables included in incremental
> backups
> > > > might be
> > > > > > > > too big
> > > > > > >
> > > >
> > >
>
Re: [DISCUSS] Blockers for 2.6.1
Posted by "张铎(Duo Zhang)" <pa...@gmail.com>.
Thanks Bryan and Nick.
I could also help if needed.
Thanks.
Nick Dimiduk <nd...@apache.org> 于2024年9月15日周日 01:14写道：
>
> I agree — we’re due.
>
> On Sat, 14 Sep 2024 at 17:05, Bryan Beaudreault <bb...@apache.org>
> wrote:
>
> > Sorry for the delay, I've been too busy with other things and expect that
> > to continue for a while. I’ve spoken to Nick Dimiduk and he’s going to take
> > over as RM for 2.6. I expect next week he’ll be able to get a sense of any
> > outstanding blockers and set a date for the release.
> >
> > On Sat, Sep 14, 2024 at 10:47 AM 张铎(Duo Zhang) <pa...@gmail.com>
> > wrote:
> >
> > > Bump.
> > >
> > > Are we ready for the 2.6.1 release now? We have delayed too much till
> > > now...
> > >
> > > 张铎(Duo Zhang) <pa...@gmail.com> 于2024年8月5日周一 22:20写道：
> > > >
> > > > Thanks Bryan!
> > > >
> > > > Bryan Beaudreault <bb...@apache.org> 于2024年8月5日周一 20:59写道：
> > > > >
> > > > > Ray is out on holiday right now, and should be returning next week.
> > I'm
> > > > > waiting for him and the team to give an update on their
> > backups-related
> > > > > fixes and reviews of Dieter's work. I've been a bit busy with an
> > > important
> > > > > internal project, so haven't had as much bandwidth myself. Once those
> > > final
> > > > > issues are merged, I'm happy to do another release
> > > > >
> > > > > On Sat, Jul 27, 2024 at 10:26 AM 张铎(Duo Zhang) <
> > palomino219@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Are we ready to make the 2.6.1 release now?
> > > > > >
> > > > > > Are there any remaining issues that need to be addressed?
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > Dieter De Paepe <di...@ngdata.com.invalid> 于2024年7月2日周二
> > 22:53写道：
> > > > > > >
> > > > > > > I found 2 more possible causes for data loss when using multiple
> > > backup
> > > > > > roots. These should also be included in the 2.6.1 in my opinion.
> > > > > > Unfortunately, I only had time to fix one of them:
> > > > > > >
> > > > > > >
> > > > > > > * HBASE-28705 (PR available)
> > > > > > >
> > > > > > > * HBASE-28706
> > > > > > >
> > > > > > > Regards,
> > > > > > > Dieter
> > > > > > > ________________________________
> > > > > > > From: Dieter De Paepe <di...@ngdata.com.INVALID>
> > > > > > > Sent: 10 June 2024 15:29
> > > > > > > To: dev@hbase.apache.org <de...@hbase.apache.org>
> > > > > > > Subject: Re: [DISCUSS] Blockers for 2.6.1
> > > > > > >
> > > > > > > Hi Bryan,
> > > > > > >
> > > > > > > I suggest to add HBASE-28389, and possibly HBASE-28103 to the
> > list
> > > as
> > > > > > well, as they both cause some kind of broken functionality.
> > > > > > >
> > > > > > > Regards,
> > > > > > > Dieter
> > > > > > > ________________________________
> > > > > > > From: Bryan Beaudreault <bb...@apache.org>
> > > > > > > Sent: 05 June 2024 21:34
> > > > > > > To: HBase Dev List <de...@hbase.apache.org>
> > > > > > > Subject: [DISCUSS] Blockers for 2.6.1
> > > > > > >
> > > > > > > Hey all,
> > > > > > >
> > > > > > > It's been 2 weeks since 2.6.0 was released. As discussed in the
> > > vote
> > > > > > > thread, there were a few outstanding backup-related issues. I
> > > believe
> > > > > > we've
> > > > > > > made some progress on some of those.
> > > > > > >
> > > > > > > I'd like to start compiling a list of important backup-related
> > > fixes to
> > > > > > > target for the 2.6.1 release so that we can track progress. Can
> > > those of
> > > > > > > you who are involved (Ray Mattingly, Nick Dimiduck, Dieter De
> > > Paepe &
> > > > > > team,
> > > > > > > and any others) please list any important jiras here?
> > > > > > >
> > > > > > > With a list of jiras in hand, we can check to make sure blockers
> > &
> > > > > > > fixVersions are set and use that to track what we need to drill
> > > down
> > > > > > before
> > > > > > > releasing.
> > > > > > >
> > > > > > > Here's what I know of so far, let me know if I'm missing
> > anything:
> > > > > > >
> > > > > > > Not yet started:
> > > > > > > - HBASE-28084: incremental backups should be forbidden after
> > > deleting
> > > > > > > backups
> > > > > > > - HBASE-28602: Incremental backup fails when WALs move
> > > > > > > - HBASE-28462: (similar to ^, but in a different part of the
> > code)
> > > > > > > - HBASE-28538: BackupHFileCleaner is very expensive
> > > > > > >
> > > > > > > Patch available:
> > > > > > > - HBASE-28539: backup merging does not work when using cloud
> > > storage as
> > > > > > > filesystem
> > > > > > > - HBASE-28562: another possible failure cause for incremental
> > > backups +
> > > > > > > possibly cause of overly big backup metadata
> > > > > > >
> > > > > > > Resolved:
> > > > > > > - HBASE-28502: backed up tables are not listed correctly in
> > backup
> > > > > > > metadata, which causes unreliable backup validation
> > > > > > > - HBASE-28568: the set of tables included in incremental backups
> > > might be
> > > > > > > too big
> > > > > >
> > >
> >
Re: [DISCUSS] Blockers for 2.6.1
Posted by Nick Dimiduk <nd...@apache.org>.
I agree — we’re due.
On Sat, 14 Sep 2024 at 17:05, Bryan Beaudreault <bb...@apache.org>
wrote:
> Sorry for the delay, I've been too busy with other things and expect that
> to continue for a while. I’ve spoken to Nick Dimiduk and he’s going to take
> over as RM for 2.6. I expect next week he’ll be able to get a sense of any
> outstanding blockers and set a date for the release.
>
> On Sat, Sep 14, 2024 at 10:47 AM 张铎(Duo Zhang) <pa...@gmail.com>
> wrote:
>
> > Bump.
> >
> > Are we ready for the 2.6.1 release now? We have delayed too much till
> > now...
> >
> > 张铎(Duo Zhang) <pa...@gmail.com> 于2024年8月5日周一 22:20写道：
> > >
> > > Thanks Bryan!
> > >
> > > Bryan Beaudreault <bb...@apache.org> 于2024年8月5日周一 20:59写道：
> > > >
> > > > Ray is out on holiday right now, and should be returning next week.
> I'm
> > > > waiting for him and the team to give an update on their
> backups-related
> > > > fixes and reviews of Dieter's work. I've been a bit busy with an
> > important
> > > > internal project, so haven't had as much bandwidth myself. Once those
> > final
> > > > issues are merged, I'm happy to do another release
> > > >
> > > > On Sat, Jul 27, 2024 at 10:26 AM 张铎(Duo Zhang) <
> palomino219@gmail.com>
> > > > wrote:
> > > >
> > > > > Are we ready to make the 2.6.1 release now?
> > > > >
> > > > > Are there any remaining issues that need to be addressed?
> > > > >
> > > > > Thanks.
> > > > >
> > > > > Dieter De Paepe <di...@ngdata.com.invalid> 于2024年7月2日周二
> 22:53写道：
> > > > > >
> > > > > > I found 2 more possible causes for data loss when using multiple
> > backup
> > > > > roots. These should also be included in the 2.6.1 in my opinion.
> > > > > Unfortunately, I only had time to fix one of them:
> > > > > >
> > > > > >
> > > > > > * HBASE-28705 (PR available)
> > > > > >
> > > > > > * HBASE-28706
> > > > > >
> > > > > > Regards,
> > > > > > Dieter
> > > > > > ________________________________
> > > > > > From: Dieter De Paepe <di...@ngdata.com.INVALID>
> > > > > > Sent: 10 June 2024 15:29
> > > > > > To: dev@hbase.apache.org <de...@hbase.apache.org>
> > > > > > Subject: Re: [DISCUSS] Blockers for 2.6.1
> > > > > >
> > > > > > Hi Bryan,
> > > > > >
> > > > > > I suggest to add HBASE-28389, and possibly HBASE-28103 to the
> list
> > as
> > > > > well, as they both cause some kind of broken functionality.
> > > > > >
> > > > > > Regards,
> > > > > > Dieter
> > > > > > ________________________________
> > > > > > From: Bryan Beaudreault <bb...@apache.org>
> > > > > > Sent: 05 June 2024 21:34
> > > > > > To: HBase Dev List <de...@hbase.apache.org>
> > > > > > Subject: [DISCUSS] Blockers for 2.6.1
> > > > > >
> > > > > > Hey all,
> > > > > >
> > > > > > It's been 2 weeks since 2.6.0 was released. As discussed in the
> > vote
> > > > > > thread, there were a few outstanding backup-related issues. I
> > believe
> > > > > we've
> > > > > > made some progress on some of those.
> > > > > >
> > > > > > I'd like to start compiling a list of important backup-related
> > fixes to
> > > > > > target for the 2.6.1 release so that we can track progress. Can
> > those of
> > > > > > you who are involved (Ray Mattingly, Nick Dimiduck, Dieter De
> > Paepe &
> > > > > team,
> > > > > > and any others) please list any important jiras here?
> > > > > >
> > > > > > With a list of jiras in hand, we can check to make sure blockers
> &
> > > > > > fixVersions are set and use that to track what we need to drill
> > down
> > > > > before
> > > > > > releasing.
> > > > > >
> > > > > > Here's what I know of so far, let me know if I'm missing
> anything:
> > > > > >
> > > > > > Not yet started:
> > > > > > - HBASE-28084: incremental backups should be forbidden after
> > deleting
> > > > > > backups
> > > > > > - HBASE-28602: Incremental backup fails when WALs move
> > > > > > - HBASE-28462: (similar to ^, but in a different part of the
> code)
> > > > > > - HBASE-28538: BackupHFileCleaner is very expensive
> > > > > >
> > > > > > Patch available:
> > > > > > - HBASE-28539: backup merging does not work when using cloud
> > storage as
> > > > > > filesystem
> > > > > > - HBASE-28562: another possible failure cause for incremental
> > backups +
> > > > > > possibly cause of overly big backup metadata
> > > > > >
> > > > > > Resolved:
> > > > > > - HBASE-28502: backed up tables are not listed correctly in
> backup
> > > > > > metadata, which causes unreliable backup validation
> > > > > > - HBASE-28568: the set of tables included in incremental backups
> > might be
> > > > > > too big
> > > > >
> >
>
Re: [DISCUSS] Blockers for 2.6.1
Posted by Bryan Beaudreault <bb...@apache.org>.
Sorry for the delay, I've been too busy with other things and expect that
to continue for a while. I’ve spoken to Nick Dimiduk and he’s going to take
over as RM for 2.6. I expect next week he’ll be able to get a sense of any
outstanding blockers and set a date for the release.
On Sat, Sep 14, 2024 at 10:47 AM 张铎(Duo Zhang) <pa...@gmail.com>
wrote:
> Bump.
>
> Are we ready for the 2.6.1 release now? We have delayed too much till
> now...
>
> 张铎(Duo Zhang) <pa...@gmail.com> 于2024年8月5日周一 22:20写道：
> >
> > Thanks Bryan!
> >
> > Bryan Beaudreault <bb...@apache.org> 于2024年8月5日周一 20:59写道：
> > >
> > > Ray is out on holiday right now, and should be returning next week. I'm
> > > waiting for him and the team to give an update on their backups-related
> > > fixes and reviews of Dieter's work. I've been a bit busy with an
> important
> > > internal project, so haven't had as much bandwidth myself. Once those
> final
> > > issues are merged, I'm happy to do another release
> > >
> > > On Sat, Jul 27, 2024 at 10:26 AM 张铎(Duo Zhang) <pa...@gmail.com>
> > > wrote:
> > >
> > > > Are we ready to make the 2.6.1 release now?
> > > >
> > > > Are there any remaining issues that need to be addressed?
> > > >
> > > > Thanks.
> > > >
> > > > Dieter De Paepe <di...@ngdata.com.invalid> 于2024年7月2日周二 22:53写道：
> > > > >
> > > > > I found 2 more possible causes for data loss when using multiple
> backup
> > > > roots. These should also be included in the 2.6.1 in my opinion.
> > > > Unfortunately, I only had time to fix one of them:
> > > > >
> > > > >
> > > > > * HBASE-28705 (PR available)
> > > > >
> > > > > * HBASE-28706
> > > > >
> > > > > Regards,
> > > > > Dieter
> > > > > ________________________________
> > > > > From: Dieter De Paepe <di...@ngdata.com.INVALID>
> > > > > Sent: 10 June 2024 15:29
> > > > > To: dev@hbase.apache.org <de...@hbase.apache.org>
> > > > > Subject: Re: [DISCUSS] Blockers for 2.6.1
> > > > >
> > > > > Hi Bryan,
> > > > >
> > > > > I suggest to add HBASE-28389, and possibly HBASE-28103 to the list
> as
> > > > well, as they both cause some kind of broken functionality.
> > > > >
> > > > > Regards,
> > > > > Dieter
> > > > > ________________________________
> > > > > From: Bryan Beaudreault <bb...@apache.org>
> > > > > Sent: 05 June 2024 21:34
> > > > > To: HBase Dev List <de...@hbase.apache.org>
> > > > > Subject: [DISCUSS] Blockers for 2.6.1
> > > > >
> > > > > Hey all,
> > > > >
> > > > > It's been 2 weeks since 2.6.0 was released. As discussed in the
> vote
> > > > > thread, there were a few outstanding backup-related issues. I
> believe
> > > > we've
> > > > > made some progress on some of those.
> > > > >
> > > > > I'd like to start compiling a list of important backup-related
> fixes to
> > > > > target for the 2.6.1 release so that we can track progress. Can
> those of
> > > > > you who are involved (Ray Mattingly, Nick Dimiduck, Dieter De
> Paepe &
> > > > team,
> > > > > and any others) please list any important jiras here?
> > > > >
> > > > > With a list of jiras in hand, we can check to make sure blockers &
> > > > > fixVersions are set and use that to track what we need to drill
> down
> > > > before
> > > > > releasing.
> > > > >
> > > > > Here's what I know of so far, let me know if I'm missing anything:
> > > > >
> > > > > Not yet started:
> > > > > - HBASE-28084: incremental backups should be forbidden after
> deleting
> > > > > backups
> > > > > - HBASE-28602: Incremental backup fails when WALs move
> > > > > - HBASE-28462: (similar to ^, but in a different part of the code)
> > > > > - HBASE-28538: BackupHFileCleaner is very expensive
> > > > >
> > > > > Patch available:
> > > > > - HBASE-28539: backup merging does not work when using cloud
> storage as
> > > > > filesystem
> > > > > - HBASE-28562: another possible failure cause for incremental
> backups +
> > > > > possibly cause of overly big backup metadata
> > > > >
> > > > > Resolved:
> > > > > - HBASE-28502: backed up tables are not listed correctly in backup
> > > > > metadata, which causes unreliable backup validation
> > > > > - HBASE-28568: the set of tables included in incremental backups
> might be
> > > > > too big
> > > >
>
Re: [DISCUSS] Blockers for 2.6.1
Posted by "张铎(Duo Zhang)" <pa...@gmail.com>.
Bump.
Are we ready for the 2.6.1 release now? We have delayed too much till now...
张铎(Duo Zhang) <pa...@gmail.com> 于2024年8月5日周一 22:20写道：
>
> Thanks Bryan!
>
> Bryan Beaudreault <bb...@apache.org> 于2024年8月5日周一 20:59写道：
> >
> > Ray is out on holiday right now, and should be returning next week. I'm
> > waiting for him and the team to give an update on their backups-related
> > fixes and reviews of Dieter's work. I've been a bit busy with an important
> > internal project, so haven't had as much bandwidth myself. Once those final
> > issues are merged, I'm happy to do another release
> >
> > On Sat, Jul 27, 2024 at 10:26 AM 张铎(Duo Zhang) <pa...@gmail.com>
> > wrote:
> >
> > > Are we ready to make the 2.6.1 release now?
> > >
> > > Are there any remaining issues that need to be addressed?
> > >
> > > Thanks.
> > >
> > > Dieter De Paepe <di...@ngdata.com.invalid> 于2024年7月2日周二 22:53写道：
> > > >
> > > > I found 2 more possible causes for data loss when using multiple backup
> > > roots. These should also be included in the 2.6.1 in my opinion.
> > > Unfortunately, I only had time to fix one of them:
> > > >
> > > >
> > > > * HBASE-28705 (PR available)
> > > >
> > > > * HBASE-28706
> > > >
> > > > Regards,
> > > > Dieter
> > > > ________________________________
> > > > From: Dieter De Paepe <di...@ngdata.com.INVALID>
> > > > Sent: 10 June 2024 15:29
> > > > To: dev@hbase.apache.org <de...@hbase.apache.org>
> > > > Subject: Re: [DISCUSS] Blockers for 2.6.1
> > > >
> > > > Hi Bryan,
> > > >
> > > > I suggest to add HBASE-28389, and possibly HBASE-28103 to the list as
> > > well, as they both cause some kind of broken functionality.
> > > >
> > > > Regards,
> > > > Dieter
> > > > ________________________________
> > > > From: Bryan Beaudreault <bb...@apache.org>
> > > > Sent: 05 June 2024 21:34
> > > > To: HBase Dev List <de...@hbase.apache.org>
> > > > Subject: [DISCUSS] Blockers for 2.6.1
> > > >
> > > > Hey all,
> > > >
> > > > It's been 2 weeks since 2.6.0 was released. As discussed in the vote
> > > > thread, there were a few outstanding backup-related issues. I believe
> > > we've
> > > > made some progress on some of those.
> > > >
> > > > I'd like to start compiling a list of important backup-related fixes to
> > > > target for the 2.6.1 release so that we can track progress. Can those of
> > > > you who are involved (Ray Mattingly, Nick Dimiduck, Dieter De Paepe &
> > > team,
> > > > and any others) please list any important jiras here?
> > > >
> > > > With a list of jiras in hand, we can check to make sure blockers &
> > > > fixVersions are set and use that to track what we need to drill down
> > > before
> > > > releasing.
> > > >
> > > > Here's what I know of so far, let me know if I'm missing anything:
> > > >
> > > > Not yet started:
> > > > - HBASE-28084: incremental backups should be forbidden after deleting
> > > > backups
> > > > - HBASE-28602: Incremental backup fails when WALs move
> > > > - HBASE-28462: (similar to ^, but in a different part of the code)
> > > > - HBASE-28538: BackupHFileCleaner is very expensive
> > > >
> > > > Patch available:
> > > > - HBASE-28539: backup merging does not work when using cloud storage as
> > > > filesystem
> > > > - HBASE-28562: another possible failure cause for incremental backups +
> > > > possibly cause of overly big backup metadata
> > > >
> > > > Resolved:
> > > > - HBASE-28502: backed up tables are not listed correctly in backup
> > > > metadata, which causes unreliable backup validation
> > > > - HBASE-28568: the set of tables included in incremental backups might be
> > > > too big
> > >
Re: [DISCUSS] Blockers for 2.6.1
Posted by "张铎(Duo Zhang)" <pa...@gmail.com>.
Thanks Bryan!
Bryan Beaudreault <bb...@apache.org> 于2024年8月5日周一 20:59写道：
>
> Ray is out on holiday right now, and should be returning next week. I'm
> waiting for him and the team to give an update on their backups-related
> fixes and reviews of Dieter's work. I've been a bit busy with an important
> internal project, so haven't had as much bandwidth myself. Once those final
> issues are merged, I'm happy to do another release
>
> On Sat, Jul 27, 2024 at 10:26 AM 张铎(Duo Zhang) <pa...@gmail.com>
> wrote:
>
> > Are we ready to make the 2.6.1 release now?
> >
> > Are there any remaining issues that need to be addressed?
> >
> > Thanks.
> >
> > Dieter De Paepe <di...@ngdata.com.invalid> 于2024年7月2日周二 22:53写道：
> > >
> > > I found 2 more possible causes for data loss when using multiple backup
> > roots. These should also be included in the 2.6.1 in my opinion.
> > Unfortunately, I only had time to fix one of them:
> > >
> > >
> > > * HBASE-28705 (PR available)
> > >
> > > * HBASE-28706
> > >
> > > Regards,
> > > Dieter
> > > ________________________________
> > > From: Dieter De Paepe <di...@ngdata.com.INVALID>
> > > Sent: 10 June 2024 15:29
> > > To: dev@hbase.apache.org <de...@hbase.apache.org>
> > > Subject: Re: [DISCUSS] Blockers for 2.6.1
> > >
> > > Hi Bryan,
> > >
> > > I suggest to add HBASE-28389, and possibly HBASE-28103 to the list as
> > well, as they both cause some kind of broken functionality.
> > >
> > > Regards,
> > > Dieter
> > > ________________________________
> > > From: Bryan Beaudreault <bb...@apache.org>
> > > Sent: 05 June 2024 21:34
> > > To: HBase Dev List <de...@hbase.apache.org>
> > > Subject: [DISCUSS] Blockers for 2.6.1
> > >
> > > Hey all,
> > >
> > > It's been 2 weeks since 2.6.0 was released. As discussed in the vote
> > > thread, there were a few outstanding backup-related issues. I believe
> > we've
> > > made some progress on some of those.
> > >
> > > I'd like to start compiling a list of important backup-related fixes to
> > > target for the 2.6.1 release so that we can track progress. Can those of
> > > you who are involved (Ray Mattingly, Nick Dimiduck, Dieter De Paepe &
> > team,
> > > and any others) please list any important jiras here?
> > >
> > > With a list of jiras in hand, we can check to make sure blockers &
> > > fixVersions are set and use that to track what we need to drill down
> > before
> > > releasing.
> > >
> > > Here's what I know of so far, let me know if I'm missing anything:
> > >
> > > Not yet started:
> > > - HBASE-28084: incremental backups should be forbidden after deleting
> > > backups
> > > - HBASE-28602: Incremental backup fails when WALs move
> > > - HBASE-28462: (similar to ^, but in a different part of the code)
> > > - HBASE-28538: BackupHFileCleaner is very expensive
> > >
> > > Patch available:
> > > - HBASE-28539: backup merging does not work when using cloud storage as
> > > filesystem
> > > - HBASE-28562: another possible failure cause for incremental backups +
> > > possibly cause of overly big backup metadata
> > >
> > > Resolved:
> > > - HBASE-28502: backed up tables are not listed correctly in backup
> > > metadata, which causes unreliable backup validation
> > > - HBASE-28568: the set of tables included in incremental backups might be
> > > too big
> >
Re: [DISCUSS] Blockers for 2.6.1
Posted by Bryan Beaudreault <bb...@apache.org>.
Ray is out on holiday right now, and should be returning next week. I'm
waiting for him and the team to give an update on their backups-related
fixes and reviews of Dieter's work. I've been a bit busy with an important
internal project, so haven't had as much bandwidth myself. Once those final
issues are merged, I'm happy to do another release
On Sat, Jul 27, 2024 at 10:26 AM 张铎(Duo Zhang) <pa...@gmail.com>
wrote:
> Are we ready to make the 2.6.1 release now?
>
> Are there any remaining issues that need to be addressed?
>
> Thanks.
>
> Dieter De Paepe <di...@ngdata.com.invalid> 于2024年7月2日周二 22:53写道：
> >
> > I found 2 more possible causes for data loss when using multiple backup
> roots. These should also be included in the 2.6.1 in my opinion.
> Unfortunately, I only had time to fix one of them:
> >
> >
> > * HBASE-28705 (PR available)
> >
> > * HBASE-28706
> >
> > Regards,
> > Dieter
> > ________________________________
> > From: Dieter De Paepe <di...@ngdata.com.INVALID>
> > Sent: 10 June 2024 15:29
> > To: dev@hbase.apache.org <de...@hbase.apache.org>
> > Subject: Re: [DISCUSS] Blockers for 2.6.1
> >
> > Hi Bryan,
> >
> > I suggest to add HBASE-28389, and possibly HBASE-28103 to the list as
> well, as they both cause some kind of broken functionality.
> >
> > Regards,
> > Dieter
> > ________________________________
> > From: Bryan Beaudreault <bb...@apache.org>
> > Sent: 05 June 2024 21:34
> > To: HBase Dev List <de...@hbase.apache.org>
> > Subject: [DISCUSS] Blockers for 2.6.1
> >
> > Hey all,
> >
> > It's been 2 weeks since 2.6.0 was released. As discussed in the vote
> > thread, there were a few outstanding backup-related issues. I believe
> we've
> > made some progress on some of those.
> >
> > I'd like to start compiling a list of important backup-related fixes to
> > target for the 2.6.1 release so that we can track progress. Can those of
> > you who are involved (Ray Mattingly, Nick Dimiduck, Dieter De Paepe &
> team,
> > and any others) please list any important jiras here?
> >
> > With a list of jiras in hand, we can check to make sure blockers &
> > fixVersions are set and use that to track what we need to drill down
> before
> > releasing.
> >
> > Here's what I know of so far, let me know if I'm missing anything:
> >
> > Not yet started:
> > - HBASE-28084: incremental backups should be forbidden after deleting
> > backups
> > - HBASE-28602: Incremental backup fails when WALs move
> > - HBASE-28462: (similar to ^, but in a different part of the code)
> > - HBASE-28538: BackupHFileCleaner is very expensive
> >
> > Patch available:
> > - HBASE-28539: backup merging does not work when using cloud storage as
> > filesystem
> > - HBASE-28562: another possible failure cause for incremental backups +
> > possibly cause of overly big backup metadata
> >
> > Resolved:
> > - HBASE-28502: backed up tables are not listed correctly in backup
> > metadata, which causes unreliable backup validation
> > - HBASE-28568: the set of tables included in incremental backups might be
> > too big
>
Re: [DISCUSS] Blockers for 2.6.1
Posted by "张铎(Duo Zhang)" <pa...@gmail.com>.
Are we ready to make the 2.6.1 release now?
Are there any remaining issues that need to be addressed?
Thanks.
Dieter De Paepe <di...@ngdata.com.invalid> 于2024年7月2日周二 22:53写道：
>
> I found 2 more possible causes for data loss when using multiple backup roots. These should also be included in the 2.6.1 in my opinion. Unfortunately, I only had time to fix one of them:
>
>
> * HBASE-28705 (PR available)
>
> * HBASE-28706
>
> Regards,
> Dieter
> ________________________________
> From: Dieter De Paepe <di...@ngdata.com.INVALID>
> Sent: 10 June 2024 15:29
> To: dev@hbase.apache.org <de...@hbase.apache.org>
> Subject: Re: [DISCUSS] Blockers for 2.6.1
>
> Hi Bryan,
>
> I suggest to add HBASE-28389, and possibly HBASE-28103 to the list as well, as they both cause some kind of broken functionality.
>
> Regards,
> Dieter
> ________________________________
> From: Bryan Beaudreault <bb...@apache.org>
> Sent: 05 June 2024 21:34
> To: HBase Dev List <de...@hbase.apache.org>
> Subject: [DISCUSS] Blockers for 2.6.1
>
> Hey all,
>
> It's been 2 weeks since 2.6.0 was released. As discussed in the vote
> thread, there were a few outstanding backup-related issues. I believe we've
> made some progress on some of those.
>
> I'd like to start compiling a list of important backup-related fixes to
> target for the 2.6.1 release so that we can track progress. Can those of
> you who are involved (Ray Mattingly, Nick Dimiduck, Dieter De Paepe & team,
> and any others) please list any important jiras here?
>
> With a list of jiras in hand, we can check to make sure blockers &
> fixVersions are set and use that to track what we need to drill down before
> releasing.
>
> Here's what I know of so far, let me know if I'm missing anything:
>
> Not yet started:
> - HBASE-28084: incremental backups should be forbidden after deleting
> backups
> - HBASE-28602: Incremental backup fails when WALs move
> - HBASE-28462: (similar to ^, but in a different part of the code)
> - HBASE-28538: BackupHFileCleaner is very expensive
>
> Patch available:
> - HBASE-28539: backup merging does not work when using cloud storage as
> filesystem
> - HBASE-28562: another possible failure cause for incremental backups +
> possibly cause of overly big backup metadata
>
> Resolved:
> - HBASE-28502: backed up tables are not listed correctly in backup
> metadata, which causes unreliable backup validation
> - HBASE-28568: the set of tables included in incremental backups might be
> too big
Re: [DISCUSS] Blockers for 2.6.1
Posted by Dieter De Paepe <di...@ngdata.com.INVALID>.
I found 2 more possible causes for data loss when using multiple backup roots. These should also be included in the 2.6.1 in my opinion. Unfortunately, I only had time to fix one of them:
* HBASE-28705 (PR available)
* HBASE-28706
Regards,
Dieter
________________________________
From: Dieter De Paepe <di...@ngdata.com.INVALID>
Sent: 10 June 2024 15:29
To: dev@hbase.apache.org <de...@hbase.apache.org>
Subject: Re: [DISCUSS] Blockers for 2.6.1
Hi Bryan,
I suggest to add HBASE-28389, and possibly HBASE-28103 to the list as well, as they both cause some kind of broken functionality.
Regards,
Dieter
________________________________
From: Bryan Beaudreault <bb...@apache.org>
Sent: 05 June 2024 21:34
To: HBase Dev List <de...@hbase.apache.org>
Subject: [DISCUSS] Blockers for 2.6.1
Hey all,
It's been 2 weeks since 2.6.0 was released. As discussed in the vote
thread, there were a few outstanding backup-related issues. I believe we've
made some progress on some of those.
I'd like to start compiling a list of important backup-related fixes to
target for the 2.6.1 release so that we can track progress. Can those of
you who are involved (Ray Mattingly, Nick Dimiduck, Dieter De Paepe & team,
and any others) please list any important jiras here?
With a list of jiras in hand, we can check to make sure blockers &
fixVersions are set and use that to track what we need to drill down before
releasing.
Here's what I know of so far, let me know if I'm missing anything:
Not yet started:
- HBASE-28084: incremental backups should be forbidden after deleting
backups
- HBASE-28602: Incremental backup fails when WALs move
- HBASE-28462: (similar to ^, but in a different part of the code)
- HBASE-28538: BackupHFileCleaner is very expensive
Patch available:
- HBASE-28539: backup merging does not work when using cloud storage as
filesystem
- HBASE-28562: another possible failure cause for incremental backups +
possibly cause of overly big backup metadata
Resolved:
- HBASE-28502: backed up tables are not listed correctly in backup
metadata, which causes unreliable backup validation
- HBASE-28568: the set of tables included in incremental backups might be
too big
Re: [DISCUSS] Blockers for 2.6.1
Posted by Dieter De Paepe <di...@ngdata.com.INVALID>.
Hi Bryan,
I suggest to add HBASE-28389, and possibly HBASE-28103 to the list as well, as they both cause some kind of broken functionality.
Regards,
Dieter
________________________________
From: Bryan Beaudreault <bb...@apache.org>
Sent: 05 June 2024 21:34
To: HBase Dev List <de...@hbase.apache.org>
Subject: [DISCUSS] Blockers for 2.6.1
Hey all,
It's been 2 weeks since 2.6.0 was released. As discussed in the vote
thread, there were a few outstanding backup-related issues. I believe we've
made some progress on some of those.
I'd like to start compiling a list of important backup-related fixes to
target for the 2.6.1 release so that we can track progress. Can those of
you who are involved (Ray Mattingly, Nick Dimiduck, Dieter De Paepe & team,
and any others) please list any important jiras here?
With a list of jiras in hand, we can check to make sure blockers &
fixVersions are set and use that to track what we need to drill down before
releasing.
Here's what I know of so far, let me know if I'm missing anything:
Not yet started:
- HBASE-28084: incremental backups should be forbidden after deleting
backups
- HBASE-28602: Incremental backup fails when WALs move
- HBASE-28462: (similar to ^, but in a different part of the code)
- HBASE-28538: BackupHFileCleaner is very expensive
Patch available:
- HBASE-28539: backup merging does not work when using cloud storage as
filesystem
- HBASE-28562: another possible failure cause for incremental backups +
possibly cause of overly big backup metadata
Resolved:
- HBASE-28502: backed up tables are not listed correctly in backup
metadata, which causes unreliable backup validation
- HBASE-28568: the set of tables included in incremental backups might be
too big
Re: [DISCUSS] Blockers for 2.6.1
Posted by Nick Dimiduk <nd...@apache.org>.
HBASE-28539 and HBASE-28562 are merged as well.
On Thu, Jun 6, 2024 at 2:57 PM Ray Mattingly <rm...@hubspot.com.invalid>
wrote:
> I found the bug documented in `HBASE-28643: An unbounded backup failure
> message can cause an irrecoverable state for the given backup` yesterday,
> and I think we should probably aim to fix this in the 2.6.1 release.
> Otherwise that list looks good to me.
>
> On Wed, Jun 5, 2024 at 3:35 PM Bryan Beaudreault <bb...@apache.org>
> wrote:
>
> > Hey all,
> >
> > It's been 2 weeks since 2.6.0 was released. As discussed in the vote
> > thread, there were a few outstanding backup-related issues. I believe
> we've
> > made some progress on some of those.
> >
> > I'd like to start compiling a list of important backup-related fixes to
> > target for the 2.6.1 release so that we can track progress. Can those of
> > you who are involved (Ray Mattingly, Nick Dimiduck, Dieter De Paepe &
> team,
> > and any others) please list any important jiras here?
> >
> > With a list of jiras in hand, we can check to make sure blockers &
> > fixVersions are set and use that to track what we need to drill down
> before
> > releasing.
> >
> > Here's what I know of so far, let me know if I'm missing anything:
> >
> > Not yet started:
> > - HBASE-28084: incremental backups should be forbidden after deleting
> > backups
> > - HBASE-28602: Incremental backup fails when WALs move
> > - HBASE-28462: (similar to ^, but in a different part of the code)
> > - HBASE-28538: BackupHFileCleaner is very expensive
> >
> > Patch available:
> > - HBASE-28539: backup merging does not work when using cloud storage as
> > filesystem
> > - HBASE-28562: another possible failure cause for incremental backups +
> > possibly cause of overly big backup metadata
> >
> > Resolved:
> > - HBASE-28502: backed up tables are not listed correctly in backup
> > metadata, which causes unreliable backup validation
> > - HBASE-28568: the set of tables included in incremental backups might be
> > too big
> >
>
Re: [DISCUSS] Blockers for 2.6.1
Posted by Ray Mattingly <rm...@hubspot.com.INVALID>.
I found the bug documented in `HBASE-28643: An unbounded backup failure
message can cause an irrecoverable state for the given backup` yesterday,
and I think we should probably aim to fix this in the 2.6.1 release.
Otherwise that list looks good to me.
On Wed, Jun 5, 2024 at 3:35 PM Bryan Beaudreault <bb...@apache.org>
wrote:
> Hey all,
>
> It's been 2 weeks since 2.6.0 was released. As discussed in the vote
> thread, there were a few outstanding backup-related issues. I believe we've
> made some progress on some of those.
>
> I'd like to start compiling a list of important backup-related fixes to
> target for the 2.6.1 release so that we can track progress. Can those of
> you who are involved (Ray Mattingly, Nick Dimiduck, Dieter De Paepe & team,
> and any others) please list any important jiras here?
>
> With a list of jiras in hand, we can check to make sure blockers &
> fixVersions are set and use that to track what we need to drill down before
> releasing.
>
> Here's what I know of so far, let me know if I'm missing anything:
>
> Not yet started:
> - HBASE-28084: incremental backups should be forbidden after deleting
> backups
> - HBASE-28602: Incremental backup fails when WALs move
> - HBASE-28462: (similar to ^, but in a different part of the code)
> - HBASE-28538: BackupHFileCleaner is very expensive
>
> Patch available:
> - HBASE-28539: backup merging does not work when using cloud storage as
> filesystem
> - HBASE-28562: another possible failure cause for incremental backups +
> possibly cause of overly big backup metadata
>
> Resolved:
> - HBASE-28502: backed up tables are not listed correctly in backup
> metadata, which causes unreliable backup validation
> - HBASE-28568: the set of tables included in incremental backups might be
> too big
>