You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Bryan Beaudreault <bb...@hubspot.com.INVALID> on 2022/10/03 18:03:21 UTC
[DISCUSS] Backporting hbase-backup module to branch-2
Hi again all,
We have a relatively full featured backup solution in master branch. It
looks like the original development had intended to be included in branch-2
[1], but did not make the deadline for 2.0.0 release and was removed [2].
Later the idea of backporting was forgotten, potentially with some of the
main devs moving onto other projects.
In the interim, one company, Flipkart (Mallikarjun works there), took it
upon themselves to backport the feature to their own fork. They've been
running that backport in production for some time now. Mallikarjun has been
trying to contribute some improvements, but has lacked committer support.
At my company, we're considering redesigning our backup/restore solution
which has been relatively static since originally built back in 2014 and is
showing its age. While investigating options, I reached out to Mallikarjun
and he was graciously willing to provide a backport PR [3]. The backport
applied cleanly with small conflicts in one file.
There were a few blockers listed in the original thread in [1] and from
what I can tell, they are all done. There is a remaining "Phase 4" umbrella
[4] with all of the issues looking like nice-to-haves. Most could just be
tackled based on community interest.
I think a big reason why there is no committer support and relatively
little uptake on this feature is because it has for years been stuck on
master, when pretty much everyone runs a 2.x release. So no one is using it
or has the ability to test it out, outside flipkart who backported it
themselves.
We are currently evaluating the backport PR in our own fork to see if the
solution as-is can be a good foundation for what we're trying to
accomplish. If the evaluation works out and we decide to move forward, I'd
like to work with Mallikarjun to get his backport committed.
Before doing that, I wanted to check if any other devs have concerns about
this initiative given the size of the patch and history of the project. Let
me know what you think.
[1] https://lists.apache.org/thread/k9dx12rj58r1qbrxqb0s1306lhhg3grn
[2] https://issues.apache.org/jira/browse/HBASE-19407
[3] https://github.com/apache/hbase/pull/4770
[4] https://issues.apache.org/jira/browse/HBASE-17362
Re: [DISCUSS] Backporting hbase-backup module to branch-2
Posted by "张铎(Duo Zhang)" <pa...@gmail.com>.
Thanks for taking care of this. I've been helping Mallikarjun to
improve backup&restore but it is a bit hard for me to catch up.
It will be good if someone could revisit the design and implementation
of the whole backup&restore feature, improve it and use it in
production, so it will be used by more users in the community.
For me, I'm glad to see we can have this feature on branch-2, the only
thing is that the design and implementation is stable enough, and we
have people in the community can keep maintaining it...
Thanks.
Bryan Beaudreault <bb...@hubspot.com.invalid> 于2022年10月4日周二 02:03写道：
>
> Hi again all,
>
> We have a relatively full featured backup solution in master branch. It
> looks like the original development had intended to be included in branch-2
> [1], but did not make the deadline for 2.0.0 release and was removed [2].
> Later the idea of backporting was forgotten, potentially with some of the
> main devs moving onto other projects.
>
> In the interim, one company, Flipkart (Mallikarjun works there), took it
> upon themselves to backport the feature to their own fork. They've been
> running that backport in production for some time now. Mallikarjun has been
> trying to contribute some improvements, but has lacked committer support.
>
> At my company, we're considering redesigning our backup/restore solution
> which has been relatively static since originally built back in 2014 and is
> showing its age. While investigating options, I reached out to Mallikarjun
> and he was graciously willing to provide a backport PR [3]. The backport
> applied cleanly with small conflicts in one file.
>
> There were a few blockers listed in the original thread in [1] and from
> what I can tell, they are all done. There is a remaining "Phase 4" umbrella
> [4] with all of the issues looking like nice-to-haves. Most could just be
> tackled based on community interest.
>
> I think a big reason why there is no committer support and relatively
> little uptake on this feature is because it has for years been stuck on
> master, when pretty much everyone runs a 2.x release. So no one is using it
> or has the ability to test it out, outside flipkart who backported it
> themselves.
>
> We are currently evaluating the backport PR in our own fork to see if the
> solution as-is can be a good foundation for what we're trying to
> accomplish. If the evaluation works out and we decide to move forward, I'd
> like to work with Mallikarjun to get his backport committed.
>
> Before doing that, I wanted to check if any other devs have concerns about
> this initiative given the size of the patch and history of the project. Let
> me know what you think.
>
> [1] https://lists.apache.org/thread/k9dx12rj58r1qbrxqb0s1306lhhg3grn
> [2] https://issues.apache.org/jira/browse/HBASE-19407
> [3] https://github.com/apache/hbase/pull/4770
> [4] https://issues.apache.org/jira/browse/HBASE-17362
Re: [DISCUSS] Backporting hbase-backup module to branch-2
Posted by Bryan Beaudreault <bb...@hubspot.com.INVALID>.
This has landed in branch-2. Your assumption on the configuration toggle is
correct: we should expect no changes to the output of these jobs (and thus
no downstream impact in other jobs). The new directory structure is toggled
via hbase.hfileoutputformat.tablename.namespace.inclusive, which I've
documented in the release note.
Thank you both for your guidance here, and big thanks to Mallikarjun for
his hard work on the backport and fixes. We've filed two follow-ons:
https://issues.apache.org/jira/browse/HBASE-27584 - document the
availability of experimental support as of 2.6.0
https://issues.apache.org/jira/browse/HBASE-27582 - errorprone cleanup in
hbase-backup, which we noticed during the backport
Members of my team have also pushed some improvements to backups in master
recently, which I'll also backport shortly. Mallikarjun has some long open
PRs for backups enhancements, which I hope to take a look at soon.
On Wed, Jan 18, 2023 at 1:35 AM Andrew Purtell <ap...@apache.org> wrote:
> I assume supporting either directory structure with a configuration toggle
> means existing code can coexist (in other jobs) with the backup feature,
> and so it will be fine to port this into a code line that will release a
> new minor. Sounds like a plan. +1
>
>
> On Tue, Jan 17, 2023 at 5:21 PM Bryan Beaudreault
> <bb...@hubspot.com.invalid> wrote:
>
> > Hey all,
> >
> > I reviewed the entirety of the backport PR. It's almost all net-new code,
> > which matches identically with what was already reviewed and approved in
> > the original implementation. There are some minor changes to
> > HFileOutputFormat2 and WALPlayer (and related tests).
> >
> > Of the changes to those 2 files, there was only 1 incompatible change --
> a
> > change to the directory structure of the output path for
> > HFileOutputFormat2. This change is necessary to differentiate output
> files
> > for tables in different namespaces. However, in discussing with
> Mallikarjun
> > we are going to wrap this change in a config to preserve backwards
> > compatibility. In addition to use by backup/restore, the config would
> > provide users of HFileOutputFormat2 to have a path towards upgrading to
> 3.0
> > in the future without breaking their downstream jobs which depend on the
> > current structure.
> >
> > Let me know if there are any other concerns. Otherwise once the
> > compatibility fix is in place and I give it a final review, I will merge
> it
> > to branch-2 (with a release note that it is experimental) and update our
> > refguide.
> >
> > On Wed, Oct 5, 2022 at 8:25 PM Andrew Purtell <ap...@apache.org>
> wrote:
> >
> > > Agreed that the open tasks are not essential before considering a
> > backport
> > > for (near term) release. We have often released backported features
> from
> > > the main branch in new minors with documentation -- release notes and
> > > updates to the online book, typically -- describing them as
> > "experimental",
> > > until something causes the community to reconsider that designation. I
> > > assume this would happen in this case too?
> > >
> > > On Mon, Oct 3, 2022 at 11:03 AM Bryan Beaudreault
> > > <bb...@hubspot.com.invalid> wrote:
> > >
> > > > Hi again all,
> > > >
> > > > We have a relatively full featured backup solution in master branch.
> It
> > > > looks like the original development had intended to be included in
> > > branch-2
> > > > [1], but did not make the deadline for 2.0.0 release and was removed
> > [2].
> > > > Later the idea of backporting was forgotten, potentially with some of
> > the
> > > > main devs moving onto other projects.
> > > >
> > > > In the interim, one company, Flipkart (Mallikarjun works there), took
> > it
> > > > upon themselves to backport the feature to their own fork. They've
> been
> > > > running that backport in production for some time now. Mallikarjun
> has
> > > been
> > > > trying to contribute some improvements, but has lacked committer
> > support.
> > > >
> > > > At my company, we're considering redesigning our backup/restore
> > solution
> > > > which has been relatively static since originally built back in 2014
> > and
> > > is
> > > > showing its age. While investigating options, I reached out to
> > > Mallikarjun
> > > > and he was graciously willing to provide a backport PR [3]. The
> > backport
> > > > applied cleanly with small conflicts in one file.
> > > >
> > > > There were a few blockers listed in the original thread in [1] and
> from
> > > > what I can tell, they are all done. There is a remaining "Phase 4"
> > > umbrella
> > > > [4] with all of the issues looking like nice-to-haves. Most could
> just
> > be
> > > > tackled based on community interest.
> > > >
> > > > I think a big reason why there is no committer support and relatively
> > > > little uptake on this feature is because it has for years been stuck
> on
> > > > master, when pretty much everyone runs a 2.x release. So no one is
> > using
> > > it
> > > > or has the ability to test it out, outside flipkart who backported it
> > > > themselves.
> > > >
> > > > We are currently evaluating the backport PR in our own fork to see if
> > the
> > > > solution as-is can be a good foundation for what we're trying to
> > > > accomplish. If the evaluation works out and we decide to move
> forward,
> > > I'd
> > > > like to work with Mallikarjun to get his backport committed.
> > > >
> > > > Before doing that, I wanted to check if any other devs have concerns
> > > about
> > > > this initiative given the size of the patch and history of the
> project.
> > > Let
> > > > me know what you think.
> > > >
> > > > [1] https://lists.apache.org/thread/k9dx12rj58r1qbrxqb0s1306lhhg3grn
> <https://lists.apache.org/thread/k9dx12rj58r1qbrxqb0s1306lhhg3grn>
> > > <https://lists.apache.org/thread/k9dx12rj58r1qbrxqb0s1306lhhg3grn
> <https://lists.apache.org/thread/k9dx12rj58r1qbrxqb0s1306lhhg3grn>
> >
> > > > [2] https://issues.apache.org/jira/browse/HBASE-19407
> <https://issues.apache.org/jira/browse/HBASE-19407>
> > > <https://issues.apache.org/jira/browse/HBASE-19407
> <https://issues.apache.org/jira/browse/HBASE-19407>
> >
> > > > [3] https://github.com/apache/hbase/pull/4770
> <https://github.com/apache/hbase/pull/4770>
> > > <https://github.com/apache/hbase/pull/4770
> <https://github.com/apache/hbase/pull/4770>
> >
> > > > [4] https://issues.apache.org/jira/browse/HBASE-17362
> <https://issues.apache.org/jira/browse/HBASE-17362>
> > > <https://issues.apache.org/jira/browse/HBASE-17362
> <https://issues.apache.org/jira/browse/HBASE-17362>
> >
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Andrew
> > >
> > > Unrest, ignorance distilled, nihilistic imbeciles -
> > > It's what we’ve earned
> > > Welcome, apocalypse, what’s taken you so long?
> > > Bring us the fitting end that we’ve been counting on
> > > - A23, Welcome, Apocalypse
> > >
> >
>
>
> --
> Best regards,
> Andrew
>
> Unrest, ignorance distilled, nihilistic imbeciles -
> It's what we’ve earned
> Welcome, apocalypse, what’s taken you so long?
> Bring us the fitting end that we’ve been counting on
> - A23, Welcome, Apocalypse
>
Re: [DISCUSS] Backporting hbase-backup module to branch-2
Posted by Andrew Purtell <ap...@apache.org>.
I assume supporting either directory structure with a configuration toggle
means existing code can coexist (in other jobs) with the backup feature,
and so it will be fine to port this into a code line that will release a
new minor. Sounds like a plan. +1
On Tue, Jan 17, 2023 at 5:21 PM Bryan Beaudreault
<bb...@hubspot.com.invalid> wrote:
> Hey all,
>
> I reviewed the entirety of the backport PR. It's almost all net-new code,
> which matches identically with what was already reviewed and approved in
> the original implementation. There are some minor changes to
> HFileOutputFormat2 and WALPlayer (and related tests).
>
> Of the changes to those 2 files, there was only 1 incompatible change -- a
> change to the directory structure of the output path for
> HFileOutputFormat2. This change is necessary to differentiate output files
> for tables in different namespaces. However, in discussing with Mallikarjun
> we are going to wrap this change in a config to preserve backwards
> compatibility. In addition to use by backup/restore, the config would
> provide users of HFileOutputFormat2 to have a path towards upgrading to 3.0
> in the future without breaking their downstream jobs which depend on the
> current structure.
>
> Let me know if there are any other concerns. Otherwise once the
> compatibility fix is in place and I give it a final review, I will merge it
> to branch-2 (with a release note that it is experimental) and update our
> refguide.
>
> On Wed, Oct 5, 2022 at 8:25 PM Andrew Purtell <ap...@apache.org> wrote:
>
> > Agreed that the open tasks are not essential before considering a
> backport
> > for (near term) release. We have often released backported features from
> > the main branch in new minors with documentation -- release notes and
> > updates to the online book, typically -- describing them as
> "experimental",
> > until something causes the community to reconsider that designation. I
> > assume this would happen in this case too?
> >
> > On Mon, Oct 3, 2022 at 11:03 AM Bryan Beaudreault
> > <bb...@hubspot.com.invalid> wrote:
> >
> > > Hi again all,
> > >
> > > We have a relatively full featured backup solution in master branch. It
> > > looks like the original development had intended to be included in
> > branch-2
> > > [1], but did not make the deadline for 2.0.0 release and was removed
> [2].
> > > Later the idea of backporting was forgotten, potentially with some of
> the
> > > main devs moving onto other projects.
> > >
> > > In the interim, one company, Flipkart (Mallikarjun works there), took
> it
> > > upon themselves to backport the feature to their own fork. They've been
> > > running that backport in production for some time now. Mallikarjun has
> > been
> > > trying to contribute some improvements, but has lacked committer
> support.
> > >
> > > At my company, we're considering redesigning our backup/restore
> solution
> > > which has been relatively static since originally built back in 2014
> and
> > is
> > > showing its age. While investigating options, I reached out to
> > Mallikarjun
> > > and he was graciously willing to provide a backport PR [3]. The
> backport
> > > applied cleanly with small conflicts in one file.
> > >
> > > There were a few blockers listed in the original thread in [1] and from
> > > what I can tell, they are all done. There is a remaining "Phase 4"
> > umbrella
> > > [4] with all of the issues looking like nice-to-haves. Most could just
> be
> > > tackled based on community interest.
> > >
> > > I think a big reason why there is no committer support and relatively
> > > little uptake on this feature is because it has for years been stuck on
> > > master, when pretty much everyone runs a 2.x release. So no one is
> using
> > it
> > > or has the ability to test it out, outside flipkart who backported it
> > > themselves.
> > >
> > > We are currently evaluating the backport PR in our own fork to see if
> the
> > > solution as-is can be a good foundation for what we're trying to
> > > accomplish. If the evaluation works out and we decide to move forward,
> > I'd
> > > like to work with Mallikarjun to get his backport committed.
> > >
> > > Before doing that, I wanted to check if any other devs have concerns
> > about
> > > this initiative given the size of the patch and history of the project.
> > Let
> > > me know what you think.
> > >
> > > [1] https://lists.apache.org/thread/k9dx12rj58r1qbrxqb0s1306lhhg3grn
> > <https://lists.apache.org/thread/k9dx12rj58r1qbrxqb0s1306lhhg3grn>
> > > [2] https://issues.apache.org/jira/browse/HBASE-19407
> > <https://issues.apache.org/jira/browse/HBASE-19407>
> > > [3] https://github.com/apache/hbase/pull/4770
> > <https://github.com/apache/hbase/pull/4770>
> > > [4] https://issues.apache.org/jira/browse/HBASE-17362
> > <https://issues.apache.org/jira/browse/HBASE-17362>
> > >
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Unrest, ignorance distilled, nihilistic imbeciles -
> > It's what we’ve earned
> > Welcome, apocalypse, what’s taken you so long?
> > Bring us the fitting end that we’ve been counting on
> > - A23, Welcome, Apocalypse
> >
>
--
Best regards,
Andrew
Unrest, ignorance distilled, nihilistic imbeciles -
It's what we’ve earned
Welcome, apocalypse, what’s taken you so long?
Bring us the fitting end that we’ve been counting on
- A23, Welcome, Apocalypse
Re: [DISCUSS] Backporting hbase-backup module to branch-2
Posted by "张铎(Duo Zhang)" <pa...@gmail.com>.
Thanks Bryan for taking care of this!
Bryan Beaudreault <bb...@hubspot.com.invalid> 于2023年1月18日周三 09:21写道：
>
> Hey all,
>
> I reviewed the entirety of the backport PR. It's almost all net-new code,
> which matches identically with what was already reviewed and approved in
> the original implementation. There are some minor changes to
> HFileOutputFormat2 and WALPlayer (and related tests).
>
> Of the changes to those 2 files, there was only 1 incompatible change -- a
> change to the directory structure of the output path for
> HFileOutputFormat2. This change is necessary to differentiate output files
> for tables in different namespaces. However, in discussing with Mallikarjun
> we are going to wrap this change in a config to preserve backwards
> compatibility. In addition to use by backup/restore, the config would
> provide users of HFileOutputFormat2 to have a path towards upgrading to 3.0
> in the future without breaking their downstream jobs which depend on the
> current structure.
>
> Let me know if there are any other concerns. Otherwise once the
> compatibility fix is in place and I give it a final review, I will merge it
> to branch-2 (with a release note that it is experimental) and update our
> refguide.
>
> On Wed, Oct 5, 2022 at 8:25 PM Andrew Purtell <ap...@apache.org> wrote:
>
> > Agreed that the open tasks are not essential before considering a backport
> > for (near term) release. We have often released backported features from
> > the main branch in new minors with documentation -- release notes and
> > updates to the online book, typically -- describing them as "experimental",
> > until something causes the community to reconsider that designation. I
> > assume this would happen in this case too?
> >
> > On Mon, Oct 3, 2022 at 11:03 AM Bryan Beaudreault
> > <bb...@hubspot.com.invalid> wrote:
> >
> > > Hi again all,
> > >
> > > We have a relatively full featured backup solution in master branch. It
> > > looks like the original development had intended to be included in
> > branch-2
> > > [1], but did not make the deadline for 2.0.0 release and was removed [2].
> > > Later the idea of backporting was forgotten, potentially with some of the
> > > main devs moving onto other projects.
> > >
> > > In the interim, one company, Flipkart (Mallikarjun works there), took it
> > > upon themselves to backport the feature to their own fork. They've been
> > > running that backport in production for some time now. Mallikarjun has
> > been
> > > trying to contribute some improvements, but has lacked committer support.
> > >
> > > At my company, we're considering redesigning our backup/restore solution
> > > which has been relatively static since originally built back in 2014 and
> > is
> > > showing its age. While investigating options, I reached out to
> > Mallikarjun
> > > and he was graciously willing to provide a backport PR [3]. The backport
> > > applied cleanly with small conflicts in one file.
> > >
> > > There were a few blockers listed in the original thread in [1] and from
> > > what I can tell, they are all done. There is a remaining "Phase 4"
> > umbrella
> > > [4] with all of the issues looking like nice-to-haves. Most could just be
> > > tackled based on community interest.
> > >
> > > I think a big reason why there is no committer support and relatively
> > > little uptake on this feature is because it has for years been stuck on
> > > master, when pretty much everyone runs a 2.x release. So no one is using
> > it
> > > or has the ability to test it out, outside flipkart who backported it
> > > themselves.
> > >
> > > We are currently evaluating the backport PR in our own fork to see if the
> > > solution as-is can be a good foundation for what we're trying to
> > > accomplish. If the evaluation works out and we decide to move forward,
> > I'd
> > > like to work with Mallikarjun to get his backport committed.
> > >
> > > Before doing that, I wanted to check if any other devs have concerns
> > about
> > > this initiative given the size of the patch and history of the project.
> > Let
> > > me know what you think.
> > >
> > > [1] https://lists.apache.org/thread/k9dx12rj58r1qbrxqb0s1306lhhg3grn
> > <https://lists.apache.org/thread/k9dx12rj58r1qbrxqb0s1306lhhg3grn>
> > > [2] https://issues.apache.org/jira/browse/HBASE-19407
> > <https://issues.apache.org/jira/browse/HBASE-19407>
> > > [3] https://github.com/apache/hbase/pull/4770
> > <https://github.com/apache/hbase/pull/4770>
> > > [4] https://issues.apache.org/jira/browse/HBASE-17362
> > <https://issues.apache.org/jira/browse/HBASE-17362>
> > >
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Unrest, ignorance distilled, nihilistic imbeciles -
> > It's what we’ve earned
> > Welcome, apocalypse, what’s taken you so long?
> > Bring us the fitting end that we’ve been counting on
> > - A23, Welcome, Apocalypse
> >
Re: [DISCUSS] Backporting hbase-backup module to branch-2
Posted by Bryan Beaudreault <bb...@hubspot.com.INVALID>.
Hey all,
I reviewed the entirety of the backport PR. It's almost all net-new code,
which matches identically with what was already reviewed and approved in
the original implementation. There are some minor changes to
HFileOutputFormat2 and WALPlayer (and related tests).
Of the changes to those 2 files, there was only 1 incompatible change -- a
change to the directory structure of the output path for
HFileOutputFormat2. This change is necessary to differentiate output files
for tables in different namespaces. However, in discussing with Mallikarjun
we are going to wrap this change in a config to preserve backwards
compatibility. In addition to use by backup/restore, the config would
provide users of HFileOutputFormat2 to have a path towards upgrading to 3.0
in the future without breaking their downstream jobs which depend on the
current structure.
Let me know if there are any other concerns. Otherwise once the
compatibility fix is in place and I give it a final review, I will merge it
to branch-2 (with a release note that it is experimental) and update our
refguide.
On Wed, Oct 5, 2022 at 8:25 PM Andrew Purtell <ap...@apache.org> wrote:
> Agreed that the open tasks are not essential before considering a backport
> for (near term) release. We have often released backported features from
> the main branch in new minors with documentation -- release notes and
> updates to the online book, typically -- describing them as "experimental",
> until something causes the community to reconsider that designation. I
> assume this would happen in this case too?
>
> On Mon, Oct 3, 2022 at 11:03 AM Bryan Beaudreault
> <bb...@hubspot.com.invalid> wrote:
>
> > Hi again all,
> >
> > We have a relatively full featured backup solution in master branch. It
> > looks like the original development had intended to be included in
> branch-2
> > [1], but did not make the deadline for 2.0.0 release and was removed [2].
> > Later the idea of backporting was forgotten, potentially with some of the
> > main devs moving onto other projects.
> >
> > In the interim, one company, Flipkart (Mallikarjun works there), took it
> > upon themselves to backport the feature to their own fork. They've been
> > running that backport in production for some time now. Mallikarjun has
> been
> > trying to contribute some improvements, but has lacked committer support.
> >
> > At my company, we're considering redesigning our backup/restore solution
> > which has been relatively static since originally built back in 2014 and
> is
> > showing its age. While investigating options, I reached out to
> Mallikarjun
> > and he was graciously willing to provide a backport PR [3]. The backport
> > applied cleanly with small conflicts in one file.
> >
> > There were a few blockers listed in the original thread in [1] and from
> > what I can tell, they are all done. There is a remaining "Phase 4"
> umbrella
> > [4] with all of the issues looking like nice-to-haves. Most could just be
> > tackled based on community interest.
> >
> > I think a big reason why there is no committer support and relatively
> > little uptake on this feature is because it has for years been stuck on
> > master, when pretty much everyone runs a 2.x release. So no one is using
> it
> > or has the ability to test it out, outside flipkart who backported it
> > themselves.
> >
> > We are currently evaluating the backport PR in our own fork to see if the
> > solution as-is can be a good foundation for what we're trying to
> > accomplish. If the evaluation works out and we decide to move forward,
> I'd
> > like to work with Mallikarjun to get his backport committed.
> >
> > Before doing that, I wanted to check if any other devs have concerns
> about
> > this initiative given the size of the patch and history of the project.
> Let
> > me know what you think.
> >
> > [1] https://lists.apache.org/thread/k9dx12rj58r1qbrxqb0s1306lhhg3grn
> <https://lists.apache.org/thread/k9dx12rj58r1qbrxqb0s1306lhhg3grn>
> > [2] https://issues.apache.org/jira/browse/HBASE-19407
> <https://issues.apache.org/jira/browse/HBASE-19407>
> > [3] https://github.com/apache/hbase/pull/4770
> <https://github.com/apache/hbase/pull/4770>
> > [4] https://issues.apache.org/jira/browse/HBASE-17362
> <https://issues.apache.org/jira/browse/HBASE-17362>
> >
>
>
> --
> Best regards,
> Andrew
>
> Unrest, ignorance distilled, nihilistic imbeciles -
> It's what we’ve earned
> Welcome, apocalypse, what’s taken you so long?
> Bring us the fitting end that we’ve been counting on
> - A23, Welcome, Apocalypse
>
Re: [DISCUSS] Backporting hbase-backup module to branch-2
Posted by Andrew Purtell <ap...@apache.org>.
Agreed that the open tasks are not essential before considering a backport
for (near term) release. We have often released backported features from
the main branch in new minors with documentation -- release notes and
updates to the online book, typically -- describing them as "experimental",
until something causes the community to reconsider that designation. I
assume this would happen in this case too?
On Mon, Oct 3, 2022 at 11:03 AM Bryan Beaudreault
<bb...@hubspot.com.invalid> wrote:
> Hi again all,
>
> We have a relatively full featured backup solution in master branch. It
> looks like the original development had intended to be included in branch-2
> [1], but did not make the deadline for 2.0.0 release and was removed [2].
> Later the idea of backporting was forgotten, potentially with some of the
> main devs moving onto other projects.
>
> In the interim, one company, Flipkart (Mallikarjun works there), took it
> upon themselves to backport the feature to their own fork. They've been
> running that backport in production for some time now. Mallikarjun has been
> trying to contribute some improvements, but has lacked committer support.
>
> At my company, we're considering redesigning our backup/restore solution
> which has been relatively static since originally built back in 2014 and is
> showing its age. While investigating options, I reached out to Mallikarjun
> and he was graciously willing to provide a backport PR [3]. The backport
> applied cleanly with small conflicts in one file.
>
> There were a few blockers listed in the original thread in [1] and from
> what I can tell, they are all done. There is a remaining "Phase 4" umbrella
> [4] with all of the issues looking like nice-to-haves. Most could just be
> tackled based on community interest.
>
> I think a big reason why there is no committer support and relatively
> little uptake on this feature is because it has for years been stuck on
> master, when pretty much everyone runs a 2.x release. So no one is using it
> or has the ability to test it out, outside flipkart who backported it
> themselves.
>
> We are currently evaluating the backport PR in our own fork to see if the
> solution as-is can be a good foundation for what we're trying to
> accomplish. If the evaluation works out and we decide to move forward, I'd
> like to work with Mallikarjun to get his backport committed.
>
> Before doing that, I wanted to check if any other devs have concerns about
> this initiative given the size of the patch and history of the project. Let
> me know what you think.
>
> [1] https://lists.apache.org/thread/k9dx12rj58r1qbrxqb0s1306lhhg3grn
> [2] https://issues.apache.org/jira/browse/HBASE-19407
> [3] https://github.com/apache/hbase/pull/4770
> [4] https://issues.apache.org/jira/browse/HBASE-17362
>
--
Best regards,
Andrew
Unrest, ignorance distilled, nihilistic imbeciles -
It's what we’ve earned
Welcome, apocalypse, what’s taken you so long?
Bring us the fitting end that we’ve been counting on
- A23, Welcome, Apocalypse