[DISCUSS] Separate Git Repository for HBCK2

[Umesh Agashe <ua...@cloudera.com.INVALID>]

Hi,

I've had the opportunity to talk about HBCK2 with a few of you. One of the
suggestions is to to have a separate git repository for HBCK2. Lets discuss
about it.

In the past when bugs were found in hbck, there is no easy way to release
patched version of just hbck (without patching HBase). If HBCK2 has a
separate git repo, HBCK2 versions will not be tightly related to HBase
versions. Fixing and releasing hbck2, may not require patching HBase.
Though tight coupling will be somewhat loosened, HBCK2 will still depend on
HBase APIs/ code. Caution will be required going forward regarding
compatibility.

What you all think?

Thanks,
Umesh

JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
Doc:
https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing

Re: [DISCUSS] Separate Git Repository for HBCK2

[Stack <st...@duboce.net>]

On Tue, Jul 24, 2018 at 8:53 AM Josh Elser <el...@apache.org> wrote:

> (-cc user as this I'm getting purely into code development topics)
>
> First off, thanks for working on an hbck2, Umesh!
>
> I like the idea of having a separate repository for tracking HBCK and
> the flexibility it gives us for making releases at a cadence of our
> choosing.
>
> There are two worries that come to mind immediately:
>
> * How often does HBCK make decisions on how to implement a correction
> based on some known functionality (e.g. a bug) in a specific version(s)
> of HBase. Concretely, would HBCK need to make corrections to an HBase
> installation that are specific to a subset of HBase 2.x.y versions that
> may not be valid for other 2.x.y versions?
>


hbck2 should be able to do this -- execute a fix ONLY if version matches. I
add your suggestion to Umesh's attached doc.
S



> * How often does HBCK need to re-use methods and constants from code in
> hbase-common, hbase-server, etc?
>      - Related: Is it a goal to firm up API stability around this shared
> code or are you planning to just copy needed code to the HBCK2 repo? I
> think you are saying that this *is* a goal -- could/should we introduce
> some new level of InterfaceAudience to assert that we don't
> inadvertently break HBCK2?
>
> Thanks!
>
> On 7/19/18 5:09 PM, Umesh Agashe wrote:
> > Hi,
> >
> > I've had the opportunity to talk about HBCK2 with a few of you. One of
> the
> > suggestions is to to have a separate git repository for HBCK2. Lets
> discuss
> > about it.
> >
> > In the past when bugs were found in hbck, there is no easy way to release
> > patched version of just hbck (without patching HBase). If HBCK2 has a
> > separate git repo, HBCK2 versions will not be tightly related to HBase
> > versions. Fixing and releasing hbck2, may not require patching HBase.
> > Though tight coupling will be somewhat loosened, HBCK2 will still depend
> on
> > HBase APIs/ code. Caution will be required going forward regarding
> > compatibility.
> >
> > What you all think?
> >
> > Thanks,
> > Umesh
> >
> > JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> > Doc:
> >
> https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> >
>

Re: [DISCUSS] Separate Git Repository for HBCK2

[Josh Elser <el...@apache.org>]

(-cc user as this I'm getting purely into code development topics)

First off, thanks for working on an hbck2, Umesh!

I like the idea of having a separate repository for tracking HBCK and 
the flexibility it gives us for making releases at a cadence of our 
choosing.

There are two worries that come to mind immediately:

* How often does HBCK make decisions on how to implement a correction 
based on some known functionality (e.g. a bug) in a specific version(s) 
of HBase. Concretely, would HBCK need to make corrections to an HBase 
installation that are specific to a subset of HBase 2.x.y versions that 
may not be valid for other 2.x.y versions?
* How often does HBCK need to re-use methods and constants from code in 
hbase-common, hbase-server, etc?
     - Related: Is it a goal to firm up API stability around this shared 
code or are you planning to just copy needed code to the HBCK2 repo? I 
think you are saying that this *is* a goal -- could/should we introduce 
some new level of InterfaceAudience to assert that we don't 
inadvertently break HBCK2?

Thanks!

On 7/19/18 5:09 PM, Umesh Agashe wrote:
> Hi,
> 
> I've had the opportunity to talk about HBCK2 with a few of you. One of the
> suggestions is to to have a separate git repository for HBCK2. Lets discuss
> about it.
> 
> In the past when bugs were found in hbck, there is no easy way to release
> patched version of just hbck (without patching HBase). If HBCK2 has a
> separate git repo, HBCK2 versions will not be tightly related to HBase
> versions. Fixing and releasing hbck2, may not require patching HBase.
> Though tight coupling will be somewhat loosened, HBCK2 will still depend on
> HBase APIs/ code. Caution will be required going forward regarding
> compatibility.
> 
> What you all think?
> 
> Thanks,
> Umesh
> 
> JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> Doc:
> https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> 

Re: [DISCUSS] Separate Git Repository for HBCK2

[Umesh Agashe <ua...@cloudera.com.INVALID>]

bq. Seems like you're saying it's not a problem now, but
you're not sure if it would become a problem. Regardless of that, it's a
goal to not be version-specific (and thus, we can have generic hbck-v1
and hbck-v2 tools). LMK if I misread, please :)

Thats right.

On Wed, Jul 25, 2018 at 11:11 AM Josh Elser <el...@apache.org> wrote:

> Thanks, Umesh. Seems like you're saying it's not a problem now, but
> you're not sure if it would become a problem. Regardless of that, it's a
> goal to not be version-specific (and thus, we can have generic hbck-v1
> and hbck-v2 tools). LMK if I misread, please :)
>
> One more thought, it would be nice to name this repository as
> "operator-tools" or similar (instead of hbck). A separate repo on its
> own release cadence is a nice vehicle for random sorts of recovery,
> slice-and-dice, one-off tools. I think HBCK is one example of
> administrator/operator tooling we provide (certainly, the most used),
> but we have the capacity to provide more than just that.
>
> On 7/24/18 5:55 PM, Umesh Agashe wrote:
> > Thanks Stack, Josh and Andrew for your suggestions and concerns.
> >
> > I share Stack's suggestions. This would be similar to hbase-thirdparty.
> The
> > new repo could be hbase-hbck/hbase-hbck2. As this tool will be used by
> > hbase users/ developers, hbase JIRA can be used for hbck issues.
> >
> > bq. How often does HBCK need to re-use methods and constants from code
> > in hbase-common, hbase-server, etc?
> > bq. Is it a goal to firm up API stability around this shared code.
> >
> > bq. If we do this can we also move out hbck version 1?
> >
> > As HBCK2 tool will be freshly written, we can try to achieve this goal. I
> > think its great idea to move hbck1 to new repo as well. Though I think
> its
> > more involved with hbck1 as the existing code already uses what it can
> from
> > hbase-common and hbase-server etc. modules.
> >
> > bq. How often does HBCK make decisions on how to implement a correction
> > based on some known functionality (e.g. a bug) in a specific version(s)
> > of HBase. Concretely, would HBCK need to make corrections to an HBase
> > installation that are specific to a subset of HBase 2.x.y versions that
> > may not be valid for other 2.x.y versions?
> >
> > I see if this happens too often, compatibility metrics will be
> complicated.
> >
> > Thanks,
> > Umesh
> >
> >
> > On Tue, Jul 24, 2018 at 10:27 AM Andrew Purtell <ap...@apache.org>
> wrote:
> >
> >> If we do this can we also move out hbck version 1? It would be really
> weird
> >> in my opinion to have v2 in a separate repo but v1 shipping with the 1.x
> >> releases. That would be a source of understandable confusion.
> >>
> >> I believe our compatibility guidelines allow us to upgrade interface
> >> annotations from private to LP or Public and from LP to Public. These
> are
> >> not changes that impact source or binary compatibility. They only change
> >> the promises we make going forward about their stability. I believe we
> can
> >> allow these in new minors, so we could potentially move hbck out in a
> >> 1.5.0.
> >>
> >>
> >> On Mon, Jul 23, 2018 at 4:46 PM Stack <st...@duboce.net> wrote:
> >>
> >>> On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
> >> <uagashe@cloudera.com.invalid
> >>>>
> >>> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> I've had the opportunity to talk about HBCK2 with a few of you. One of
> >>> the
> >>>> suggestions is to to have a separate git repository for HBCK2. Lets
> >>> discuss
> >>>> about it.
> >>>>
> >>>> In the past when bugs were found in hbck, there is no easy way to
> >> release
> >>>> patched version of just hbck (without patching HBase). If HBCK2 has a
> >>>> separate git repo, HBCK2 versions will not be tightly related to HBase
> >>>> versions. Fixing and releasing hbck2, may not require patching HBase.
> >>>> Though tight coupling will be somewhat loosened, HBCK2 will still
> >> depend
> >>> on
> >>>> HBase APIs/ code. Caution will be required going forward regarding
> >>>> compatibility.
> >>>>
> >>>> What you all think?
> >>>>
> >>>>
> >>> I think this the way to go.
> >>>
> >>> We'd make a new hbase-hbck2 repo as we did for hbase-thirdparty?
> >>>
> >>> We'd use the hbase JIRA for hbase-hbck2 issues?
> >>>
> >>> We'd make hbase-hbck2 releases on occasion that the PMC voted on?
> >>>
> >>> Sounds great!
> >>> St.Ack
> >>>
> >>> Thanks,
> >>>> Umesh
> >>>>
> >>>> JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> >>>> Doc:
> >>>>
> >>>>
> >>>
> >>
> https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> >>>>
> >>>
> >>
> >>
> >> --
> >> Best regards,
> >> Andrew
> >>
> >> Words like orphans lost among the crosstalk, meaning torn from truth's
> >> decrepit hands
> >>     - A23, Crosstalk
> >>
> >
>

Re: [DISCUSS] Separate Git Repository for HBCK2

[Umesh Agashe <ua...@cloudera.com.INVALID>]

Thanks Josh! separate 'operator-tools' repo for hbase tools is a great
suggestion. We can work towards it starting with hbck2. Each existing tool
needs to be looked in detail regarding how much code it shares with HBase.

On Wed, Jul 25, 2018 at 11:11 AM Josh Elser <el...@apache.org> wrote:

> Thanks, Umesh. Seems like you're saying it's not a problem now, but
> you're not sure if it would become a problem. Regardless of that, it's a
> goal to not be version-specific (and thus, we can have generic hbck-v1
> and hbck-v2 tools). LMK if I misread, please :)
>
> One more thought, it would be nice to name this repository as
> "operator-tools" or similar (instead of hbck). A separate repo on its
> own release cadence is a nice vehicle for random sorts of recovery,
> slice-and-dice, one-off tools. I think HBCK is one example of
> administrator/operator tooling we provide (certainly, the most used),
> but we have the capacity to provide more than just that.
>
> On 7/24/18 5:55 PM, Umesh Agashe wrote:
> > Thanks Stack, Josh and Andrew for your suggestions and concerns.
> >
> > I share Stack's suggestions. This would be similar to hbase-thirdparty.
> The
> > new repo could be hbase-hbck/hbase-hbck2. As this tool will be used by
> > hbase users/ developers, hbase JIRA can be used for hbck issues.
> >
> > bq. How often does HBCK need to re-use methods and constants from code
> > in hbase-common, hbase-server, etc?
> > bq. Is it a goal to firm up API stability around this shared code.
> >
> > bq. If we do this can we also move out hbck version 1?
> >
> > As HBCK2 tool will be freshly written, we can try to achieve this goal. I
> > think its great idea to move hbck1 to new repo as well. Though I think
> its
> > more involved with hbck1 as the existing code already uses what it can
> from
> > hbase-common and hbase-server etc. modules.
> >
> > bq. How often does HBCK make decisions on how to implement a correction
> > based on some known functionality (e.g. a bug) in a specific version(s)
> > of HBase. Concretely, would HBCK need to make corrections to an HBase
> > installation that are specific to a subset of HBase 2.x.y versions that
> > may not be valid for other 2.x.y versions?
> >
> > I see if this happens too often, compatibility metrics will be
> complicated.
> >
> > Thanks,
> > Umesh
> >
> >
> > On Tue, Jul 24, 2018 at 10:27 AM Andrew Purtell <ap...@apache.org>
> wrote:
> >
> >> If we do this can we also move out hbck version 1? It would be really
> weird
> >> in my opinion to have v2 in a separate repo but v1 shipping with the 1.x
> >> releases. That would be a source of understandable confusion.
> >>
> >> I believe our compatibility guidelines allow us to upgrade interface
> >> annotations from private to LP or Public and from LP to Public. These
> are
> >> not changes that impact source or binary compatibility. They only change
> >> the promises we make going forward about their stability. I believe we
> can
> >> allow these in new minors, so we could potentially move hbck out in a
> >> 1.5.0.
> >>
> >>
> >> On Mon, Jul 23, 2018 at 4:46 PM Stack <st...@duboce.net> wrote:
> >>
> >>> On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
> >> <uagashe@cloudera.com.invalid
> >>>>
> >>> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> I've had the opportunity to talk about HBCK2 with a few of you. One of
> >>> the
> >>>> suggestions is to to have a separate git repository for HBCK2. Lets
> >>> discuss
> >>>> about it.
> >>>>
> >>>> In the past when bugs were found in hbck, there is no easy way to
> >> release
> >>>> patched version of just hbck (without patching HBase). If HBCK2 has a
> >>>> separate git repo, HBCK2 versions will not be tightly related to HBase
> >>>> versions. Fixing and releasing hbck2, may not require patching HBase.
> >>>> Though tight coupling will be somewhat loosened, HBCK2 will still
> >> depend
> >>> on
> >>>> HBase APIs/ code. Caution will be required going forward regarding
> >>>> compatibility.
> >>>>
> >>>> What you all think?
> >>>>
> >>>>
> >>> I think this the way to go.
> >>>
> >>> We'd make a new hbase-hbck2 repo as we did for hbase-thirdparty?
> >>>
> >>> We'd use the hbase JIRA for hbase-hbck2 issues?
> >>>
> >>> We'd make hbase-hbck2 releases on occasion that the PMC voted on?
> >>>
> >>> Sounds great!
> >>> St.Ack
> >>>
> >>> Thanks,
> >>>> Umesh
> >>>>
> >>>> JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> >>>> Doc:
> >>>>
> >>>>
> >>>
> >>
> https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> >>>>
> >>>
> >>
> >>
> >> --
> >> Best regards,
> >> Andrew
> >>
> >> Words like orphans lost among the crosstalk, meaning torn from truth's
> >> decrepit hands
> >>     - A23, Crosstalk
> >>
> >
>

Re: [DISCUSS] Separate Git Repository for HBCK2

[Andrew Purtell <ap...@apache.org>]

Yes, and in that vein also VerifyReplication and tools of that nature.


On Wed, Jul 25, 2018 at 11:11 AM Josh Elser <el...@apache.org> wrote:

> Thanks, Umesh. Seems like you're saying it's not a problem now, but
> you're not sure if it would become a problem. Regardless of that, it's a
> goal to not be version-specific (and thus, we can have generic hbck-v1
> and hbck-v2 tools). LMK if I misread, please :)
>
> One more thought, it would be nice to name this repository as
> "operator-tools" or similar (instead of hbck). A separate repo on its
> own release cadence is a nice vehicle for random sorts of recovery,
> slice-and-dice, one-off tools. I think HBCK is one example of
> administrator/operator tooling we provide (certainly, the most used),
> but we have the capacity to provide more than just that.
>
> On 7/24/18 5:55 PM, Umesh Agashe wrote:
> > Thanks Stack, Josh and Andrew for your suggestions and concerns.
> >
> > I share Stack's suggestions. This would be similar to hbase-thirdparty.
> The
> > new repo could be hbase-hbck/hbase-hbck2. As this tool will be used by
> > hbase users/ developers, hbase JIRA can be used for hbck issues.
> >
> > bq. How often does HBCK need to re-use methods and constants from code
> > in hbase-common, hbase-server, etc?
> > bq. Is it a goal to firm up API stability around this shared code.
> >
> > bq. If we do this can we also move out hbck version 1?
> >
> > As HBCK2 tool will be freshly written, we can try to achieve this goal. I
> > think its great idea to move hbck1 to new repo as well. Though I think
> its
> > more involved with hbck1 as the existing code already uses what it can
> from
> > hbase-common and hbase-server etc. modules.
> >
> > bq. How often does HBCK make decisions on how to implement a correction
> > based on some known functionality (e.g. a bug) in a specific version(s)
> > of HBase. Concretely, would HBCK need to make corrections to an HBase
> > installation that are specific to a subset of HBase 2.x.y versions that
> > may not be valid for other 2.x.y versions?
> >
> > I see if this happens too often, compatibility metrics will be
> complicated.
> >
> > Thanks,
> > Umesh
> >
> >
> > On Tue, Jul 24, 2018 at 10:27 AM Andrew Purtell <ap...@apache.org>
> wrote:
> >
> >> If we do this can we also move out hbck version 1? It would be really
> weird
> >> in my opinion to have v2 in a separate repo but v1 shipping with the 1.x
> >> releases. That would be a source of understandable confusion.
> >>
> >> I believe our compatibility guidelines allow us to upgrade interface
> >> annotations from private to LP or Public and from LP to Public. These
> are
> >> not changes that impact source or binary compatibility. They only change
> >> the promises we make going forward about their stability. I believe we
> can
> >> allow these in new minors, so we could potentially move hbck out in a
> >> 1.5.0.
> >>
> >>
> >> On Mon, Jul 23, 2018 at 4:46 PM Stack <st...@duboce.net> wrote:
> >>
> >>> On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
> >> <uagashe@cloudera.com.invalid
> >>>>
> >>> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> I've had the opportunity to talk about HBCK2 with a few of you. One of
> >>> the
> >>>> suggestions is to to have a separate git repository for HBCK2. Lets
> >>> discuss
> >>>> about it.
> >>>>
> >>>> In the past when bugs were found in hbck, there is no easy way to
> >> release
> >>>> patched version of just hbck (without patching HBase). If HBCK2 has a
> >>>> separate git repo, HBCK2 versions will not be tightly related to HBase
> >>>> versions. Fixing and releasing hbck2, may not require patching HBase.
> >>>> Though tight coupling will be somewhat loosened, HBCK2 will still
> >> depend
> >>> on
> >>>> HBase APIs/ code. Caution will be required going forward regarding
> >>>> compatibility.
> >>>>
> >>>> What you all think?
> >>>>
> >>>>
> >>> I think this the way to go.
> >>>
> >>> We'd make a new hbase-hbck2 repo as we did for hbase-thirdparty?
> >>>
> >>> We'd use the hbase JIRA for hbase-hbck2 issues?
> >>>
> >>> We'd make hbase-hbck2 releases on occasion that the PMC voted on?
> >>>
> >>> Sounds great!
> >>> St.Ack
> >>>
> >>> Thanks,
> >>>> Umesh
> >>>>
> >>>> JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> >>>> Doc:
> >>>>
> >>>>
> >>>
> >>
> https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> >>>>
> >>>
> >>
> >>
> >> --
> >> Best regards,
> >> Andrew
> >>
> >> Words like orphans lost among the crosstalk, meaning torn from truth's
> >> decrepit hands
> >>     - A23, Crosstalk
> >>
> >
>


-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

Re: [DISCUSS] Separate Git Repository for HBCK2

[Umesh Agashe <ua...@cloudera.com.INVALID>]

Thanks Josh! separate 'operator-tools' repo for hbase tools is a great
suggestion. We can work towards it starting with hbck2. Each existing tool
needs to be looked in detail regarding how much code it shares with HBase.

On Wed, Jul 25, 2018 at 11:11 AM Josh Elser <el...@apache.org> wrote:

> Thanks, Umesh. Seems like you're saying it's not a problem now, but
> you're not sure if it would become a problem. Regardless of that, it's a
> goal to not be version-specific (and thus, we can have generic hbck-v1
> and hbck-v2 tools). LMK if I misread, please :)
>
> One more thought, it would be nice to name this repository as
> "operator-tools" or similar (instead of hbck). A separate repo on its
> own release cadence is a nice vehicle for random sorts of recovery,
> slice-and-dice, one-off tools. I think HBCK is one example of
> administrator/operator tooling we provide (certainly, the most used),
> but we have the capacity to provide more than just that.
>
> On 7/24/18 5:55 PM, Umesh Agashe wrote:
> > Thanks Stack, Josh and Andrew for your suggestions and concerns.
> >
> > I share Stack's suggestions. This would be similar to hbase-thirdparty.
> The
> > new repo could be hbase-hbck/hbase-hbck2. As this tool will be used by
> > hbase users/ developers, hbase JIRA can be used for hbck issues.
> >
> > bq. How often does HBCK need to re-use methods and constants from code
> > in hbase-common, hbase-server, etc?
> > bq. Is it a goal to firm up API stability around this shared code.
> >
> > bq. If we do this can we also move out hbck version 1?
> >
> > As HBCK2 tool will be freshly written, we can try to achieve this goal. I
> > think its great idea to move hbck1 to new repo as well. Though I think
> its
> > more involved with hbck1 as the existing code already uses what it can
> from
> > hbase-common and hbase-server etc. modules.
> >
> > bq. How often does HBCK make decisions on how to implement a correction
> > based on some known functionality (e.g. a bug) in a specific version(s)
> > of HBase. Concretely, would HBCK need to make corrections to an HBase
> > installation that are specific to a subset of HBase 2.x.y versions that
> > may not be valid for other 2.x.y versions?
> >
> > I see if this happens too often, compatibility metrics will be
> complicated.
> >
> > Thanks,
> > Umesh
> >
> >
> > On Tue, Jul 24, 2018 at 10:27 AM Andrew Purtell <ap...@apache.org>
> wrote:
> >
> >> If we do this can we also move out hbck version 1? It would be really
> weird
> >> in my opinion to have v2 in a separate repo but v1 shipping with the 1.x
> >> releases. That would be a source of understandable confusion.
> >>
> >> I believe our compatibility guidelines allow us to upgrade interface
> >> annotations from private to LP or Public and from LP to Public. These
> are
> >> not changes that impact source or binary compatibility. They only change
> >> the promises we make going forward about their stability. I believe we
> can
> >> allow these in new minors, so we could potentially move hbck out in a
> >> 1.5.0.
> >>
> >>
> >> On Mon, Jul 23, 2018 at 4:46 PM Stack <st...@duboce.net> wrote:
> >>
> >>> On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
> >> <uagashe@cloudera.com.invalid
> >>>>
> >>> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> I've had the opportunity to talk about HBCK2 with a few of you. One of
> >>> the
> >>>> suggestions is to to have a separate git repository for HBCK2. Lets
> >>> discuss
> >>>> about it.
> >>>>
> >>>> In the past when bugs were found in hbck, there is no easy way to
> >> release
> >>>> patched version of just hbck (without patching HBase). If HBCK2 has a
> >>>> separate git repo, HBCK2 versions will not be tightly related to HBase
> >>>> versions. Fixing and releasing hbck2, may not require patching HBase.
> >>>> Though tight coupling will be somewhat loosened, HBCK2 will still
> >> depend
> >>> on
> >>>> HBase APIs/ code. Caution will be required going forward regarding
> >>>> compatibility.
> >>>>
> >>>> What you all think?
> >>>>
> >>>>
> >>> I think this the way to go.
> >>>
> >>> We'd make a new hbase-hbck2 repo as we did for hbase-thirdparty?
> >>>
> >>> We'd use the hbase JIRA for hbase-hbck2 issues?
> >>>
> >>> We'd make hbase-hbck2 releases on occasion that the PMC voted on?
> >>>
> >>> Sounds great!
> >>> St.Ack
> >>>
> >>> Thanks,
> >>>> Umesh
> >>>>
> >>>> JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> >>>> Doc:
> >>>>
> >>>>
> >>>
> >>
> https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> >>>>
> >>>
> >>
> >>
> >> --
> >> Best regards,
> >> Andrew
> >>
> >> Words like orphans lost among the crosstalk, meaning torn from truth's
> >> decrepit hands
> >>     - A23, Crosstalk
> >>
> >
>

Re: [DISCUSS] Separate Git Repository for HBCK2

[Andrew Purtell <ap...@apache.org>]

Yes, and in that vein also VerifyReplication and tools of that nature.


On Wed, Jul 25, 2018 at 11:11 AM Josh Elser <el...@apache.org> wrote:

> Thanks, Umesh. Seems like you're saying it's not a problem now, but
> you're not sure if it would become a problem. Regardless of that, it's a
> goal to not be version-specific (and thus, we can have generic hbck-v1
> and hbck-v2 tools). LMK if I misread, please :)
>
> One more thought, it would be nice to name this repository as
> "operator-tools" or similar (instead of hbck). A separate repo on its
> own release cadence is a nice vehicle for random sorts of recovery,
> slice-and-dice, one-off tools. I think HBCK is one example of
> administrator/operator tooling we provide (certainly, the most used),
> but we have the capacity to provide more than just that.
>
> On 7/24/18 5:55 PM, Umesh Agashe wrote:
> > Thanks Stack, Josh and Andrew for your suggestions and concerns.
> >
> > I share Stack's suggestions. This would be similar to hbase-thirdparty.
> The
> > new repo could be hbase-hbck/hbase-hbck2. As this tool will be used by
> > hbase users/ developers, hbase JIRA can be used for hbck issues.
> >
> > bq. How often does HBCK need to re-use methods and constants from code
> > in hbase-common, hbase-server, etc?
> > bq. Is it a goal to firm up API stability around this shared code.
> >
> > bq. If we do this can we also move out hbck version 1?
> >
> > As HBCK2 tool will be freshly written, we can try to achieve this goal. I
> > think its great idea to move hbck1 to new repo as well. Though I think
> its
> > more involved with hbck1 as the existing code already uses what it can
> from
> > hbase-common and hbase-server etc. modules.
> >
> > bq. How often does HBCK make decisions on how to implement a correction
> > based on some known functionality (e.g. a bug) in a specific version(s)
> > of HBase. Concretely, would HBCK need to make corrections to an HBase
> > installation that are specific to a subset of HBase 2.x.y versions that
> > may not be valid for other 2.x.y versions?
> >
> > I see if this happens too often, compatibility metrics will be
> complicated.
> >
> > Thanks,
> > Umesh
> >
> >
> > On Tue, Jul 24, 2018 at 10:27 AM Andrew Purtell <ap...@apache.org>
> wrote:
> >
> >> If we do this can we also move out hbck version 1? It would be really
> weird
> >> in my opinion to have v2 in a separate repo but v1 shipping with the 1.x
> >> releases. That would be a source of understandable confusion.
> >>
> >> I believe our compatibility guidelines allow us to upgrade interface
> >> annotations from private to LP or Public and from LP to Public. These
> are
> >> not changes that impact source or binary compatibility. They only change
> >> the promises we make going forward about their stability. I believe we
> can
> >> allow these in new minors, so we could potentially move hbck out in a
> >> 1.5.0.
> >>
> >>
> >> On Mon, Jul 23, 2018 at 4:46 PM Stack <st...@duboce.net> wrote:
> >>
> >>> On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
> >> <uagashe@cloudera.com.invalid
> >>>>
> >>> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> I've had the opportunity to talk about HBCK2 with a few of you. One of
> >>> the
> >>>> suggestions is to to have a separate git repository for HBCK2. Lets
> >>> discuss
> >>>> about it.
> >>>>
> >>>> In the past when bugs were found in hbck, there is no easy way to
> >> release
> >>>> patched version of just hbck (without patching HBase). If HBCK2 has a
> >>>> separate git repo, HBCK2 versions will not be tightly related to HBase
> >>>> versions. Fixing and releasing hbck2, may not require patching HBase.
> >>>> Though tight coupling will be somewhat loosened, HBCK2 will still
> >> depend
> >>> on
> >>>> HBase APIs/ code. Caution will be required going forward regarding
> >>>> compatibility.
> >>>>
> >>>> What you all think?
> >>>>
> >>>>
> >>> I think this the way to go.
> >>>
> >>> We'd make a new hbase-hbck2 repo as we did for hbase-thirdparty?
> >>>
> >>> We'd use the hbase JIRA for hbase-hbck2 issues?
> >>>
> >>> We'd make hbase-hbck2 releases on occasion that the PMC voted on?
> >>>
> >>> Sounds great!
> >>> St.Ack
> >>>
> >>> Thanks,
> >>>> Umesh
> >>>>
> >>>> JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> >>>> Doc:
> >>>>
> >>>>
> >>>
> >>
> https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> >>>>
> >>>
> >>
> >>
> >> --
> >> Best regards,
> >> Andrew
> >>
> >> Words like orphans lost among the crosstalk, meaning torn from truth's
> >> decrepit hands
> >>     - A23, Crosstalk
> >>
> >
>


-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

Re: [DISCUSS] Separate Git Repository for HBCK2

[Umesh Agashe <ua...@cloudera.com.INVALID>]

bq. Seems like you're saying it's not a problem now, but
you're not sure if it would become a problem. Regardless of that, it's a
goal to not be version-specific (and thus, we can have generic hbck-v1
and hbck-v2 tools). LMK if I misread, please :)

Thats right.

On Wed, Jul 25, 2018 at 11:11 AM Josh Elser <el...@apache.org> wrote:

> Thanks, Umesh. Seems like you're saying it's not a problem now, but
> you're not sure if it would become a problem. Regardless of that, it's a
> goal to not be version-specific (and thus, we can have generic hbck-v1
> and hbck-v2 tools). LMK if I misread, please :)
>
> One more thought, it would be nice to name this repository as
> "operator-tools" or similar (instead of hbck). A separate repo on its
> own release cadence is a nice vehicle for random sorts of recovery,
> slice-and-dice, one-off tools. I think HBCK is one example of
> administrator/operator tooling we provide (certainly, the most used),
> but we have the capacity to provide more than just that.
>
> On 7/24/18 5:55 PM, Umesh Agashe wrote:
> > Thanks Stack, Josh and Andrew for your suggestions and concerns.
> >
> > I share Stack's suggestions. This would be similar to hbase-thirdparty.
> The
> > new repo could be hbase-hbck/hbase-hbck2. As this tool will be used by
> > hbase users/ developers, hbase JIRA can be used for hbck issues.
> >
> > bq. How often does HBCK need to re-use methods and constants from code
> > in hbase-common, hbase-server, etc?
> > bq. Is it a goal to firm up API stability around this shared code.
> >
> > bq. If we do this can we also move out hbck version 1?
> >
> > As HBCK2 tool will be freshly written, we can try to achieve this goal. I
> > think its great idea to move hbck1 to new repo as well. Though I think
> its
> > more involved with hbck1 as the existing code already uses what it can
> from
> > hbase-common and hbase-server etc. modules.
> >
> > bq. How often does HBCK make decisions on how to implement a correction
> > based on some known functionality (e.g. a bug) in a specific version(s)
> > of HBase. Concretely, would HBCK need to make corrections to an HBase
> > installation that are specific to a subset of HBase 2.x.y versions that
> > may not be valid for other 2.x.y versions?
> >
> > I see if this happens too often, compatibility metrics will be
> complicated.
> >
> > Thanks,
> > Umesh
> >
> >
> > On Tue, Jul 24, 2018 at 10:27 AM Andrew Purtell <ap...@apache.org>
> wrote:
> >
> >> If we do this can we also move out hbck version 1? It would be really
> weird
> >> in my opinion to have v2 in a separate repo but v1 shipping with the 1.x
> >> releases. That would be a source of understandable confusion.
> >>
> >> I believe our compatibility guidelines allow us to upgrade interface
> >> annotations from private to LP or Public and from LP to Public. These
> are
> >> not changes that impact source or binary compatibility. They only change
> >> the promises we make going forward about their stability. I believe we
> can
> >> allow these in new minors, so we could potentially move hbck out in a
> >> 1.5.0.
> >>
> >>
> >> On Mon, Jul 23, 2018 at 4:46 PM Stack <st...@duboce.net> wrote:
> >>
> >>> On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
> >> <uagashe@cloudera.com.invalid
> >>>>
> >>> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> I've had the opportunity to talk about HBCK2 with a few of you. One of
> >>> the
> >>>> suggestions is to to have a separate git repository for HBCK2. Lets
> >>> discuss
> >>>> about it.
> >>>>
> >>>> In the past when bugs were found in hbck, there is no easy way to
> >> release
> >>>> patched version of just hbck (without patching HBase). If HBCK2 has a
> >>>> separate git repo, HBCK2 versions will not be tightly related to HBase
> >>>> versions. Fixing and releasing hbck2, may not require patching HBase.
> >>>> Though tight coupling will be somewhat loosened, HBCK2 will still
> >> depend
> >>> on
> >>>> HBase APIs/ code. Caution will be required going forward regarding
> >>>> compatibility.
> >>>>
> >>>> What you all think?
> >>>>
> >>>>
> >>> I think this the way to go.
> >>>
> >>> We'd make a new hbase-hbck2 repo as we did for hbase-thirdparty?
> >>>
> >>> We'd use the hbase JIRA for hbase-hbck2 issues?
> >>>
> >>> We'd make hbase-hbck2 releases on occasion that the PMC voted on?
> >>>
> >>> Sounds great!
> >>> St.Ack
> >>>
> >>> Thanks,
> >>>> Umesh
> >>>>
> >>>> JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> >>>> Doc:
> >>>>
> >>>>
> >>>
> >>
> https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> >>>>
> >>>
> >>
> >>
> >> --
> >> Best regards,
> >> Andrew
> >>
> >> Words like orphans lost among the crosstalk, meaning torn from truth's
> >> decrepit hands
> >>     - A23, Crosstalk
> >>
> >
>

Re: [DISCUSS] Separate Git Repository for HBCK2

[Josh Elser <el...@apache.org>]

Thanks, Umesh. Seems like you're saying it's not a problem now, but 
you're not sure if it would become a problem. Regardless of that, it's a 
goal to not be version-specific (and thus, we can have generic hbck-v1 
and hbck-v2 tools). LMK if I misread, please :)

One more thought, it would be nice to name this repository as 
"operator-tools" or similar (instead of hbck). A separate repo on its 
own release cadence is a nice vehicle for random sorts of recovery, 
slice-and-dice, one-off tools. I think HBCK is one example of 
administrator/operator tooling we provide (certainly, the most used), 
but we have the capacity to provide more than just that.

On 7/24/18 5:55 PM, Umesh Agashe wrote:
> Thanks Stack, Josh and Andrew for your suggestions and concerns.
> 
> I share Stack's suggestions. This would be similar to hbase-thirdparty. The
> new repo could be hbase-hbck/hbase-hbck2. As this tool will be used by
> hbase users/ developers, hbase JIRA can be used for hbck issues.
> 
> bq. How often does HBCK need to re-use methods and constants from code
> in hbase-common, hbase-server, etc?
> bq. Is it a goal to firm up API stability around this shared code.
> 
> bq. If we do this can we also move out hbck version 1?
> 
> As HBCK2 tool will be freshly written, we can try to achieve this goal. I
> think its great idea to move hbck1 to new repo as well. Though I think its
> more involved with hbck1 as the existing code already uses what it can from
> hbase-common and hbase-server etc. modules.
> 
> bq. How often does HBCK make decisions on how to implement a correction
> based on some known functionality (e.g. a bug) in a specific version(s)
> of HBase. Concretely, would HBCK need to make corrections to an HBase
> installation that are specific to a subset of HBase 2.x.y versions that
> may not be valid for other 2.x.y versions?
> 
> I see if this happens too often, compatibility metrics will be complicated.
> 
> Thanks,
> Umesh
> 
> 
> On Tue, Jul 24, 2018 at 10:27 AM Andrew Purtell <ap...@apache.org> wrote:
> 
>> If we do this can we also move out hbck version 1? It would be really weird
>> in my opinion to have v2 in a separate repo but v1 shipping with the 1.x
>> releases. That would be a source of understandable confusion.
>>
>> I believe our compatibility guidelines allow us to upgrade interface
>> annotations from private to LP or Public and from LP to Public. These are
>> not changes that impact source or binary compatibility. They only change
>> the promises we make going forward about their stability. I believe we can
>> allow these in new minors, so we could potentially move hbck out in a
>> 1.5.0.
>>
>>
>> On Mon, Jul 23, 2018 at 4:46 PM Stack <st...@duboce.net> wrote:
>>
>>> On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
>> <uagashe@cloudera.com.invalid
>>>>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I've had the opportunity to talk about HBCK2 with a few of you. One of
>>> the
>>>> suggestions is to to have a separate git repository for HBCK2. Lets
>>> discuss
>>>> about it.
>>>>
>>>> In the past when bugs were found in hbck, there is no easy way to
>> release
>>>> patched version of just hbck (without patching HBase). If HBCK2 has a
>>>> separate git repo, HBCK2 versions will not be tightly related to HBase
>>>> versions. Fixing and releasing hbck2, may not require patching HBase.
>>>> Though tight coupling will be somewhat loosened, HBCK2 will still
>> depend
>>> on
>>>> HBase APIs/ code. Caution will be required going forward regarding
>>>> compatibility.
>>>>
>>>> What you all think?
>>>>
>>>>
>>> I think this the way to go.
>>>
>>> We'd make a new hbase-hbck2 repo as we did for hbase-thirdparty?
>>>
>>> We'd use the hbase JIRA for hbase-hbck2 issues?
>>>
>>> We'd make hbase-hbck2 releases on occasion that the PMC voted on?
>>>
>>> Sounds great!
>>> St.Ack
>>>
>>> Thanks,
>>>> Umesh
>>>>
>>>> JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
>>>> Doc:
>>>>
>>>>
>>>
>> https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
>>>>
>>>
>>
>>
>> --
>> Best regards,
>> Andrew
>>
>> Words like orphans lost among the crosstalk, meaning torn from truth's
>> decrepit hands
>>     - A23, Crosstalk
>>
> 

Re: [DISCUSS] Separate Git Repository for HBCK2

[Josh Elser <el...@apache.org>]

Thanks, Umesh. Seems like you're saying it's not a problem now, but 
you're not sure if it would become a problem. Regardless of that, it's a 
goal to not be version-specific (and thus, we can have generic hbck-v1 
and hbck-v2 tools). LMK if I misread, please :)

One more thought, it would be nice to name this repository as 
"operator-tools" or similar (instead of hbck). A separate repo on its 
own release cadence is a nice vehicle for random sorts of recovery, 
slice-and-dice, one-off tools. I think HBCK is one example of 
administrator/operator tooling we provide (certainly, the most used), 
but we have the capacity to provide more than just that.

On 7/24/18 5:55 PM, Umesh Agashe wrote:
> Thanks Stack, Josh and Andrew for your suggestions and concerns.
> 
> I share Stack's suggestions. This would be similar to hbase-thirdparty. The
> new repo could be hbase-hbck/hbase-hbck2. As this tool will be used by
> hbase users/ developers, hbase JIRA can be used for hbck issues.
> 
> bq. How often does HBCK need to re-use methods and constants from code
> in hbase-common, hbase-server, etc?
> bq. Is it a goal to firm up API stability around this shared code.
> 
> bq. If we do this can we also move out hbck version 1?
> 
> As HBCK2 tool will be freshly written, we can try to achieve this goal. I
> think its great idea to move hbck1 to new repo as well. Though I think its
> more involved with hbck1 as the existing code already uses what it can from
> hbase-common and hbase-server etc. modules.
> 
> bq. How often does HBCK make decisions on how to implement a correction
> based on some known functionality (e.g. a bug) in a specific version(s)
> of HBase. Concretely, would HBCK need to make corrections to an HBase
> installation that are specific to a subset of HBase 2.x.y versions that
> may not be valid for other 2.x.y versions?
> 
> I see if this happens too often, compatibility metrics will be complicated.
> 
> Thanks,
> Umesh
> 
> 
> On Tue, Jul 24, 2018 at 10:27 AM Andrew Purtell <ap...@apache.org> wrote:
> 
>> If we do this can we also move out hbck version 1? It would be really weird
>> in my opinion to have v2 in a separate repo but v1 shipping with the 1.x
>> releases. That would be a source of understandable confusion.
>>
>> I believe our compatibility guidelines allow us to upgrade interface
>> annotations from private to LP or Public and from LP to Public. These are
>> not changes that impact source or binary compatibility. They only change
>> the promises we make going forward about their stability. I believe we can
>> allow these in new minors, so we could potentially move hbck out in a
>> 1.5.0.
>>
>>
>> On Mon, Jul 23, 2018 at 4:46 PM Stack <st...@duboce.net> wrote:
>>
>>> On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
>> <uagashe@cloudera.com.invalid
>>>>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I've had the opportunity to talk about HBCK2 with a few of you. One of
>>> the
>>>> suggestions is to to have a separate git repository for HBCK2. Lets
>>> discuss
>>>> about it.
>>>>
>>>> In the past when bugs were found in hbck, there is no easy way to
>> release
>>>> patched version of just hbck (without patching HBase). If HBCK2 has a
>>>> separate git repo, HBCK2 versions will not be tightly related to HBase
>>>> versions. Fixing and releasing hbck2, may not require patching HBase.
>>>> Though tight coupling will be somewhat loosened, HBCK2 will still
>> depend
>>> on
>>>> HBase APIs/ code. Caution will be required going forward regarding
>>>> compatibility.
>>>>
>>>> What you all think?
>>>>
>>>>
>>> I think this the way to go.
>>>
>>> We'd make a new hbase-hbck2 repo as we did for hbase-thirdparty?
>>>
>>> We'd use the hbase JIRA for hbase-hbck2 issues?
>>>
>>> We'd make hbase-hbck2 releases on occasion that the PMC voted on?
>>>
>>> Sounds great!
>>> St.Ack
>>>
>>> Thanks,
>>>> Umesh
>>>>
>>>> JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
>>>> Doc:
>>>>
>>>>
>>>
>> https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
>>>>
>>>
>>
>>
>> --
>> Best regards,
>> Andrew
>>
>> Words like orphans lost among the crosstalk, meaning torn from truth's
>> decrepit hands
>>     - A23, Crosstalk
>>
> 

Re: [DISCUSS] Separate Git Repository for HBCK2

[Umesh Agashe <ua...@cloudera.com.INVALID>]

Thanks Stack, Josh and Andrew for your suggestions and concerns.

I share Stack's suggestions. This would be similar to hbase-thirdparty. The
new repo could be hbase-hbck/hbase-hbck2. As this tool will be used by
hbase users/ developers, hbase JIRA can be used for hbck issues.

bq. How often does HBCK need to re-use methods and constants from code
in hbase-common, hbase-server, etc?
bq. Is it a goal to firm up API stability around this shared code.

bq. If we do this can we also move out hbck version 1?

As HBCK2 tool will be freshly written, we can try to achieve this goal. I
think its great idea to move hbck1 to new repo as well. Though I think its
more involved with hbck1 as the existing code already uses what it can from
hbase-common and hbase-server etc. modules.

bq. How often does HBCK make decisions on how to implement a correction
based on some known functionality (e.g. a bug) in a specific version(s)
of HBase. Concretely, would HBCK need to make corrections to an HBase
installation that are specific to a subset of HBase 2.x.y versions that
may not be valid for other 2.x.y versions?

I see if this happens too often, compatibility metrics will be complicated.

Thanks,
Umesh


On Tue, Jul 24, 2018 at 10:27 AM Andrew Purtell <ap...@apache.org> wrote:

> If we do this can we also move out hbck version 1? It would be really weird
> in my opinion to have v2 in a separate repo but v1 shipping with the 1.x
> releases. That would be a source of understandable confusion.
>
> I believe our compatibility guidelines allow us to upgrade interface
> annotations from private to LP or Public and from LP to Public. These are
> not changes that impact source or binary compatibility. They only change
> the promises we make going forward about their stability. I believe we can
> allow these in new minors, so we could potentially move hbck out in a
> 1.5.0.
>
>
> On Mon, Jul 23, 2018 at 4:46 PM Stack <st...@duboce.net> wrote:
>
> > On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
> <uagashe@cloudera.com.invalid
> > >
> > wrote:
> >
> > > Hi,
> > >
> > > I've had the opportunity to talk about HBCK2 with a few of you. One of
> > the
> > > suggestions is to to have a separate git repository for HBCK2. Lets
> > discuss
> > > about it.
> > >
> > > In the past when bugs were found in hbck, there is no easy way to
> release
> > > patched version of just hbck (without patching HBase). If HBCK2 has a
> > > separate git repo, HBCK2 versions will not be tightly related to HBase
> > > versions. Fixing and releasing hbck2, may not require patching HBase.
> > > Though tight coupling will be somewhat loosened, HBCK2 will still
> depend
> > on
> > > HBase APIs/ code. Caution will be required going forward regarding
> > > compatibility.
> > >
> > > What you all think?
> > >
> > >
> > I think this the way to go.
> >
> > We'd make a new hbase-hbck2 repo as we did for hbase-thirdparty?
> >
> > We'd use the hbase JIRA for hbase-hbck2 issues?
> >
> > We'd make hbase-hbck2 releases on occasion that the PMC voted on?
> >
> > Sounds great!
> > St.Ack
> >
> > Thanks,
> > > Umesh
> > >
> > > JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> > > Doc:
> > >
> > >
> >
> https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> > >
> >
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>    - A23, Crosstalk
>

Re: [DISCUSS] Separate Git Repository for HBCK2

[Andrew Purtell <ap...@apache.org>]

We could try LimitedPrivate(HBCK) and see how it goes.


On Thu, Jul 26, 2018 at 2:46 AM 张铎(Duo Zhang) <pa...@gmail.com> wrote:

> I feel the same way with stack.
>
> In general, if we want HBCK to fix something, then it must depend on some
> internal stuffs of HBase, we need to keep HBCK in sync(I agree that this
> may not be the truth in the past, but we have to do this...)
>
> Anyway, let's try it first. If it does not work then we can move it back...
>
> 2018-07-26 13:54 GMT+08:00 Stack <st...@duboce.net>:
>
> > On Tue, Jul 24, 2018 at 10:27 AM Andrew Purtell <ap...@apache.org>
> > wrote:
> >
> > > If we do this can we also move out hbck version 1? It would be really
> > weird
> > > in my opinion to have v2 in a separate repo but v1 shipping with the
> 1.x
> > > releases. That would be a source of understandable confusion.
> > >
> > >
> > My sense is that hbck1 is not externalizable; we'd not be able to move it
> > out of core because it does dirty tricks all over the shop. But lets
> see...
> > S
> >
> >
> >
> > > I believe our compatibility guidelines allow us to upgrade interface
> > > annotations from private to LP or Public and from LP to Public. These
> are
> > > not changes that impact source or binary compatibility. They only
> change
> > > the promises we make going forward about their stability. I believe we
> > can
> > > allow these in new minors, so we could potentially move hbck out in a
> > > 1.5.0.
> > >
> > >
> > > On Mon, Jul 23, 2018 at 4:46 PM Stack <st...@duboce.net> wrote:
> > >
> > > > On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
> > > <uagashe@cloudera.com.invalid
> > > > >
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I've had the opportunity to talk about HBCK2 with a few of you. One
> > of
> > > > the
> > > > > suggestions is to to have a separate git repository for HBCK2. Lets
> > > > discuss
> > > > > about it.
> > > > >
> > > > > In the past when bugs were found in hbck, there is no easy way to
> > > release
> > > > > patched version of just hbck (without patching HBase). If HBCK2
> has a
> > > > > separate git repo, HBCK2 versions will not be tightly related to
> > HBase
> > > > > versions. Fixing and releasing hbck2, may not require patching
> HBase.
> > > > > Though tight coupling will be somewhat loosened, HBCK2 will still
> > > depend
> > > > on
> > > > > HBase APIs/ code. Caution will be required going forward regarding
> > > > > compatibility.
> > > > >
> > > > > What you all think?
> > > > >
> > > > >
> > > > I think this the way to go.
> > > >
> > > > We'd make a new hbase-hbck2 repo as we did for hbase-thirdparty?
> > > >
> > > > We'd use the hbase JIRA for hbase-hbck2 issues?
> > > >
> > > > We'd make hbase-hbck2 releases on occasion that the PMC voted on?
> > > >
> > > > Sounds great!
> > > > St.Ack
> > > >
> > > > Thanks,
> > > > > Umesh
> > > > >
> > > > > JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> > > > > Doc:
> > > > >
> > > > >
> > > >
> > > https://docs.google.com/document/d/1NxSFu4TKQ6lY-
> > 9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Andrew
> > >
> > > Words like orphans lost among the crosstalk, meaning torn from truth's
> > > decrepit hands
> > >    - A23, Crosstalk
> > >
> >
>


-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

Re: [DISCUSS] Separate Git Repository for HBCK2

[Andrew Purtell <ap...@apache.org>]

We could try LimitedPrivate(HBCK) and see how it goes.


On Thu, Jul 26, 2018 at 2:46 AM 张铎(Duo Zhang) <pa...@gmail.com> wrote:

> I feel the same way with stack.
>
> In general, if we want HBCK to fix something, then it must depend on some
> internal stuffs of HBase, we need to keep HBCK in sync(I agree that this
> may not be the truth in the past, but we have to do this...)
>
> Anyway, let's try it first. If it does not work then we can move it back...
>
> 2018-07-26 13:54 GMT+08:00 Stack <st...@duboce.net>:
>
> > On Tue, Jul 24, 2018 at 10:27 AM Andrew Purtell <ap...@apache.org>
> > wrote:
> >
> > > If we do this can we also move out hbck version 1? It would be really
> > weird
> > > in my opinion to have v2 in a separate repo but v1 shipping with the
> 1.x
> > > releases. That would be a source of understandable confusion.
> > >
> > >
> > My sense is that hbck1 is not externalizable; we'd not be able to move it
> > out of core because it does dirty tricks all over the shop. But lets
> see...
> > S
> >
> >
> >
> > > I believe our compatibility guidelines allow us to upgrade interface
> > > annotations from private to LP or Public and from LP to Public. These
> are
> > > not changes that impact source or binary compatibility. They only
> change
> > > the promises we make going forward about their stability. I believe we
> > can
> > > allow these in new minors, so we could potentially move hbck out in a
> > > 1.5.0.
> > >
> > >
> > > On Mon, Jul 23, 2018 at 4:46 PM Stack <st...@duboce.net> wrote:
> > >
> > > > On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
> > > <uagashe@cloudera.com.invalid
> > > > >
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I've had the opportunity to talk about HBCK2 with a few of you. One
> > of
> > > > the
> > > > > suggestions is to to have a separate git repository for HBCK2. Lets
> > > > discuss
> > > > > about it.
> > > > >
> > > > > In the past when bugs were found in hbck, there is no easy way to
> > > release
> > > > > patched version of just hbck (without patching HBase). If HBCK2
> has a
> > > > > separate git repo, HBCK2 versions will not be tightly related to
> > HBase
> > > > > versions. Fixing and releasing hbck2, may not require patching
> HBase.
> > > > > Though tight coupling will be somewhat loosened, HBCK2 will still
> > > depend
> > > > on
> > > > > HBase APIs/ code. Caution will be required going forward regarding
> > > > > compatibility.
> > > > >
> > > > > What you all think?
> > > > >
> > > > >
> > > > I think this the way to go.
> > > >
> > > > We'd make a new hbase-hbck2 repo as we did for hbase-thirdparty?
> > > >
> > > > We'd use the hbase JIRA for hbase-hbck2 issues?
> > > >
> > > > We'd make hbase-hbck2 releases on occasion that the PMC voted on?
> > > >
> > > > Sounds great!
> > > > St.Ack
> > > >
> > > > Thanks,
> > > > > Umesh
> > > > >
> > > > > JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> > > > > Doc:
> > > > >
> > > > >
> > > >
> > > https://docs.google.com/document/d/1NxSFu4TKQ6lY-
> > 9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Andrew
> > >
> > > Words like orphans lost among the crosstalk, meaning torn from truth's
> > > decrepit hands
> > >    - A23, Crosstalk
> > >
> >
>


-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

Re: [DISCUSS] Separate Git Repository for HBCK2

["张铎 (Duo Zhang)" <pa...@gmail.com>]

I feel the same way with stack.

In general, if we want HBCK to fix something, then it must depend on some
internal stuffs of HBase, we need to keep HBCK in sync(I agree that this
may not be the truth in the past, but we have to do this...)

Anyway, let's try it first. If it does not work then we can move it back...

2018-07-26 13:54 GMT+08:00 Stack <st...@duboce.net>:

> On Tue, Jul 24, 2018 at 10:27 AM Andrew Purtell <ap...@apache.org>
> wrote:
>
> > If we do this can we also move out hbck version 1? It would be really
> weird
> > in my opinion to have v2 in a separate repo but v1 shipping with the 1.x
> > releases. That would be a source of understandable confusion.
> >
> >
> My sense is that hbck1 is not externalizable; we'd not be able to move it
> out of core because it does dirty tricks all over the shop. But lets see...
> S
>
>
>
> > I believe our compatibility guidelines allow us to upgrade interface
> > annotations from private to LP or Public and from LP to Public. These are
> > not changes that impact source or binary compatibility. They only change
> > the promises we make going forward about their stability. I believe we
> can
> > allow these in new minors, so we could potentially move hbck out in a
> > 1.5.0.
> >
> >
> > On Mon, Jul 23, 2018 at 4:46 PM Stack <st...@duboce.net> wrote:
> >
> > > On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
> > <uagashe@cloudera.com.invalid
> > > >
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I've had the opportunity to talk about HBCK2 with a few of you. One
> of
> > > the
> > > > suggestions is to to have a separate git repository for HBCK2. Lets
> > > discuss
> > > > about it.
> > > >
> > > > In the past when bugs were found in hbck, there is no easy way to
> > release
> > > > patched version of just hbck (without patching HBase). If HBCK2 has a
> > > > separate git repo, HBCK2 versions will not be tightly related to
> HBase
> > > > versions. Fixing and releasing hbck2, may not require patching HBase.
> > > > Though tight coupling will be somewhat loosened, HBCK2 will still
> > depend
> > > on
> > > > HBase APIs/ code. Caution will be required going forward regarding
> > > > compatibility.
> > > >
> > > > What you all think?
> > > >
> > > >
> > > I think this the way to go.
> > >
> > > We'd make a new hbase-hbck2 repo as we did for hbase-thirdparty?
> > >
> > > We'd use the hbase JIRA for hbase-hbck2 issues?
> > >
> > > We'd make hbase-hbck2 releases on occasion that the PMC voted on?
> > >
> > > Sounds great!
> > > St.Ack
> > >
> > > Thanks,
> > > > Umesh
> > > >
> > > > JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> > > > Doc:
> > > >
> > > >
> > >
> > https://docs.google.com/document/d/1NxSFu4TKQ6lY-
> 9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Words like orphans lost among the crosstalk, meaning torn from truth's
> > decrepit hands
> >    - A23, Crosstalk
> >
>

Re: [DISCUSS] Separate Git Repository for HBCK2

["张铎 (Duo Zhang)" <pa...@gmail.com>]

I feel the same way with stack.

In general, if we want HBCK to fix something, then it must depend on some
internal stuffs of HBase, we need to keep HBCK in sync(I agree that this
may not be the truth in the past, but we have to do this...)

Anyway, let's try it first. If it does not work then we can move it back...

2018-07-26 13:54 GMT+08:00 Stack <st...@duboce.net>:

> On Tue, Jul 24, 2018 at 10:27 AM Andrew Purtell <ap...@apache.org>
> wrote:
>
> > If we do this can we also move out hbck version 1? It would be really
> weird
> > in my opinion to have v2 in a separate repo but v1 shipping with the 1.x
> > releases. That would be a source of understandable confusion.
> >
> >
> My sense is that hbck1 is not externalizable; we'd not be able to move it
> out of core because it does dirty tricks all over the shop. But lets see...
> S
>
>
>
> > I believe our compatibility guidelines allow us to upgrade interface
> > annotations from private to LP or Public and from LP to Public. These are
> > not changes that impact source or binary compatibility. They only change
> > the promises we make going forward about their stability. I believe we
> can
> > allow these in new minors, so we could potentially move hbck out in a
> > 1.5.0.
> >
> >
> > On Mon, Jul 23, 2018 at 4:46 PM Stack <st...@duboce.net> wrote:
> >
> > > On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
> > <uagashe@cloudera.com.invalid
> > > >
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I've had the opportunity to talk about HBCK2 with a few of you. One
> of
> > > the
> > > > suggestions is to to have a separate git repository for HBCK2. Lets
> > > discuss
> > > > about it.
> > > >
> > > > In the past when bugs were found in hbck, there is no easy way to
> > release
> > > > patched version of just hbck (without patching HBase). If HBCK2 has a
> > > > separate git repo, HBCK2 versions will not be tightly related to
> HBase
> > > > versions. Fixing and releasing hbck2, may not require patching HBase.
> > > > Though tight coupling will be somewhat loosened, HBCK2 will still
> > depend
> > > on
> > > > HBase APIs/ code. Caution will be required going forward regarding
> > > > compatibility.
> > > >
> > > > What you all think?
> > > >
> > > >
> > > I think this the way to go.
> > >
> > > We'd make a new hbase-hbck2 repo as we did for hbase-thirdparty?
> > >
> > > We'd use the hbase JIRA for hbase-hbck2 issues?
> > >
> > > We'd make hbase-hbck2 releases on occasion that the PMC voted on?
> > >
> > > Sounds great!
> > > St.Ack
> > >
> > > Thanks,
> > > > Umesh
> > > >
> > > > JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> > > > Doc:
> > > >
> > > >
> > >
> > https://docs.google.com/document/d/1NxSFu4TKQ6lY-
> 9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Words like orphans lost among the crosstalk, meaning torn from truth's
> > decrepit hands
> >    - A23, Crosstalk
> >
>

Re: [DISCUSS] Separate Git Repository for HBCK2

[Stack <st...@duboce.net>]

On Tue, Jul 24, 2018 at 10:27 AM Andrew Purtell <ap...@apache.org> wrote:

> If we do this can we also move out hbck version 1? It would be really weird
> in my opinion to have v2 in a separate repo but v1 shipping with the 1.x
> releases. That would be a source of understandable confusion.
>
>
My sense is that hbck1 is not externalizable; we'd not be able to move it
out of core because it does dirty tricks all over the shop. But lets see...
S



> I believe our compatibility guidelines allow us to upgrade interface
> annotations from private to LP or Public and from LP to Public. These are
> not changes that impact source or binary compatibility. They only change
> the promises we make going forward about their stability. I believe we can
> allow these in new minors, so we could potentially move hbck out in a
> 1.5.0.
>
>
> On Mon, Jul 23, 2018 at 4:46 PM Stack <st...@duboce.net> wrote:
>
> > On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
> <uagashe@cloudera.com.invalid
> > >
> > wrote:
> >
> > > Hi,
> > >
> > > I've had the opportunity to talk about HBCK2 with a few of you. One of
> > the
> > > suggestions is to to have a separate git repository for HBCK2. Lets
> > discuss
> > > about it.
> > >
> > > In the past when bugs were found in hbck, there is no easy way to
> release
> > > patched version of just hbck (without patching HBase). If HBCK2 has a
> > > separate git repo, HBCK2 versions will not be tightly related to HBase
> > > versions. Fixing and releasing hbck2, may not require patching HBase.
> > > Though tight coupling will be somewhat loosened, HBCK2 will still
> depend
> > on
> > > HBase APIs/ code. Caution will be required going forward regarding
> > > compatibility.
> > >
> > > What you all think?
> > >
> > >
> > I think this the way to go.
> >
> > We'd make a new hbase-hbck2 repo as we did for hbase-thirdparty?
> >
> > We'd use the hbase JIRA for hbase-hbck2 issues?
> >
> > We'd make hbase-hbck2 releases on occasion that the PMC voted on?
> >
> > Sounds great!
> > St.Ack
> >
> > Thanks,
> > > Umesh
> > >
> > > JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> > > Doc:
> > >
> > >
> >
> https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> > >
> >
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>    - A23, Crosstalk
>

Re: [DISCUSS] Separate Git Repository for HBCK2

[Umesh Agashe <ua...@cloudera.com.INVALID>]

Thanks Stack, Josh and Andrew for your suggestions and concerns.

I share Stack's suggestions. This would be similar to hbase-thirdparty. The
new repo could be hbase-hbck/hbase-hbck2. As this tool will be used by
hbase users/ developers, hbase JIRA can be used for hbck issues.

bq. How often does HBCK need to re-use methods and constants from code
in hbase-common, hbase-server, etc?
bq. Is it a goal to firm up API stability around this shared code.

bq. If we do this can we also move out hbck version 1?

As HBCK2 tool will be freshly written, we can try to achieve this goal. I
think its great idea to move hbck1 to new repo as well. Though I think its
more involved with hbck1 as the existing code already uses what it can from
hbase-common and hbase-server etc. modules.

bq. How often does HBCK make decisions on how to implement a correction
based on some known functionality (e.g. a bug) in a specific version(s)
of HBase. Concretely, would HBCK need to make corrections to an HBase
installation that are specific to a subset of HBase 2.x.y versions that
may not be valid for other 2.x.y versions?

I see if this happens too often, compatibility metrics will be complicated.

Thanks,
Umesh


On Tue, Jul 24, 2018 at 10:27 AM Andrew Purtell <ap...@apache.org> wrote:

> If we do this can we also move out hbck version 1? It would be really weird
> in my opinion to have v2 in a separate repo but v1 shipping with the 1.x
> releases. That would be a source of understandable confusion.
>
> I believe our compatibility guidelines allow us to upgrade interface
> annotations from private to LP or Public and from LP to Public. These are
> not changes that impact source or binary compatibility. They only change
> the promises we make going forward about their stability. I believe we can
> allow these in new minors, so we could potentially move hbck out in a
> 1.5.0.
>
>
> On Mon, Jul 23, 2018 at 4:46 PM Stack <st...@duboce.net> wrote:
>
> > On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
> <uagashe@cloudera.com.invalid
> > >
> > wrote:
> >
> > > Hi,
> > >
> > > I've had the opportunity to talk about HBCK2 with a few of you. One of
> > the
> > > suggestions is to to have a separate git repository for HBCK2. Lets
> > discuss
> > > about it.
> > >
> > > In the past when bugs were found in hbck, there is no easy way to
> release
> > > patched version of just hbck (without patching HBase). If HBCK2 has a
> > > separate git repo, HBCK2 versions will not be tightly related to HBase
> > > versions. Fixing and releasing hbck2, may not require patching HBase.
> > > Though tight coupling will be somewhat loosened, HBCK2 will still
> depend
> > on
> > > HBase APIs/ code. Caution will be required going forward regarding
> > > compatibility.
> > >
> > > What you all think?
> > >
> > >
> > I think this the way to go.
> >
> > We'd make a new hbase-hbck2 repo as we did for hbase-thirdparty?
> >
> > We'd use the hbase JIRA for hbase-hbck2 issues?
> >
> > We'd make hbase-hbck2 releases on occasion that the PMC voted on?
> >
> > Sounds great!
> > St.Ack
> >
> > Thanks,
> > > Umesh
> > >
> > > JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> > > Doc:
> > >
> > >
> >
> https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> > >
> >
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>    - A23, Crosstalk
>

Re: [DISCUSS] Separate Git Repository for HBCK2

[Stack <st...@duboce.net>]

On Tue, Jul 24, 2018 at 10:27 AM Andrew Purtell <ap...@apache.org> wrote:

> If we do this can we also move out hbck version 1? It would be really weird
> in my opinion to have v2 in a separate repo but v1 shipping with the 1.x
> releases. That would be a source of understandable confusion.
>
>
My sense is that hbck1 is not externalizable; we'd not be able to move it
out of core because it does dirty tricks all over the shop. But lets see...
S



> I believe our compatibility guidelines allow us to upgrade interface
> annotations from private to LP or Public and from LP to Public. These are
> not changes that impact source or binary compatibility. They only change
> the promises we make going forward about their stability. I believe we can
> allow these in new minors, so we could potentially move hbck out in a
> 1.5.0.
>
>
> On Mon, Jul 23, 2018 at 4:46 PM Stack <st...@duboce.net> wrote:
>
> > On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
> <uagashe@cloudera.com.invalid
> > >
> > wrote:
> >
> > > Hi,
> > >
> > > I've had the opportunity to talk about HBCK2 with a few of you. One of
> > the
> > > suggestions is to to have a separate git repository for HBCK2. Lets
> > discuss
> > > about it.
> > >
> > > In the past when bugs were found in hbck, there is no easy way to
> release
> > > patched version of just hbck (without patching HBase). If HBCK2 has a
> > > separate git repo, HBCK2 versions will not be tightly related to HBase
> > > versions. Fixing and releasing hbck2, may not require patching HBase.
> > > Though tight coupling will be somewhat loosened, HBCK2 will still
> depend
> > on
> > > HBase APIs/ code. Caution will be required going forward regarding
> > > compatibility.
> > >
> > > What you all think?
> > >
> > >
> > I think this the way to go.
> >
> > We'd make a new hbase-hbck2 repo as we did for hbase-thirdparty?
> >
> > We'd use the hbase JIRA for hbase-hbck2 issues?
> >
> > We'd make hbase-hbck2 releases on occasion that the PMC voted on?
> >
> > Sounds great!
> > St.Ack
> >
> > Thanks,
> > > Umesh
> > >
> > > JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> > > Doc:
> > >
> > >
> >
> https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> > >
> >
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>    - A23, Crosstalk
>

Re: [DISCUSS] Separate Git Repository for HBCK2

[Andrew Purtell <ap...@apache.org>]

If we do this can we also move out hbck version 1? It would be really weird
in my opinion to have v2 in a separate repo but v1 shipping with the 1.x
releases. That would be a source of understandable confusion.

I believe our compatibility guidelines allow us to upgrade interface
annotations from private to LP or Public and from LP to Public. These are
not changes that impact source or binary compatibility. They only change
the promises we make going forward about their stability. I believe we can
allow these in new minors, so we could potentially move hbck out in a
1.5.0.


On Mon, Jul 23, 2018 at 4:46 PM Stack <st...@duboce.net> wrote:

> On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe <uagashe@cloudera.com.invalid
> >
> wrote:
>
> > Hi,
> >
> > I've had the opportunity to talk about HBCK2 with a few of you. One of
> the
> > suggestions is to to have a separate git repository for HBCK2. Lets
> discuss
> > about it.
> >
> > In the past when bugs were found in hbck, there is no easy way to release
> > patched version of just hbck (without patching HBase). If HBCK2 has a
> > separate git repo, HBCK2 versions will not be tightly related to HBase
> > versions. Fixing and releasing hbck2, may not require patching HBase.
> > Though tight coupling will be somewhat loosened, HBCK2 will still depend
> on
> > HBase APIs/ code. Caution will be required going forward regarding
> > compatibility.
> >
> > What you all think?
> >
> >
> I think this the way to go.
>
> We'd make a new hbase-hbck2 repo as we did for hbase-thirdparty?
>
> We'd use the hbase JIRA for hbase-hbck2 issues?
>
> We'd make hbase-hbck2 releases on occasion that the PMC voted on?
>
> Sounds great!
> St.Ack
>
> Thanks,
> > Umesh
> >
> > JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> > Doc:
> >
> >
> https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> >
>


-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

Re: [DISCUSS] Separate Git Repository for HBCK2

[Andrew Purtell <ap...@apache.org>]

If we do this can we also move out hbck version 1? It would be really weird
in my opinion to have v2 in a separate repo but v1 shipping with the 1.x
releases. That would be a source of understandable confusion.

I believe our compatibility guidelines allow us to upgrade interface
annotations from private to LP or Public and from LP to Public. These are
not changes that impact source or binary compatibility. They only change
the promises we make going forward about their stability. I believe we can
allow these in new minors, so we could potentially move hbck out in a
1.5.0.


On Mon, Jul 23, 2018 at 4:46 PM Stack <st...@duboce.net> wrote:

> On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe <uagashe@cloudera.com.invalid
> >
> wrote:
>
> > Hi,
> >
> > I've had the opportunity to talk about HBCK2 with a few of you. One of
> the
> > suggestions is to to have a separate git repository for HBCK2. Lets
> discuss
> > about it.
> >
> > In the past when bugs were found in hbck, there is no easy way to release
> > patched version of just hbck (without patching HBase). If HBCK2 has a
> > separate git repo, HBCK2 versions will not be tightly related to HBase
> > versions. Fixing and releasing hbck2, may not require patching HBase.
> > Though tight coupling will be somewhat loosened, HBCK2 will still depend
> on
> > HBase APIs/ code. Caution will be required going forward regarding
> > compatibility.
> >
> > What you all think?
> >
> >
> I think this the way to go.
>
> We'd make a new hbase-hbck2 repo as we did for hbase-thirdparty?
>
> We'd use the hbase JIRA for hbase-hbck2 issues?
>
> We'd make hbase-hbck2 releases on occasion that the PMC voted on?
>
> Sounds great!
> St.Ack
>
> Thanks,
> > Umesh
> >
> > JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> > Doc:
> >
> >
> https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> >
>


-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

Re: [DISCUSS] Separate Git Repository for HBCK2

[Stack <st...@duboce.net>]

On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe <ua...@cloudera.com.invalid>
wrote:

> Hi,
>
> I've had the opportunity to talk about HBCK2 with a few of you. One of the
> suggestions is to to have a separate git repository for HBCK2. Lets discuss
> about it.
>
> In the past when bugs were found in hbck, there is no easy way to release
> patched version of just hbck (without patching HBase). If HBCK2 has a
> separate git repo, HBCK2 versions will not be tightly related to HBase
> versions. Fixing and releasing hbck2, may not require patching HBase.
> Though tight coupling will be somewhat loosened, HBCK2 will still depend on
> HBase APIs/ code. Caution will be required going forward regarding
> compatibility.
>
> What you all think?
>
>
I think this the way to go.

We'd make a new hbase-hbck2 repo as we did for hbase-thirdparty?

We'd use the hbase JIRA for hbase-hbck2 issues?

We'd make hbase-hbck2 releases on occasion that the PMC voted on?

Sounds great!
St.Ack

Thanks,
> Umesh
>
> JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> Doc:
>
> https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
>

Re: [DISCUSS] Separate Git Repository for HBCK2

[Stack <st...@duboce.net>]

I have made more progress on the hbck2 tool. It can actually do damage now.
Would appreciate input, especially while it is early days, on the form it
is taking. Please see
https://github.com/apache/hbase-operator-tools/tree/master/hbase-hbck2
There is also a start on a doc to outline the hbck shape here:
https://docs.google.com/document/d/1Oun4G3M5fyrM0OxXcCKYF8td0KD7gJQjnU9Ad-2t-uk/edit#heading=h.64hssul3ontj
Would appreciate any input here too.

Thanks,
S


On Thu, Aug 2, 2018 at 5:28 PM Stack <st...@duboce.net> wrote:

> I put up new repo under rubric of
> https://issues.apache.org/jira/browse/HBASE-20969.
>
> It has barebones license and a skeleton hbase-hbck2 module at the
> moment. I made an hbase-operator-tools component up in JIRA. Flag
> items with this if operator tooling patches.
>
> TODO:
>
>  * Figure layout (minimize copy from core).
>  * Would be coolio if tools could make do w/ minimal dependency on
> hbase (shaded hbase client or build fat jars so could do java -jar
> hbase_excellent_tool.jar -Dhbase.zookeeper.ensemble=X.Y.Z do_magic or
> download all it needs before running -- run via mvn?)
>
> What else?
>
> Thanks.
>
> St.Ack
> On Tue, Jul 31, 2018 at 11:17 PM Yu Li <ca...@gmail.com> wrote:
> >
> > Just notice this one from git notification. I believe it's a great move,
> > that operators could share their tools out now. Tools are also important
> > for production usage besides java features (smile).
> >
> > Best Regards,
> > Yu
> >
> > On 29 July 2018 at 05:57, Stack <st...@duboce.net> wrote:
> >
> > > I filed HBASE-20969 "Create an hbase-operator-tools repo to host hbck2
> > > and later, other toolings". Will create repo in a day or two unless
> > > objection.
> > > Thanks,
> > > S
> > >
> > > On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
> > > <ua...@cloudera.com.invalid> wrote:
> > > >
> > > > Hi,
> > > >
> > > > I've had the opportunity to talk about HBCK2 with a few of you. One
> of
> > > the
> > > > suggestions is to to have a separate git repository for HBCK2. Lets
> > > discuss
> > > > about it.
> > > >
> > > > In the past when bugs were found in hbck, there is no easy way to
> release
> > > > patched version of just hbck (without patching HBase). If HBCK2 has a
> > > > separate git repo, HBCK2 versions will not be tightly related to
> HBase
> > > > versions. Fixing and releasing hbck2, may not require patching HBase.
> > > > Though tight coupling will be somewhat loosened, HBCK2 will still
> depend
> > > on
> > > > HBase APIs/ code. Caution will be required going forward regarding
> > > > compatibility.
> > > >
> > > > What you all think?
> > > >
> > > > Thanks,
> > > > Umesh
> > > >
> > > > JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> > > > Doc:
> > > > https://docs.google.com/document/d/1NxSFu4TKQ6lY-
> > > 9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> > >
>

Re: [DISCUSS] Separate Git Repository for HBCK2

[Stack <st...@duboce.net>]

I have made more progress on the hbck2 tool. It can actually do damage now.
Would appreciate input, especially while it is early days, on the form it
is taking. Please see
https://github.com/apache/hbase-operator-tools/tree/master/hbase-hbck2
There is also a start on a doc to outline the hbck shape here:
https://docs.google.com/document/d/1Oun4G3M5fyrM0OxXcCKYF8td0KD7gJQjnU9Ad-2t-uk/edit#heading=h.64hssul3ontj
Would appreciate any input here too.

Thanks,
S


On Thu, Aug 2, 2018 at 5:28 PM Stack <st...@duboce.net> wrote:

> I put up new repo under rubric of
> https://issues.apache.org/jira/browse/HBASE-20969.
>
> It has barebones license and a skeleton hbase-hbck2 module at the
> moment. I made an hbase-operator-tools component up in JIRA. Flag
> items with this if operator tooling patches.
>
> TODO:
>
>  * Figure layout (minimize copy from core).
>  * Would be coolio if tools could make do w/ minimal dependency on
> hbase (shaded hbase client or build fat jars so could do java -jar
> hbase_excellent_tool.jar -Dhbase.zookeeper.ensemble=X.Y.Z do_magic or
> download all it needs before running -- run via mvn?)
>
> What else?
>
> Thanks.
>
> St.Ack
> On Tue, Jul 31, 2018 at 11:17 PM Yu Li <ca...@gmail.com> wrote:
> >
> > Just notice this one from git notification. I believe it's a great move,
> > that operators could share their tools out now. Tools are also important
> > for production usage besides java features (smile).
> >
> > Best Regards,
> > Yu
> >
> > On 29 July 2018 at 05:57, Stack <st...@duboce.net> wrote:
> >
> > > I filed HBASE-20969 "Create an hbase-operator-tools repo to host hbck2
> > > and later, other toolings". Will create repo in a day or two unless
> > > objection.
> > > Thanks,
> > > S
> > >
> > > On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
> > > <ua...@cloudera.com.invalid> wrote:
> > > >
> > > > Hi,
> > > >
> > > > I've had the opportunity to talk about HBCK2 with a few of you. One
> of
> > > the
> > > > suggestions is to to have a separate git repository for HBCK2. Lets
> > > discuss
> > > > about it.
> > > >
> > > > In the past when bugs were found in hbck, there is no easy way to
> release
> > > > patched version of just hbck (without patching HBase). If HBCK2 has a
> > > > separate git repo, HBCK2 versions will not be tightly related to
> HBase
> > > > versions. Fixing and releasing hbck2, may not require patching HBase.
> > > > Though tight coupling will be somewhat loosened, HBCK2 will still
> depend
> > > on
> > > > HBase APIs/ code. Caution will be required going forward regarding
> > > > compatibility.
> > > >
> > > > What you all think?
> > > >
> > > > Thanks,
> > > > Umesh
> > > >
> > > > JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> > > > Doc:
> > > > https://docs.google.com/document/d/1NxSFu4TKQ6lY-
> > > 9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> > >
>

Re: [DISCUSS] Separate Git Repository for HBCK2

[Stack <st...@duboce.net>]

I put up new repo under rubric of
https://issues.apache.org/jira/browse/HBASE-20969.

It has barebones license and a skeleton hbase-hbck2 module at the
moment. I made an hbase-operator-tools component up in JIRA. Flag
items with this if operator tooling patches.

TODO:

 * Figure layout (minimize copy from core).
 * Would be coolio if tools could make do w/ minimal dependency on
hbase (shaded hbase client or build fat jars so could do java -jar
hbase_excellent_tool.jar -Dhbase.zookeeper.ensemble=X.Y.Z do_magic or
download all it needs before running -- run via mvn?)

What else?

Thanks.

St.Ack
On Tue, Jul 31, 2018 at 11:17 PM Yu Li <ca...@gmail.com> wrote:
>
> Just notice this one from git notification. I believe it's a great move,
> that operators could share their tools out now. Tools are also important
> for production usage besides java features (smile).
>
> Best Regards,
> Yu
>
> On 29 July 2018 at 05:57, Stack <st...@duboce.net> wrote:
>
> > I filed HBASE-20969 "Create an hbase-operator-tools repo to host hbck2
> > and later, other toolings". Will create repo in a day or two unless
> > objection.
> > Thanks,
> > S
> >
> > On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
> > <ua...@cloudera.com.invalid> wrote:
> > >
> > > Hi,
> > >
> > > I've had the opportunity to talk about HBCK2 with a few of you. One of
> > the
> > > suggestions is to to have a separate git repository for HBCK2. Lets
> > discuss
> > > about it.
> > >
> > > In the past when bugs were found in hbck, there is no easy way to release
> > > patched version of just hbck (without patching HBase). If HBCK2 has a
> > > separate git repo, HBCK2 versions will not be tightly related to HBase
> > > versions. Fixing and releasing hbck2, may not require patching HBase.
> > > Though tight coupling will be somewhat loosened, HBCK2 will still depend
> > on
> > > HBase APIs/ code. Caution will be required going forward regarding
> > > compatibility.
> > >
> > > What you all think?
> > >
> > > Thanks,
> > > Umesh
> > >
> > > JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> > > Doc:
> > > https://docs.google.com/document/d/1NxSFu4TKQ6lY-
> > 9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> >

Re: [DISCUSS] Separate Git Repository for HBCK2

[Stack <st...@duboce.net>]

I put up new repo under rubric of
https://issues.apache.org/jira/browse/HBASE-20969.

It has barebones license and a skeleton hbase-hbck2 module at the
moment. I made an hbase-operator-tools component up in JIRA. Flag
items with this if operator tooling patches.

TODO:

 * Figure layout (minimize copy from core).
 * Would be coolio if tools could make do w/ minimal dependency on
hbase (shaded hbase client or build fat jars so could do java -jar
hbase_excellent_tool.jar -Dhbase.zookeeper.ensemble=X.Y.Z do_magic or
download all it needs before running -- run via mvn?)

What else?

Thanks.

St.Ack
On Tue, Jul 31, 2018 at 11:17 PM Yu Li <ca...@gmail.com> wrote:
>
> Just notice this one from git notification. I believe it's a great move,
> that operators could share their tools out now. Tools are also important
> for production usage besides java features (smile).
>
> Best Regards,
> Yu
>
> On 29 July 2018 at 05:57, Stack <st...@duboce.net> wrote:
>
> > I filed HBASE-20969 "Create an hbase-operator-tools repo to host hbck2
> > and later, other toolings". Will create repo in a day or two unless
> > objection.
> > Thanks,
> > S
> >
> > On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
> > <ua...@cloudera.com.invalid> wrote:
> > >
> > > Hi,
> > >
> > > I've had the opportunity to talk about HBCK2 with a few of you. One of
> > the
> > > suggestions is to to have a separate git repository for HBCK2. Lets
> > discuss
> > > about it.
> > >
> > > In the past when bugs were found in hbck, there is no easy way to release
> > > patched version of just hbck (without patching HBase). If HBCK2 has a
> > > separate git repo, HBCK2 versions will not be tightly related to HBase
> > > versions. Fixing and releasing hbck2, may not require patching HBase.
> > > Though tight coupling will be somewhat loosened, HBCK2 will still depend
> > on
> > > HBase APIs/ code. Caution will be required going forward regarding
> > > compatibility.
> > >
> > > What you all think?
> > >
> > > Thanks,
> > > Umesh
> > >
> > > JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> > > Doc:
> > > https://docs.google.com/document/d/1NxSFu4TKQ6lY-
> > 9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
> >

Re: [DISCUSS] Separate Git Repository for HBCK2

[Yu Li <ca...@gmail.com>]

Just notice this one from git notification. I believe it's a great move,
that operators could share their tools out now. Tools are also important
for production usage besides java features (smile).

Best Regards,
Yu

On 29 July 2018 at 05:57, Stack <st...@duboce.net> wrote:

> I filed HBASE-20969 "Create an hbase-operator-tools repo to host hbck2
> and later, other toolings". Will create repo in a day or two unless
> objection.
> Thanks,
> S
>
> On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
> <ua...@cloudera.com.invalid> wrote:
> >
> > Hi,
> >
> > I've had the opportunity to talk about HBCK2 with a few of you. One of
> the
> > suggestions is to to have a separate git repository for HBCK2. Lets
> discuss
> > about it.
> >
> > In the past when bugs were found in hbck, there is no easy way to release
> > patched version of just hbck (without patching HBase). If HBCK2 has a
> > separate git repo, HBCK2 versions will not be tightly related to HBase
> > versions. Fixing and releasing hbck2, may not require patching HBase.
> > Though tight coupling will be somewhat loosened, HBCK2 will still depend
> on
> > HBase APIs/ code. Caution will be required going forward regarding
> > compatibility.
> >
> > What you all think?
> >
> > Thanks,
> > Umesh
> >
> > JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> > Doc:
> > https://docs.google.com/document/d/1NxSFu4TKQ6lY-
> 9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
>

Re: [DISCUSS] Separate Git Repository for HBCK2

[Yu Li <ca...@gmail.com>]

Just notice this one from git notification. I believe it's a great move,
that operators could share their tools out now. Tools are also important
for production usage besides java features (smile).

Best Regards,
Yu

On 29 July 2018 at 05:57, Stack <st...@duboce.net> wrote:

> I filed HBASE-20969 "Create an hbase-operator-tools repo to host hbck2
> and later, other toolings". Will create repo in a day or two unless
> objection.
> Thanks,
> S
>
> On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
> <ua...@cloudera.com.invalid> wrote:
> >
> > Hi,
> >
> > I've had the opportunity to talk about HBCK2 with a few of you. One of
> the
> > suggestions is to to have a separate git repository for HBCK2. Lets
> discuss
> > about it.
> >
> > In the past when bugs were found in hbck, there is no easy way to release
> > patched version of just hbck (without patching HBase). If HBCK2 has a
> > separate git repo, HBCK2 versions will not be tightly related to HBase
> > versions. Fixing and releasing hbck2, may not require patching HBase.
> > Though tight coupling will be somewhat loosened, HBCK2 will still depend
> on
> > HBase APIs/ code. Caution will be required going forward regarding
> > compatibility.
> >
> > What you all think?
> >
> > Thanks,
> > Umesh
> >
> > JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> > Doc:
> > https://docs.google.com/document/d/1NxSFu4TKQ6lY-
> 9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
>

Re: [DISCUSS] Separate Git Repository for HBCK2

[Stack <st...@duboce.net>]

I filed HBASE-20969 "Create an hbase-operator-tools repo to host hbck2
and later, other toolings". Will create repo in a day or two unless
objection.
Thanks,
S

On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
<ua...@cloudera.com.invalid> wrote:
>
> Hi,
>
> I've had the opportunity to talk about HBCK2 with a few of you. One of the
> suggestions is to to have a separate git repository for HBCK2. Lets discuss
> about it.
>
> In the past when bugs were found in hbck, there is no easy way to release
> patched version of just hbck (without patching HBase). If HBCK2 has a
> separate git repo, HBCK2 versions will not be tightly related to HBase
> versions. Fixing and releasing hbck2, may not require patching HBase.
> Though tight coupling will be somewhat loosened, HBCK2 will still depend on
> HBase APIs/ code. Caution will be required going forward regarding
> compatibility.
>
> What you all think?
>
> Thanks,
> Umesh
>
> JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> Doc:
> https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing

Re: [DISCUSS] Separate Git Repository for HBCK2

[Stack <st...@duboce.net>]

On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe <ua...@cloudera.com.invalid>
wrote:

> Hi,
>
> I've had the opportunity to talk about HBCK2 with a few of you. One of the
> suggestions is to to have a separate git repository for HBCK2. Lets discuss
> about it.
>
> In the past when bugs were found in hbck, there is no easy way to release
> patched version of just hbck (without patching HBase). If HBCK2 has a
> separate git repo, HBCK2 versions will not be tightly related to HBase
> versions. Fixing and releasing hbck2, may not require patching HBase.
> Though tight coupling will be somewhat loosened, HBCK2 will still depend on
> HBase APIs/ code. Caution will be required going forward regarding
> compatibility.
>
> What you all think?
>
>
I think this the way to go.

We'd make a new hbase-hbck2 repo as we did for hbase-thirdparty?

We'd use the hbase JIRA for hbase-hbck2 issues?

We'd make hbase-hbck2 releases on occasion that the PMC voted on?

Sounds great!
St.Ack

Thanks,
> Umesh
>
> JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> Doc:
>
> https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing
>

Re: [DISCUSS] Separate Git Repository for HBCK2

[Stack <st...@duboce.net>]

I filed HBASE-20969 "Create an hbase-operator-tools repo to host hbck2
and later, other toolings". Will create repo in a day or two unless
objection.
Thanks,
S

On Thu, Jul 19, 2018 at 2:09 PM Umesh Agashe
<ua...@cloudera.com.invalid> wrote:
>
> Hi,
>
> I've had the opportunity to talk about HBCK2 with a few of you. One of the
> suggestions is to to have a separate git repository for HBCK2. Lets discuss
> about it.
>
> In the past when bugs were found in hbck, there is no easy way to release
> patched version of just hbck (without patching HBase). If HBCK2 has a
> separate git repo, HBCK2 versions will not be tightly related to HBase
> versions. Fixing and releasing hbck2, may not require patching HBase.
> Though tight coupling will be somewhat loosened, HBCK2 will still depend on
> HBase APIs/ code. Caution will be required going forward regarding
> compatibility.
>
> What you all think?
>
> Thanks,
> Umesh
>
> JIRA:  https://issues.apache.org/jira/browse/HBASE-19121.
> Doc:
> https://docs.google.com/document/d/1NxSFu4TKQ6lY-9J5qsCcJb9kZOnkfX66KMYsiVxBy0Y/edit?usp=sharing