You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by James Taylor <ja...@apache.org> on 2016/06/07 15:42:14 UTC

Re: [DISCUSS] Regarding PHOENIX-1734

I don't understand why HBASE-15600 is needed if we're ok with the local
index updates not being transactional with older versions of HBase. Why
can't we do the htable.batch() call in ParallelWriterIndexCommitter as we
do now? The row keys are different. If this is an issue, then we could just
enqueue the mutations and let them be submitted outside the row lock. We
don't need to have the row lock.

Seems to me that it could be made to work without the branches diverging so
radically.

Thanks,
James

On Tuesday, June 7, 2016, rajeshbabu@apache.org <ch...@gmail.com>
wrote:

> Hi Team,
>
> As you know prior to PHOENIX-1734 we are storing all  local index data in
> separate table and colocate data and index regions. To ensure proper
> colocation we need customer balancer, custom split and merge logic still
> these will not ensure 100% colocation. That's why we started working
> PHOENIX-1734 to store local index data in separate column families in the
> same table to ensure 100% colocation.
>
> As part of PHOENIX-1734 we need to write local index udpates to same region
> in preBatchMutate. When I am working with master branch(The HBase version
> is 1.2.x) it went smooth and I continued the development and ensure new
> implementation of local index works properly and committed the patch to
> master branch as well.
>
> But when I started porting it to other branches(HBase versions less than
> 1.2) I realized that write to same region is in preBatchMutate is not
> allowed by HBase versions less than 1.2. There is a dependency of
> HBASE-15600 to make the new local index implementation work properly for
> versions less than 1.2. So I am not able to port it to other branches. I
> will complete it once the next HBase point releases have HBASE-15600.
>
> Now code in master branch is not in sync with other branches but the local
> indexing feature is much more stable in master branch after PHOENIX-1734 so
> I think it would be better to have PHOENIX-1734 in master branch even
> porting it to other branches might delay till upcoming HBase releases
> available.
>
> Wdyt?
>
> Thanks,
> Rajeshbabu.
>

Re: Regarding PHOENIX-1734

Posted by Nick Dimiduk <nd...@gmail.com>.
On Sun, Jun 12, 2016 at 4:33 PM, James Taylor <ja...@apache.org>
wrote:

> Users not relying on local indexes can upgrade the server ahead of the
> client, but users depending on local indexes will need to upgrade the
> client and server together. The problem with bumping up the major version
> is that existing users not using local indexes will not be able to move to
> 4.8 without down time.
>

Explaining to someone that they have to treat a minor release like a major
release just because they happened to add the LOCAL keyword without
explanation of the consequences sounds like a very bad time. This sounds
like an argument to have local index improvements wait for a major release.
Do we have a pile of other features waiting for breaking changes to ship?

On Sun, Jun 12, 2016 at 4:17 PM, Sergey Soldatov <se...@gmail.com>
> wrote:
>
> > I have an another question. New implementation of local indexes affects
> > both server and client parts. Wouldn't it break the compatibility between
> > old 4.x clients and new servers? I.e. clients are creating/writing local
> > indexes into a separate table, but on server side the index updater will
> > try to update indexes in the user table. Shouldn't we bump the major
> > version  to avoid such incompatibility?
> >
> > On Sat, Jun 11, 2016 at 7:05 PM, Andrew Purtell <
> andrew.purtell@gmail.com>
> > wrote:
> >
> > > I'm pretty sure the HBASE-15600 change wouldn't be accepted by the
> branch
> > > RM for 1.1, but that is my opinion of course, not his. I could be
> > > surprised. With 0.98 the bar would be lower as each 0.98.x release is a
> > > minor release as defined by the HBase compatibility policy, but even so
> > the
> > > suggestion makes me nervous offhand.
> > >
> > > > I don't understand why HBASE-15600 is needed if we're ok with the
> local
> > > index updates not being transactional with older versions of HBase
> > >
> > > If you're able to take that position that would be good, I think. It
> > might
> > > help motivate people to move up to newer versions of HBase.
> > >
> > >
> > > > On Jun 8, 2016, at 9:31 PM, James Taylor <ja...@apache.org>
> > wrote:
> > > >
> > > > Another factor is documentation. It's hard enough to document new
> > > features
> > > > and behavior as it is, but we'd have to explain that for the *same*
> > > > release, it depends on which version of HBase you're using to know
> how
> > to
> > > > correctly configure local indexes - sometimes you need modify
> > > > the hbase-site.xml on the master and sometimes you wouldn't.
> > > >
> > > > It's one thing if we're EOLing these branches, but I don't think
> we're
> > at
> > > > that point yet.
> > > >
> > > >> On Wednesday, June 8, 2016, Samarth Jain <sa...@gmail.com>
> > > wrote:
> > > >>
> > > >> Considering the footprint of PHOENIX-1734, it would make development
> > > >> onerous if the feature is merged only to 4.x-HBase-1.2. There is
> > > already an
> > > >> overhead of having to merge patches in 4 branches. It also has the
> > > >> potential of increasing the likelihood of bugs being introduced
> > arising
> > > out
> > > >> of having to handle many more merge conflicts.
> > > >>
> > > >> Just my 2 cents :)
> > > >>
> > > >>
> > > >>
> > > >> On Tue, Jun 7, 2016 at 8:42 AM, James Taylor <
> jamestaylor@apache.org
> > > >> <javascript:;>> wrote:
> > > >>
> > > >>> I don't understand why HBASE-15600 is needed if we're ok with the
> > local
> > > >>> index updates not being transactional with older versions of HBase.
> > Why
> > > >>> can't we do the htable.batch() call in ParallelWriterIndexCommitter
> > as
> > > we
> > > >>> do now? The row keys are different. If this is an issue, then we
> > could
> > > >> just
> > > >>> enqueue the mutations and let them be submitted outside the row
> lock.
> > > We
> > > >>> don't need to have the row lock.
> > > >>>
> > > >>> Seems to me that it could be made to work without the branches
> > > diverging
> > > >> so
> > > >>> radically.
> > > >>>
> > > >>> Thanks,
> > > >>> James
> > > >>>
> > > >>> On Tuesday, June 7, 2016, rajeshbabu@apache.org <javascript:;> <
> > > >> chrajeshbabu32@gmail.com <javascript:;>>
> > > >>> wrote:
> > > >>>
> > > >>>> Hi Team,
> > > >>>>
> > > >>>> As you know prior to PHOENIX-1734 we are storing all  local index
> > data
> > > >> in
> > > >>>> separate table and colocate data and index regions. To ensure
> proper
> > > >>>> colocation we need customer balancer, custom split and merge logic
> > > >> still
> > > >>>> these will not ensure 100% colocation. That's why we started
> working
> > > >>>> PHOENIX-1734 to store local index data in separate column families
> > in
> > > >> the
> > > >>>> same table to ensure 100% colocation.
> > > >>>>
> > > >>>> As part of PHOENIX-1734 we need to write local index udpates to
> same
> > > >>> region
> > > >>>> in preBatchMutate. When I am working with master branch(The HBase
> > > >> version
> > > >>>> is 1.2.x) it went smooth and I continued the development and
> ensure
> > > new
> > > >>>> implementation of local index works properly and committed the
> patch
> > > to
> > > >>>> master branch as well.
> > > >>>>
> > > >>>> But when I started porting it to other branches(HBase versions
> less
> > > >> than
> > > >>>> 1.2) I realized that write to same region is in preBatchMutate is
> > not
> > > >>>> allowed by HBase versions less than 1.2. There is a dependency of
> > > >>>> HBASE-15600 to make the new local index implementation work
> properly
> > > >> for
> > > >>>> versions less than 1.2. So I am not able to port it to other
> > branches.
> > > >> I
> > > >>>> will complete it once the next HBase point releases have
> > HBASE-15600.
> > > >>>>
> > > >>>> Now code in master branch is not in sync with other branches but
> the
> > > >>> local
> > > >>>> indexing feature is much more stable in master branch after
> > > >> PHOENIX-1734
> > > >>> so
> > > >>>> I think it would be better to have PHOENIX-1734 in master branch
> > even
> > > >>>> porting it to other branches might delay till upcoming HBase
> > releases
> > > >>>> available.
> > > >>>>
> > > >>>> Wdyt?
> > > >>>>
> > > >>>> Thanks,
> > > >>>> Rajeshbabu.
> > > >>
> > >
> >
>

Re: Regarding PHOENIX-1734

Posted by James Taylor <ja...@apache.org>.
Users not relying on local indexes can upgrade the server ahead of the
client, but users depending on local indexes will need to upgrade the
client and server together. The problem with bumping up the major version
is that existing users not using local indexes will not be able to move to
4.8 without down time.

On Sun, Jun 12, 2016 at 4:17 PM, Sergey Soldatov <se...@gmail.com>
wrote:

> I have an another question. New implementation of local indexes affects
> both server and client parts. Wouldn't it break the compatibility between
> old 4.x clients and new servers? I.e. clients are creating/writing local
> indexes into a separate table, but on server side the index updater will
> try to update indexes in the user table. Shouldn't we bump the major
> version  to avoid such incompatibility?
>
> On Sat, Jun 11, 2016 at 7:05 PM, Andrew Purtell <an...@gmail.com>
> wrote:
>
> > I'm pretty sure the HBASE-15600 change wouldn't be accepted by the branch
> > RM for 1.1, but that is my opinion of course, not his. I could be
> > surprised. With 0.98 the bar would be lower as each 0.98.x release is a
> > minor release as defined by the HBase compatibility policy, but even so
> the
> > suggestion makes me nervous offhand.
> >
> > > I don't understand why HBASE-15600 is needed if we're ok with the local
> > index updates not being transactional with older versions of HBase
> >
> > If you're able to take that position that would be good, I think. It
> might
> > help motivate people to move up to newer versions of HBase.
> >
> >
> > > On Jun 8, 2016, at 9:31 PM, James Taylor <ja...@apache.org>
> wrote:
> > >
> > > Another factor is documentation. It's hard enough to document new
> > features
> > > and behavior as it is, but we'd have to explain that for the *same*
> > > release, it depends on which version of HBase you're using to know how
> to
> > > correctly configure local indexes - sometimes you need modify
> > > the hbase-site.xml on the master and sometimes you wouldn't.
> > >
> > > It's one thing if we're EOLing these branches, but I don't think we're
> at
> > > that point yet.
> > >
> > >> On Wednesday, June 8, 2016, Samarth Jain <sa...@gmail.com>
> > wrote:
> > >>
> > >> Considering the footprint of PHOENIX-1734, it would make development
> > >> onerous if the feature is merged only to 4.x-HBase-1.2. There is
> > already an
> > >> overhead of having to merge patches in 4 branches. It also has the
> > >> potential of increasing the likelihood of bugs being introduced
> arising
> > out
> > >> of having to handle many more merge conflicts.
> > >>
> > >> Just my 2 cents :)
> > >>
> > >>
> > >>
> > >> On Tue, Jun 7, 2016 at 8:42 AM, James Taylor <jamestaylor@apache.org
> > >> <javascript:;>> wrote:
> > >>
> > >>> I don't understand why HBASE-15600 is needed if we're ok with the
> local
> > >>> index updates not being transactional with older versions of HBase.
> Why
> > >>> can't we do the htable.batch() call in ParallelWriterIndexCommitter
> as
> > we
> > >>> do now? The row keys are different. If this is an issue, then we
> could
> > >> just
> > >>> enqueue the mutations and let them be submitted outside the row lock.
> > We
> > >>> don't need to have the row lock.
> > >>>
> > >>> Seems to me that it could be made to work without the branches
> > diverging
> > >> so
> > >>> radically.
> > >>>
> > >>> Thanks,
> > >>> James
> > >>>
> > >>> On Tuesday, June 7, 2016, rajeshbabu@apache.org <javascript:;> <
> > >> chrajeshbabu32@gmail.com <javascript:;>>
> > >>> wrote:
> > >>>
> > >>>> Hi Team,
> > >>>>
> > >>>> As you know prior to PHOENIX-1734 we are storing all  local index
> data
> > >> in
> > >>>> separate table and colocate data and index regions. To ensure proper
> > >>>> colocation we need customer balancer, custom split and merge logic
> > >> still
> > >>>> these will not ensure 100% colocation. That's why we started working
> > >>>> PHOENIX-1734 to store local index data in separate column families
> in
> > >> the
> > >>>> same table to ensure 100% colocation.
> > >>>>
> > >>>> As part of PHOENIX-1734 we need to write local index udpates to same
> > >>> region
> > >>>> in preBatchMutate. When I am working with master branch(The HBase
> > >> version
> > >>>> is 1.2.x) it went smooth and I continued the development and ensure
> > new
> > >>>> implementation of local index works properly and committed the patch
> > to
> > >>>> master branch as well.
> > >>>>
> > >>>> But when I started porting it to other branches(HBase versions less
> > >> than
> > >>>> 1.2) I realized that write to same region is in preBatchMutate is
> not
> > >>>> allowed by HBase versions less than 1.2. There is a dependency of
> > >>>> HBASE-15600 to make the new local index implementation work properly
> > >> for
> > >>>> versions less than 1.2. So I am not able to port it to other
> branches.
> > >> I
> > >>>> will complete it once the next HBase point releases have
> HBASE-15600.
> > >>>>
> > >>>> Now code in master branch is not in sync with other branches but the
> > >>> local
> > >>>> indexing feature is much more stable in master branch after
> > >> PHOENIX-1734
> > >>> so
> > >>>> I think it would be better to have PHOENIX-1734 in master branch
> even
> > >>>> porting it to other branches might delay till upcoming HBase
> releases
> > >>>> available.
> > >>>>
> > >>>> Wdyt?
> > >>>>
> > >>>> Thanks,
> > >>>> Rajeshbabu.
> > >>
> >
>

Re: Regarding PHOENIX-1734

Posted by Sergey Soldatov <se...@gmail.com>.
I have an another question. New implementation of local indexes affects
both server and client parts. Wouldn't it break the compatibility between
old 4.x clients and new servers? I.e. clients are creating/writing local
indexes into a separate table, but on server side the index updater will
try to update indexes in the user table. Shouldn't we bump the major
version  to avoid such incompatibility?

On Sat, Jun 11, 2016 at 7:05 PM, Andrew Purtell <an...@gmail.com>
wrote:

> I'm pretty sure the HBASE-15600 change wouldn't be accepted by the branch
> RM for 1.1, but that is my opinion of course, not his. I could be
> surprised. With 0.98 the bar would be lower as each 0.98.x release is a
> minor release as defined by the HBase compatibility policy, but even so the
> suggestion makes me nervous offhand.
>
> > I don't understand why HBASE-15600 is needed if we're ok with the local
> index updates not being transactional with older versions of HBase
>
> If you're able to take that position that would be good, I think. It might
> help motivate people to move up to newer versions of HBase.
>
>
> > On Jun 8, 2016, at 9:31 PM, James Taylor <ja...@apache.org> wrote:
> >
> > Another factor is documentation. It's hard enough to document new
> features
> > and behavior as it is, but we'd have to explain that for the *same*
> > release, it depends on which version of HBase you're using to know how to
> > correctly configure local indexes - sometimes you need modify
> > the hbase-site.xml on the master and sometimes you wouldn't.
> >
> > It's one thing if we're EOLing these branches, but I don't think we're at
> > that point yet.
> >
> >> On Wednesday, June 8, 2016, Samarth Jain <sa...@gmail.com>
> wrote:
> >>
> >> Considering the footprint of PHOENIX-1734, it would make development
> >> onerous if the feature is merged only to 4.x-HBase-1.2. There is
> already an
> >> overhead of having to merge patches in 4 branches. It also has the
> >> potential of increasing the likelihood of bugs being introduced arising
> out
> >> of having to handle many more merge conflicts.
> >>
> >> Just my 2 cents :)
> >>
> >>
> >>
> >> On Tue, Jun 7, 2016 at 8:42 AM, James Taylor <jamestaylor@apache.org
> >> <javascript:;>> wrote:
> >>
> >>> I don't understand why HBASE-15600 is needed if we're ok with the local
> >>> index updates not being transactional with older versions of HBase. Why
> >>> can't we do the htable.batch() call in ParallelWriterIndexCommitter as
> we
> >>> do now? The row keys are different. If this is an issue, then we could
> >> just
> >>> enqueue the mutations and let them be submitted outside the row lock.
> We
> >>> don't need to have the row lock.
> >>>
> >>> Seems to me that it could be made to work without the branches
> diverging
> >> so
> >>> radically.
> >>>
> >>> Thanks,
> >>> James
> >>>
> >>> On Tuesday, June 7, 2016, rajeshbabu@apache.org <javascript:;> <
> >> chrajeshbabu32@gmail.com <javascript:;>>
> >>> wrote:
> >>>
> >>>> Hi Team,
> >>>>
> >>>> As you know prior to PHOENIX-1734 we are storing all  local index data
> >> in
> >>>> separate table and colocate data and index regions. To ensure proper
> >>>> colocation we need customer balancer, custom split and merge logic
> >> still
> >>>> these will not ensure 100% colocation. That's why we started working
> >>>> PHOENIX-1734 to store local index data in separate column families in
> >> the
> >>>> same table to ensure 100% colocation.
> >>>>
> >>>> As part of PHOENIX-1734 we need to write local index udpates to same
> >>> region
> >>>> in preBatchMutate. When I am working with master branch(The HBase
> >> version
> >>>> is 1.2.x) it went smooth and I continued the development and ensure
> new
> >>>> implementation of local index works properly and committed the patch
> to
> >>>> master branch as well.
> >>>>
> >>>> But when I started porting it to other branches(HBase versions less
> >> than
> >>>> 1.2) I realized that write to same region is in preBatchMutate is not
> >>>> allowed by HBase versions less than 1.2. There is a dependency of
> >>>> HBASE-15600 to make the new local index implementation work properly
> >> for
> >>>> versions less than 1.2. So I am not able to port it to other branches.
> >> I
> >>>> will complete it once the next HBase point releases have HBASE-15600.
> >>>>
> >>>> Now code in master branch is not in sync with other branches but the
> >>> local
> >>>> indexing feature is much more stable in master branch after
> >> PHOENIX-1734
> >>> so
> >>>> I think it would be better to have PHOENIX-1734 in master branch even
> >>>> porting it to other branches might delay till upcoming HBase releases
> >>>> available.
> >>>>
> >>>> Wdyt?
> >>>>
> >>>> Thanks,
> >>>> Rajeshbabu.
> >>
>

Re: Regarding PHOENIX-1734

Posted by Andrew Purtell <an...@gmail.com>.
I'm pretty sure the HBASE-15600 change wouldn't be accepted by the branch RM for 1.1, but that is my opinion of course, not his. I could be surprised. With 0.98 the bar would be lower as each 0.98.x release is a minor release as defined by the HBase compatibility policy, but even so the suggestion makes me nervous offhand. 

> I don't understand why HBASE-15600 is needed if we're ok with the local index updates not being transactional with older versions of HBase

If you're able to take that position that would be good, I think. It might help motivate people to move up to newer versions of HBase. 


> On Jun 8, 2016, at 9:31 PM, James Taylor <ja...@apache.org> wrote:
> 
> Another factor is documentation. It's hard enough to document new features
> and behavior as it is, but we'd have to explain that for the *same*
> release, it depends on which version of HBase you're using to know how to
> correctly configure local indexes - sometimes you need modify
> the hbase-site.xml on the master and sometimes you wouldn't.
> 
> It's one thing if we're EOLing these branches, but I don't think we're at
> that point yet.
> 
>> On Wednesday, June 8, 2016, Samarth Jain <sa...@gmail.com> wrote:
>> 
>> Considering the footprint of PHOENIX-1734, it would make development
>> onerous if the feature is merged only to 4.x-HBase-1.2. There is already an
>> overhead of having to merge patches in 4 branches. It also has the
>> potential of increasing the likelihood of bugs being introduced arising out
>> of having to handle many more merge conflicts.
>> 
>> Just my 2 cents :)
>> 
>> 
>> 
>> On Tue, Jun 7, 2016 at 8:42 AM, James Taylor <jamestaylor@apache.org
>> <javascript:;>> wrote:
>> 
>>> I don't understand why HBASE-15600 is needed if we're ok with the local
>>> index updates not being transactional with older versions of HBase. Why
>>> can't we do the htable.batch() call in ParallelWriterIndexCommitter as we
>>> do now? The row keys are different. If this is an issue, then we could
>> just
>>> enqueue the mutations and let them be submitted outside the row lock. We
>>> don't need to have the row lock.
>>> 
>>> Seems to me that it could be made to work without the branches diverging
>> so
>>> radically.
>>> 
>>> Thanks,
>>> James
>>> 
>>> On Tuesday, June 7, 2016, rajeshbabu@apache.org <javascript:;> <
>> chrajeshbabu32@gmail.com <javascript:;>>
>>> wrote:
>>> 
>>>> Hi Team,
>>>> 
>>>> As you know prior to PHOENIX-1734 we are storing all  local index data
>> in
>>>> separate table and colocate data and index regions. To ensure proper
>>>> colocation we need customer balancer, custom split and merge logic
>> still
>>>> these will not ensure 100% colocation. That's why we started working
>>>> PHOENIX-1734 to store local index data in separate column families in
>> the
>>>> same table to ensure 100% colocation.
>>>> 
>>>> As part of PHOENIX-1734 we need to write local index udpates to same
>>> region
>>>> in preBatchMutate. When I am working with master branch(The HBase
>> version
>>>> is 1.2.x) it went smooth and I continued the development and ensure new
>>>> implementation of local index works properly and committed the patch to
>>>> master branch as well.
>>>> 
>>>> But when I started porting it to other branches(HBase versions less
>> than
>>>> 1.2) I realized that write to same region is in preBatchMutate is not
>>>> allowed by HBase versions less than 1.2. There is a dependency of
>>>> HBASE-15600 to make the new local index implementation work properly
>> for
>>>> versions less than 1.2. So I am not able to port it to other branches.
>> I
>>>> will complete it once the next HBase point releases have HBASE-15600.
>>>> 
>>>> Now code in master branch is not in sync with other branches but the
>>> local
>>>> indexing feature is much more stable in master branch after
>> PHOENIX-1734
>>> so
>>>> I think it would be better to have PHOENIX-1734 in master branch even
>>>> porting it to other branches might delay till upcoming HBase releases
>>>> available.
>>>> 
>>>> Wdyt?
>>>> 
>>>> Thanks,
>>>> Rajeshbabu.
>> 

Re: Regarding PHOENIX-1734

Posted by James Taylor <ja...@apache.org>.
Another factor is documentation. It's hard enough to document new features
and behavior as it is, but we'd have to explain that for the *same*
release, it depends on which version of HBase you're using to know how to
correctly configure local indexes - sometimes you need modify
the hbase-site.xml on the master and sometimes you wouldn't.

It's one thing if we're EOLing these branches, but I don't think we're at
that point yet.

On Wednesday, June 8, 2016, Samarth Jain <sa...@gmail.com> wrote:

> Considering the footprint of PHOENIX-1734, it would make development
> onerous if the feature is merged only to 4.x-HBase-1.2. There is already an
> overhead of having to merge patches in 4 branches. It also has the
> potential of increasing the likelihood of bugs being introduced arising out
> of having to handle many more merge conflicts.
>
> Just my 2 cents :)
>
>
>
> On Tue, Jun 7, 2016 at 8:42 AM, James Taylor <jamestaylor@apache.org
> <javascript:;>> wrote:
>
> > I don't understand why HBASE-15600 is needed if we're ok with the local
> > index updates not being transactional with older versions of HBase. Why
> > can't we do the htable.batch() call in ParallelWriterIndexCommitter as we
> > do now? The row keys are different. If this is an issue, then we could
> just
> > enqueue the mutations and let them be submitted outside the row lock. We
> > don't need to have the row lock.
> >
> > Seems to me that it could be made to work without the branches diverging
> so
> > radically.
> >
> > Thanks,
> > James
> >
> > On Tuesday, June 7, 2016, rajeshbabu@apache.org <javascript:;> <
> chrajeshbabu32@gmail.com <javascript:;>>
> > wrote:
> >
> > > Hi Team,
> > >
> > > As you know prior to PHOENIX-1734 we are storing all  local index data
> in
> > > separate table and colocate data and index regions. To ensure proper
> > > colocation we need customer balancer, custom split and merge logic
> still
> > > these will not ensure 100% colocation. That's why we started working
> > > PHOENIX-1734 to store local index data in separate column families in
> the
> > > same table to ensure 100% colocation.
> > >
> > > As part of PHOENIX-1734 we need to write local index udpates to same
> > region
> > > in preBatchMutate. When I am working with master branch(The HBase
> version
> > > is 1.2.x) it went smooth and I continued the development and ensure new
> > > implementation of local index works properly and committed the patch to
> > > master branch as well.
> > >
> > > But when I started porting it to other branches(HBase versions less
> than
> > > 1.2) I realized that write to same region is in preBatchMutate is not
> > > allowed by HBase versions less than 1.2. There is a dependency of
> > > HBASE-15600 to make the new local index implementation work properly
> for
> > > versions less than 1.2. So I am not able to port it to other branches.
> I
> > > will complete it once the next HBase point releases have HBASE-15600.
> > >
> > > Now code in master branch is not in sync with other branches but the
> > local
> > > indexing feature is much more stable in master branch after
> PHOENIX-1734
> > so
> > > I think it would be better to have PHOENIX-1734 in master branch even
> > > porting it to other branches might delay till upcoming HBase releases
> > > available.
> > >
> > > Wdyt?
> > >
> > > Thanks,
> > > Rajeshbabu.
> > >
> >
>

Re: [DISCUSS] Regarding PHOENIX-1734

Posted by Samarth Jain <sa...@gmail.com>.
Considering the footprint of PHOENIX-1734, it would make development
onerous if the feature is merged only to 4.x-HBase-1.2. There is already an
overhead of having to merge patches in 4 branches. It also has the
potential of increasing the likelihood of bugs being introduced arising out
of having to handle many more merge conflicts.

Just my 2 cents :)



On Tue, Jun 7, 2016 at 8:42 AM, James Taylor <ja...@apache.org> wrote:

> I don't understand why HBASE-15600 is needed if we're ok with the local
> index updates not being transactional with older versions of HBase. Why
> can't we do the htable.batch() call in ParallelWriterIndexCommitter as we
> do now? The row keys are different. If this is an issue, then we could just
> enqueue the mutations and let them be submitted outside the row lock. We
> don't need to have the row lock.
>
> Seems to me that it could be made to work without the branches diverging so
> radically.
>
> Thanks,
> James
>
> On Tuesday, June 7, 2016, rajeshbabu@apache.org <ch...@gmail.com>
> wrote:
>
> > Hi Team,
> >
> > As you know prior to PHOENIX-1734 we are storing all  local index data in
> > separate table and colocate data and index regions. To ensure proper
> > colocation we need customer balancer, custom split and merge logic still
> > these will not ensure 100% colocation. That's why we started working
> > PHOENIX-1734 to store local index data in separate column families in the
> > same table to ensure 100% colocation.
> >
> > As part of PHOENIX-1734 we need to write local index udpates to same
> region
> > in preBatchMutate. When I am working with master branch(The HBase version
> > is 1.2.x) it went smooth and I continued the development and ensure new
> > implementation of local index works properly and committed the patch to
> > master branch as well.
> >
> > But when I started porting it to other branches(HBase versions less than
> > 1.2) I realized that write to same region is in preBatchMutate is not
> > allowed by HBase versions less than 1.2. There is a dependency of
> > HBASE-15600 to make the new local index implementation work properly for
> > versions less than 1.2. So I am not able to port it to other branches. I
> > will complete it once the next HBase point releases have HBASE-15600.
> >
> > Now code in master branch is not in sync with other branches but the
> local
> > indexing feature is much more stable in master branch after PHOENIX-1734
> so
> > I think it would be better to have PHOENIX-1734 in master branch even
> > porting it to other branches might delay till upcoming HBase releases
> > available.
> >
> > Wdyt?
> >
> > Thanks,
> > Rajeshbabu.
> >
>