You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by Sergey Soldatov <se...@gmail.com> on 2017/10/09 19:36:47 UTC
[DISCUSS] dropping 0.98 support?
I remember that we discussed that a couple times already without any final
decision, so let me raise this question again. HBase 2.0 is getting close
to the release and to support it we will need to do a lot of refactoring in
the code even just to get Phoenix compiled. And already we may start to
remove all deprecated API that was removed from 2.0 to minimize the further
changes that are directly related to 2.0 support.
Thoughts?
Thanks,
Sergey
Re: [DISCUSS] dropping 0.98 support?
Posted by Josh Elser <el...@apache.org>.
+1 to the "getting lighter" sentiment.
Selectively bringing back stuff to 0.98 is a nice way to keep the burden
low.
On 10/10/17 1:32 PM, James Taylor wrote:
> I'm -1 on a shim layer for 0.98. It would needlessly complicate the code
> when all we need for the 0.98 branch are critical bug fixes. It's not just
> the effort of putting it in place, but the complication of understanding it
> going forward (not to mention the testing efforts - we've spent a huge
> effort just getting our unit tests passing again). It puts a tax on future
> commits that's too high.
>
> As I mentioned in my previous email, I don't believe we need to do 1.1 or
> 1.2 releases. I don't think we should spend time keeping the branches in
> sync - it's too much effort for too little value.
>
> I think instead we should focus on a 5.0 release that sheds as much baggage
> as possible. We can do it in a manner that still supports rolling restarts
> from the last 4.x release.
>
> We need to get lighter, not heavier given the limited resources we have.
>
>
>
> On Tue, Oct 10, 2017 at 10:18 AM, Andrew Purtell <ap...@apache.org>
> wrote:
>
>> I volunteer to develop and maintain a shim for 0.98 if nobody else will. If
>> Phoenix is going to shim anyway for 1.2 vs 1.3 vs 1.4 vs ... then we need
>> one for 0.98 for as long as we have 0.98 in production where I work, and
>> I'm pessimistically estimating another year at least in some places. I
>> think a year is too long to go without taking on at least a subset of new
>> features.
>>
>> We can have varying support for server-side features by shim. That is an
>> interesting compromise. I'm thinking especially of local indexes. Note that
>> at some point we (at my workplace) will only have 0.98 on the client side,
>> so it doesn't matter if we are not supporting something on the server in
>> the 0.98 shim as long as the client can drive the necessary query plans.
>> The shims for things like server side support for local indexes could throw
>> UnsupportedOperationException to signal that the feature(s) do not have the
>> necessary server side support. In such a scenario if running a 1.3 client
>> and a 0.98 server, for example, attempting to use such a feature would fail
>> with runtime errors as expected and documented. If running a 0.98 client
>> and a 1.3 server, for example, then there would be no problem.
>>
>> Like I said, if Phoenix is going to shim anyway to handle the differences
>> in the various 1.x, which I think is a good idea, because there are going
>> to be more I sadly promise you, then adding one more shim for 0.98 isn't
>> onerous, if you have a volunteer to do the work, and you do.
>>
>> On Tue, Oct 10, 2017 at 10:12 AM, James Taylor <ja...@apache.org>
>> wrote:
>>
>>> We won't need features ported over to the 0.98 branch, so I think it's
>>> better all around if we just let the branches diverge rather than
>>> implementing some kind of shim layer. We'll likely just need to do one or
>>> two point releases on 0.98. We can do that from the 4.12-HBase-0.98
>> branch.
>>>
>>> Here are some thoughts I have on releases going forward:
>>> - stop doing parallel releases on 0.98, 1.1, 1.2, and 1.3.
>>> - stop committing across all branches
>>> - if someone out there needs 1.1 and 1.2 release, they can volunteer as
>> RM
>>> and back port what they need. There's very little due diligence done on
>>> those releases currently, so there's not a lot of value IMHO.
>>> - continue forward with releases on master for 1.3.
>>> - consider a shim layer when 1.4 releases land.
>>> - target a 5.0 release as there are some big improvements coming to the
>>> system catalog in PHOENIX-4198 and PHOENIX-3534. We need a volunteer for
>> RM
>>> - I've done 14 or so of our 17 releases so it's time for someone else to
>>> step up.
>>>
>>> Thanks,
>>> James
>>>
>>> On Mon, Oct 9, 2017 at 8:02 PM, Sergey Soldatov <
>> sergeysoldatov@gmail.com>
>>> wrote:
>>>
>>>> Heh. I just remember that last time in August you said that you already
>>>> migrating to more recent version of HBase and have no problems to
>>> maintain
>>>> 0.98 internally during for short time. As an alternative can we add
>>> shims,
>>>> so most of the difference in API can be grouped there and will be easy
>> to
>>>> maintain. Most of the changes are related to removing deprecated API
>> in
>>>> Put/Get/Delete and name changes like HAdmin -> Admin, so it may be
>>> covered
>>>> by a couple of support classes. After that it will be possible to get
>>>> Phoenix compiled against 2.0 with some little changes related to shaded
>>>> coprocessor API (possible may be added to shims as well) and tephra
>> which
>>>> is also using deprecated API.
>>>>
>>>> Thanks,
>>>> Sergey
>>>>
>>>> On Tue, Oct 10, 2017 at 4:49 AM, Andrew Purtell <ap...@apache.org>
>>>> wrote:
>>>>
>>>>> Until we migrate our production at Salesforce off of 0.98 we will
>> still
>>>>> have to support 0.98 internally, and a number of our staff are
>>>> committers,
>>>>> so I suspect there will be adequate support for the 0.98 branch for
>>>> another
>>>>> couple of releases.
>>>>>
>>>>> On Mon, Oct 9, 2017 at 12:36 PM, Sergey Soldatov <
>>>> sergeysoldatov@gmail.com
>>>>>>
>>>>> wrote:
>>>>>
>>>>>> I remember that we discussed that a couple times already without
>> any
>>>>> final
>>>>>> decision, so let me raise this question again. HBase 2.0 is getting
>>>> close
>>>>>> to the release and to support it we will need to do a lot of
>>>> refactoring
>>>>> in
>>>>>> the code even just to get Phoenix compiled. And already we may
>> start
>>> to
>>>>>> remove all deprecated API that was removed from 2.0 to minimize the
>>>>> further
>>>>>> changes that are directly related to 2.0 support.
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>> Thanks,
>>>>>> Sergey
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Andrew
>>>>>
>>>>> Words like orphans lost among the crosstalk, meaning torn from
>> truth's
>>>>> decrepit hands
>>>>> - A23, Crosstalk
>>>>>
>>>>
>>>
>>
>>
>>
>> --
>> Best regards,
>> Andrew
>>
>> Words like orphans lost among the crosstalk, meaning torn from truth's
>> decrepit hands
>> - A23, Crosstalk
>>
>
Re: [DISCUSS] dropping 0.98 support?
Posted by James Taylor <ja...@apache.org>.
I'm -1 on a shim layer for 0.98. It would needlessly complicate the code
when all we need for the 0.98 branch are critical bug fixes. It's not just
the effort of putting it in place, but the complication of understanding it
going forward (not to mention the testing efforts - we've spent a huge
effort just getting our unit tests passing again). It puts a tax on future
commits that's too high.
As I mentioned in my previous email, I don't believe we need to do 1.1 or
1.2 releases. I don't think we should spend time keeping the branches in
sync - it's too much effort for too little value.
I think instead we should focus on a 5.0 release that sheds as much baggage
as possible. We can do it in a manner that still supports rolling restarts
from the last 4.x release.
We need to get lighter, not heavier given the limited resources we have.
On Tue, Oct 10, 2017 at 10:18 AM, Andrew Purtell <ap...@apache.org>
wrote:
> I volunteer to develop and maintain a shim for 0.98 if nobody else will. If
> Phoenix is going to shim anyway for 1.2 vs 1.3 vs 1.4 vs ... then we need
> one for 0.98 for as long as we have 0.98 in production where I work, and
> I'm pessimistically estimating another year at least in some places. I
> think a year is too long to go without taking on at least a subset of new
> features.
>
> We can have varying support for server-side features by shim. That is an
> interesting compromise. I'm thinking especially of local indexes. Note that
> at some point we (at my workplace) will only have 0.98 on the client side,
> so it doesn't matter if we are not supporting something on the server in
> the 0.98 shim as long as the client can drive the necessary query plans.
> The shims for things like server side support for local indexes could throw
> UnsupportedOperationException to signal that the feature(s) do not have the
> necessary server side support. In such a scenario if running a 1.3 client
> and a 0.98 server, for example, attempting to use such a feature would fail
> with runtime errors as expected and documented. If running a 0.98 client
> and a 1.3 server, for example, then there would be no problem.
>
> Like I said, if Phoenix is going to shim anyway to handle the differences
> in the various 1.x, which I think is a good idea, because there are going
> to be more I sadly promise you, then adding one more shim for 0.98 isn't
> onerous, if you have a volunteer to do the work, and you do.
>
> On Tue, Oct 10, 2017 at 10:12 AM, James Taylor <ja...@apache.org>
> wrote:
>
> > We won't need features ported over to the 0.98 branch, so I think it's
> > better all around if we just let the branches diverge rather than
> > implementing some kind of shim layer. We'll likely just need to do one or
> > two point releases on 0.98. We can do that from the 4.12-HBase-0.98
> branch.
> >
> > Here are some thoughts I have on releases going forward:
> > - stop doing parallel releases on 0.98, 1.1, 1.2, and 1.3.
> > - stop committing across all branches
> > - if someone out there needs 1.1 and 1.2 release, they can volunteer as
> RM
> > and back port what they need. There's very little due diligence done on
> > those releases currently, so there's not a lot of value IMHO.
> > - continue forward with releases on master for 1.3.
> > - consider a shim layer when 1.4 releases land.
> > - target a 5.0 release as there are some big improvements coming to the
> > system catalog in PHOENIX-4198 and PHOENIX-3534. We need a volunteer for
> RM
> > - I've done 14 or so of our 17 releases so it's time for someone else to
> > step up.
> >
> > Thanks,
> > James
> >
> > On Mon, Oct 9, 2017 at 8:02 PM, Sergey Soldatov <
> sergeysoldatov@gmail.com>
> > wrote:
> >
> > > Heh. I just remember that last time in August you said that you already
> > > migrating to more recent version of HBase and have no problems to
> > maintain
> > > 0.98 internally during for short time. As an alternative can we add
> > shims,
> > > so most of the difference in API can be grouped there and will be easy
> to
> > > maintain. Most of the changes are related to removing deprecated API
> in
> > > Put/Get/Delete and name changes like HAdmin -> Admin, so it may be
> > covered
> > > by a couple of support classes. After that it will be possible to get
> > > Phoenix compiled against 2.0 with some little changes related to shaded
> > > coprocessor API (possible may be added to shims as well) and tephra
> which
> > > is also using deprecated API.
> > >
> > > Thanks,
> > > Sergey
> > >
> > > On Tue, Oct 10, 2017 at 4:49 AM, Andrew Purtell <ap...@apache.org>
> > > wrote:
> > >
> > > > Until we migrate our production at Salesforce off of 0.98 we will
> still
> > > > have to support 0.98 internally, and a number of our staff are
> > > committers,
> > > > so I suspect there will be adequate support for the 0.98 branch for
> > > another
> > > > couple of releases.
> > > >
> > > > On Mon, Oct 9, 2017 at 12:36 PM, Sergey Soldatov <
> > > sergeysoldatov@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > I remember that we discussed that a couple times already without
> any
> > > > final
> > > > > decision, so let me raise this question again. HBase 2.0 is getting
> > > close
> > > > > to the release and to support it we will need to do a lot of
> > > refactoring
> > > > in
> > > > > the code even just to get Phoenix compiled. And already we may
> start
> > to
> > > > > remove all deprecated API that was removed from 2.0 to minimize the
> > > > further
> > > > > changes that are directly related to 2.0 support.
> > > > >
> > > > > Thoughts?
> > > > >
> > > > > Thanks,
> > > > > Sergey
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Andrew
> > > >
> > > > Words like orphans lost among the crosstalk, meaning torn from
> truth's
> > > > decrepit hands
> > > > - A23, Crosstalk
> > > >
> > >
> >
>
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
> - A23, Crosstalk
>
Re: [DISCUSS] dropping 0.98 support?
Posted by Andrew Purtell <ap...@apache.org>.
I volunteer to develop and maintain a shim for 0.98 if nobody else will. If
Phoenix is going to shim anyway for 1.2 vs 1.3 vs 1.4 vs ... then we need
one for 0.98 for as long as we have 0.98 in production where I work, and
I'm pessimistically estimating another year at least in some places. I
think a year is too long to go without taking on at least a subset of new
features.
We can have varying support for server-side features by shim. That is an
interesting compromise. I'm thinking especially of local indexes. Note that
at some point we (at my workplace) will only have 0.98 on the client side,
so it doesn't matter if we are not supporting something on the server in
the 0.98 shim as long as the client can drive the necessary query plans.
The shims for things like server side support for local indexes could throw
UnsupportedOperationException to signal that the feature(s) do not have the
necessary server side support. In such a scenario if running a 1.3 client
and a 0.98 server, for example, attempting to use such a feature would fail
with runtime errors as expected and documented. If running a 0.98 client
and a 1.3 server, for example, then there would be no problem.
Like I said, if Phoenix is going to shim anyway to handle the differences
in the various 1.x, which I think is a good idea, because there are going
to be more I sadly promise you, then adding one more shim for 0.98 isn't
onerous, if you have a volunteer to do the work, and you do.
On Tue, Oct 10, 2017 at 10:12 AM, James Taylor <ja...@apache.org>
wrote:
> We won't need features ported over to the 0.98 branch, so I think it's
> better all around if we just let the branches diverge rather than
> implementing some kind of shim layer. We'll likely just need to do one or
> two point releases on 0.98. We can do that from the 4.12-HBase-0.98 branch.
>
> Here are some thoughts I have on releases going forward:
> - stop doing parallel releases on 0.98, 1.1, 1.2, and 1.3.
> - stop committing across all branches
> - if someone out there needs 1.1 and 1.2 release, they can volunteer as RM
> and back port what they need. There's very little due diligence done on
> those releases currently, so there's not a lot of value IMHO.
> - continue forward with releases on master for 1.3.
> - consider a shim layer when 1.4 releases land.
> - target a 5.0 release as there are some big improvements coming to the
> system catalog in PHOENIX-4198 and PHOENIX-3534. We need a volunteer for RM
> - I've done 14 or so of our 17 releases so it's time for someone else to
> step up.
>
> Thanks,
> James
>
> On Mon, Oct 9, 2017 at 8:02 PM, Sergey Soldatov <se...@gmail.com>
> wrote:
>
> > Heh. I just remember that last time in August you said that you already
> > migrating to more recent version of HBase and have no problems to
> maintain
> > 0.98 internally during for short time. As an alternative can we add
> shims,
> > so most of the difference in API can be grouped there and will be easy to
> > maintain. Most of the changes are related to removing deprecated API in
> > Put/Get/Delete and name changes like HAdmin -> Admin, so it may be
> covered
> > by a couple of support classes. After that it will be possible to get
> > Phoenix compiled against 2.0 with some little changes related to shaded
> > coprocessor API (possible may be added to shims as well) and tephra which
> > is also using deprecated API.
> >
> > Thanks,
> > Sergey
> >
> > On Tue, Oct 10, 2017 at 4:49 AM, Andrew Purtell <ap...@apache.org>
> > wrote:
> >
> > > Until we migrate our production at Salesforce off of 0.98 we will still
> > > have to support 0.98 internally, and a number of our staff are
> > committers,
> > > so I suspect there will be adequate support for the 0.98 branch for
> > another
> > > couple of releases.
> > >
> > > On Mon, Oct 9, 2017 at 12:36 PM, Sergey Soldatov <
> > sergeysoldatov@gmail.com
> > > >
> > > wrote:
> > >
> > > > I remember that we discussed that a couple times already without any
> > > final
> > > > decision, so let me raise this question again. HBase 2.0 is getting
> > close
> > > > to the release and to support it we will need to do a lot of
> > refactoring
> > > in
> > > > the code even just to get Phoenix compiled. And already we may start
> to
> > > > remove all deprecated API that was removed from 2.0 to minimize the
> > > further
> > > > changes that are directly related to 2.0 support.
> > > >
> > > > Thoughts?
> > > >
> > > > Thanks,
> > > > Sergey
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Andrew
> > >
> > > Words like orphans lost among the crosstalk, meaning torn from truth's
> > > decrepit hands
> > > - A23, Crosstalk
> > >
> >
>
--
Best regards,
Andrew
Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
- A23, Crosstalk
Re: [DISCUSS] dropping 0.98 support?
Posted by James Taylor <ja...@apache.org>.
We won't need features ported over to the 0.98 branch, so I think it's
better all around if we just let the branches diverge rather than
implementing some kind of shim layer. We'll likely just need to do one or
two point releases on 0.98. We can do that from the 4.12-HBase-0.98 branch.
Here are some thoughts I have on releases going forward:
- stop doing parallel releases on 0.98, 1.1, 1.2, and 1.3.
- stop committing across all branches
- if someone out there needs 1.1 and 1.2 release, they can volunteer as RM
and back port what they need. There's very little due diligence done on
those releases currently, so there's not a lot of value IMHO.
- continue forward with releases on master for 1.3.
- consider a shim layer when 1.4 releases land.
- target a 5.0 release as there are some big improvements coming to the
system catalog in PHOENIX-4198 and PHOENIX-3534. We need a volunteer for RM
- I've done 14 or so of our 17 releases so it's time for someone else to
step up.
Thanks,
James
On Mon, Oct 9, 2017 at 8:02 PM, Sergey Soldatov <se...@gmail.com>
wrote:
> Heh. I just remember that last time in August you said that you already
> migrating to more recent version of HBase and have no problems to maintain
> 0.98 internally during for short time. As an alternative can we add shims,
> so most of the difference in API can be grouped there and will be easy to
> maintain. Most of the changes are related to removing deprecated API in
> Put/Get/Delete and name changes like HAdmin -> Admin, so it may be covered
> by a couple of support classes. After that it will be possible to get
> Phoenix compiled against 2.0 with some little changes related to shaded
> coprocessor API (possible may be added to shims as well) and tephra which
> is also using deprecated API.
>
> Thanks,
> Sergey
>
> On Tue, Oct 10, 2017 at 4:49 AM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> > Until we migrate our production at Salesforce off of 0.98 we will still
> > have to support 0.98 internally, and a number of our staff are
> committers,
> > so I suspect there will be adequate support for the 0.98 branch for
> another
> > couple of releases.
> >
> > On Mon, Oct 9, 2017 at 12:36 PM, Sergey Soldatov <
> sergeysoldatov@gmail.com
> > >
> > wrote:
> >
> > > I remember that we discussed that a couple times already without any
> > final
> > > decision, so let me raise this question again. HBase 2.0 is getting
> close
> > > to the release and to support it we will need to do a lot of
> refactoring
> > in
> > > the code even just to get Phoenix compiled. And already we may start to
> > > remove all deprecated API that was removed from 2.0 to minimize the
> > further
> > > changes that are directly related to 2.0 support.
> > >
> > > Thoughts?
> > >
> > > Thanks,
> > > Sergey
> > >
> >
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Words like orphans lost among the crosstalk, meaning torn from truth's
> > decrepit hands
> > - A23, Crosstalk
> >
>
Re: [DISCUSS] dropping 0.98 support?
Posted by Sergey Soldatov <se...@gmail.com>.
Heh. I just remember that last time in August you said that you already
migrating to more recent version of HBase and have no problems to maintain
0.98 internally during for short time. As an alternative can we add shims,
so most of the difference in API can be grouped there and will be easy to
maintain. Most of the changes are related to removing deprecated API in
Put/Get/Delete and name changes like HAdmin -> Admin, so it may be covered
by a couple of support classes. After that it will be possible to get
Phoenix compiled against 2.0 with some little changes related to shaded
coprocessor API (possible may be added to shims as well) and tephra which
is also using deprecated API.
Thanks,
Sergey
On Tue, Oct 10, 2017 at 4:49 AM, Andrew Purtell <ap...@apache.org> wrote:
> Until we migrate our production at Salesforce off of 0.98 we will still
> have to support 0.98 internally, and a number of our staff are committers,
> so I suspect there will be adequate support for the 0.98 branch for another
> couple of releases.
>
> On Mon, Oct 9, 2017 at 12:36 PM, Sergey Soldatov <sergeysoldatov@gmail.com
> >
> wrote:
>
> > I remember that we discussed that a couple times already without any
> final
> > decision, so let me raise this question again. HBase 2.0 is getting close
> > to the release and to support it we will need to do a lot of refactoring
> in
> > the code even just to get Phoenix compiled. And already we may start to
> > remove all deprecated API that was removed from 2.0 to minimize the
> further
> > changes that are directly related to 2.0 support.
> >
> > Thoughts?
> >
> > Thanks,
> > Sergey
> >
>
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
> - A23, Crosstalk
>
Re: [DISCUSS] dropping 0.98 support?
Posted by Andrew Purtell <ap...@apache.org>.
Until we migrate our production at Salesforce off of 0.98 we will still
have to support 0.98 internally, and a number of our staff are committers,
so I suspect there will be adequate support for the 0.98 branch for another
couple of releases.
On Mon, Oct 9, 2017 at 12:36 PM, Sergey Soldatov <se...@gmail.com>
wrote:
> I remember that we discussed that a couple times already without any final
> decision, so let me raise this question again. HBase 2.0 is getting close
> to the release and to support it we will need to do a lot of refactoring in
> the code even just to get Phoenix compiled. And already we may start to
> remove all deprecated API that was removed from 2.0 to minimize the further
> changes that are directly related to 2.0 support.
>
> Thoughts?
>
> Thanks,
> Sergey
>
--
Best regards,
Andrew
Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
- A23, Crosstalk