You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Ted Yu <yu...@gmail.com> on 2012/07/07 16:23:37 UTC
deprecating (old) metrics in favor of metrics2 framework
Hi,
See the following comment for background:
https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
metrics2 framework is in both hadoop 1.0 and hadoop 2.0
We would like to migrate to metrics2 framework. But maintaining two metrics
frameworks in HBase is costly and results in lower readability.
However, if you have cluster monitoring tool that depends on (old) metrics
framework, we want to give you smooth migration experience.
Please provide your opinions on whether (old) metrics framework should be
deprecated in 0.94 or in 0.96.
Thanks
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by lars hofhansl <lh...@yahoo.com>.
It seems that if we want to make that switch 0.96 (the singularity) should be release.
Sorry, I should give a clear vote.
+1 on metrics2 in 0.96 only (if we're ready by then of course)
________________________________
From: Ted Yu <yu...@gmail.com>
To: dev@hbase.apache.org; lars hofhansl <lh...@yahoo.com>
Sent: Monday, July 9, 2012 4:52 PM
Subject: Re: deprecating (old) metrics in favor of metrics2 framework
Lars:
Do you think 0.96 should only support metrics2 framework ?
Cheers
On Mon, Jul 9, 2012 at 4:47 PM, lars hofhansl <lh...@yahoo.com> wrote:
Let's not deprecate until the replacement is ready (including docs).
>
>-0
>
>
>-- Lars
>
>
>
>----- Original Message -----
>From: Elliott Clark <ec...@stumbleupon.com>
>To: dev@hbase.apache.org
>Cc:
>Sent: Monday, July 9, 2012 2:34 PM
>Subject: Re: deprecating (old) metrics in favor of metrics2 framework
>
>
>HBASE-6323 has the first Metrics2 implementation that I've gotten to. We
>will have to add lots of documentation around hadoop-metrics2.properties
>and the jmx locations are not consistent, however Metrics2 is much nicer to
>use for dynamic metrics.
>
>On Mon, Jul 9, 2012 at 12:28 PM, Ted Yu <yu...@gmail.com> wrote:
>
>> If I don't get objection by Friday the 13th @ mid night (PST), I will
>> create a JIRA that marks HBase metric classes that depend on
>> org.apache.hadoop.metrics.* deprecated.
>>
>> Thanks
>>
>> On Mon, Jul 9, 2012 at 9:35 AM, Ted Yu <yu...@gmail.com> wrote:
>>
>> > Thanks for the response, Jon.
>> >
>> > One more detail: the metrics2 classes in hadoop 1.0 and hadoop 2.0 are
>> > different. Alex is working on a shim layer to make the difference
>> > transparent. If we were to keep 0.96 metric (1) compatible, we would be
>> > maintaining 2.5 metric systems.
>> >
>> > Cheers
>> >
>> >
>> > On Mon, Jul 9, 2012 at 9:28 AM, Jonathan Hsieh <jo...@cloudera.com> wrote:
>> >
>> >> I'm ok if we moved solely to metrics2 in 0.96 and if we deprecated in
>> >> 0.94.
>> >>
>> >> Jon.
>> >>
>> >> On Sat, Jul 7, 2012 at 7:23 AM, Ted Yu <yu...@gmail.com> wrote:
>> >>
>> >> > Hi,
>> >> > See the following comment for background:
>> >> >
>> >> >
>> >>
>> https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
>> >> >
>> >> > metrics2 framework is in both hadoop 1.0 and hadoop 2.0
>> >> > We would like to migrate to metrics2 framework. But maintaining two
>> >> metrics
>> >> > frameworks in HBase is costly and results in lower readability.
>> >> >
>> >> > However, if you have cluster monitoring tool that depends on (old)
>> >> metrics
>> >> > framework, we want to give you smooth migration experience.
>> >> >
>> >> > Please provide your opinions on whether (old) metrics framework should
>> >> be
>> >> > deprecated in 0.94 or in 0.96.
>> >> >
>> >> > Thanks
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> // Jonathan Hsieh (shay)
>> >> // Software Engineer, Cloudera
>> >> // jon@cloudera.com
>> >>
>> >
>> >
>>
>
>
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Ted Yu <yu...@gmail.com>.
Lars:
Do you think 0.96 should only support metrics2 framework ?
Cheers
On Mon, Jul 9, 2012 at 4:47 PM, lars hofhansl <lh...@yahoo.com> wrote:
> Let's not deprecate until the replacement is ready (including docs).
>
> -0
>
>
> -- Lars
>
>
> ----- Original Message -----
> From: Elliott Clark <ec...@stumbleupon.com>
> To: dev@hbase.apache.org
> Cc:
> Sent: Monday, July 9, 2012 2:34 PM
> Subject: Re: deprecating (old) metrics in favor of metrics2 framework
>
> HBASE-6323 has the first Metrics2 implementation that I've gotten to. We
> will have to add lots of documentation around hadoop-metrics2.properties
> and the jmx locations are not consistent, however Metrics2 is much nicer to
> use for dynamic metrics.
>
> On Mon, Jul 9, 2012 at 12:28 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > If I don't get objection by Friday the 13th @ mid night (PST), I will
> > create a JIRA that marks HBase metric classes that depend on
> > org.apache.hadoop.metrics.* deprecated.
> >
> > Thanks
> >
> > On Mon, Jul 9, 2012 at 9:35 AM, Ted Yu <yu...@gmail.com> wrote:
> >
> > > Thanks for the response, Jon.
> > >
> > > One more detail: the metrics2 classes in hadoop 1.0 and hadoop 2.0 are
> > > different. Alex is working on a shim layer to make the difference
> > > transparent. If we were to keep 0.96 metric (1) compatible, we would be
> > > maintaining 2.5 metric systems.
> > >
> > > Cheers
> > >
> > >
> > > On Mon, Jul 9, 2012 at 9:28 AM, Jonathan Hsieh <jo...@cloudera.com>
> wrote:
> > >
> > >> I'm ok if we moved solely to metrics2 in 0.96 and if we deprecated in
> > >> 0.94.
> > >>
> > >> Jon.
> > >>
> > >> On Sat, Jul 7, 2012 at 7:23 AM, Ted Yu <yu...@gmail.com> wrote:
> > >>
> > >> > Hi,
> > >> > See the following comment for background:
> > >> >
> > >> >
> > >>
> >
> https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
> > >> >
> > >> > metrics2 framework is in both hadoop 1.0 and hadoop 2.0
> > >> > We would like to migrate to metrics2 framework. But maintaining two
> > >> metrics
> > >> > frameworks in HBase is costly and results in lower readability.
> > >> >
> > >> > However, if you have cluster monitoring tool that depends on (old)
> > >> metrics
> > >> > framework, we want to give you smooth migration experience.
> > >> >
> > >> > Please provide your opinions on whether (old) metrics framework
> should
> > >> be
> > >> > deprecated in 0.94 or in 0.96.
> > >> >
> > >> > Thanks
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> // Jonathan Hsieh (shay)
> > >> // Software Engineer, Cloudera
> > >> // jon@cloudera.com
> > >>
> > >
> > >
> >
>
>
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by lars hofhansl <lh...@yahoo.com>.
Let's not deprecate until the replacement is ready (including docs).
-0
-- Lars
----- Original Message -----
From: Elliott Clark <ec...@stumbleupon.com>
To: dev@hbase.apache.org
Cc:
Sent: Monday, July 9, 2012 2:34 PM
Subject: Re: deprecating (old) metrics in favor of metrics2 framework
HBASE-6323 has the first Metrics2 implementation that I've gotten to. We
will have to add lots of documentation around hadoop-metrics2.properties
and the jmx locations are not consistent, however Metrics2 is much nicer to
use for dynamic metrics.
On Mon, Jul 9, 2012 at 12:28 PM, Ted Yu <yu...@gmail.com> wrote:
> If I don't get objection by Friday the 13th @ mid night (PST), I will
> create a JIRA that marks HBase metric classes that depend on
> org.apache.hadoop.metrics.* deprecated.
>
> Thanks
>
> On Mon, Jul 9, 2012 at 9:35 AM, Ted Yu <yu...@gmail.com> wrote:
>
> > Thanks for the response, Jon.
> >
> > One more detail: the metrics2 classes in hadoop 1.0 and hadoop 2.0 are
> > different. Alex is working on a shim layer to make the difference
> > transparent. If we were to keep 0.96 metric (1) compatible, we would be
> > maintaining 2.5 metric systems.
> >
> > Cheers
> >
> >
> > On Mon, Jul 9, 2012 at 9:28 AM, Jonathan Hsieh <jo...@cloudera.com> wrote:
> >
> >> I'm ok if we moved solely to metrics2 in 0.96 and if we deprecated in
> >> 0.94.
> >>
> >> Jon.
> >>
> >> On Sat, Jul 7, 2012 at 7:23 AM, Ted Yu <yu...@gmail.com> wrote:
> >>
> >> > Hi,
> >> > See the following comment for background:
> >> >
> >> >
> >>
> https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
> >> >
> >> > metrics2 framework is in both hadoop 1.0 and hadoop 2.0
> >> > We would like to migrate to metrics2 framework. But maintaining two
> >> metrics
> >> > frameworks in HBase is costly and results in lower readability.
> >> >
> >> > However, if you have cluster monitoring tool that depends on (old)
> >> metrics
> >> > framework, we want to give you smooth migration experience.
> >> >
> >> > Please provide your opinions on whether (old) metrics framework should
> >> be
> >> > deprecated in 0.94 or in 0.96.
> >> >
> >> > Thanks
> >> >
> >>
> >>
> >>
> >> --
> >> // Jonathan Hsieh (shay)
> >> // Software Engineer, Cloudera
> >> // jon@cloudera.com
> >>
> >
> >
>
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Elliott Clark <ec...@stumbleupon.com>.
HBASE-6323 has the first Metrics2 implementation that I've gotten to. We
will have to add lots of documentation around hadoop-metrics2.properties
and the jmx locations are not consistent, however Metrics2 is much nicer to
use for dynamic metrics.
On Mon, Jul 9, 2012 at 12:28 PM, Ted Yu <yu...@gmail.com> wrote:
> If I don't get objection by Friday the 13th @ mid night (PST), I will
> create a JIRA that marks HBase metric classes that depend on
> org.apache.hadoop.metrics.* deprecated.
>
> Thanks
>
> On Mon, Jul 9, 2012 at 9:35 AM, Ted Yu <yu...@gmail.com> wrote:
>
> > Thanks for the response, Jon.
> >
> > One more detail: the metrics2 classes in hadoop 1.0 and hadoop 2.0 are
> > different. Alex is working on a shim layer to make the difference
> > transparent. If we were to keep 0.96 metric (1) compatible, we would be
> > maintaining 2.5 metric systems.
> >
> > Cheers
> >
> >
> > On Mon, Jul 9, 2012 at 9:28 AM, Jonathan Hsieh <jo...@cloudera.com> wrote:
> >
> >> I'm ok if we moved solely to metrics2 in 0.96 and if we deprecated in
> >> 0.94.
> >>
> >> Jon.
> >>
> >> On Sat, Jul 7, 2012 at 7:23 AM, Ted Yu <yu...@gmail.com> wrote:
> >>
> >> > Hi,
> >> > See the following comment for background:
> >> >
> >> >
> >>
> https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
> >> >
> >> > metrics2 framework is in both hadoop 1.0 and hadoop 2.0
> >> > We would like to migrate to metrics2 framework. But maintaining two
> >> metrics
> >> > frameworks in HBase is costly and results in lower readability.
> >> >
> >> > However, if you have cluster monitoring tool that depends on (old)
> >> metrics
> >> > framework, we want to give you smooth migration experience.
> >> >
> >> > Please provide your opinions on whether (old) metrics framework should
> >> be
> >> > deprecated in 0.94 or in 0.96.
> >> >
> >> > Thanks
> >> >
> >>
> >>
> >>
> >> --
> >> // Jonathan Hsieh (shay)
> >> // Software Engineer, Cloudera
> >> // jon@cloudera.com
> >>
> >
> >
>
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Ted Yu <yu...@gmail.com>.
If I don't get objection by Friday the 13th @ mid night (PST), I will
create a JIRA that marks HBase metric classes that depend on
org.apache.hadoop.metrics.* deprecated.
Thanks
On Mon, Jul 9, 2012 at 9:35 AM, Ted Yu <yu...@gmail.com> wrote:
> Thanks for the response, Jon.
>
> One more detail: the metrics2 classes in hadoop 1.0 and hadoop 2.0 are
> different. Alex is working on a shim layer to make the difference
> transparent. If we were to keep 0.96 metric (1) compatible, we would be
> maintaining 2.5 metric systems.
>
> Cheers
>
>
> On Mon, Jul 9, 2012 at 9:28 AM, Jonathan Hsieh <jo...@cloudera.com> wrote:
>
>> I'm ok if we moved solely to metrics2 in 0.96 and if we deprecated in
>> 0.94.
>>
>> Jon.
>>
>> On Sat, Jul 7, 2012 at 7:23 AM, Ted Yu <yu...@gmail.com> wrote:
>>
>> > Hi,
>> > See the following comment for background:
>> >
>> >
>> https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
>> >
>> > metrics2 framework is in both hadoop 1.0 and hadoop 2.0
>> > We would like to migrate to metrics2 framework. But maintaining two
>> metrics
>> > frameworks in HBase is costly and results in lower readability.
>> >
>> > However, if you have cluster monitoring tool that depends on (old)
>> metrics
>> > framework, we want to give you smooth migration experience.
>> >
>> > Please provide your opinions on whether (old) metrics framework should
>> be
>> > deprecated in 0.94 or in 0.96.
>> >
>> > Thanks
>> >
>>
>>
>>
>> --
>> // Jonathan Hsieh (shay)
>> // Software Engineer, Cloudera
>> // jon@cloudera.com
>>
>
>
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Ted Yu <yu...@gmail.com>.
Thanks for the response, Jon.
One more detail: the metrics2 classes in hadoop 1.0 and hadoop 2.0 are
different. Alex is working on a shim layer to make the difference
transparent. If we were to keep 0.96 metric (1) compatible, we would be
maintaining 2.5 metric systems.
Cheers
On Mon, Jul 9, 2012 at 9:28 AM, Jonathan Hsieh <jo...@cloudera.com> wrote:
> I'm ok if we moved solely to metrics2 in 0.96 and if we deprecated in 0.94.
>
> Jon.
>
> On Sat, Jul 7, 2012 at 7:23 AM, Ted Yu <yu...@gmail.com> wrote:
>
> > Hi,
> > See the following comment for background:
> >
> >
> https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
> >
> > metrics2 framework is in both hadoop 1.0 and hadoop 2.0
> > We would like to migrate to metrics2 framework. But maintaining two
> metrics
> > frameworks in HBase is costly and results in lower readability.
> >
> > However, if you have cluster monitoring tool that depends on (old)
> metrics
> > framework, we want to give you smooth migration experience.
> >
> > Please provide your opinions on whether (old) metrics framework should be
> > deprecated in 0.94 or in 0.96.
> >
> > Thanks
> >
>
>
>
> --
> // Jonathan Hsieh (shay)
> // Software Engineer, Cloudera
> // jon@cloudera.com
>
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Jonathan Hsieh <jo...@cloudera.com>.
I'm ok if we moved solely to metrics2 in 0.96 and if we deprecated in 0.94.
Jon.
On Sat, Jul 7, 2012 at 7:23 AM, Ted Yu <yu...@gmail.com> wrote:
> Hi,
> See the following comment for background:
>
> https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
>
> metrics2 framework is in both hadoop 1.0 and hadoop 2.0
> We would like to migrate to metrics2 framework. But maintaining two metrics
> frameworks in HBase is costly and results in lower readability.
>
> However, if you have cluster monitoring tool that depends on (old) metrics
> framework, we want to give you smooth migration experience.
>
> Please provide your opinions on whether (old) metrics framework should be
> deprecated in 0.94 or in 0.96.
>
> Thanks
>
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Elliott Clark <ec...@stumbleupon.com>.
I think Gary here, and Stack in the other thread have raised enough points
that if I could vote I would vote in favor of holding onto the classes that
might be used externally past 0.96.0. Right now there's no alternative to
several of our Metrics classes and the deprecations weren't out in 0.94.0.
With all that said I don't see a large down side with keeping those
classes around while the actual hbase code base starts using something else
(the metrics2 replacements). I added comment in HBASE-6365 with all of the
classes that I could find that would be deprecated (They all derive from or
use the old metrics code). There are a lot
On Tue, Jul 10, 2012 at 1:55 PM, Gary Helmling <gh...@gmail.com> wrote:
> I agree that having a new metrics2 implementation in 0.96 would be
> great to see and seems like a natural fit. I'm 100% for that. But I
> do think that having metrics2 and (deprecated) metrics v1 in the same
> release would be very helpful to users making the transition. So to
> me it seems more natural for 0.96 to be that release with both
> implementations, since that's where it seems like the metrics2
> implementation will land.
>
> Otherwise it seems like we risk introducing the same disruptions that
> Hadoop did when metrics2 initially replaced the metrics v1
> implementation, instead of living along side. This did cause us as a
> project some trouble until metrics v1 was added back in. So it would
> be unfortunate to repeat the same mistake ourselves.
>
> If there's considerable pain or overhead in having both
> implementations live in parallel, maybe it's worth doing a straight
> switch over in 0.96. I haven't looked at the differences enough
> myself to know. But otherwise it seems like an easier migration path
> to deprecate v1 in 0.96 and remove the release after.
>
>
> On Tue, Jul 10, 2012 at 1:35 PM, Ted Yu <yu...@gmail.com> wrote:
> > Gary:
> > Your comment makes sense.
> >
> > Part of this poll originates from the fact that 0.96 is our singularity
> > release. RPC, coprocessor, etc have undergone considerable changes.
> > Users migrating to 0.96 would have to deal with a lot of updates in their
> > codebase.
> >
> > It seems to me that doing all upgrades in one shot is almost the same as
> > upgrading components other than metrics framework.
> >
> > Cheers
> >
> > On Tue, Jul 10, 2012 at 1:29 PM, Gary Helmling <gh...@gmail.com>
> wrote:
> >
> >> >
> >> > Whether we support 2 (actually more than 2) metrics frameworks in 0.96
> >> can
> >> > be debated in the next 2 months.
> >> >
> >>
> >> I'm not sure I agree that deprecating without having something in
> >> place for users to move to makes sense.
> >>
> >> > As Todd mentioned in the thread 'HBase 0.94.1', we will try our best
> to
> >> > keep JMX interface the same across 0.94 and 0.96. Does this somehow
> >> reduce
> >> > the concern you raised ?
> >> >
> >>
> >> I think that maintaining consistency with the existing JMX naming
> >> conventions (to the extent possible) is important for operational
> >> concerns, but it's independent of the MetricsContext question and the
> >> question of whether other metrics classes of our own need a proper
> >> deprecation cycle.
> >>
> >> > As for using MetricsContext, I assume the user also uses hadoop in
> his /
> >> > her deployment. Then he / she should be aware of the deprecation of
> >> > metrics.* classes in both hadoop 1.0 and 2.0
> >> > Meaning he / she should be prepared to endorse metrics2 framework.
> >> >
> >>
> >> Hadoop deprecating metrics in favor of metrics2 is independent of us
> >> deprecating HBase metrics classes. TimeStampingFileContext is one
> >> MetricsContext implementation in HBase that would need to be
> >> deprecated and could be used or possibly extended by current users.
> >>
> >> Ultimately it's up to Lars H as RM for 0.94 to decide what he wants to
> >> include. It just feels to me like we're rushing to deprecate metrics
> >> in 0.94 so that it can be removed in 0.96, instead of what seems to me
> >> like the more standard path of deprecating metrics in 0.96, while also
> >> including new metrics2 implementations, which would give users a
> >> smoother path to actually switch over. I'm just not sure I understand
> >> the motivation for deprecating in 0.94 instead of 0.96.
> >>
>
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Ted Yu <yu...@gmail.com>.
Thanks for sharing this, Elliot.
7734 is marked blocked by HADOOP-7742: Evolve metrics2 in 0.23 and trunk to
allow coexistence with 0.20-security releases
On Tue, Jul 10, 2012 at 3:33 PM, Elliott Clark <ec...@stumbleupon.com>wrote:
> https://issues.apache.org/jira/browse/HADOOP-7734
>
> As far as I can tell this basically sinks all Metrics2 usage in HBase.
>
> On Tue, Jul 10, 2012 at 3:24 PM, Alex Baranau <alex.baranov.v@gmail.com
> >wrote:
>
> > > If there's considerable pain or overhead in having both
> > > implementations live in parallel, maybe it's worth doing a straight
> > > switch over in 0.96.
> >
> > As far as I can tell, they can live together easily. This should not be a
> > big issue. E.g. it should be much smaller issue than the fact that code
> > implemented on top of metrics2 will not compile against 1.0+, 2.0+ and
> 3.0+
> > hadoop at the same time because class names, etc. changed between 1.0 and
> > 2.0. But that's a separate story, will look at it tomorrow (Ted pointed
> me
> > to smth to look at).
> >
> > Alex Baranau
> >
> > On Tue, Jul 10, 2012 at 5:03 PM, Ted Yu <yu...@gmail.com> wrote:
> >
> > > Points taken.
> > >
> > > Thanks for the education of metrics framework history.
> > >
> > > On Tue, Jul 10, 2012 at 1:55 PM, Gary Helmling <gh...@gmail.com>
> > > wrote:
> > >
> > > > I agree that having a new metrics2 implementation in 0.96 would be
> > > > great to see and seems like a natural fit. I'm 100% for that. But I
> > > > do think that having metrics2 and (deprecated) metrics v1 in the same
> > > > release would be very helpful to users making the transition. So to
> > > > me it seems more natural for 0.96 to be that release with both
> > > > implementations, since that's where it seems like the metrics2
> > > > implementation will land.
> > > >
> > > > Otherwise it seems like we risk introducing the same disruptions that
> > > > Hadoop did when metrics2 initially replaced the metrics v1
> > > > implementation, instead of living along side. This did cause us as a
> > > > project some trouble until metrics v1 was added back in. So it would
> > > > be unfortunate to repeat the same mistake ourselves.
> > > >
> > > > If there's considerable pain or overhead in having both
> > > > implementations live in parallel, maybe it's worth doing a straight
> > > > switch over in 0.96. I haven't looked at the differences enough
> > > > myself to know. But otherwise it seems like an easier migration path
> > > > to deprecate v1 in 0.96 and remove the release after.
> > > >
> > > >
> > > > On Tue, Jul 10, 2012 at 1:35 PM, Ted Yu <yu...@gmail.com> wrote:
> > > > > Gary:
> > > > > Your comment makes sense.
> > > > >
> > > > > Part of this poll originates from the fact that 0.96 is our
> > singularity
> > > > > release. RPC, coprocessor, etc have undergone considerable changes.
> > > > > Users migrating to 0.96 would have to deal with a lot of updates in
> > > their
> > > > > codebase.
> > > > >
> > > > > It seems to me that doing all upgrades in one shot is almost the
> same
> > > as
> > > > > upgrading components other than metrics framework.
> > > > >
> > > > > Cheers
> > > > >
> > > > > On Tue, Jul 10, 2012 at 1:29 PM, Gary Helmling <
> ghelmling@gmail.com>
> > > > wrote:
> > > > >
> > > > >> >
> > > > >> > Whether we support 2 (actually more than 2) metrics frameworks
> in
> > > 0.96
> > > > >> can
> > > > >> > be debated in the next 2 months.
> > > > >> >
> > > > >>
> > > > >> I'm not sure I agree that deprecating without having something in
> > > > >> place for users to move to makes sense.
> > > > >>
> > > > >> > As Todd mentioned in the thread 'HBase 0.94.1', we will try our
> > best
> > > > to
> > > > >> > keep JMX interface the same across 0.94 and 0.96. Does this
> > somehow
> > > > >> reduce
> > > > >> > the concern you raised ?
> > > > >> >
> > > > >>
> > > > >> I think that maintaining consistency with the existing JMX naming
> > > > >> conventions (to the extent possible) is important for operational
> > > > >> concerns, but it's independent of the MetricsContext question and
> > the
> > > > >> question of whether other metrics classes of our own need a proper
> > > > >> deprecation cycle.
> > > > >>
> > > > >> > As for using MetricsContext, I assume the user also uses hadoop
> in
> > > > his /
> > > > >> > her deployment. Then he / she should be aware of the deprecation
> > of
> > > > >> > metrics.* classes in both hadoop 1.0 and 2.0
> > > > >> > Meaning he / she should be prepared to endorse metrics2
> framework.
> > > > >> >
> > > > >>
> > > > >> Hadoop deprecating metrics in favor of metrics2 is independent of
> us
> > > > >> deprecating HBase metrics classes. TimeStampingFileContext is one
> > > > >> MetricsContext implementation in HBase that would need to be
> > > > >> deprecated and could be used or possibly extended by current
> users.
> > > > >>
> > > > >> Ultimately it's up to Lars H as RM for 0.94 to decide what he
> wants
> > to
> > > > >> include. It just feels to me like we're rushing to deprecate
> > metrics
> > > > >> in 0.94 so that it can be removed in 0.96, instead of what seems
> to
> > me
> > > > >> like the more standard path of deprecating metrics in 0.96, while
> > also
> > > > >> including new metrics2 implementations, which would give users a
> > > > >> smoother path to actually switch over. I'm just not sure I
> > understand
> > > > >> the motivation for deprecating in 0.94 instead of 0.96.
> > > > >>
> > > >
> > >
> >
>
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Elliott Clark <ec...@stumbleupon.com>.
>From what I know about using guice it's best to only have on injector per
application. So It's not really possible to only implement metrics with
guice. There would need to be a medium amount of of work to pipe it all
the way from HMasterCommandLine/HRegionserverCommandLine alll the way to
metrics. Probably something we should look into after the current patch
goes in so that we don't couple the two discussions too tightly.
On Thu, Jul 12, 2012 at 1:23 AM, Stack <st...@duboce.net> wrote:
> On Thu, Jul 12, 2012 at 4:07 AM, Elliott Clark <ec...@stumbleupon.com>
> wrote:
> > Yep. This is pretty darn close to DI. I would be willing to change the
> > patch that I just put up to use Guice if that's what people wanted. I
> know
> > some of our classes could use some DI loving to help with testability.
> >
>
> So, you think we could Guice the metrics and then over time, Guice
> everything else Elliott?
> St.Ack
>
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Stack <st...@duboce.net>.
On Thu, Jul 12, 2012 at 4:07 AM, Elliott Clark <ec...@stumbleupon.com> wrote:
> Yep. This is pretty darn close to DI. I would be willing to change the
> patch that I just put up to use Guice if that's what people wanted. I know
> some of our classes could use some DI loving to help with testability.
>
So, you think we could Guice the metrics and then over time, Guice
everything else Elliott?
St.Ack
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Elliott Clark <ec...@stumbleupon.com>.
Yep. This is pretty darn close to DI. I would be willing to change the
patch that I just put up to use Guice if that's what people wanted. I know
some of our classes could use some DI loving to help with testability.
On Wed, Jul 11, 2012 at 11:57 AM, Andrew Purtell <ap...@apache.org>wrote:
> On Wed, Jul 11, 2012 at 10:50 AM, Elliott Clark <ec...@stumbleupon.com>
> wrote:
> [...]
> >
> > The reflection option seems like it would be a big perf hit. Reflection
> > basically means that nothing ever gets inlined so all function calls into
> > metrics2 code would be very expensive. Since it seems like we are adding
> > more and more instrumentation this perf impact would only grow.
> > In addition as more hadoop versions come out all of our reflection code
> > would get much more complicated and brittle.
> >
> > The conditionally loaded jar would be nice in that the JIT would only see
> > one version of the factory classes on the classpath and everything could
> be
> > optimized just like any other jvm run jit'd code. In addition there are
> > other places that we use reflection to do things conditionally and a
> > conditionally loaded jar would be nice.
>
> Coprocessors are a solution close to conditionally loaded JARs: there
> is not reflection, very frequently executed code would be JITed, the
> only thing missing is direct inlining.
>
> More and more (useful) metrics are going into hot paths, we want all
> three I think.
>
> We keep inching toward DI.
>
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein (via Tom White)
>
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Andrew Purtell <ap...@apache.org>.
On Wed, Jul 11, 2012 at 10:50 AM, Elliott Clark <ec...@stumbleupon.com> wrote:
[...]
>
> The reflection option seems like it would be a big perf hit. Reflection
> basically means that nothing ever gets inlined so all function calls into
> metrics2 code would be very expensive. Since it seems like we are adding
> more and more instrumentation this perf impact would only grow.
> In addition as more hadoop versions come out all of our reflection code
> would get much more complicated and brittle.
>
> The conditionally loaded jar would be nice in that the JIT would only see
> one version of the factory classes on the classpath and everything could be
> optimized just like any other jvm run jit'd code. In addition there are
> other places that we use reflection to do things conditionally and a
> conditionally loaded jar would be nice.
Coprocessors are a solution close to conditionally loaded JARs: there
is not reflection, very frequently executed code would be JITed, the
only thing missing is direct inlining.
More and more (useful) metrics are going into hot paths, we want all
three I think.
We keep inching toward DI.
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet
Hein (via Tom White)
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Elliott Clark <ec...@stumbleupon.com>.
He and I had discussed something close to #2. (This is my understanding
and is partially based on Todd's irc thoughts so feel free to correct me if
I get anything wrong)
The reflection option seems like it would be a big perf hit. Reflection
basically means that nothing ever gets inlined so all function calls into
metrics2 code would be very expensive. Since it seems like we are adding
more and more instrumentation this perf impact would only grow.
In addition as more hadoop versions come out all of our reflection code
would get much more complicated and brittle.
The conditionally loaded jar would be nice in that the JIT would only see
one version of the factory classes on the classpath and everything could be
optimized just like any other jvm run jit'd code. In addition there are
other places that we use reflection to do things conditionally and a
conditionally loaded jar would be nice.
(putting this on jira as well.)
On Wed, Jul 11, 2012 at 9:29 AM, Alex Baranau <al...@gmail.com>wrote:
> Hi, I don't know the details of what Todd suggested, could you please take
> a look at last two comments at
> https://issues.apache.org/jira/browse/HBASE-4050, to see if I understand
> correctly the options here.
>
> Thank you in advance,
> Alex Baranau
> ------
> Sematext :: http://blog.sematext.com/ : Hadoop - HBase - ElasticSearch -
> Solr - Lucene
>
> On Tue, Jul 10, 2012 at 7:58 PM, Elliott Clark <eclark@stumbleupon.com
> >wrote:
>
> > I would think so yes. Talked with Todd on irc and he suggested a good
> > solution, shim jars that are loaded based on what version of hadoop is on
> > the class path. However to get that working is a lot more work than just
> > rewriting some classes to use the metrics2 namespace. As such it seems
> way
> > too early to think about removing the working code.
> >
> > On Tue, Jul 10, 2012 at 4:54 PM, Andrew Purtell <ap...@apache.org>
> > wrote:
> >
> > > On Tue, Jul 10, 2012 at 3:33 PM, Elliott Clark <eclark@stumbleupon.com
> >
> > > wrote:
> > > > https://issues.apache.org/jira/browse/HADOOP-7734
> > > >
> > > > As far as I can tell this basically sinks all Metrics2 usage in
> HBase.
> > >
> > > So can we settle this then as something to do 0.96 and beyond and get
> > > the RC out?
> > >
> > > Best regards,
> > >
> > > - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> > > Hein (via Tom White)
> > >
> >
>
>
>
> --
> Alex Baranau
> ------
> Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - ElasticSearch -
> Solr - Lucene
>
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Alex Baranau <al...@gmail.com>.
Hi, I don't know the details of what Todd suggested, could you please take
a look at last two comments at
https://issues.apache.org/jira/browse/HBASE-4050, to see if I understand
correctly the options here.
Thank you in advance,
Alex Baranau
------
Sematext :: http://blog.sematext.com/ : Hadoop - HBase - ElasticSearch -
Solr - Lucene
On Tue, Jul 10, 2012 at 7:58 PM, Elliott Clark <ec...@stumbleupon.com>wrote:
> I would think so yes. Talked with Todd on irc and he suggested a good
> solution, shim jars that are loaded based on what version of hadoop is on
> the class path. However to get that working is a lot more work than just
> rewriting some classes to use the metrics2 namespace. As such it seems way
> too early to think about removing the working code.
>
> On Tue, Jul 10, 2012 at 4:54 PM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> > On Tue, Jul 10, 2012 at 3:33 PM, Elliott Clark <ec...@stumbleupon.com>
> > wrote:
> > > https://issues.apache.org/jira/browse/HADOOP-7734
> > >
> > > As far as I can tell this basically sinks all Metrics2 usage in HBase.
> >
> > So can we settle this then as something to do 0.96 and beyond and get
> > the RC out?
> >
> > Best regards,
> >
> > - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein (via Tom White)
> >
>
--
Alex Baranau
------
Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - ElasticSearch -
Solr - Lucene
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Elliott Clark <ec...@stumbleupon.com>.
I would think so yes. Talked with Todd on irc and he suggested a good
solution, shim jars that are loaded based on what version of hadoop is on
the class path. However to get that working is a lot more work than just
rewriting some classes to use the metrics2 namespace. As such it seems way
too early to think about removing the working code.
On Tue, Jul 10, 2012 at 4:54 PM, Andrew Purtell <ap...@apache.org> wrote:
> On Tue, Jul 10, 2012 at 3:33 PM, Elliott Clark <ec...@stumbleupon.com>
> wrote:
> > https://issues.apache.org/jira/browse/HADOOP-7734
> >
> > As far as I can tell this basically sinks all Metrics2 usage in HBase.
>
> So can we settle this then as something to do 0.96 and beyond and get
> the RC out?
>
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein (via Tom White)
>
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Andrew Purtell <ap...@apache.org>.
On Tue, Jul 10, 2012 at 3:33 PM, Elliott Clark <ec...@stumbleupon.com> wrote:
> https://issues.apache.org/jira/browse/HADOOP-7734
>
> As far as I can tell this basically sinks all Metrics2 usage in HBase.
So can we settle this then as something to do 0.96 and beyond and get
the RC out?
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet
Hein (via Tom White)
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Elliott Clark <ec...@stumbleupon.com>.
https://issues.apache.org/jira/browse/HADOOP-7734
As far as I can tell this basically sinks all Metrics2 usage in HBase.
On Tue, Jul 10, 2012 at 3:24 PM, Alex Baranau <al...@gmail.com>wrote:
> > If there's considerable pain or overhead in having both
> > implementations live in parallel, maybe it's worth doing a straight
> > switch over in 0.96.
>
> As far as I can tell, they can live together easily. This should not be a
> big issue. E.g. it should be much smaller issue than the fact that code
> implemented on top of metrics2 will not compile against 1.0+, 2.0+ and 3.0+
> hadoop at the same time because class names, etc. changed between 1.0 and
> 2.0. But that's a separate story, will look at it tomorrow (Ted pointed me
> to smth to look at).
>
> Alex Baranau
>
> On Tue, Jul 10, 2012 at 5:03 PM, Ted Yu <yu...@gmail.com> wrote:
>
> > Points taken.
> >
> > Thanks for the education of metrics framework history.
> >
> > On Tue, Jul 10, 2012 at 1:55 PM, Gary Helmling <gh...@gmail.com>
> > wrote:
> >
> > > I agree that having a new metrics2 implementation in 0.96 would be
> > > great to see and seems like a natural fit. I'm 100% for that. But I
> > > do think that having metrics2 and (deprecated) metrics v1 in the same
> > > release would be very helpful to users making the transition. So to
> > > me it seems more natural for 0.96 to be that release with both
> > > implementations, since that's where it seems like the metrics2
> > > implementation will land.
> > >
> > > Otherwise it seems like we risk introducing the same disruptions that
> > > Hadoop did when metrics2 initially replaced the metrics v1
> > > implementation, instead of living along side. This did cause us as a
> > > project some trouble until metrics v1 was added back in. So it would
> > > be unfortunate to repeat the same mistake ourselves.
> > >
> > > If there's considerable pain or overhead in having both
> > > implementations live in parallel, maybe it's worth doing a straight
> > > switch over in 0.96. I haven't looked at the differences enough
> > > myself to know. But otherwise it seems like an easier migration path
> > > to deprecate v1 in 0.96 and remove the release after.
> > >
> > >
> > > On Tue, Jul 10, 2012 at 1:35 PM, Ted Yu <yu...@gmail.com> wrote:
> > > > Gary:
> > > > Your comment makes sense.
> > > >
> > > > Part of this poll originates from the fact that 0.96 is our
> singularity
> > > > release. RPC, coprocessor, etc have undergone considerable changes.
> > > > Users migrating to 0.96 would have to deal with a lot of updates in
> > their
> > > > codebase.
> > > >
> > > > It seems to me that doing all upgrades in one shot is almost the same
> > as
> > > > upgrading components other than metrics framework.
> > > >
> > > > Cheers
> > > >
> > > > On Tue, Jul 10, 2012 at 1:29 PM, Gary Helmling <gh...@gmail.com>
> > > wrote:
> > > >
> > > >> >
> > > >> > Whether we support 2 (actually more than 2) metrics frameworks in
> > 0.96
> > > >> can
> > > >> > be debated in the next 2 months.
> > > >> >
> > > >>
> > > >> I'm not sure I agree that deprecating without having something in
> > > >> place for users to move to makes sense.
> > > >>
> > > >> > As Todd mentioned in the thread 'HBase 0.94.1', we will try our
> best
> > > to
> > > >> > keep JMX interface the same across 0.94 and 0.96. Does this
> somehow
> > > >> reduce
> > > >> > the concern you raised ?
> > > >> >
> > > >>
> > > >> I think that maintaining consistency with the existing JMX naming
> > > >> conventions (to the extent possible) is important for operational
> > > >> concerns, but it's independent of the MetricsContext question and
> the
> > > >> question of whether other metrics classes of our own need a proper
> > > >> deprecation cycle.
> > > >>
> > > >> > As for using MetricsContext, I assume the user also uses hadoop in
> > > his /
> > > >> > her deployment. Then he / she should be aware of the deprecation
> of
> > > >> > metrics.* classes in both hadoop 1.0 and 2.0
> > > >> > Meaning he / she should be prepared to endorse metrics2 framework.
> > > >> >
> > > >>
> > > >> Hadoop deprecating metrics in favor of metrics2 is independent of us
> > > >> deprecating HBase metrics classes. TimeStampingFileContext is one
> > > >> MetricsContext implementation in HBase that would need to be
> > > >> deprecated and could be used or possibly extended by current users.
> > > >>
> > > >> Ultimately it's up to Lars H as RM for 0.94 to decide what he wants
> to
> > > >> include. It just feels to me like we're rushing to deprecate
> metrics
> > > >> in 0.94 so that it can be removed in 0.96, instead of what seems to
> me
> > > >> like the more standard path of deprecating metrics in 0.96, while
> also
> > > >> including new metrics2 implementations, which would give users a
> > > >> smoother path to actually switch over. I'm just not sure I
> understand
> > > >> the motivation for deprecating in 0.94 instead of 0.96.
> > > >>
> > >
> >
>
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Alex Baranau <al...@gmail.com>.
> If there's considerable pain or overhead in having both
> implementations live in parallel, maybe it's worth doing a straight
> switch over in 0.96.
As far as I can tell, they can live together easily. This should not be a
big issue. E.g. it should be much smaller issue than the fact that code
implemented on top of metrics2 will not compile against 1.0+, 2.0+ and 3.0+
hadoop at the same time because class names, etc. changed between 1.0 and
2.0. But that's a separate story, will look at it tomorrow (Ted pointed me
to smth to look at).
Alex Baranau
On Tue, Jul 10, 2012 at 5:03 PM, Ted Yu <yu...@gmail.com> wrote:
> Points taken.
>
> Thanks for the education of metrics framework history.
>
> On Tue, Jul 10, 2012 at 1:55 PM, Gary Helmling <gh...@gmail.com>
> wrote:
>
> > I agree that having a new metrics2 implementation in 0.96 would be
> > great to see and seems like a natural fit. I'm 100% for that. But I
> > do think that having metrics2 and (deprecated) metrics v1 in the same
> > release would be very helpful to users making the transition. So to
> > me it seems more natural for 0.96 to be that release with both
> > implementations, since that's where it seems like the metrics2
> > implementation will land.
> >
> > Otherwise it seems like we risk introducing the same disruptions that
> > Hadoop did when metrics2 initially replaced the metrics v1
> > implementation, instead of living along side. This did cause us as a
> > project some trouble until metrics v1 was added back in. So it would
> > be unfortunate to repeat the same mistake ourselves.
> >
> > If there's considerable pain or overhead in having both
> > implementations live in parallel, maybe it's worth doing a straight
> > switch over in 0.96. I haven't looked at the differences enough
> > myself to know. But otherwise it seems like an easier migration path
> > to deprecate v1 in 0.96 and remove the release after.
> >
> >
> > On Tue, Jul 10, 2012 at 1:35 PM, Ted Yu <yu...@gmail.com> wrote:
> > > Gary:
> > > Your comment makes sense.
> > >
> > > Part of this poll originates from the fact that 0.96 is our singularity
> > > release. RPC, coprocessor, etc have undergone considerable changes.
> > > Users migrating to 0.96 would have to deal with a lot of updates in
> their
> > > codebase.
> > >
> > > It seems to me that doing all upgrades in one shot is almost the same
> as
> > > upgrading components other than metrics framework.
> > >
> > > Cheers
> > >
> > > On Tue, Jul 10, 2012 at 1:29 PM, Gary Helmling <gh...@gmail.com>
> > wrote:
> > >
> > >> >
> > >> > Whether we support 2 (actually more than 2) metrics frameworks in
> 0.96
> > >> can
> > >> > be debated in the next 2 months.
> > >> >
> > >>
> > >> I'm not sure I agree that deprecating without having something in
> > >> place for users to move to makes sense.
> > >>
> > >> > As Todd mentioned in the thread 'HBase 0.94.1', we will try our best
> > to
> > >> > keep JMX interface the same across 0.94 and 0.96. Does this somehow
> > >> reduce
> > >> > the concern you raised ?
> > >> >
> > >>
> > >> I think that maintaining consistency with the existing JMX naming
> > >> conventions (to the extent possible) is important for operational
> > >> concerns, but it's independent of the MetricsContext question and the
> > >> question of whether other metrics classes of our own need a proper
> > >> deprecation cycle.
> > >>
> > >> > As for using MetricsContext, I assume the user also uses hadoop in
> > his /
> > >> > her deployment. Then he / she should be aware of the deprecation of
> > >> > metrics.* classes in both hadoop 1.0 and 2.0
> > >> > Meaning he / she should be prepared to endorse metrics2 framework.
> > >> >
> > >>
> > >> Hadoop deprecating metrics in favor of metrics2 is independent of us
> > >> deprecating HBase metrics classes. TimeStampingFileContext is one
> > >> MetricsContext implementation in HBase that would need to be
> > >> deprecated and could be used or possibly extended by current users.
> > >>
> > >> Ultimately it's up to Lars H as RM for 0.94 to decide what he wants to
> > >> include. It just feels to me like we're rushing to deprecate metrics
> > >> in 0.94 so that it can be removed in 0.96, instead of what seems to me
> > >> like the more standard path of deprecating metrics in 0.96, while also
> > >> including new metrics2 implementations, which would give users a
> > >> smoother path to actually switch over. I'm just not sure I understand
> > >> the motivation for deprecating in 0.94 instead of 0.96.
> > >>
> >
>
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Ted Yu <yu...@gmail.com>.
Points taken.
Thanks for the education of metrics framework history.
On Tue, Jul 10, 2012 at 1:55 PM, Gary Helmling <gh...@gmail.com> wrote:
> I agree that having a new metrics2 implementation in 0.96 would be
> great to see and seems like a natural fit. I'm 100% for that. But I
> do think that having metrics2 and (deprecated) metrics v1 in the same
> release would be very helpful to users making the transition. So to
> me it seems more natural for 0.96 to be that release with both
> implementations, since that's where it seems like the metrics2
> implementation will land.
>
> Otherwise it seems like we risk introducing the same disruptions that
> Hadoop did when metrics2 initially replaced the metrics v1
> implementation, instead of living along side. This did cause us as a
> project some trouble until metrics v1 was added back in. So it would
> be unfortunate to repeat the same mistake ourselves.
>
> If there's considerable pain or overhead in having both
> implementations live in parallel, maybe it's worth doing a straight
> switch over in 0.96. I haven't looked at the differences enough
> myself to know. But otherwise it seems like an easier migration path
> to deprecate v1 in 0.96 and remove the release after.
>
>
> On Tue, Jul 10, 2012 at 1:35 PM, Ted Yu <yu...@gmail.com> wrote:
> > Gary:
> > Your comment makes sense.
> >
> > Part of this poll originates from the fact that 0.96 is our singularity
> > release. RPC, coprocessor, etc have undergone considerable changes.
> > Users migrating to 0.96 would have to deal with a lot of updates in their
> > codebase.
> >
> > It seems to me that doing all upgrades in one shot is almost the same as
> > upgrading components other than metrics framework.
> >
> > Cheers
> >
> > On Tue, Jul 10, 2012 at 1:29 PM, Gary Helmling <gh...@gmail.com>
> wrote:
> >
> >> >
> >> > Whether we support 2 (actually more than 2) metrics frameworks in 0.96
> >> can
> >> > be debated in the next 2 months.
> >> >
> >>
> >> I'm not sure I agree that deprecating without having something in
> >> place for users to move to makes sense.
> >>
> >> > As Todd mentioned in the thread 'HBase 0.94.1', we will try our best
> to
> >> > keep JMX interface the same across 0.94 and 0.96. Does this somehow
> >> reduce
> >> > the concern you raised ?
> >> >
> >>
> >> I think that maintaining consistency with the existing JMX naming
> >> conventions (to the extent possible) is important for operational
> >> concerns, but it's independent of the MetricsContext question and the
> >> question of whether other metrics classes of our own need a proper
> >> deprecation cycle.
> >>
> >> > As for using MetricsContext, I assume the user also uses hadoop in
> his /
> >> > her deployment. Then he / she should be aware of the deprecation of
> >> > metrics.* classes in both hadoop 1.0 and 2.0
> >> > Meaning he / she should be prepared to endorse metrics2 framework.
> >> >
> >>
> >> Hadoop deprecating metrics in favor of metrics2 is independent of us
> >> deprecating HBase metrics classes. TimeStampingFileContext is one
> >> MetricsContext implementation in HBase that would need to be
> >> deprecated and could be used or possibly extended by current users.
> >>
> >> Ultimately it's up to Lars H as RM for 0.94 to decide what he wants to
> >> include. It just feels to me like we're rushing to deprecate metrics
> >> in 0.94 so that it can be removed in 0.96, instead of what seems to me
> >> like the more standard path of deprecating metrics in 0.96, while also
> >> including new metrics2 implementations, which would give users a
> >> smoother path to actually switch over. I'm just not sure I understand
> >> the motivation for deprecating in 0.94 instead of 0.96.
> >>
>
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Gary Helmling <gh...@gmail.com>.
I agree that having a new metrics2 implementation in 0.96 would be
great to see and seems like a natural fit. I'm 100% for that. But I
do think that having metrics2 and (deprecated) metrics v1 in the same
release would be very helpful to users making the transition. So to
me it seems more natural for 0.96 to be that release with both
implementations, since that's where it seems like the metrics2
implementation will land.
Otherwise it seems like we risk introducing the same disruptions that
Hadoop did when metrics2 initially replaced the metrics v1
implementation, instead of living along side. This did cause us as a
project some trouble until metrics v1 was added back in. So it would
be unfortunate to repeat the same mistake ourselves.
If there's considerable pain or overhead in having both
implementations live in parallel, maybe it's worth doing a straight
switch over in 0.96. I haven't looked at the differences enough
myself to know. But otherwise it seems like an easier migration path
to deprecate v1 in 0.96 and remove the release after.
On Tue, Jul 10, 2012 at 1:35 PM, Ted Yu <yu...@gmail.com> wrote:
> Gary:
> Your comment makes sense.
>
> Part of this poll originates from the fact that 0.96 is our singularity
> release. RPC, coprocessor, etc have undergone considerable changes.
> Users migrating to 0.96 would have to deal with a lot of updates in their
> codebase.
>
> It seems to me that doing all upgrades in one shot is almost the same as
> upgrading components other than metrics framework.
>
> Cheers
>
> On Tue, Jul 10, 2012 at 1:29 PM, Gary Helmling <gh...@gmail.com> wrote:
>
>> >
>> > Whether we support 2 (actually more than 2) metrics frameworks in 0.96
>> can
>> > be debated in the next 2 months.
>> >
>>
>> I'm not sure I agree that deprecating without having something in
>> place for users to move to makes sense.
>>
>> > As Todd mentioned in the thread 'HBase 0.94.1', we will try our best to
>> > keep JMX interface the same across 0.94 and 0.96. Does this somehow
>> reduce
>> > the concern you raised ?
>> >
>>
>> I think that maintaining consistency with the existing JMX naming
>> conventions (to the extent possible) is important for operational
>> concerns, but it's independent of the MetricsContext question and the
>> question of whether other metrics classes of our own need a proper
>> deprecation cycle.
>>
>> > As for using MetricsContext, I assume the user also uses hadoop in his /
>> > her deployment. Then he / she should be aware of the deprecation of
>> > metrics.* classes in both hadoop 1.0 and 2.0
>> > Meaning he / she should be prepared to endorse metrics2 framework.
>> >
>>
>> Hadoop deprecating metrics in favor of metrics2 is independent of us
>> deprecating HBase metrics classes. TimeStampingFileContext is one
>> MetricsContext implementation in HBase that would need to be
>> deprecated and could be used or possibly extended by current users.
>>
>> Ultimately it's up to Lars H as RM for 0.94 to decide what he wants to
>> include. It just feels to me like we're rushing to deprecate metrics
>> in 0.94 so that it can be removed in 0.96, instead of what seems to me
>> like the more standard path of deprecating metrics in 0.96, while also
>> including new metrics2 implementations, which would give users a
>> smoother path to actually switch over. I'm just not sure I understand
>> the motivation for deprecating in 0.94 instead of 0.96.
>>
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Ted Yu <yu...@gmail.com>.
Gary:
Your comment makes sense.
Part of this poll originates from the fact that 0.96 is our singularity
release. RPC, coprocessor, etc have undergone considerable changes.
Users migrating to 0.96 would have to deal with a lot of updates in their
codebase.
It seems to me that doing all upgrades in one shot is almost the same as
upgrading components other than metrics framework.
Cheers
On Tue, Jul 10, 2012 at 1:29 PM, Gary Helmling <gh...@gmail.com> wrote:
> >
> > Whether we support 2 (actually more than 2) metrics frameworks in 0.96
> can
> > be debated in the next 2 months.
> >
>
> I'm not sure I agree that deprecating without having something in
> place for users to move to makes sense.
>
> > As Todd mentioned in the thread 'HBase 0.94.1', we will try our best to
> > keep JMX interface the same across 0.94 and 0.96. Does this somehow
> reduce
> > the concern you raised ?
> >
>
> I think that maintaining consistency with the existing JMX naming
> conventions (to the extent possible) is important for operational
> concerns, but it's independent of the MetricsContext question and the
> question of whether other metrics classes of our own need a proper
> deprecation cycle.
>
> > As for using MetricsContext, I assume the user also uses hadoop in his /
> > her deployment. Then he / she should be aware of the deprecation of
> > metrics.* classes in both hadoop 1.0 and 2.0
> > Meaning he / she should be prepared to endorse metrics2 framework.
> >
>
> Hadoop deprecating metrics in favor of metrics2 is independent of us
> deprecating HBase metrics classes. TimeStampingFileContext is one
> MetricsContext implementation in HBase that would need to be
> deprecated and could be used or possibly extended by current users.
>
> Ultimately it's up to Lars H as RM for 0.94 to decide what he wants to
> include. It just feels to me like we're rushing to deprecate metrics
> in 0.94 so that it can be removed in 0.96, instead of what seems to me
> like the more standard path of deprecating metrics in 0.96, while also
> including new metrics2 implementations, which would give users a
> smoother path to actually switch over. I'm just not sure I understand
> the motivation for deprecating in 0.94 instead of 0.96.
>
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Gary Helmling <gh...@gmail.com>.
>
> Whether we support 2 (actually more than 2) metrics frameworks in 0.96 can
> be debated in the next 2 months.
>
I'm not sure I agree that deprecating without having something in
place for users to move to makes sense.
> As Todd mentioned in the thread 'HBase 0.94.1', we will try our best to
> keep JMX interface the same across 0.94 and 0.96. Does this somehow reduce
> the concern you raised ?
>
I think that maintaining consistency with the existing JMX naming
conventions (to the extent possible) is important for operational
concerns, but it's independent of the MetricsContext question and the
question of whether other metrics classes of our own need a proper
deprecation cycle.
> As for using MetricsContext, I assume the user also uses hadoop in his /
> her deployment. Then he / she should be aware of the deprecation of
> metrics.* classes in both hadoop 1.0 and 2.0
> Meaning he / she should be prepared to endorse metrics2 framework.
>
Hadoop deprecating metrics in favor of metrics2 is independent of us
deprecating HBase metrics classes. TimeStampingFileContext is one
MetricsContext implementation in HBase that would need to be
deprecated and could be used or possibly extended by current users.
Ultimately it's up to Lars H as RM for 0.94 to decide what he wants to
include. It just feels to me like we're rushing to deprecate metrics
in 0.94 so that it can be removed in 0.96, instead of what seems to me
like the more standard path of deprecating metrics in 0.96, while also
including new metrics2 implementations, which would give users a
smoother path to actually switch over. I'm just not sure I understand
the motivation for deprecating in 0.94 instead of 0.96.
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Ted Yu <yu...@gmail.com>.
Thanks for the analysis, Gary.
Whether we support 2 (actually more than 2) metrics frameworks in 0.96 can
be debated in the next 2 months.
As Todd mentioned in the thread 'HBase 0.94.1', we will try our best to
keep JMX interface the same across 0.94 and 0.96. Does this somehow reduce
the concern you raised ?
As for using MetricsContext, I assume the user also uses hadoop in his /
her deployment. Then he / she should be aware of the deprecation of
metrics.* classes in both hadoop 1.0 and 2.0
Meaning he / she should be prepared to endorse metrics2 framework.
Correct me if I am wrong.
On Tue, Jul 10, 2012 at 11:26 AM, Gary Helmling <gh...@gmail.com> wrote:
> Do we have any usage of metrics2 currently in 0.94? If not, it seems
> odd to deprecate existing metrics implementations. Aside from whether
> or not we should deprecate in a point release, normally deprecation
> happens when the new implementation you're supposed to move to has
> already been added, so that you have both old and new versions in
> parallel.
>
> If we deprecate existing metrics in 0.94 and only have metrics2
> implementation in 0.96, that gives no chance for switching back and
> forth between the two in the same release. While most of the metrics
> classes can be seen as private, I think that MetricsContext is
> arguably a public interface. Implementing your own MetricsContext to
> plugin to your own monitoring system is not that crazy a thing to do.
> So if that is changing in the move from metrics to metrics2, I think
> it would be hurting our users to not have both implementations
> available at the same time.
>
>
> On Tue, Jul 10, 2012 at 1:45 AM, Ted Yu <yu...@gmail.com> wrote:
> > In another email thread 'HBase 0.94.1', Lars has kindly accepted the
> motion
> > that deprecating current metric classes be done for 0.94.1
> >
> > Since 0.94.1 contains critical bug fixes, I want to get public opinion on
> > closing this poll earlier than Friday.
> >
> > Thanks for the understanding.
> >
> > On Mon, Jul 9, 2012 at 9:05 PM, Otis Gospodnetic <
> otis_gospodnetic@yahoo.com
> >> wrote:
> >
> >> +1 for deprecating and moving to metrics2 faster.
> >> I'd normally make the same argument about deprecating that Lars H. made,
> >> but I think current HBase metrics really need improvement, so if I had
> >> voting power I'd vote to drop old stuff and move to metrics2 ASAP.
> >>
> >> Otis
> >>
> >> ----- Original Message -----
> >> > From: Ted Yu <yu...@gmail.com>
> >> > To: dev@hbase.apache.org
> >> > Cc:
> >> > Sent: Saturday, July 7, 2012 10:23 AM
> >> > Subject: deprecating (old) metrics in favor of metrics2 framework
> >> >
> >> > Hi,
> >> > See the following comment for background:
> >> >
> >>
> https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
> >> >
> >> > metrics2 framework is in both hadoop 1.0 and hadoop 2.0
> >> > We would like to migrate to metrics2 framework. But maintaining two
> >> metrics
> >> > frameworks in HBase is costly and results in lower readability.
> >> >
> >> > However, if you have cluster monitoring tool that depends on (old)
> >> metrics
> >> > framework, we want to give you smooth migration experience.
> >> >
> >> > Please provide your opinions on whether (old) metrics framework
> should be
> >> > deprecated in 0.94 or in 0.96.
> >> >
> >> > Thanks
> >> >
> >>
>
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Gary Helmling <gh...@gmail.com>.
Do we have any usage of metrics2 currently in 0.94? If not, it seems
odd to deprecate existing metrics implementations. Aside from whether
or not we should deprecate in a point release, normally deprecation
happens when the new implementation you're supposed to move to has
already been added, so that you have both old and new versions in
parallel.
If we deprecate existing metrics in 0.94 and only have metrics2
implementation in 0.96, that gives no chance for switching back and
forth between the two in the same release. While most of the metrics
classes can be seen as private, I think that MetricsContext is
arguably a public interface. Implementing your own MetricsContext to
plugin to your own monitoring system is not that crazy a thing to do.
So if that is changing in the move from metrics to metrics2, I think
it would be hurting our users to not have both implementations
available at the same time.
On Tue, Jul 10, 2012 at 1:45 AM, Ted Yu <yu...@gmail.com> wrote:
> In another email thread 'HBase 0.94.1', Lars has kindly accepted the motion
> that deprecating current metric classes be done for 0.94.1
>
> Since 0.94.1 contains critical bug fixes, I want to get public opinion on
> closing this poll earlier than Friday.
>
> Thanks for the understanding.
>
> On Mon, Jul 9, 2012 at 9:05 PM, Otis Gospodnetic <otis_gospodnetic@yahoo.com
>> wrote:
>
>> +1 for deprecating and moving to metrics2 faster.
>> I'd normally make the same argument about deprecating that Lars H. made,
>> but I think current HBase metrics really need improvement, so if I had
>> voting power I'd vote to drop old stuff and move to metrics2 ASAP.
>>
>> Otis
>>
>> ----- Original Message -----
>> > From: Ted Yu <yu...@gmail.com>
>> > To: dev@hbase.apache.org
>> > Cc:
>> > Sent: Saturday, July 7, 2012 10:23 AM
>> > Subject: deprecating (old) metrics in favor of metrics2 framework
>> >
>> > Hi,
>> > See the following comment for background:
>> >
>> https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
>> >
>> > metrics2 framework is in both hadoop 1.0 and hadoop 2.0
>> > We would like to migrate to metrics2 framework. But maintaining two
>> metrics
>> > frameworks in HBase is costly and results in lower readability.
>> >
>> > However, if you have cluster monitoring tool that depends on (old)
>> metrics
>> > framework, we want to give you smooth migration experience.
>> >
>> > Please provide your opinions on whether (old) metrics framework should be
>> > deprecated in 0.94 or in 0.96.
>> >
>> > Thanks
>> >
>>
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Ted Yu <yu...@gmail.com>.
In another email thread 'HBase 0.94.1', Lars has kindly accepted the motion
that deprecating current metric classes be done for 0.94.1
Since 0.94.1 contains critical bug fixes, I want to get public opinion on
closing this poll earlier than Friday.
Thanks for the understanding.
On Mon, Jul 9, 2012 at 9:05 PM, Otis Gospodnetic <otis_gospodnetic@yahoo.com
> wrote:
> +1 for deprecating and moving to metrics2 faster.
> I'd normally make the same argument about deprecating that Lars H. made,
> but I think current HBase metrics really need improvement, so if I had
> voting power I'd vote to drop old stuff and move to metrics2 ASAP.
>
> Otis
>
> ----- Original Message -----
> > From: Ted Yu <yu...@gmail.com>
> > To: dev@hbase.apache.org
> > Cc:
> > Sent: Saturday, July 7, 2012 10:23 AM
> > Subject: deprecating (old) metrics in favor of metrics2 framework
> >
> > Hi,
> > See the following comment for background:
> >
> https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
> >
> > metrics2 framework is in both hadoop 1.0 and hadoop 2.0
> > We would like to migrate to metrics2 framework. But maintaining two
> metrics
> > frameworks in HBase is costly and results in lower readability.
> >
> > However, if you have cluster monitoring tool that depends on (old)
> metrics
> > framework, we want to give you smooth migration experience.
> >
> > Please provide your opinions on whether (old) metrics framework should be
> > deprecated in 0.94 or in 0.96.
> >
> > Thanks
> >
>
Re: deprecating (old) metrics in favor of metrics2 framework
Posted by Otis Gospodnetic <ot...@yahoo.com>.
+1 for deprecating and moving to metrics2 faster.
I'd normally make the same argument about deprecating that Lars H. made, but I think current HBase metrics really need improvement, so if I had voting power I'd vote to drop old stuff and move to metrics2 ASAP.
Otis
----- Original Message -----
> From: Ted Yu <yu...@gmail.com>
> To: dev@hbase.apache.org
> Cc:
> Sent: Saturday, July 7, 2012 10:23 AM
> Subject: deprecating (old) metrics in favor of metrics2 framework
>
> Hi,
> See the following comment for background:
> https://issues.apache.org/jira/browse/HBASE-4050?focusedCommentId=13408679&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13408679
>
> metrics2 framework is in both hadoop 1.0 and hadoop 2.0
> We would like to migrate to metrics2 framework. But maintaining two metrics
> frameworks in HBase is costly and results in lower readability.
>
> However, if you have cluster monitoring tool that depends on (old) metrics
> framework, we want to give you smooth migration experience.
>
> Please provide your opinions on whether (old) metrics framework should be
> deprecated in 0.94 or in 0.96.
>
> Thanks
>