You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@deltacloud.apache.org by "Koper, Dies" <di...@fast.au.fujitsu.com> on 2012/08/16 05:36:07 UTC
normalised metric names?
Hi Michal, all,
A while ago I implemented statistics for the mock and fgcp drivers.
As suggested by Michal at the time, the fgcp driver lists the stats that
FGCP supports, using the metric names the FGCP uses.
Therefore, the metric name to query to the cpu utilization ratio
parameter for an instance running on ec2 is different than for an
instance on the fgcp.
Many metrics may be provider specific, or at least not commonly
supported, some are very common.
Could you define and expose a set of common metric names?
Regards,
Dies Koper
RE: normalised metric names?
Posted by David Lutterkort <lu...@redhat.com>.
On Sun, 2012-09-02 at 13:53 +1000, Koper, Dies wrote:
> Looking at the EC2 Cloudwatch sample Michal once shared with me, I'm
> still not sure how to interpret their data and how it should be modelled
> in DC.
> Let me copy and paste part of the XML he sent me:
>
> <?xml version="1.0" encoding="UTF-8"?>
> <metric href="http://localhost:3002/api/metrics/i-5cac1c24"
> id="i-5cac1c24">
> <entity>i-5cac1c24</entity>
> <properties>
> <NetworkIn>
> <sample>
> <property name="timestamp" value="2012-07-09T12:38:00Z"/>
> <property name="unit" value="Bytes"/>
> <property name="minimum" value="42.0"/>
> <property name="maximum" value="42.0"/>
> <property name="average" value="42.0"/>
> <property name="samples" value="1.0"/>
> </sample>
> <sample>
> <property name="timestamp" value="2012-07-09T12:34:00Z"/>
> <property name="unit" value="Bytes"/>
> <property name="minimum" value="84.0"/>
> <property name="maximum" value="84.0"/>
> <property name="average" value="84.0"/>
> <property name="samples" value="1.0"/>
> </sample>
> <sample>
> ...
>
> What I observe is:
> 1. The 'samples' value is always 1. I suppose that's why a sample's min,
> max and average are the same. In what case would samples not be 1?
> 2. Even though the 'samples' value is 1, clearly it returns multiple
> samples (=measurements?) over a period (every 6 min. judging from the
> timestamp).
>
> The only useful data I get is the 42 and 84 and their timestamps. FGCP
> returns that data too, it just doesn't call them average, min and max.
> Do we need to include it three times in the DC API?
The important thing to me in the XML format is to prepare the format for
when the backend does only produce aggregates for multiple measurements.
I am not sure under what circumstances EC2 produces metrics for more
than one sample, but I think the general idea is sound: allow the
backend to do some aggregation of samples, and only report certain
staistics for them.
That in the case of nsamples = 1 average, min, and max are redundant is
somewhat unfortunate, and I would be perfectly fine to say that we only
expect backends to report an average, with min and max being optional.
IOW, for FGCP, I'd be fine if the XML looked something like
<sample timestamp="..." count="1" unit="...">
<entry name="average" value="42.0"/>
</sample>
That gives us the flexibility to report more staistics for count > 1.
David
RE: normalised metric names?
Posted by "Koper, Dies" <di...@fast.au.fujitsu.com>.
Looking at the EC2 Cloudwatch sample Michal once shared with me, I'm
still not sure how to interpret their data and how it should be modelled
in DC.
Let me copy and paste part of the XML he sent me:
<?xml version="1.0" encoding="UTF-8"?>
<metric href="http://localhost:3002/api/metrics/i-5cac1c24"
id="i-5cac1c24">
<entity>i-5cac1c24</entity>
<properties>
<NetworkIn>
<sample>
<property name="timestamp" value="2012-07-09T12:38:00Z"/>
<property name="unit" value="Bytes"/>
<property name="minimum" value="42.0"/>
<property name="maximum" value="42.0"/>
<property name="average" value="42.0"/>
<property name="samples" value="1.0"/>
</sample>
<sample>
<property name="timestamp" value="2012-07-09T12:34:00Z"/>
<property name="unit" value="Bytes"/>
<property name="minimum" value="84.0"/>
<property name="maximum" value="84.0"/>
<property name="average" value="84.0"/>
<property name="samples" value="1.0"/>
</sample>
<sample>
...
What I observe is:
1. The 'samples' value is always 1. I suppose that's why a sample's min,
max and average are the same. In what case would samples not be 1?
2. Even though the 'samples' value is 1, clearly it returns multiple
samples (=measurements?) over a period (every 6 min. judging from the
timestamp).
The only useful data I get is the 42 and 84 and their timestamps. FGCP
returns that data too, it just doesn't call them average, min and max.
Do we need to include it three times in the DC API?
<sample timestamp="..." count="..." unit="...">
<entry name="average" value="42.0"/>
<!--
<entry name="minimum" value="42.0"/>
<entry name="maximum" value="42.0"/>
-->
</sample>
Regards,
Dies Koper
> -----Original Message-----
> From: David Lutterkort [mailto:lutter@redhat.com]
> Sent: Tuesday, 28 August 2012 10:32 AM
> To: dev@deltacloud.apache.org
> Subject: RE: normalised metric names?
>
> On Fri, 2012-08-24 at 16:46 +1000, Koper, Dies wrote:
> > Hi David,
> >
> > > normalized name should not contain a unit ("nicInputByte"), and
I'd
> > > prefer if it was underscored, but whether we call it network_in or
> > > nic_input doesn't make a difference to me.
> >
> > I've made a new list (below).
> > I'm suggesting nic instead of network for consistency with cpu (vs.
> > compute).
> > I wonder if that will turn out to be the better choice if providers
> > start returning samples per nic (when a server has several), or not
:)
> >
> > EC2 "provider_probe" FGCP "provider_probe"
Normalised
> > "probe"
> > CPUUtilization cpuUtilization
cpu_utilization
> >
> > NetworkIn nicInputByte nic_in
> > NetworkOut nicOutputByte nic_out
> > - nicInputPacket -
> > - nicOutputPacket -
> >
> > DiskReadOps diskReadRequestCount disk_read_ops
> > DiskWriteOps diskWriteRequestCount
disk_write_ops
> > DiskReadBytes - -
> > DiskWriteBytes - -
> > - diskReadSector -
> > - diskWriteSector -
>
> Those names work for me.
>
> > > seems clearer; this would indicate that we took a sample at
> > @timestamp,
> > > consisting of @count measurements, and the values are reported in
> > @unit.
> > > The body of <sample/> only contains the statistics for that one
> > sample.
> > >
> > > What isn't clear to me is how we can talk about taking a sample at
a
> > > certain time when that sample consists of multiple measurements -
> > > shouldn't that sample then have an underlying time period (from
time X
> > > to time Y, we looked N times, and here are aggregate stats of what
we
> > > saw ?)
> >
> > FGCP doesn't return "average", min and max for metrics. It just
returns
> > a number for each.
> > I assumed each sample returned subtotals since the previous
measurement.
> > Except for cpu utilization, not sure how that's determined.
>
> What EC2 does, is that it takes a number N of samples, and then tells
> you N and aggregate statistics for those N. If FGCP returns
measurements
> directly, we can just indicate that by saying we took 1 measurement
and
> return the value as 'average' (or 'max' or 'min' ;)
>
> David
>
>
RE: normalised metric names?
Posted by David Lutterkort <lu...@redhat.com>.
On Fri, 2012-08-24 at 16:46 +1000, Koper, Dies wrote:
> Hi David,
>
> > normalized name should not contain a unit ("nicInputByte"), and I'd
> > prefer if it was underscored, but whether we call it network_in or
> > nic_input doesn't make a difference to me.
>
> I've made a new list (below).
> I'm suggesting nic instead of network for consistency with cpu (vs.
> compute).
> I wonder if that will turn out to be the better choice if providers
> start returning samples per nic (when a server has several), or not :)
>
> EC2 "provider_probe" FGCP "provider_probe" Normalised
> "probe"
> CPUUtilization cpuUtilization cpu_utilization
>
> NetworkIn nicInputByte nic_in
> NetworkOut nicOutputByte nic_out
> - nicInputPacket -
> - nicOutputPacket -
>
> DiskReadOps diskReadRequestCount disk_read_ops
> DiskWriteOps diskWriteRequestCount disk_write_ops
> DiskReadBytes - -
> DiskWriteBytes - -
> - diskReadSector -
> - diskWriteSector -
Those names work for me.
> > seems clearer; this would indicate that we took a sample at
> @timestamp,
> > consisting of @count measurements, and the values are reported in
> @unit.
> > The body of <sample/> only contains the statistics for that one
> sample.
> >
> > What isn't clear to me is how we can talk about taking a sample at a
> > certain time when that sample consists of multiple measurements -
> > shouldn't that sample then have an underlying time period (from time X
> > to time Y, we looked N times, and here are aggregate stats of what we
> > saw ?)
>
> FGCP doesn't return "average", min and max for metrics. It just returns
> a number for each.
> I assumed each sample returned subtotals since the previous measurement.
> Except for cpu utilization, not sure how that's determined.
What EC2 does, is that it takes a number N of samples, and then tells
you N and aggregate statistics for those N. If FGCP returns measurements
directly, we can just indicate that by saying we took 1 measurement and
return the value as 'average' (or 'max' or 'min' ;)
David
RE: normalised metric names?
Posted by "Koper, Dies" <di...@fast.au.fujitsu.com>.
Hi David,
> normalized name should not contain a unit ("nicInputByte"), and I'd
> prefer if it was underscored, but whether we call it network_in or
> nic_input doesn't make a difference to me.
I've made a new list (below).
I'm suggesting nic instead of network for consistency with cpu (vs.
compute).
I wonder if that will turn out to be the better choice if providers
start returning samples per nic (when a server has several), or not :)
EC2 "provider_probe" FGCP "provider_probe" Normalised
"probe"
CPUUtilization cpuUtilization cpu_utilization
NetworkIn nicInputByte nic_in
NetworkOut nicOutputByte nic_out
- nicInputPacket -
- nicOutputPacket -
DiskReadOps diskReadRequestCount disk_read_ops
DiskWriteOps diskWriteRequestCount disk_write_ops
DiskReadBytes - -
DiskWriteBytes - -
- diskReadSector -
- diskWriteSector -
> Each driver will need a table to map known probe names to normalized
> names; if the backend provider surprises us with a new probe, we'll
only
> have <samples provider_probe="StarsInTheSky"/>
Sounds good.
> seems clearer; this would indicate that we took a sample at
@timestamp,
> consisting of @count measurements, and the values are reported in
@unit.
> The body of <sample/> only contains the statistics for that one
sample.
>
> What isn't clear to me is how we can talk about taking a sample at a
> certain time when that sample consists of multiple measurements -
> shouldn't that sample then have an underlying time period (from time X
> to time Y, we looked N times, and here are aggregate stats of what we
> saw ?)
FGCP doesn't return "average", min and max for metrics. It just returns
a number for each.
I assumed each sample returned subtotals since the previous measurement.
Except for cpu utilization, not sure how that's determined.
Regards,
Dies Koper
> -----Original Message-----
> From: David Lutterkort [mailto:lutter@redhat.com]
> Sent: Wednesday, 22 August 2012 10:23 AM
> To: dev@deltacloud.apache.org
> Subject: RE: normalised metric names?
>
> On Sat, 2012-08-18 at 18:03 +1000, Koper, Dies wrote:
> > Here is my comparison:
> >
> > EC2 FGCP Mock
> > CPUUtilization cpuUtilization
cpuUtilization
> >
> > NetworkIn nicInputByte nicInputByte
> > NetworkOut nicOutputByte nicOutputByte
> > - nicInputPacket nicInputPacket
> > - nicOutputPacket nicOutputPacket
> >
> > DiskReadOps diskReadRequestCount
> > diskReadRequestCount
> > DiskWriteOps diskWriteRequestCount
diskWriteRequestCount
> > DiskReadBytes - -
> > DiskWriteBytes - -
> > - diskReadSector diskReadSector
> > - diskWriteSector diskWriteSector
> >
> > (I've taken the EC2 metric names from output Michal sent to me at
the
> > time).
> >
> > Five look like a match.
> > I'm thinking two options:
> >
> > 1. Come up with 5 new metric names that capture those and won't
clash
> > with any other providers' current or future metric names, and add
them
> > to the collection returned by DC. Future proof, but would mean a lot
of
> > double data being returned.
> >
> > 2. Take those 5 names from EC2 (as they were already supported in DC
> > v1.0) and replace the equivalents in FGCP and mock. The disadvantage
of
> > that is that if EC2 adds e.g. a DiskReadSector metric, FGCP and mock
> > would need to have two to keep backwards compatibility. Also, EC2
and
> > FGCP metrics (and names) are obtained dynamically using an API so we
may
> > not know when new metrics are added. (Well, I would know for FGCP
but
> > not for EC2.)
> >
> > If you don't mind breaking compatibility for EC2, there may be an
option
> > 1b: Come up with 5 new metric names that capture those and won't
clash
> > with any other providers' current or future metric names, and use
those
> > instead of the provider specific metric names. But that's postponing
the
> > decision: what would we do if ec2 adds a e.g. DiskReadSector metric
in a
> > next version and we introduce a new common metric for it, what would
we
> > do with the current diskReadSector metric in FGCP? Return both?
> >
> > Clearly this is something that should have been considered and
decided
> > when the ec2 metrics were introduced.
>
> I agree; the current XML has a few problems. I'd prefer we fix them,
> even if it means breaking backwards compatibility - I doubt that there
> are any users of the metrics collection right now.
>
> The current XML looks like
>
> <metric href="http://localhost:3001/api/metrics/ami-5e837b37" id="ami-
> 5e837b37">
> <entity>ImageId</entity>
> <properties>
> <NetworkIn>
> <sample>
> <property name="average" value="42.0"/>
> <property name="maximum" value="42.0"/>
> ...
>
> The reason I think nobody is using them is that it's not possible to
> correlate them back to the underlying entity - there's no link to the
> instance whose network traffic is being monitored.
>
> The following would make more sense:
>
> <metric href="http://localhost:3001/api/metrics/ami-5e837b37" id="ami-
> 5e837b37">
> <entity>
> <image href="/api/images/ami-5e837b37"/>
> </entity>
> <samples probe="network_in" provider_probe="NetworkIn">
> <sample>
> <property name="average" value="42.0"/>
> <property name="maximum" value="42.0"/>
> ...
>
>
> To get back to your initial question: I think we should include both a
> normalized name and the provider's name when we list statistics. The
> normalized name should not contain a unit ("nicInputByte"), and I'd
> prefer if it was underscored, but whether we call it network_in or
> nic_input doesn't make a difference to me.
>
> Each driver will need a table to map known probe names to normalized
> names; if the backend provider surprises us with a new probe, we'll
only
> have <samples provider_probe="StarsInTheSky"/>
>
> We also need to clamp down on what goes into <sample/> elements - the
> EC2 driver just dumps out whatever EC2 sends back. To me something
like
>
> <sample timestamp="..." count="..." unit="...">
> <entry name="average" value="42.0"/>
> <entry name="minimum" value="42.0"/>
> <entry name="maximum" value="42.0"/>
> </sample>
>
> seems clearer; this would indicate that we took a sample at
@timestamp,
> consisting of @count measurements, and the values are reported in
@unit.
> The body of <sample/> only contains the statistics for that one
sample.
>
> What isn't clear to me is how we can talk about taking a sample at a
> certain time when that sample consists of multiple measurements -
> shouldn't that sample then have an underlying time period (from time X
> to time Y, we looked N times, and here are aggregate stats of what we
> saw ?)
>
> Probably not what you were looking for ... but I think we should make
a
> breaking change to the XML for this.
>
> David
>
>
RE: normalised metric names?
Posted by David Lutterkort <lu...@redhat.com>.
On Sat, 2012-08-18 at 18:03 +1000, Koper, Dies wrote:
> Here is my comparison:
>
> EC2 FGCP Mock
> CPUUtilization cpuUtilization cpuUtilization
>
> NetworkIn nicInputByte nicInputByte
> NetworkOut nicOutputByte nicOutputByte
> - nicInputPacket nicInputPacket
> - nicOutputPacket nicOutputPacket
>
> DiskReadOps diskReadRequestCount
> diskReadRequestCount
> DiskWriteOps diskWriteRequestCount diskWriteRequestCount
> DiskReadBytes - -
> DiskWriteBytes - -
> - diskReadSector diskReadSector
> - diskWriteSector diskWriteSector
>
> (I've taken the EC2 metric names from output Michal sent to me at the
> time).
>
> Five look like a match.
> I'm thinking two options:
>
> 1. Come up with 5 new metric names that capture those and won't clash
> with any other providers' current or future metric names, and add them
> to the collection returned by DC. Future proof, but would mean a lot of
> double data being returned.
>
> 2. Take those 5 names from EC2 (as they were already supported in DC
> v1.0) and replace the equivalents in FGCP and mock. The disadvantage of
> that is that if EC2 adds e.g. a DiskReadSector metric, FGCP and mock
> would need to have two to keep backwards compatibility. Also, EC2 and
> FGCP metrics (and names) are obtained dynamically using an API so we may
> not know when new metrics are added. (Well, I would know for FGCP but
> not for EC2.)
>
> If you don't mind breaking compatibility for EC2, there may be an option
> 1b: Come up with 5 new metric names that capture those and won't clash
> with any other providers' current or future metric names, and use those
> instead of the provider specific metric names. But that's postponing the
> decision: what would we do if ec2 adds a e.g. DiskReadSector metric in a
> next version and we introduce a new common metric for it, what would we
> do with the current diskReadSector metric in FGCP? Return both?
>
> Clearly this is something that should have been considered and decided
> when the ec2 metrics were introduced.
I agree; the current XML has a few problems. I'd prefer we fix them,
even if it means breaking backwards compatibility - I doubt that there
are any users of the metrics collection right now.
The current XML looks like
<metric href="http://localhost:3001/api/metrics/ami-5e837b37" id="ami-5e837b37">
<entity>ImageId</entity>
<properties>
<NetworkIn>
<sample>
<property name="average" value="42.0"/>
<property name="maximum" value="42.0"/>
...
The reason I think nobody is using them is that it's not possible to
correlate them back to the underlying entity - there's no link to the
instance whose network traffic is being monitored.
The following would make more sense:
<metric href="http://localhost:3001/api/metrics/ami-5e837b37" id="ami-5e837b37">
<entity>
<image href="/api/images/ami-5e837b37"/>
</entity>
<samples probe="network_in" provider_probe="NetworkIn">
<sample>
<property name="average" value="42.0"/>
<property name="maximum" value="42.0"/>
...
To get back to your initial question: I think we should include both a
normalized name and the provider's name when we list statistics. The
normalized name should not contain a unit ("nicInputByte"), and I'd
prefer if it was underscored, but whether we call it network_in or
nic_input doesn't make a difference to me.
Each driver will need a table to map known probe names to normalized
names; if the backend provider surprises us with a new probe, we'll only
have <samples provider_probe="StarsInTheSky"/>
We also need to clamp down on what goes into <sample/> elements - the
EC2 driver just dumps out whatever EC2 sends back. To me something like
<sample timestamp="..." count="..." unit="...">
<entry name="average" value="42.0"/>
<entry name="minimum" value="42.0"/>
<entry name="maximum" value="42.0"/>
</sample>
seems clearer; this would indicate that we took a sample at @timestamp,
consisting of @count measurements, and the values are reported in @unit.
The body of <sample/> only contains the statistics for that one sample.
What isn't clear to me is how we can talk about taking a sample at a
certain time when that sample consists of multiple measurements -
shouldn't that sample then have an underlying time period (from time X
to time Y, we looked N times, and here are aggregate stats of what we
saw ?)
Probably not what you were looking for ... but I think we should make a
breaking change to the XML for this.
David
RE: normalised metric names?
Posted by "Koper, Dies" <di...@fast.au.fujitsu.com>.
Here is my comparison:
EC2 FGCP Mock
CPUUtilization cpuUtilization cpuUtilization
NetworkIn nicInputByte nicInputByte
NetworkOut nicOutputByte nicOutputByte
- nicInputPacket nicInputPacket
- nicOutputPacket nicOutputPacket
DiskReadOps diskReadRequestCount
diskReadRequestCount
DiskWriteOps diskWriteRequestCount diskWriteRequestCount
DiskReadBytes - -
DiskWriteBytes - -
- diskReadSector diskReadSector
- diskWriteSector diskWriteSector
(I've taken the EC2 metric names from output Michal sent to me at the
time).
Five look like a match.
I'm thinking two options:
1. Come up with 5 new metric names that capture those and won't clash
with any other providers' current or future metric names, and add them
to the collection returned by DC. Future proof, but would mean a lot of
double data being returned.
2. Take those 5 names from EC2 (as they were already supported in DC
v1.0) and replace the equivalents in FGCP and mock. The disadvantage of
that is that if EC2 adds e.g. a DiskReadSector metric, FGCP and mock
would need to have two to keep backwards compatibility. Also, EC2 and
FGCP metrics (and names) are obtained dynamically using an API so we may
not know when new metrics are added. (Well, I would know for FGCP but
not for EC2.)
If you don't mind breaking compatibility for EC2, there may be an option
1b: Come up with 5 new metric names that capture those and won't clash
with any other providers' current or future metric names, and use those
instead of the provider specific metric names. But that's postponing the
decision: what would we do if ec2 adds a e.g. DiskReadSector metric in a
next version and we introduce a new common metric for it, what would we
do with the current diskReadSector metric in FGCP? Return both?
Clearly this is something that should have been considered and decided
when the ec2 metrics were introduced.
Regards,
Dies
> -----Original Message-----
> From: David Lutterkort [mailto:lutter@redhat.com]
> Sent: Saturday, 18 August 2012 4:35 AM
> To: dev@deltacloud.apache.org
> Subject: Re: normalised metric names?
>
> On Thu, 2012-08-16 at 13:36 +1000, Koper, Dies wrote:
> > A while ago I implemented statistics for the mock and fgcp drivers.
> > As suggested by Michal at the time, the fgcp driver lists the stats
that
> > FGCP supports, using the metric names the FGCP uses.
> > Therefore, the metric name to query to the cpu utilization ratio
> > parameter for an instance running on ec2 is different than for an
> > instance on the fgcp.
> > Many metrics may be provider specific, or at least not commonly
> > supported, some are very common.
> > Could you define and expose a set of common metric names?
>
> It would bre really helpful (if you have it already) if you could post
a
> list that compars the FGCP metric names with the EC2 ones ... since
you
> are probably the most familiar with them at this points, I think you
> should get to choose, too ;)
>
> David
>
Re: normalised metric names?
Posted by David Lutterkort <lu...@redhat.com>.
On Thu, 2012-08-16 at 13:36 +1000, Koper, Dies wrote:
> A while ago I implemented statistics for the mock and fgcp drivers.
> As suggested by Michal at the time, the fgcp driver lists the stats that
> FGCP supports, using the metric names the FGCP uses.
> Therefore, the metric name to query to the cpu utilization ratio
> parameter for an instance running on ec2 is different than for an
> instance on the fgcp.
> Many metrics may be provider specific, or at least not commonly
> supported, some are very common.
> Could you define and expose a set of common metric names?
It would bre really helpful (if you have it already) if you could post a
list that compars the FGCP metric names with the EC2 ones ... since you
are probably the most familiar with them at this points, I think you
should get to choose, too ;)
David