You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mynewt.apache.org by Sterling Hughes <st...@apache.org> on 2016/10/04 18:06:47 UTC
Version on statistics structures
As I\u2019m going through the statistics module and documenting it, one
thing that occurs to me is for the case where STATS_NAME_ENABLE = 0, and
statistic names are omitted, we should create the statistic name ->
number mapping, in the target\u2019s directory, in a machine parseable
format (i.e. JSON.) I\u2019m thinking we should put this into the
manifest.json when we create an image, that way if a management system
wants to ingest this data, it can easily take this information, along
with the created image.
I thought about versioning the structure, however decided against it,
because:
- Manually updating the statistics version when it changes is error
prone, because people forget to change version numbers
- I think it is likely that any management system or user who wants to
interpret these values, will likely have the image definition for that
image, _AND_ will probably not do an exhaustive search of all images, to
find one that has a matching definition of version of statistics that
are found on that device. (likely:)
Comments welcome. Below is the documentation I\u2019ve written at the top
of the module.
NOTE: Newt does not yet support generating statistics mappings in the
manifest.json, I\u2019m adding that support to create-image now.
Best,
Sterling
/**
* Declare an example statistics section, which is fittingly, the
number
* statistics registered in the system. There are two, largely
duplicated,
* statistics sections in order to provide the optional ability to
* name statistics.
*
* STATS_SECT_START/END actually declare the statistics structure
definition,
* STATS_SECT_DECL() creates the structure declaration so you can
declare
* these statistics as a global structure, and STATS_NAME_START/END are
how
* you name the statistics themselves.
*
* Statistics entries can be declared as any of the following values,
however,
* all statistics in a given structure must be of the same size, and
they are
* all unsigned.
*
* - STATS_SECT_ENTRY(): default statistic entry, 32-bits. This
* is the good stuff. Two factors to consider:
* - With 16-bit numbers, rollovers happen, frequently. Whether
its
* testing a pathological condition, or just a long time since
you've
* collected a statistic: it really sucks to not have a crucial
piece
* of information.
* - 64-bit numbers are wonderful things. However, the gods did
not see
* fit to bless us with unlimited memory. 64-bit statistics are
useful
* when you want to store non-statistics in a statistics entry
(i.e. time),
* because its convenient...
*
* - STATS_SECT_ENTRY16(): 16-bits. Smaller statistics if you need to
fit into
* specific RAM or code size numbers.
*
* - STATS_SECT_ENTRY32(): 32-bits, if you want to force it.
*
* - STATS_SECT_ENTRY64(): 64-bits. Useful for storing chunks of data.
*
* Following the statics entry declaration is the statistic names
declaration.
* This is compiled out when STATS_NAME_ENABLE is set to 0. This
declaration
* is const, and therefore can be located in .text, not .data.
*
* In cases where the system configuration variable STATS_NAME_ENABLE
is set
* to 1, the statistics names are stored and returned to both the
console
* and management APIs. Whereas, when STATS_NAME_ENABLE = 0, these
statistics
* are numbered, s0, s1, etc. The newt tool will create a list of
statistic #
* -> statistic name in the
* bin/targets/<your-target>/app/apps/<your-app>/manifest.json file.
* That way, in cases where you want to save on code size, you can
store the
* manifest file for the image you've generated, and management tools
will
* be able to display the named statistic from the map.
*/
STATS_SECT_START(stats)
STATS_SECT_ENTRY(num_registered)
STATS_SECT_END
STATS_SECT_DECL(stats) g_stats_stats;
STATS_NAME_START(stats)
STATS_NAME(stats, num_registered)
STATS_NAME_END(stats)
Re: Version on statistics structures
Posted by Kevin Townsend <ke...@adafruit.com>.
Generating a map file during 'create-image' makes a lot of sense. I
haven't tried without names enabled, but a .json file would make life
easier compared to digging around in an .elf file or something similar.
On 04/10/16 20:06, Sterling Hughes wrote:
> As I\u2019m going through the statistics module and documenting it, one
> thing that occurs to me is for the case where STATS_NAME_ENABLE = 0,
> and statistic names are omitted, we should create the statistic name
> -> number mapping, in the target\u2019s directory, in a machine parseable
> format (i.e. JSON.) I\u2019m thinking we should put this into the
> manifest.json when we create an image, that way if a management system
> wants to ingest this data, it can easily take this information, along
> with the created image.
>
> I thought about versioning the structure, however decided against it,
> because:
>
> - Manually updating the statistics version when it changes is error
> prone, because people forget to change version numbers
>
> - I think it is likely that any management system or user who wants to
> interpret these values, will likely have the image definition for that
> image, _AND_ will probably not do an exhaustive search of all images,
> to find one that has a matching definition of version of statistics
> that are found on that device. (likely:)
>
> Comments welcome. Below is the documentation I\u2019ve written at the top
> of the module.
>
> NOTE: Newt does not yet support generating statistics mappings in the
> manifest.json, I\u2019m adding that support to create-image now.
Re: Version on statistics structures
Posted by Kevin Townsend <ke...@adafruit.com>.
On 04/10/16 21:01, Sterling Hughes wrote:
> Also, unless people object I\u2019m going to make the stats_name_map a
> const structure (so it can be located in .text), pack it, as the name
> map is rarely referenced, and it should save a bunch of space given
> the id is only 16-bits.
+1
Re: Version on statistics structures
Posted by Vipul Rahane <vi...@runtime.io>.
+1 as well.
> On Oct 5, 2016, at 6:30 AM, will sanfilippo <wi...@runtime.io> wrote:
>
> +1
>
>> On Oct 4, 2016, at 12:01 PM, Sterling Hughes <st...@apache.org> wrote:
>>
>> Also, unless people object I’m going to make the stats_name_map a const structure (so it can be located in .text), pack it, as the name map is rarely referenced, and it should save a bunch of space given the id is only 16-bits.
>>
>> On 4 Oct 2016, at 11:06, Sterling Hughes wrote:
>>
>>> As I’m going through the statistics module and documenting it, one thing that occurs to me is for the case where STATS_NAME_ENABLE = 0, and statistic names are omitted, we should create the statistic name -> number mapping, in the target’s directory, in a machine parseable format (i.e. JSON.) I’m thinking we should put this into the manifest.json when we create an image, that way if a management system wants to ingest this data, it can easily take this information, along with the created image.
>>>
>>> I thought about versioning the structure, however decided against it, because:
>>>
>>> - Manually updating the statistics version when it changes is error prone, because people forget to change version numbers
>>>
>>> - I think it is likely that any management system or user who wants to interpret these values, will likely have the image definition for that image, _AND_ will probably not do an exhaustive search of all images, to find one that has a matching definition of version of statistics that are found on that device. (likely:)
>>>
>>> Comments welcome. Below is the documentation I’ve written at the top of the module.
>>>
>>> NOTE: Newt does not yet support generating statistics mappings in the manifest.json, I’m adding that support to create-image now.
>>>
>>> Best,
>>>
>>> Sterling
>>>
>>> /**
>>> * Declare an example statistics section, which is fittingly, the number
>>> * statistics registered in the system. There are two, largely duplicated,
>>> * statistics sections in order to provide the optional ability to
>>> * name statistics.
>>> *
>>> * STATS_SECT_START/END actually declare the statistics structure definition,
>>> * STATS_SECT_DECL() creates the structure declaration so you can declare
>>> * these statistics as a global structure, and STATS_NAME_START/END are how
>>> * you name the statistics themselves.
>>> *
>>> * Statistics entries can be declared as any of the following values, however,
>>> * all statistics in a given structure must be of the same size, and they are
>>> * all unsigned.
>>> *
>>> * - STATS_SECT_ENTRY(): default statistic entry, 32-bits. This
>>> * is the good stuff. Two factors to consider:
>>> * - With 16-bit numbers, rollovers happen, frequently. Whether its
>>> * testing a pathological condition, or just a long time since you've
>>> * collected a statistic: it really sucks to not have a crucial piece
>>> * of information.
>>> * - 64-bit numbers are wonderful things. However, the gods did not see
>>> * fit to bless us with unlimited memory. 64-bit statistics are useful
>>> * when you want to store non-statistics in a statistics entry (i.e. time),
>>> * because its convenient...
>>> *
>>> * - STATS_SECT_ENTRY16(): 16-bits. Smaller statistics if you need to fit into
>>> * specific RAM or code size numbers.
>>> *
>>> * - STATS_SECT_ENTRY32(): 32-bits, if you want to force it.
>>> *
>>> * - STATS_SECT_ENTRY64(): 64-bits. Useful for storing chunks of data.
>>> *
>>> * Following the statics entry declaration is the statistic names declaration.
>>> * This is compiled out when STATS_NAME_ENABLE is set to 0. This declaration
>>> * is const, and therefore can be located in .text, not .data.
>>> *
>>> * In cases where the system configuration variable STATS_NAME_ENABLE is set
>>> * to 1, the statistics names are stored and returned to both the console
>>> * and management APIs. Whereas, when STATS_NAME_ENABLE = 0, these statistics
>>> * are numbered, s0, s1, etc. The newt tool will create a list of statistic #
>>> * -> statistic name in the
>>> * bin/targets/<your-target>/app/apps/<your-app>/manifest.json file.
>>> * That way, in cases where you want to save on code size, you can store the
>>> * manifest file for the image you've generated, and management tools will
>>> * be able to display the named statistic from the map.
>>> */
>>> STATS_SECT_START(stats)
>>> STATS_SECT_ENTRY(num_registered)
>>> STATS_SECT_END
>>>
>>> STATS_SECT_DECL(stats) g_stats_stats;
>>>
>>> STATS_NAME_START(stats)
>>> STATS_NAME(stats, num_registered)
>>> STATS_NAME_END(stats)
>>>
>
Re: Version on statistics structures
Posted by will sanfilippo <wi...@runtime.io>.
+1
> On Oct 4, 2016, at 12:01 PM, Sterling Hughes <st...@apache.org> wrote:
>
> Also, unless people object I’m going to make the stats_name_map a const structure (so it can be located in .text), pack it, as the name map is rarely referenced, and it should save a bunch of space given the id is only 16-bits.
>
> On 4 Oct 2016, at 11:06, Sterling Hughes wrote:
>
>> As I’m going through the statistics module and documenting it, one thing that occurs to me is for the case where STATS_NAME_ENABLE = 0, and statistic names are omitted, we should create the statistic name -> number mapping, in the target’s directory, in a machine parseable format (i.e. JSON.) I’m thinking we should put this into the manifest.json when we create an image, that way if a management system wants to ingest this data, it can easily take this information, along with the created image.
>>
>> I thought about versioning the structure, however decided against it, because:
>>
>> - Manually updating the statistics version when it changes is error prone, because people forget to change version numbers
>>
>> - I think it is likely that any management system or user who wants to interpret these values, will likely have the image definition for that image, _AND_ will probably not do an exhaustive search of all images, to find one that has a matching definition of version of statistics that are found on that device. (likely:)
>>
>> Comments welcome. Below is the documentation I’ve written at the top of the module.
>>
>> NOTE: Newt does not yet support generating statistics mappings in the manifest.json, I’m adding that support to create-image now.
>>
>> Best,
>>
>> Sterling
>>
>> /**
>> * Declare an example statistics section, which is fittingly, the number
>> * statistics registered in the system. There are two, largely duplicated,
>> * statistics sections in order to provide the optional ability to
>> * name statistics.
>> *
>> * STATS_SECT_START/END actually declare the statistics structure definition,
>> * STATS_SECT_DECL() creates the structure declaration so you can declare
>> * these statistics as a global structure, and STATS_NAME_START/END are how
>> * you name the statistics themselves.
>> *
>> * Statistics entries can be declared as any of the following values, however,
>> * all statistics in a given structure must be of the same size, and they are
>> * all unsigned.
>> *
>> * - STATS_SECT_ENTRY(): default statistic entry, 32-bits. This
>> * is the good stuff. Two factors to consider:
>> * - With 16-bit numbers, rollovers happen, frequently. Whether its
>> * testing a pathological condition, or just a long time since you've
>> * collected a statistic: it really sucks to not have a crucial piece
>> * of information.
>> * - 64-bit numbers are wonderful things. However, the gods did not see
>> * fit to bless us with unlimited memory. 64-bit statistics are useful
>> * when you want to store non-statistics in a statistics entry (i.e. time),
>> * because its convenient...
>> *
>> * - STATS_SECT_ENTRY16(): 16-bits. Smaller statistics if you need to fit into
>> * specific RAM or code size numbers.
>> *
>> * - STATS_SECT_ENTRY32(): 32-bits, if you want to force it.
>> *
>> * - STATS_SECT_ENTRY64(): 64-bits. Useful for storing chunks of data.
>> *
>> * Following the statics entry declaration is the statistic names declaration.
>> * This is compiled out when STATS_NAME_ENABLE is set to 0. This declaration
>> * is const, and therefore can be located in .text, not .data.
>> *
>> * In cases where the system configuration variable STATS_NAME_ENABLE is set
>> * to 1, the statistics names are stored and returned to both the console
>> * and management APIs. Whereas, when STATS_NAME_ENABLE = 0, these statistics
>> * are numbered, s0, s1, etc. The newt tool will create a list of statistic #
>> * -> statistic name in the
>> * bin/targets/<your-target>/app/apps/<your-app>/manifest.json file.
>> * That way, in cases where you want to save on code size, you can store the
>> * manifest file for the image you've generated, and management tools will
>> * be able to display the named statistic from the map.
>> */
>> STATS_SECT_START(stats)
>> STATS_SECT_ENTRY(num_registered)
>> STATS_SECT_END
>>
>> STATS_SECT_DECL(stats) g_stats_stats;
>>
>> STATS_NAME_START(stats)
>> STATS_NAME(stats, num_registered)
>> STATS_NAME_END(stats)
>>
Re: Version on statistics structures
Posted by Sterling Hughes <st...@apache.org>.
Also, unless people object I\u2019m going to make the stats_name_map a
const structure (so it can be located in .text), pack it, as the name
map is rarely referenced, and it should save a bunch of space given the
id is only 16-bits.
On 4 Oct 2016, at 11:06, Sterling Hughes wrote:
> As I\u2019m going through the statistics module and documenting it, one
> thing that occurs to me is for the case where STATS_NAME_ENABLE = 0,
> and statistic names are omitted, we should create the statistic name
> -> number mapping, in the target\u2019s directory, in a machine parseable
> format (i.e. JSON.) I\u2019m thinking we should put this into the
> manifest.json when we create an image, that way if a management system
> wants to ingest this data, it can easily take this information, along
> with the created image.
>
> I thought about versioning the structure, however decided against it,
> because:
>
> - Manually updating the statistics version when it changes is error
> prone, because people forget to change version numbers
>
> - I think it is likely that any management system or user who wants to
> interpret these values, will likely have the image definition for that
> image, _AND_ will probably not do an exhaustive search of all images,
> to find one that has a matching definition of version of statistics
> that are found on that device. (likely:)
>
> Comments welcome. Below is the documentation I\u2019ve written at the
> top of the module.
>
> NOTE: Newt does not yet support generating statistics mappings in the
> manifest.json, I\u2019m adding that support to create-image now.
>
> Best,
>
> Sterling
>
> /**
> * Declare an example statistics section, which is fittingly, the
> number
> * statistics registered in the system. There are two, largely
> duplicated,
> * statistics sections in order to provide the optional ability to
> * name statistics.
> *
> * STATS_SECT_START/END actually declare the statistics structure
> definition,
> * STATS_SECT_DECL() creates the structure declaration so you can
> declare
> * these statistics as a global structure, and STATS_NAME_START/END
> are how
> * you name the statistics themselves.
> *
> * Statistics entries can be declared as any of the following values,
> however,
> * all statistics in a given structure must be of the same size, and
> they are
> * all unsigned.
> *
> * - STATS_SECT_ENTRY(): default statistic entry, 32-bits. This
> * is the good stuff. Two factors to consider:
> * - With 16-bit numbers, rollovers happen, frequently. Whether
> its
> * testing a pathological condition, or just a long time since
> you've
> * collected a statistic: it really sucks to not have a crucial
> piece
> * of information.
> * - 64-bit numbers are wonderful things. However, the gods did
> not see
> * fit to bless us with unlimited memory. 64-bit statistics are
> useful
> * when you want to store non-statistics in a statistics entry
> (i.e. time),
> * because its convenient...
> *
> * - STATS_SECT_ENTRY16(): 16-bits. Smaller statistics if you need to
> fit into
> * specific RAM or code size numbers.
> *
> * - STATS_SECT_ENTRY32(): 32-bits, if you want to force it.
> *
> * - STATS_SECT_ENTRY64(): 64-bits. Useful for storing chunks of
> data.
> *
> * Following the statics entry declaration is the statistic names
> declaration.
> * This is compiled out when STATS_NAME_ENABLE is set to 0. This
> declaration
> * is const, and therefore can be located in .text, not .data.
> *
> * In cases where the system configuration variable STATS_NAME_ENABLE
> is set
> * to 1, the statistics names are stored and returned to both the
> console
> * and management APIs. Whereas, when STATS_NAME_ENABLE = 0, these
> statistics
> * are numbered, s0, s1, etc. The newt tool will create a list of
> statistic #
> * -> statistic name in the
> * bin/targets/<your-target>/app/apps/<your-app>/manifest.json file.
> * That way, in cases where you want to save on code size, you can
> store the
> * manifest file for the image you've generated, and management tools
> will
> * be able to display the named statistic from the map.
> */
> STATS_SECT_START(stats)
> STATS_SECT_ENTRY(num_registered)
> STATS_SECT_END
>
> STATS_SECT_DECL(stats) g_stats_stats;
>
> STATS_NAME_START(stats)
> STATS_NAME(stats, num_registered)
> STATS_NAME_END(stats)
>