You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mynewt.apache.org by Vipul Rahane <vi...@runtime.io> on 2016/04/08 06:42:26 UTC
Timestamp for logs
Hello,
Currently the timestamp for logs is a 64 bit value which comes from a 32 bit os_time counter. This counter counts ticks as opposed to seconds since boot or UTC timestamp. These are 1 ms ticks and so will rollover in 49 days. We have an API os_get_timeofday() which fills up a structure with UTC timestamp in seconds and microseconds since the last second.
I am going to change the log structure so that it stores both(UTC timestamp in seconds - 64 bit, Microseconds since last second - 32 bit)
This way looking at the logs we get a good resolution for time and also, we don’t have to worry about rollovers. Is everyone fine with this change ?
Regards,
Vipul Rahane
Re: Timestamp for logs
Posted by will sanfilippo <wi...@runtime.io>.
It does seem like all those bits are unnecessary. Either 48 or 64 bits seems plenty. One note: if we want to use the same logging infrastructure for the controller in the ble stack we will need sub-millisecond precision. 1 microsecond units would be preferable; 16 bits for time since last second is not great given BLE timing requirements but could work… That is like 15 or 16 usec precision, right?
> On Apr 8, 2016, at 7:33 AM, Sterling Hughes <st...@gmail.com> wrote:
>
>
>
>> On Apr 8, 2016, at 6:11 AM, Christopher Collins <cc...@apache.org> wrote:
>>
>>> On Thu, Apr 07, 2016 at 10:33:26PM -0700, Vipul Rahane wrote:
>>> Hello,
>>>
>>> I agree with your statement. We do not know on what kind of devices
>>> Mynewt would be ported to. Sleepy devices which are meant to work for
>>> 20 years running on a single coin cell battery will rollover the time
>>> stamp in 2038. We want to be able to take care of such a situation.
>>> While there are other solutions which can be implemented that are more
>>> efficient, keeping it as simple as possible is better from an end to
>>> end perspective as these logs would be used by applications to
>>> understand the state of the devices.
>>>
>>> I was planning on storing microseconds because the OS currently
>>> populates OS time in seconds and microseconds. For microseconds we do
>>> require 32 bits. I agree for milliseconds 16 bits are enough but
>>> higher resolution is always better.
>>
>> I think 12 bytes of time is more than necessary. A few notes:
>>
>> * A single 64-bit microsecond counter allows for 584942 years before
>> rollover.
>>
>> * A single 32-bit second counter won't actually roll over until 2106
>> (the 2038 issue only applies to signed 32-bit timestamps).
>>
>> If we want microsecond precision, I would just go with a single 64-bit
>> counter. Otherwise, 32 bits of seconds is sufficient in my opinion.
>>
>
> +1. That was the original thought. Underlying counters may not be at that precision- but that doesn't mean you can't store it as microseconds
>
>> Chris
>>
>>>
>>> Regards,
>>> Vipul Rahane
>>>
>>>> On Apr 7, 2016, at 10:02 PM, Justin Mclean <ju...@me.com> wrote:
>>>>
>>>> HI,
>>>>
>>>>> I am going to change the log structure so that it stores both(UTC timestamp in seconds - 64 bit, Microseconds since last second - 32 bit)
>>>>
>>>> NO objection, but just out of interest why 64 bit for seconds (when 32 bit of seconds = 60+ years and good until 2038) and 32 bits for milliseconds when 16 bits will do? See also [1]
>>>>
>>>> Thanks,
>>>> Justin
>>>>
>>>> 1. https://en.wikipedia.org/wiki/Year_2038_problem#Solutions
>>>
Re: Timestamp for logs
Posted by Sterling Hughes <st...@gmail.com>.
> On Apr 8, 2016, at 6:11 AM, Christopher Collins <cc...@apache.org> wrote:
>
>> On Thu, Apr 07, 2016 at 10:33:26PM -0700, Vipul Rahane wrote:
>> Hello,
>>
>> I agree with your statement. We do not know on what kind of devices
>> Mynewt would be ported to. Sleepy devices which are meant to work for
>> 20 years running on a single coin cell battery will rollover the time
>> stamp in 2038. We want to be able to take care of such a situation.
>> While there are other solutions which can be implemented that are more
>> efficient, keeping it as simple as possible is better from an end to
>> end perspective as these logs would be used by applications to
>> understand the state of the devices.
>>
>> I was planning on storing microseconds because the OS currently
>> populates OS time in seconds and microseconds. For microseconds we do
>> require 32 bits. I agree for milliseconds 16 bits are enough but
>> higher resolution is always better.
>
> I think 12 bytes of time is more than necessary. A few notes:
>
> * A single 64-bit microsecond counter allows for 584942 years before
> rollover.
>
> * A single 32-bit second counter won't actually roll over until 2106
> (the 2038 issue only applies to signed 32-bit timestamps).
>
> If we want microsecond precision, I would just go with a single 64-bit
> counter. Otherwise, 32 bits of seconds is sufficient in my opinion.
>
+1. That was the original thought. Underlying counters may not be at that precision- but that doesn't mean you can't store it as microseconds
> Chris
>
>>
>> Regards,
>> Vipul Rahane
>>
>>> On Apr 7, 2016, at 10:02 PM, Justin Mclean <ju...@me.com> wrote:
>>>
>>> HI,
>>>
>>>> I am going to change the log structure so that it stores both(UTC timestamp in seconds - 64 bit, Microseconds since last second - 32 bit)
>>>
>>> NO objection, but just out of interest why 64 bit for seconds (when 32 bit of seconds = 60+ years and good until 2038) and 32 bits for milliseconds when 16 bits will do? See also [1]
>>>
>>> Thanks,
>>> Justin
>>>
>>> 1. https://en.wikipedia.org/wiki/Year_2038_problem#Solutions
>>
Re: Timestamp for logs
Posted by Christopher Collins <cc...@apache.org>.
On Thu, Apr 07, 2016 at 10:33:26PM -0700, Vipul Rahane wrote:
> Hello,
>
> I agree with your statement. We do not know on what kind of devices
> Mynewt would be ported to. Sleepy devices which are meant to work for
> 20 years running on a single coin cell battery will rollover the time
> stamp in 2038. We want to be able to take care of such a situation.
> While there are other solutions which can be implemented that are more
> efficient, keeping it as simple as possible is better from an end to
> end perspective as these logs would be used by applications to
> understand the state of the devices.
>
> I was planning on storing microseconds because the OS currently
> populates OS time in seconds and microseconds. For microseconds we do
> require 32 bits. I agree for milliseconds 16 bits are enough but
> higher resolution is always better.
I think 12 bytes of time is more than necessary. A few notes:
* A single 64-bit microsecond counter allows for 584942 years before
rollover.
* A single 32-bit second counter won't actually roll over until 2106
(the 2038 issue only applies to signed 32-bit timestamps).
If we want microsecond precision, I would just go with a single 64-bit
counter. Otherwise, 32 bits of seconds is sufficient in my opinion.
Chris
>
> Regards,
> Vipul Rahane
>
> > On Apr 7, 2016, at 10:02 PM, Justin Mclean <ju...@me.com> wrote:
> >
> > HI,
> >
> >> I am going to change the log structure so that it stores both(UTC timestamp in seconds - 64 bit, Microseconds since last second - 32 bit)
> >
> > NO objection, but just out of interest why 64 bit for seconds (when 32 bit of seconds = 60+ years and good until 2038) and 32 bits for milliseconds when 16 bits will do? See also [1]
> >
> > Thanks,
> > Justin
> >
> > 1. https://en.wikipedia.org/wiki/Year_2038_problem#Solutions
> >
>
Re: Timestamp for logs
Posted by Vipul Rahane <vi...@runtime.io>.
Hello,
I agree with your statement. We do not know on what kind of devices Mynewt would be ported to. Sleepy devices which are meant to work for 20 years running on a single coin cell battery will rollover the time stamp in 2038. We want to be able to take care of such a situation. While there are other solutions which can be implemented that are more efficient, keeping it as simple as possible is better from an end to end perspective as these logs would be used by applications to understand the state of the devices.
I was planning on storing microseconds because the OS currently populates OS time in seconds and microseconds. For microseconds we do require 32 bits. I agree for milliseconds 16 bits are enough but higher resolution is always better.
Regards,
Vipul Rahane
> On Apr 7, 2016, at 10:02 PM, Justin Mclean <ju...@me.com> wrote:
>
> HI,
>
>> I am going to change the log structure so that it stores both(UTC timestamp in seconds - 64 bit, Microseconds since last second - 32 bit)
>
> NO objection, but just out of interest why 64 bit for seconds (when 32 bit of seconds = 60+ years and good until 2038) and 32 bits for milliseconds when 16 bits will do? See also [1]
>
> Thanks,
> Justin
>
> 1. https://en.wikipedia.org/wiki/Year_2038_problem#Solutions
>
Re: Timestamp for logs
Posted by Justin Mclean <ju...@me.com>.
HI,
> I am going to change the log structure so that it stores both(UTC timestamp in seconds - 64 bit, Microseconds since last second - 32 bit)
NO objection, but just out of interest why 64 bit for seconds (when 32 bit of seconds = 60+ years and good until 2038) and 32 bits for milliseconds when 16 bits will do? See also [1]
Thanks,
Justin
1. https://en.wikipedia.org/wiki/Year_2038_problem#Solutions