Should we increase timestamp precision to Micro or nanosec ?

[Philippe Mouawad <ph...@gmail.com>]

Hello,
What do you think of increasing timestamp precision up to micro or nano
seconds ?

For example:
- When sending data to TimeSeries DB like InfluxDB, with current precision
of millis, we can end up overwriting some value if user wants to send data
to InfluxDB as in the same MS you can trigger many same Transactions.

If we go up to Nanos we would be able to handle such cases without issues.
With improvement in performances and systems being more able to handle very
high throughput I think we should consider this.

And this should also probably include updating Response time precision to
nanos.


-- 
Regards
Philippe M

RE: Should we increase timestamp precision to Micro or nanosec ?

[Paulo Maia Borges <pm...@temenos.com>]

> - When sending data to TimeSeries DB like InfluxDB, with current
> precision of millis, we can end up overwriting some value if user
> wants to send data to InfluxDB as in the same MS you can trigger many same Transactions.
>
> If we go up to Nanos we would be able to handle such cases without issues.
^ Referring to the precision of "time of event", so not the precision of the likes of "response time".

JMeter should not have the responsibility of changing the precision of its time_of_event to solve a problem of interaction with InfluxDB. It would actually not entirely solve the problem and is also not the correct way to use InfluxDB - read below why and how.


InfluxDB (or Graphite or Prometheus too?) will only overwrite the data point if all "tags" + time are the same. Example:
Tag - Hostname
Tag - ThreadName
Tag - ThreadNumber
Tag - sample Label
Value ("field") - response time
Time - time_of_event_ms
^ in order for the data point to be overwritten, the Tags *plus* the Time of the data point all need to be the same. So the very same thread would have to produce the same data point within the very same millisecond. We can make this occur although not under normal circumstances (zero pauses in-between requests combined with sub-ms response times). If we were to increase the precision, collisions would happen more rarely but it would still not be guaranteed, collisions could still happen.

Identifying data points with a reasonably unique set of Tags decreases the likelihood of collisions but it will never be guaranteed, and that is regardless of the precision of the time_of_event. <- are you by chance trying to upload the data points with a small set of identification Tags? That could indeed run into frequent collisions...

The proper way to handle this and the *only* way to guarantee that no data points get lost is to adjust how the data is exported to InfluxDB - the data should be pre-aggregated before shipping to InfluxDB, data that isn't a time-series should be transformed from non-time-series data into time-series-data - that's how Influx is meant to be used, it's a time-series DB.

Ex, transforming the non-time-series JMeter results set into a time-series data of «every minute» rate:
Tag - Hostname
Tag - ThreadName
Tag - ThreadNumber
Tag - sample Label
Value ("field") - Min response time
Value ("field") - Mean response time
Value ("field") - Median response time
Value ("field") - 90th Percentile response time
Value ("field") - 95th Percentile response time
Value ("field") - Max response time
Value ("field") - Number of successful requests
Value ("field") - Number of failed requests
Time - time_of_event_minute


I wrote my own plugin to export the data to InfluxDB but I'm under the impression that JMeter's built-in BackendListener plugin is already doing these aggregations, transforming the JMeter data into a time-series. Are you by change using something like Logstash or Telegraf to monitor the produced CSV files and collect that data? If you are, make sure you are parsing the right columns into appropriate Tags identifying the data points and also you should have some manner of pause in-between requests - or alternatively consider using the built-in BackendListener plugin that exports the results directly into InfluxDB (or Graphite).




Regards,
Paulo Augusto Maia Borges

The information in this e-mail and any attachments is confidential and may be legally privileged. It is intended solely for the addressee or addressees. Any use or disclosure of the contents of this e-mail/attachments by a not intended recipient is unauthorized and may be unlawful. If you have received this e-mail in error please notify the sender. Please note that any views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of TEMENOS. We recommend that you check this e-mail and any attachments against viruses. TEMENOS accepts no liability for any damage caused by any malicious code or virus transmitted by this e-mail.

Re: Should we increase timestamp precision to Micro or nanosec ?

[sebb <se...@gmail.com>]

On 27 September 2018 at 15:16, Philippe Mouawad
<ph...@gmail.com> wrote:
> Hello,
> What do you think of increasing timestamp precision up to micro or nano
> seconds ?

We looked at this a few years ago.
There is already some code in SampleResult.
There were quite a few issues with getNanoTime, e,g, significant drift
on some systems, which is why there is the thread to keep the offset
updated.
Also nanoTime is only designed for elapsed time, not wall-clock time,
so has to be used in conjunction with getTimeMillis for timestamps.

> For example:
> - When sending data to TimeSeries DB like InfluxDB, with current precision
> of millis, we can end up overwriting some value if user wants to send data
> to InfluxDB as in the same MS you can trigger many same Transactions.
>
> If we go up to Nanos we would be able to handle such cases without issues.

That would help, but there is still the possibility of multiple
transactions with the same timestamp.
(Many OSes don't support nano-second granularity)

In any case, if duplicate times are a problem for databases such as
InfluxDB, then it seems to me that is a separate issue which needs a
different solution.

> With improvement in performances and systems being more able to handle very
> high throughput I think we should consider this.

And if the systems get fast enough they may be able to send multiple
requests in a nanosecond, thus causing problems for InfluxDB again.

So whilst it should help, it's not a complete solution, and I think it
is orthogonal to the issue of JMeter timestamp precision.

I'm not saying we should not consider increasing the timer resolution.

> And this should also probably include updating Response time precision to
> nanos.

However any solution needs to be backward compatible.

AFAIK it's not possible to support microseconds without using
getNanoTime, so I don't see the point of limiting the resolution to
microseconds.

One possible approach would be to add another method to return the
nano second remainder.

>
> --
> Regards
> Philippe M