[jira] [Created] (IOTDB-286) Rich Datatypes API / Logical Datatypes

["Julian Feinauer (Jira)" <ji...@apache.org>]

Julian Feinauer created IOTDB-286:
-------------------------------------

             Summary: Rich Datatypes API / Logical Datatypes
                 Key: IOTDB-286
                 URL: https://issues.apache.org/jira/browse/IOTDB-286
             Project: Apache IoTDB
          Issue Type: New Feature
            Reporter: Julian Feinauer


I worked a lot in the automotive industry and there is a broadly adopted open Standard called ASAM MDF (

[https://www.asam.net/standards/detail/mdf/]

). It is a format which is targeted at the efficient storage but at the same time it supports VERY complex types (which are often used in automotive controllers). Take something as simple as a boolean. We could store it as a boolean (as java bool) in 1 bit. BUT we have overall 4 possibilities: * No value is available for a timestamp (NULL / nothing stored) * We had a successful request but the Controller does not know whether true or false (or had an internal error), this is a bit like Optional.isPresent() == false * True * False In Java we could model it as a variable Optional<> x which could be null, Optional.empty(), Optional.of(true), Optional.of(false). Other examples are discrete values like “ON”, “OFF” (which are handled as “lookup tables” on integer rows, internally). Or scale formulas like a*x+b which allows to leverage the precision even for “small” double values or even integers. A formula but also a “fallback” lookup value like “NV”.

 

Example:

 

We know that we measure e.g. a voltage between 3V and 4.2V with a precision of 0.01 or something.
Then its most efficient to store integers and a formula like a * x + b with e.g. b = 3 and a = 1/100.
So 3V would be stored as x = 0, 3.01V -> x = 1, ... 4.2V as x = 1200.
So we only store 0 to 1200 and no decimals and stuff which would be very easily compressable I thnk.

 

See the full Discussion on the mailing list below

https://lists.apache.org/thread.html/b8b0fc1dc854ef5e255f906e85848db73d5c6c20cf0e602a3a8f75e8@%3Cdev.iotdb.apache.org%3E



--
This message was sent by Atlassian Jira
(v8.3.4#803005)