You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@streampipes.apache.org by "obermeier (via GitHub)" <gi...@apache.org> on 2023/01/26 14:19:35 UTC

[GitHub] [streampipes] obermeier opened a new issue, #1164: More specific types for numerical values when guessing schema

obermeier opened a new issue, #1164:
URL: https://github.com/apache/streampipes/issues/1164

   ### Apache StreamPipes version
   
   dev (current development state)
   
   ### Affected StreamPipes components
   
   Connect
   
   ### What happened?
   
   When the schema of data is guessed, many numerical formats are subsumed by the float type.
   Integer, Double, Float, Long. [Code location](https://github.com/apache/streampipes/blob/a23795b43a4a3d67021cc961ea2d956b639e390f/streampipes-extensions-management/src/main/java/org/apache/streampipes/extensions/management/connect/adapter/format/util/JsonEventProperty.java#L58)
   
   This leads sometimes to error  when the data does not fit into the range of float.
   
   E.g. When a timestamp attribute is given in ms
   `{
     "tree_size": 162451767, 
     "timestamp": 1674637204330, 
     "sha256_root_hash": "mIYlnASHw5nD\/9etlz\/BUoDhrAkER5H09IIK1I+jfmE="
    }`
   
   In this case it is not possible to use this attribute as timestamp since an exception occurs when the data is converted.
   
   What do you think to use more specific types?
   
   ### How to reproduce?
   
   Guess a schema from data with contains a number greater than float.
   E.g. https://ct.googleapis.com/logs/argon2023/ct/v1/get-sth
   
   ### Expected behavior
   
   In this example the guessed type schould be of type Long
   
   ### Additional technical information
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   Yes, I am willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@streampipes.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] More specific types for numerical values when guessing schema (streampipes)

Posted by "obermeier (via GitHub)" <gi...@apache.org>.
obermeier commented on issue #1164:
URL: https://github.com/apache/streampipes/issues/1164#issuecomment-1407776715

   Hi, I ralized that it must have ben an other problem on my system.
   I could not reproduce it with the latest version.
   Sorry


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] More specific types for numerical values when guessing schema (streampipes)

Posted by "obermeier (via GitHub)" <gi...@apache.org>.
obermeier closed issue #1164: More specific types for numerical values when guessing schema
URL: https://github.com/apache/streampipes/issues/1164


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org