You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/04/19 16:47:38 UTC
[GitHub] [incubator-pinot] amrishlal opened a new issue #6814: FLOAT missing from Literal datatypes, but defined in Column datatypes.
amrishlal opened a new issue #6814:
URL: https://github.com/apache/incubator-pinot/issues/6814
FLOAT is missing from Literal datatypes (org.apache.pinot.common.request.Literal) but defined as a column datatype (org.apache.pinot.spi.data.FieldSpec.DataType). This may create type conversion issues since the only way to specify values for FLOAT column is by using DOUBLE literal values.
Literal defines the following datatypes:
BOOL, BYTE, SHORT, INT, LONG, DOUBLE, STRING, BINARY
FieldSpec.DataType defines the following datatypes:
INT, LONG, FLOAT, DOUBLE, BOOLEAN/* Stored as STRING */, STRING, BYTES, STRUCT, MAP, LIST;
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6814: FLOAT missing from Literal datatypes, but defined in Column datatypes.
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #6814:
URL: https://github.com/apache/incubator-pinot/issues/6814#issuecomment-822778999
The literal type should not matter. We should convert the literal values based on the field data type, and as long as the values can be represented by the literal, we are good. Actually on the server side, literal is always stored as string
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] amrishlal edited a comment on issue #6814: FLOAT missing from Literal datatypes, but defined in Column datatypes.
Posted by GitBox <gi...@apache.org>.
amrishlal edited a comment on issue #6814:
URL: https://github.com/apache/incubator-pinot/issues/6814#issuecomment-822989275
From what I saw server is converting a string literal that appears in the predicate at least twice to an integer value and this is after Broker/Calcite has already parsed the string as number and possibly operated on (plus/minus scalar functions). So while it works, there is a bit of cost associated with it. There are also potential correctness issues. For example long to double conversion (which is happening in ScalarFunction.java on the Broker side) is a lossy conversion. None of these issues by themselves are major, but combined together they seem to indicate that some cleanup with type handling would be useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] amrishlal commented on issue #6814: FLOAT missing from Literal datatypes, but defined in Column datatypes.
Posted by GitBox <gi...@apache.org>.
amrishlal commented on issue #6814:
URL: https://github.com/apache/incubator-pinot/issues/6814#issuecomment-822989275
From what I saw server is converting a string literal that appears in the predicate at least twice to an integer value and this is after Broker/Calcite has already parsed the string as number and possibly operated on (plus/minus scalar functions). So while it works, there is a bit of cost associated with it. There are also potential correctness issues. For example long to double conversion (which is happening in ScalarFunction.java on the Broker side due to ScalarFunctions.java) is a lossy conversion. None of these issues by themselves are major, but combined together they seem to indicate that some cleanup with type handling would be useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6814: FLOAT missing from Literal datatypes, but defined in Column datatypes.
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #6814:
URL: https://github.com/apache/incubator-pinot/issues/6814#issuecomment-823537346
I agree that always using string literal is not efficient (legacy behavior carried over from PQL). For literal, we should keep only the necessary types as long as the value can be represented correctly without losing accuracy. Keeping `LONG`, `DOUBLE` and `STRING` should be able to represent all types supported by Pinot. We don't want to add too many literal types because that will complicate the logic of type conversion from literal to pinot type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org