You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "lewy-startree (via GitHub)" <gi...@apache.org> on 2023/05/08 15:51:40 UTC
[GitHub] [pinot] lewy-startree opened a new issue, #10743: RFE: Pinot Java client: option to avoid converting BigDecimal to double when serializing results into JSON
lewy-startree opened a new issue, #10743:
URL: https://github.com/apache/pinot/issues/10743
We are seeing some different behaviors between 0.9.0 and latest Pinot:
Some of our queries are selecting on bytesToBigDecimal(amount_decimal_hex) where amount_decimal_hex is BYTES column, values are hex strings like `001200e1905c5fd123ecb1`, and
I believe the related conversion logic on Pinot is [here](https://github.com/apache/pinot/blob/master/pinot-common/src/main/java/org/apache/pinot/common/function/scalar/DataTypeConversionFunctions.java#L89).
We're using Pinot java client to make the query, and reading the results with ResultSet.getString , in the particular example, our query to
0.9.0 Pinot got result: 16.253592621780561073, while query to
latest Pinot got result: 16.253592621780562,
notice the difference at the last few digits.
But re-running the same SQL from control UI, both clusters return 16.253592621780561073 , the same results as 0.9.0 returned from Pinot client.
I did further experiment in local IDE with Pinot repo,
```
BigDecimal amount = deserialize(bytes);
System.out.println(hexString + " decoded to " + amount + ", to double:" + amount.doubleValue());
```
outputs
```
001200e1905c5fd123ecb1 decoded to 16.253592621780561073, to double:16.253592621780562
```
It seems somehow the latest Pinot converts BigDecimal to double when serializing results into JSON?
Judging from [getString on ResultTableResultSet](https://github.com/apache/pinot/blob/master/pinot-clients/pinot-java-client/src/main/java/org/apache/pinot/client/ResultTableResultSet.java#L62) where it gets the value from Jackson JsonNode.
Is there a way we can address this discrepancy?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] lewy-startree commented on issue #10743: RFE: Pinot Java client: option to avoid converting BigDecimal to double when serializing results into JSON
Posted by "lewy-startree (via GitHub)" <gi...@apache.org>.
lewy-startree commented on issue #10743:
URL: https://github.com/apache/pinot/issues/10743#issuecomment-1561417576
What is the correct behavior? Is current behavior converting BigDecimal to Double and returning that (vs. earlier return full precision string) intended ? Should we provide an option for preserving precision?
I filed JIRA DOC-86 to document current behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] navina commented on issue #10743: RFE: Pinot Java client: option to avoid converting BigDecimal to double when serializing results into JSON
Posted by "navina (via GitHub)" <gi...@apache.org>.
navina commented on issue #10743:
URL: https://github.com/apache/pinot/issues/10743#issuecomment-1560233129
@lewy-startree : is there any action item from this issue? Do we want to document the behavior change in the release notes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] lewy-startree commented on issue #10743: RFE: Pinot Java client: option to avoid converting BigDecimal to double when serializing results into JSON
Posted by "lewy-startree (via GitHub)" <gi...@apache.org>.
lewy-startree commented on issue #10743:
URL: https://github.com/apache/pinot/issues/10743#issuecomment-1544118709
The implementation difference is in [bytesToBigDecimal](https://github.com/apache/pinot/blob/master/pinot-common/src/main/java/org/apache/pinot/common/function/scalar/DataTypeConversionFunctions.java#L89) .
The 0.9.0 version converts it to string, whereas the latest version returns BigDecimal directly
`return BigDecimalUtils.deserialize(bytes).toString;`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org