You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/11/02 00:34:47 UTC
[GitHub] [arrow-datafusion] viirya opened a new issue, #4072: Floating value literals without postfix should be parsed as decimal
viirya opened a new issue, #4072:
URL: https://github.com/apache/arrow-datafusion/issues/4072
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
(This section helps Arrow developers understand the context and *why* for this feature, in addition to the *what*)
Related to #4024, #4071.
A literal like 0.06 is parsed as double in DataFusion. It causes some counter-intuitive result as 0.06 - 0.01 = 0.049999999, 0.06 + 0.01 = 0.069999999 in DataFusion. This result is correct, though. (If you ask Spark to treat them as double (i.e., `0.06f`), you will get same result).
Such literals are parsed as decimal in Spark. I think for floating literals without postfix, we should parse them as decimal in DataFusion
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] viirya commented on issue #4072: Floating value literals without postfix should be parsed as decimal
Posted by GitBox <gi...@apache.org>.
viirya commented on issue #4072:
URL: https://github.com/apache/arrow-datafusion/issues/4072#issuecomment-1299412915
Yes, I played this a bit yesterday. With some tweak, I can make it parsed as decimal and get 0.06 - 0.01 = 0.05 correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] andygrove commented on issue #4072: Floating value literals without postfix should be parsed as decimal
Posted by GitBox <gi...@apache.org>.
andygrove commented on issue #4072:
URL: https://github.com/apache/arrow-datafusion/issues/4072#issuecomment-1301281410
Here is the spec for SQL numeric literals:
```
<signed numeric literal> ::=
[ <sign> ] <unsigned numeric literal>
<unsigned numeric literal> ::=
<exact numeric literal>
| <approximate numeric literal>
<exact numeric literal> ::=
<unsigned integer> [ <period> [ <unsigned integer> ] ]
| <period> <unsigned integer>
<sign> ::=
<plus sign>
| <minus sign>
<approximate numeric literal> ::=
<mantissa> E <exponent>
<mantissa> ::=
<exact numeric literal>
<exponent> ::=
<signed integer>
<signed integer> ::=
[ <sign> ] <unsigned integer>
<unsigned integer> ::=
<digit>...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] andygrove commented on issue #4072: Floating value literals without postfix should be parsed as decimal
Posted by GitBox <gi...@apache.org>.
andygrove commented on issue #4072:
URL: https://github.com/apache/arrow-datafusion/issues/4072#issuecomment-1299405633
Thanks @viirya I had forgotten about this ... this is the relevant code in sql/planner.rs, for reference
```rust
// Parse number in sql string, convert to Expr::Literal
fn parse_sql_number(n: &str) -> Result<Expr> {
// parse first as i64
n.parse::<i64>()
.map(lit)
// if parsing as i64 fails try f64
.or_else(|_| n.parse::<f64>().map(lit))
.map_err(|_| {
DataFusionError::from(ParserError(format!(
"Cannot parse {} as i64 or f64",
n
)))
})
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] liukun4515 commented on issue #4072: Floating value literals without postfix should be parsed as decimal
Posted by GitBox <gi...@apache.org>.
liukun4515 commented on issue #4072:
URL: https://github.com/apache/arrow-datafusion/issues/4072#issuecomment-1306830442
agree with your suggestion that if there is no postfix, we will convert the data to decimal by default.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org