You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "jdcasale (via GitHub)" <gi...@apache.org> on 2024/04/15 16:01:12 UTC
[I] DECIMAL regex in csv reader does not accept positive exponent specifier [arrow-rs]
jdcasale opened a new issue, #5648:
URL: https://github.com/apache/arrow-rs/issues/5648
**Describe the bug**
<!--
A clear and concise description of what the bug is.
-->
Decimals in scientific notation are frequently expressed with a (admittedly unnecessary) positive exponent specifier, e.g "3.106e+04". [The existing regex ](https://github.com/apache/arrow-rs/blame/master/arrow-csv/src/reader/mod.rs#L151)allows for negative exponent specifiers, but does not recognize a number with a positive specifier. This causes the parser to infer the type of any column with positive exponent specifiers as a Utf8 instead of float.
As a sanity check, I tried the same thing in DuckDB, and their csv parser does not make this error.
**To Reproduce**
<!--
Steps to reproduce the behavior:
-->
Either attempt to infer schema for a csv file containing the offending pattern (like I have done [here in this provided example](https://github.com/jdcasale/arrow-csv-parse-bug/blob/develop/src/main.rs)) or just run the existing regex directly against the example offender: "3.106e+04", it will not match.
**Expected behavior**
<!--
A clear and concise description of what you expected to happen.
-->
The decimal regex recognizes "3.106e+04" as a float value, not a Utf8 string.
**Additional context**
<!--
Add any other context about the problem here.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] DECIMAL regex in csv reader does not accept positive exponent specifier [arrow-rs]
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold closed issue #5648: DECIMAL regex in csv reader does not accept positive exponent specifier
URL: https://github.com/apache/arrow-rs/issues/5648
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] DECIMAL regex in csv reader does not accept positive exponent specifier [arrow-rs]
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #5648:
URL: https://github.com/apache/arrow-rs/issues/5648#issuecomment-2057394839
It is worth noting that we only supported parsing scientific notation as of this morning - https://github.com/apache/arrow-rs/pull/5611
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] DECIMAL regex in csv reader does not accept positive exponent specifier [arrow-rs]
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #5648:
URL: https://github.com/apache/arrow-rs/issues/5648#issuecomment-2061310335
`label_issue.py` automatically added labels {'arrow'} from #5649
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org