You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "bbannier (via GitHub)" <gi...@apache.org> on 2024/04/27 07:01:02 UTC
[I] Support skipping comments in `CsvReader` [datafusion]
bbannier opened a new issue, #10262:
URL: https://github.com/apache/datafusion/issues/10262
It would be great if datafusion supported out-of-the-box support for skipping comment lines. While non of this is "standardized" many CSV readers support skipping full comment lines. An often used comment indentifier is a `#` prefix (default in e.g. [pandas] or [R]).
I originally posted this as a comment https://github.com/apache/datafusion/issues/8824#issuecomment-2078835858_.
[pandas]: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html
[R]: https://stat.ethz.ch/R-manual/R-devel/library/utils/html/read.table.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Support skipping comments in `CsvReader` [datafusion]
Posted by "pingsutw (via GitHub)" <gi...@apache.org>.
pingsutw commented on issue #10262:
URL: https://github.com/apache/datafusion/issues/10262#issuecomment-2080726223
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Support skipping comments in `CsvReader` [datafusion]
Posted by "bbannier (via GitHub)" <gi...@apache.org>.
bbannier commented on issue #10262:
URL: https://github.com/apache/datafusion/issues/10262#issuecomment-2106180515
I opened https://github.com/apache/arrow-rs/pull/5759 to add comment support to Arrow's CSV reader. With that the work here is mostly around passing that flag from user code to the actual reader, and implementing support for the flag to be serialized in protos.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Support skipping comments in `CsvReader` [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #10262: Support skipping comments in `CsvReader`
URL: https://github.com/apache/datafusion/issues/10262
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org