You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@drill.apache.org by "Charles Givre (Jira)" <ji...@apache.org> on 2021/02/24 15:48:00 UTC

[jira] [Commented] (DRILL-7869) CSV files can't mix line breaks \x0d Vs. \x0d\x0a

    [ https://issues.apache.org/jira/browse/DRILL-7869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17290031#comment-17290031 ] 

Charles Givre commented on DRILL-7869:
--------------------------------------

[~curtis283] It seems to me that the best way to solve this would be to autodetect the line ending.  I believe the underlying reader can actually do this but we'll have to make some mods to the plugin.

> CSV files can't mix line breaks \x0d Vs. \x0d\x0a
> -------------------------------------------------
>
>                 Key: DRILL-7869
>                 URL: https://issues.apache.org/jira/browse/DRILL-7869
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Storage - Text &amp; CSV
>    Affects Versions: 1.19.0
>            Reporter: Curtis Lambert
>            Priority: Minor
>
> Querying CSV files with \x0d new line delimiters results in "DATA_READ ERROR: Column exceeds maximum length of 1024" with the default configuration.
> The \x0d new line isn't used to break lines resulting in the entire file being read in as a single record. This is configurable as "delimiter" in the format but if you have mixed csv files with different line endings it's problematic. If I have files with both \x0d and \x0d\x0a new lines (\r\n) and need to be able to read both without having to change the configuration between queries.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)