[jira] [Commented] (SPARK-15493) Allow setting the quoteEscapingEnabled flag when writing CSV

["Jacob Wellington (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15785951#comment-15785951 ] 

Jacob Wellington commented on SPARK-15493:
------------------------------------------

I'm running into an issue where this doesn't seem to be working for the SQL interface. I'm connecting to the thrift server using beeline and submitting the following sql:
{quote}
        CREATE TABLE e2
        USING csv 
        OPTIONS (path 'test.csv', quote '"', escapeQuotes 'false', quoteEscapingEnabled 'false') AS
        SELECT 
        '"G"'
        FROM parquet.`test.parquet`;
      DROP TABLE e2;
{quote}
When I look at the test.csv output I get this:
{quote}
"\"G\""
{quote}

I'm using spark 2.0.2 with its version of beeline and its hive server. I've also tried multiple variations of the options.


> Allow setting the quoteEscapingEnabled flag when writing CSV
> ------------------------------------------------------------
>
>                 Key: SPARK-15493
>                 URL: https://issues.apache.org/jira/browse/SPARK-15493
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>    Affects Versions: 2.0.0
>            Reporter: Jurriaan Pruis
>            Assignee: Jurriaan Pruis
>             Fix For: 2.0.0
>
>
> See https://github.com/uniVocity/univocity-parsers/blob/f3eb2af26374940e60d91d1703bde54619f50c51/src/main/java/com/univocity/parsers/csv/CsvWriterSettings.java#L231-L247
> This kind of functionality is needed to be able to write RFC 4180 (https://tools.ietf.org/html/rfc4180#section-2) / Amazon Redshift compatible CSV files (https://docs.aws.amazon.com/redshift/latest/dg/copy-parameters-data-format.html#copy-csv)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org