You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2016/03/03 06:47:18 UTC
[jira] [Updated] (SPARK-13638) Support for saving with a quote mode
[ https://issues.apache.org/jira/browse/SPARK-13638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon updated SPARK-13638:
---------------------------------
Description:
https://github.com/databricks/spark-csv/pull/254
tobithiel reported this.
{quote}
I'm dealing with some messy csv files and being able to just quote all fields is very useful,
so that other applications don't misunderstand the file because of some sketchy characters....
{quote}
When writing there are several quote modes in apache commons csv. (See https://commons.apache.org/proper/commons-csv/apidocs/org/apache/commons/csv/QuoteMode.html)
This might have to be supported.
However, it looks univocity parser used for writing does not support this quote mode. I think we can drop this backwards compatibility if we are not going to add apache commons csv.
This is a reminder that it will break backwards compatibility for the options, {{quoteMode}} and {{parserLib}}.
was:
https://github.com/databricks/spark-csv/pull/254
tobithiel reported this.
>I'm dealing with some messy csv files and being able to just quote all fields is very useful, so that other applications don't misunderstand the file because of some sketchy characters....
When writing there are several quote modes in apache commons csv. (See https://commons.apache.org/proper/commons-csv/apidocs/org/apache/commons/csv/QuoteMode.html)
This might have to be supported.
However, it looks univocity parser used for writing does not support this quote mode. I think we can drop this backwards compatibility if we are not going to add apache commons csv.
This is a reminder that it will break backwards compatibility for the options, {{quoteMode}} and {{parserLib}}.
> Support for saving with a quote mode
> ------------------------------------
>
> Key: SPARK-13638
> URL: https://issues.apache.org/jira/browse/SPARK-13638
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 2.0.0
> Reporter: Hyukjin Kwon
> Priority: Minor
>
> https://github.com/databricks/spark-csv/pull/254
> tobithiel reported this.
> {quote}
> I'm dealing with some messy csv files and being able to just quote all fields is very useful,
> so that other applications don't misunderstand the file because of some sketchy characters....
> {quote}
> When writing there are several quote modes in apache commons csv. (See https://commons.apache.org/proper/commons-csv/apidocs/org/apache/commons/csv/QuoteMode.html)
> This might have to be supported.
> However, it looks univocity parser used for writing does not support this quote mode. I think we can drop this backwards compatibility if we are not going to add apache commons csv.
> This is a reminder that it will break backwards compatibility for the options, {{quoteMode}} and {{parserLib}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org