[jira] [Closed] (CARBONDATA-2109) config of dataframe load with tempCSV is invalid ,such as QUOTECHAR

["Chenjian Qiu (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CARBONDATA-2109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chenjian Qiu closed CARBONDATA-2109.
------------------------------------
       Resolution: Fixed
    Fix Version/s: 1.3.0

discard tempCSV option 

> config of dataframe load with tempCSV is invalid ,such as QUOTECHAR
> -------------------------------------------------------------------
>
>                 Key: CARBONDATA-2109
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2109
>             Project: CarbonData
>          Issue Type: Improvement
>          Components: data-load
>    Affects Versions: 1.2.0
>            Reporter: Chenjian Qiu
>            Priority: Minor
>             Fix For: 1.3.0
>
>   Original Estimate: 2h
>          Time Spent: 2h
>  Remaining Estimate: 0h
>
> use datafame to load data with tempCSV true(dataset.writer..format("carbondata").save), carbondata will generate sql to load data
> the sql's configs only have SINGLE_PASS, other configs is invalid,such as QUOTECHAR.
> the code is in CarbonDataFrameWriter:
>   private def makeLoadString(csvFolder: String, options: CarbonOption): String = {
>     val dbName = CarbonEnv.getDatabaseName(options.dbName)(sqlContext.sparkSession)
>     s"""
>        | LOAD DATA INPATH '$csvFolder'
>        | INTO TABLE $dbName.${options.tableName}
>        | OPTIONS ('FILEHEADER' = '${dataFrame.columns.mkString(",")}',
>        | 'SINGLE_PASS' = '${options.singlePass}')
>      """.stripMargin
>   }



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)