[jira] [Created] (CARBONDATA-1168) Driver Delete data operation is failed due to failure in creating delete delta file for segment

["Kingsley Chen (JIRA)" <ji...@apache.org>]

Kingsley Chen created CARBONDATA-1168:
-----------------------------------------

             Summary: Driver Delete data operation is failed due to failure in creating delete delta file for segment
                 Key: CARBONDATA-1168
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1168
             Project: CarbonData
          Issue Type: Bug
          Components: sql
    Affects Versions: 1.1.0
         Environment: spark1.6+carbon1.1.0
we have a 20 nodes cluster and 32GB ram each
            Reporter: Kingsley Chen
             Fix For: NONE


We use spark code to delete data from table as below
------------------spark code----------------------
val deleteSql = s"DELETE FROM $tableName WHERE $rowkeyName IN (${rowKeyVals.mkString(",")})"
    cc.sql(deleteSql).show()
------------------spark code----------------------

when the array size of rowKeyVals is larger than 200, the delete operation will be failed, and print the log:
Delete data request has been received for default.item
Delete data operation is failed for default.item
Driver Delete data operation is failed due to failure in creating delete delta file for segment : null block : null
++
||
++
++

That is to say, it only delete success maximun at 200 a batch,and took about 1min which is
too slow. So my question is how to tuning the performance to make the batch larger 
and delete faster







--
This message was sent by Atlassian JIRA
(v6.4.14#64029)