[carbondata] branch master updated: [HOTFIX] Fix 2 random failures in CI

[qi...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

qiangcai pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/carbondata.git


The following commit(s) were added to refs/heads/master by this push:
     new f0ed756  [HOTFIX] Fix 2 random failures in CI
f0ed756 is described below

commit f0ed75664078a0a9872118800734e214b91ee3a4
Author: ajantha-bhat <aj...@gmail.com>
AuthorDate: Tue Sep 22 17:23:01 2020 +0530

    [HOTFIX] Fix 2 random failures in CI
    
    Why is this PR needed?
    There is a random 11 testcase failure in CI.
    
    What changes were proposed in this PR?
    a) After analyzing and adding logs, found out that test case added in #3608 has not reset the carbon.enable.auto.load.merge carbon property.
    This issue is random as if any other testcase (which resets that carbon property) in CarbonIndexFileMergeTestCase is ran before issue testcase, it will set the value to false and mask the issue. Also in a suite, order of testcase execution is not same always. So, it is random.
    
    b) Test case added by #3832 has a problem. test cdc with compaction fails randomly because 12.2 segment will not be there and merge carbon property is not reset when this is failed.
    It is because auto compaction sometimes 0.1 goes for marked for delete during merge operation. There is no record mismatch. Just that 12.2 doesn't exist. It will be in some other segment name. currently removed checking that 12.2 segment.
    
    Does this PR introduce any user interface change?
    No
    
    Is any new testcase added?
    No
    
    this closes #3948
---
 .../spark/testsuite/mergedata/CarbonDataFileMergeTestCaseOnSI.scala     | 2 ++
 .../org/apache/carbondata/spark/testsuite/merge/MergeTestCase.scala     | 1 -
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/index/secondary-index/src/test/scala/org/apache/carbondata/spark/testsuite/mergedata/CarbonDataFileMergeTestCaseOnSI.scala b/index/secondary-index/src/test/scala/org/apache/carbondata/spark/testsuite/mergedata/CarbonDataFileMergeTestCaseOnSI.scala
index 00c7d4a..da8c13b 100644
--- a/index/secondary-index/src/test/scala/org/apache/carbondata/spark/testsuite/mergedata/CarbonDataFileMergeTestCaseOnSI.scala
+++ b/index/secondary-index/src/test/scala/org/apache/carbondata/spark/testsuite/mergedata/CarbonDataFileMergeTestCaseOnSI.scala
@@ -223,6 +223,8 @@ class CarbonDataFileMergeTestCaseOnSI
     CarbonProperties.getInstance()
       .addProperty(CarbonCommonConstants.COMPACTION_SEGMENT_LEVEL_THRESHOLD,
         CarbonCommonConstants.DEFAULT_SEGMENT_LEVEL_THRESHOLD)
+      .addProperty(CarbonCommonConstants.ENABLE_AUTO_LOAD_MERGE,
+        CarbonCommonConstants.DEFAULT_ENABLE_AUTO_LOAD_MERGE)
   }
 
   private def getDataFileCount(tableName: String, segment: String): Int = {
diff --git a/integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/merge/MergeTestCase.scala b/integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/merge/MergeTestCase.scala
index 829c815..a81cd73 100644
--- a/integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/merge/MergeTestCase.scala
+++ b/integration/spark/src/test/scala/org/apache/carbondata/spark/testsuite/merge/MergeTestCase.scala
@@ -1044,7 +1044,6 @@ class MergeTestCase extends QueryTest with BeforeAndAfterAll {
     }
 
     sql("clean files for table target").collect()
-    assert(getDeleteDeltaFileCount("target", "12.2") == 1)
     checkAnswer(sql("select count(*) from target"), Seq(Row(70)))
 
     CarbonProperties.getInstance().addProperty("carbon.enable.auto.load.merge", "false")