[spark] branch master updated: [SPARK-32305][BUILD] Make `mvn clean` remove `metastore_db` and `spark-warehouse`

[do...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 5e0cb3e  [SPARK-32305][BUILD] Make `mvn clean` remove `metastore_db` and `spark-warehouse`
5e0cb3e is described below

commit 5e0cb3ee16dde3666af3bf5f2b152c7d0dfe9d7b
Author: yangjie01 <ya...@baidu.com>
AuthorDate: Tue Jul 14 12:40:47 2020 -0700

    [SPARK-32305][BUILD] Make `mvn clean` remove `metastore_db` and `spark-warehouse`
    
    ### What changes were proposed in this pull request?
    
    Add additional configuration to `maven-clean-plugin` to ensure cleanup `metastore_db` and `spark-warehouse` directory when execute `mvn clean` command.
    
    ### Why are the changes needed?
    Now Spark support two version of build-in hive and there are some test generated meta data not in target dir like `metastore_db`,  they don't clean up automatically when we run `mvn clean` command.
    
    So if we run `mvn clean test -pl sql/hive -am -Phadoop-2.7 -Phive -Phive-1.2 ` , the `metastore_db` dir will created and meta data will remains after test complete.
    
    Then we need manual cleanup `metastore_db` directory to ensure `mvn clean test -pl sql/hive -am -Phadoop-2.7 -Phive` command use hive2.3 profile can succeed because the residual metastore data is not compatible.
    
    `spark-warehouse` will also cause test failure in some data residual scenarios because test case thinks that meta data should not exist.
    
    This pr is used to simplify manual cleanup `metastore_db` and `spark-warehouse` directory operation.
    
    ### How was this patch tested?
    
    Manual execute `mvn clean test -pl sql/hive -am -Phadoop-2.7 -Phive -Phive-1.2`, then execute `mvn clean test -pl sql/hive -am -Phadoop-2.7 -Phive`, both commands should succeed.
    
    Closes #29103 from LuciferYang/add-clean-directory.
    
    Authored-by: yangjie01 <ya...@baidu.com>
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
 pom.xml | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/pom.xml b/pom.xml
index 279d76d..cfcb55b 100644
--- a/pom.xml
+++ b/pom.xml
@@ -2594,6 +2594,12 @@
               <fileset>
                 <directory>lib_managed</directory>
               </fileset>
+              <fileset>
+                <directory>metastore_db</directory>
+              </fileset>
+              <fileset>
+                <directory>spark-warehouse</directory>
+              </fileset>
             </filesets>
           </configuration>
         </plugin>


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org