You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/11/02 00:09:55 UTC

[GitHub] [hudi] nsivabalan opened a new pull request, #7109: [MINOR] Adding tags to assist in filtering tests from maven command line

nsivabalan opened a new pull request, #7109:
URL: https://github.com/apache/hudi/pull/7109

   ### Change Logs
   
   Adding support to filter tests based on tags. We can now filter both java and scala tests from maven command line. 
   
   Eg for java: Add 
   ```
   @Tag("FlakyTests")
   ```
   to any java tests. and then run command as below. 
   ```
   mvn  -Pflaky-tests  -Dcheckstyle.skip -DfailIfNoTests=false  -Drat.ignoreErrors=true test -pl hudi-spark-datasource/hudi-spark/
   ```
   
   For scala tests, some more manual step is required. 
   Depending on whether you are running unit tests or functional tests, add flaky test group to corresponding one (this is to ignore all java tests while executing "test" goal. 
   
   For instance incase of unit tests, 
   ```
   iff --git a/pom.xml b/pom.xml
   index bf751b5d0b..50ee937bbe 100644
   --- a/pom.xml
   +++ b/pom.xml
   @@ -1683,6 +1683,7 @@
                  <skip>${skipUTs}</skip>
                  <forkedProcessExitTimeoutInSeconds>120</forkedProcessExitTimeoutInSeconds>
                  <excludedGroups>functional</excludedGroups>
   +              <groups>FlakyTests</groups>
                  <excludes>
                    <exclude>**/*FunctionalTestSuite.java</exclude>
                    <exclude>**/IT*.java</exclude>
   ```
   
   And then, we need to include scala test tags to the test of interest. for eg, Lets say we want to run just 
   TestInsertTable.Test Insert Into with values
   
   ```
   diff --git a/hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestInsertTable.scala b/hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestInsertTable.scala
   index c6eccac2bc..b7fb562725 100644
   --- a/hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestInsertTable.scala
   +++ b/hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestInsertTable.scala
   @@ -25,12 +25,13 @@ import org.apache.hudi.exception.HoodieDuplicateKeyException
    import org.apache.hudi.keygen.ComplexKeyGenerator
    import org.apache.spark.sql.SaveMode
    import org.apache.spark.sql.internal.SQLConf
   +import org.scalatest.Tag
    
    import java.io.File
    
    class TestInsertTable extends HoodieSparkSqlTestBase {
    
   -  test("Test Insert Into with values") {
   +  test("Test Insert Into with values", Tag("org.apache.hudi.FlakyTestTag")) {
        withTempDir { tmp =>
          val tableName = generateTableName
          // Create a partitioned table
   ```
   
   And then we need to uncomment "tagsToInclude" in root pom
   ```
   <tagsToInclude>org.apache.hudi.FlakyTestTag</tagsToInclude>
   ```
   
   After this, we can run just 1 scala test from command line. 
   ```
   mvn  -Punit-tests  -Dcheckstyle.skip -DfailIfNoTests=false  -Drat.ignoreErrors=true test -pl hudi-spark-datasource/hudi-spark/
   ```
   
   
   ### Impact
   
   Assist Developers to run specific test from command line using maven. 
   
   ### Risk level (write none, low medium or high below)
   
   low.
   
   ### Documentation Update
   
   Will put up an update to our website to add this info to dev set up page. 
   
   ### Contributor's checklist
   
   - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [ ] Change Logs and Impact were stated clearly
   - [ ] Adequate tests were added if applicable
   - [ ] CI passed
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7109: [MINOR] Adding tags to assist in filtering tests from maven command line

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7109:
URL: https://github.com/apache/hudi/pull/7109#issuecomment-1299421210

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "d30d7c50600fe776995460702f24ba62888d64e1",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12710",
       "triggerID" : "d30d7c50600fe776995460702f24ba62888d64e1",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d30d7c50600fe776995460702f24ba62888d64e1 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12710) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7109: [MINOR] Adding tags to assist in filtering tests from maven command line

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7109:
URL: https://github.com/apache/hudi/pull/7109#issuecomment-1299417508

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "d30d7c50600fe776995460702f24ba62888d64e1",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "d30d7c50600fe776995460702f24ba62888d64e1",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d30d7c50600fe776995460702f24ba62888d64e1 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] nsivabalan closed pull request #7109: [MINOR] Adding tags to assist in filtering tests from maven command line

Posted by GitBox <gi...@apache.org>.
nsivabalan closed pull request #7109: [MINOR] Adding tags to assist in filtering tests from maven command line
URL: https://github.com/apache/hudi/pull/7109


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #7109: [MINOR] Adding tags to assist in filtering tests from maven command line

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7109:
URL: https://github.com/apache/hudi/pull/7109#issuecomment-1299528265

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "d30d7c50600fe776995460702f24ba62888d64e1",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12710",
       "triggerID" : "d30d7c50600fe776995460702f24ba62888d64e1",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * d30d7c50600fe776995460702f24ba62888d64e1 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12710) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org