You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2020/05/04 02:27:19 UTC
[GitHub] [incubator-hudi] xushiyan opened a new pull request #1590: [HUDI-812] Migrate hudi common tests
xushiyan opened a new pull request #1590:
URL: https://github.com/apache/incubator-hudi/pull/1590
Migrate the test cases in hudi-common to JUnit 5.
Follows #1589
### Migration status (after merging)
| Package | JUnit 5 lib | API migration | Restructure packages |
| --- | --- | --- | --- |
| `hudi-cli` | ✅ | ✅ | - |
| `hudi-client` | ✅ | ✅ | - |
| `hudi-common` | ✅ | ✅ | 🚧 |
| `hudi-hadoop-mr` | ✅ | ✅ | - |
| `hudi-hive-sync` | ✅ | ✅ | - |
| `hudi-integ-test` | ✅ | ✅ | N.A. |
| `hudi-spark` | ✅ | ✅ | - |
| `hudi-timeline-service` | ✅ | ✅ | - |
| `hudi-utilities` | ✅ | ✅ | - |
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] codecov-io edited a comment on pull request #1590: [HUDI-812] Migrate hudi common tests to JUnit 5
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #1590:
URL: https://github.com/apache/incubator-hudi/pull/1590#issuecomment-623236291
# [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=h1) Report
> Merging [#1590](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-hudi/commit/506447fd4fde4cd922f7aa8f4e17a7f06666dc97&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #1590 +/- ##
============================================
+ Coverage 71.82% 71.84% +0.01%
Complexity 294 294
============================================
Files 385 385
Lines 16549 16549
Branches 1661 1661
============================================
+ Hits 11886 11889 +3
+ Misses 3931 3928 -3
Partials 732 732
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ache/hudi/common/fs/inline/InMemoryFileSystem.java](https://codecov.io/gh/apache/incubator-hudi/pull/1590/diff?src=pr&el=tree#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL2ZzL2lubGluZS9Jbk1lbW9yeUZpbGVTeXN0ZW0uamF2YQ==) | `89.65% <0.00%> (+10.34%)` | `0.00% <0.00%> (ø%)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=footer). Last update [506447f...717bddb](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] codecov-io edited a comment on pull request #1590: [HUDI-812] Migrate hudi common tests to JUnit 5
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #1590:
URL: https://github.com/apache/incubator-hudi/pull/1590#issuecomment-623236291
# [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=h1) Report
> Merging [#1590](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-hudi/commit/506447fd4fde4cd922f7aa8f4e17a7f06666dc97&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #1590 +/- ##
============================================
+ Coverage 71.82% 71.84% +0.01%
Complexity 294 294
============================================
Files 385 385
Lines 16549 16549
Branches 1661 1661
============================================
+ Hits 11886 11889 +3
+ Misses 3931 3928 -3
Partials 732 732
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ache/hudi/common/fs/inline/InMemoryFileSystem.java](https://codecov.io/gh/apache/incubator-hudi/pull/1590/diff?src=pr&el=tree#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL2ZzL2lubGluZS9Jbk1lbW9yeUZpbGVTeXN0ZW0uamF2YQ==) | `89.65% <0.00%> (+10.34%)` | `0.00% <0.00%> (ø%)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=footer). Last update [506447f...717bddb](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] codecov-io edited a comment on pull request #1590: [HUDI-812] Migrate hudi common tests to JUnit 5
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #1590:
URL: https://github.com/apache/incubator-hudi/pull/1590#issuecomment-623236291
# [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=h1) Report
> Merging [#1590](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-hudi/commit/506447fd4fde4cd922f7aa8f4e17a7f06666dc97&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #1590 +/- ##
============================================
+ Coverage 71.82% 71.83% +0.01%
Complexity 294 294
============================================
Files 385 385
Lines 16549 16549
Branches 1661 1661
============================================
+ Hits 11886 11888 +2
+ Misses 3931 3929 -2
Partials 732 732
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...che/hudi/common/util/BufferedRandomAccessFile.java](https://codecov.io/gh/apache/incubator-hudi/pull/1590/diff?src=pr&el=tree#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3V0aWwvQnVmZmVyZWRSYW5kb21BY2Nlc3NGaWxlLmphdmE=) | `54.38% <0.00%> (-0.88%)` | `0.00% <0.00%> (ø%)` | |
| [...ache/hudi/common/fs/inline/InMemoryFileSystem.java](https://codecov.io/gh/apache/incubator-hudi/pull/1590/diff?src=pr&el=tree#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL2ZzL2lubGluZS9Jbk1lbW9yeUZpbGVTeXN0ZW0uamF2YQ==) | `89.65% <0.00%> (+10.34%)` | `0.00% <0.00%> (ø%)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=footer). Last update [506447f...717bddb](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] xushiyan commented on pull request #1590: [HUDI-812] Migrate hudi common tests to JUnit 5
Posted by GitBox <gi...@apache.org>.
xushiyan commented on pull request #1590:
URL: https://github.com/apache/incubator-hudi/pull/1590#issuecomment-623231117
@yanghua This marks completion of API migration. Once #1589 merged, I'll rebase this on master and update checkstyle to ban junit 4 import.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1590: [HUDI-812] Migrate hudi common tests to JUnit 5
Posted by GitBox <gi...@apache.org>.
xushiyan commented on a change in pull request #1590:
URL: https://github.com/apache/incubator-hudi/pull/1590#discussion_r419542895
##########
File path: hudi-common/src/test/java/org/apache/hudi/common/fs/TestFSUtils.java
##########
@@ -18,25 +18,25 @@
package org.apache.hudi.common.fs;
-import org.apache.hudi.common.HoodieCommonTestHarness;
import org.apache.hudi.common.model.HoodieLogFile;
import org.apache.hudi.common.model.HoodieTestUtils;
import org.apache.hudi.common.table.HoodieTableMetaClient;
import org.apache.hudi.common.table.timeline.HoodieInstant;
import org.apache.hudi.common.table.timeline.HoodieTimeline;
+import org.apache.hudi.common.testutils.HoodieCommonTestHarness;
import org.apache.hudi.exception.HoodieException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
-import org.junit.Assert;
-import org.junit.Before;
import org.junit.Rule;
-import org.junit.Test;
import org.junit.contrib.java.lang.system.EnvironmentVariables;
Review comment:
https://github.com/junit-pioneer/junit-pioneer
tried this but didn't work. its reflection logic caused env var contains `null` kv pair which led to error.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] codecov-io commented on pull request #1590: [HUDI-812] Migrate hudi common tests to JUnit 5
Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #1590:
URL: https://github.com/apache/incubator-hudi/pull/1590#issuecomment-623236291
# [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=h1) Report
> Merging [#1590](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-hudi/commit/506447fd4fde4cd922f7aa8f4e17a7f06666dc97&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #1590 +/- ##
============================================
+ Coverage 71.82% 71.83% +0.01%
Complexity 294 294
============================================
Files 385 385
Lines 16549 16549
Branches 1661 1661
============================================
+ Hits 11886 11888 +2
+ Misses 3931 3929 -2
Partials 732 732
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...che/hudi/common/util/BufferedRandomAccessFile.java](https://codecov.io/gh/apache/incubator-hudi/pull/1590/diff?src=pr&el=tree#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL3V0aWwvQnVmZmVyZWRSYW5kb21BY2Nlc3NGaWxlLmphdmE=) | `54.38% <0.00%> (-0.88%)` | `0.00% <0.00%> (ø%)` | |
| [...ache/hudi/common/fs/inline/InMemoryFileSystem.java](https://codecov.io/gh/apache/incubator-hudi/pull/1590/diff?src=pr&el=tree#diff-aHVkaS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY29tbW9uL2ZzL2lubGluZS9Jbk1lbW9yeUZpbGVTeXN0ZW0uamF2YQ==) | `89.65% <0.00%> (+10.34%)` | `0.00% <0.00%> (ø%)` | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=footer). Last update [506447f...717bddb](https://codecov.io/gh/apache/incubator-hudi/pull/1590?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1590: [HUDI-812] Migrate hudi common tests to JUnit 5
Posted by GitBox <gi...@apache.org>.
xushiyan commented on a change in pull request #1590:
URL: https://github.com/apache/incubator-hudi/pull/1590#discussion_r419227760
##########
File path: hudi-common/src/test/java/org/apache/hudi/common/fs/TestFSUtils.java
##########
@@ -18,25 +18,25 @@
package org.apache.hudi.common.fs;
-import org.apache.hudi.common.HoodieCommonTestHarness;
import org.apache.hudi.common.model.HoodieLogFile;
import org.apache.hudi.common.model.HoodieTestUtils;
import org.apache.hudi.common.table.HoodieTableMetaClient;
import org.apache.hudi.common.table.timeline.HoodieInstant;
import org.apache.hudi.common.table.timeline.HoodieTimeline;
+import org.apache.hudi.common.testutils.HoodieCommonTestHarness;
import org.apache.hudi.exception.HoodieException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
-import org.junit.Assert;
-import org.junit.Before;
import org.junit.Rule;
-import org.junit.Test;
import org.junit.contrib.java.lang.system.EnvironmentVariables;
Review comment:
["system-rules"](https://stefanbirkner.github.io/system-rules/) is the only Junit 4 extension needed for setting env vars to run a test case in this class. It works fine with Junit 5. Hence these 2 imports ( org.junit.Rule and org.junit.contrib.*) are still here. Also exclude this in checkstyle.xml
##########
File path: hudi-hive-sync/src/test/java/org/apache/hudi/hive/TestUtil.java
##########
@@ -59,7 +59,7 @@
import org.joda.time.DateTime;
import org.joda.time.format.DateTimeFormat;
import org.joda.time.format.DateTimeFormatter;
-import org.junit.runners.model.InitializationError;
+import org.junit.platform.commons.JUnitException;
Review comment:
Change to a RuntimeException in Junit 5
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org