You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/10/22 17:14:19 UTC
[GitHub] [hudi] the-other-tim-brown opened a new pull request, #7033: [minor] test cleanup
the-other-tim-brown opened a new pull request, #7033:
URL: https://github.com/apache/hudi/pull/7033
### Change Logs
Some minor changes for some tests to make sure that resources are cleaned up properly.
### Impact
Only impact is more consistent test execution.
### Risk level
None
### Documentation Update
None
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan merged pull request #7033: [HUDI-5081] Tests clean up in hudi-utilities
Posted by GitBox <gi...@apache.org>.
xushiyan merged PR #7033:
URL: https://github.com/apache/hudi/pull/7033
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7033: [MINOR] test cleanup
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7033:
URL: https://github.com/apache/hudi/pull/7033#issuecomment-1288232199
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "74c5d9c3ef7845d36a97e27f86399f45050e8dff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12469",
"triggerID" : "74c5d9c3ef7845d36a97e27f86399f45050e8dff",
"triggerType" : "PUSH"
}, {
"hash" : "94bac61f150805e4ec82f3d5bf44b55699257534",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12471",
"triggerID" : "94bac61f150805e4ec82f3d5bf44b55699257534",
"triggerType" : "PUSH"
}, {
"hash" : "c4dcf26eba06562edb428e668fccbc94ed48f07b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12499",
"triggerID" : "c4dcf26eba06562edb428e668fccbc94ed48f07b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* c4dcf26eba06562edb428e668fccbc94ed48f07b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12499)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on a diff in pull request #7033: [MINOR] test cleanup
Posted by GitBox <gi...@apache.org>.
xushiyan commented on code in PR #7033:
URL: https://github.com/apache/hudi/pull/7033#discussion_r1002697715
##########
hudi-utilities/src/test/java/org/apache/hudi/utilities/TestHoodieIndexer.java:
##########
@@ -100,6 +101,16 @@ public void init() throws IOException {
initMetaClient();
}
+ @AfterAll
+ public static void cleanup() {
Review Comment:
this test class can actually extend `org.apache.hudi.testutils.SparkClientFunctionalTestHarness` to make use of most of the resources like spark session and context. should just migrate to use it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7033: [MINOR] test cleanup
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7033:
URL: https://github.com/apache/hudi/pull/7033#issuecomment-1287870374
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "74c5d9c3ef7845d36a97e27f86399f45050e8dff",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12469",
"triggerID" : "74c5d9c3ef7845d36a97e27f86399f45050e8dff",
"triggerType" : "PUSH"
}, {
"hash" : "94bac61f150805e4ec82f3d5bf44b55699257534",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "94bac61f150805e4ec82f3d5bf44b55699257534",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 74c5d9c3ef7845d36a97e27f86399f45050e8dff Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12469)
* 94bac61f150805e4ec82f3d5bf44b55699257534 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7033: [MINOR] test cleanup
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7033:
URL: https://github.com/apache/hudi/pull/7033#issuecomment-1287868400
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "74c5d9c3ef7845d36a97e27f86399f45050e8dff",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "74c5d9c3ef7845d36a97e27f86399f45050e8dff",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 74c5d9c3ef7845d36a97e27f86399f45050e8dff UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7033: [MINOR] test cleanup
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7033:
URL: https://github.com/apache/hudi/pull/7033#issuecomment-1288200718
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "74c5d9c3ef7845d36a97e27f86399f45050e8dff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12469",
"triggerID" : "74c5d9c3ef7845d36a97e27f86399f45050e8dff",
"triggerType" : "PUSH"
}, {
"hash" : "94bac61f150805e4ec82f3d5bf44b55699257534",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12471",
"triggerID" : "94bac61f150805e4ec82f3d5bf44b55699257534",
"triggerType" : "PUSH"
}, {
"hash" : "c4dcf26eba06562edb428e668fccbc94ed48f07b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12499",
"triggerID" : "c4dcf26eba06562edb428e668fccbc94ed48f07b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 94bac61f150805e4ec82f3d5bf44b55699257534 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12471)
* c4dcf26eba06562edb428e668fccbc94ed48f07b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12499)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7033: [MINOR] test cleanup
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7033:
URL: https://github.com/apache/hudi/pull/7033#issuecomment-1287869434
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "74c5d9c3ef7845d36a97e27f86399f45050e8dff",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12469",
"triggerID" : "74c5d9c3ef7845d36a97e27f86399f45050e8dff",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 74c5d9c3ef7845d36a97e27f86399f45050e8dff Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12469)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on pull request #7033: [HUDI-5081] Tests clean up in hudi-utilities
Posted by GitBox <gi...@apache.org>.
xushiyan commented on PR #7033:
URL: https://github.com/apache/hudi/pull/7033#issuecomment-1288403021
CI failure due to unrelated flakiness. this change is only for utilities tests, which have passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7033: [MINOR] test cleanup
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7033:
URL: https://github.com/apache/hudi/pull/7033#issuecomment-1288199741
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "74c5d9c3ef7845d36a97e27f86399f45050e8dff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12469",
"triggerID" : "74c5d9c3ef7845d36a97e27f86399f45050e8dff",
"triggerType" : "PUSH"
}, {
"hash" : "94bac61f150805e4ec82f3d5bf44b55699257534",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12471",
"triggerID" : "94bac61f150805e4ec82f3d5bf44b55699257534",
"triggerType" : "PUSH"
}, {
"hash" : "c4dcf26eba06562edb428e668fccbc94ed48f07b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "c4dcf26eba06562edb428e668fccbc94ed48f07b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 94bac61f150805e4ec82f3d5bf44b55699257534 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12471)
* c4dcf26eba06562edb428e668fccbc94ed48f07b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7033: [MINOR] test cleanup
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7033:
URL: https://github.com/apache/hudi/pull/7033#issuecomment-1287885276
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "74c5d9c3ef7845d36a97e27f86399f45050e8dff",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12469",
"triggerID" : "74c5d9c3ef7845d36a97e27f86399f45050e8dff",
"triggerType" : "PUSH"
}, {
"hash" : "94bac61f150805e4ec82f3d5bf44b55699257534",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12471",
"triggerID" : "94bac61f150805e4ec82f3d5bf44b55699257534",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 74c5d9c3ef7845d36a97e27f86399f45050e8dff Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12469)
* 94bac61f150805e4ec82f3d5bf44b55699257534 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12471)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] the-other-tim-brown commented on a diff in pull request #7033: [MINOR] test cleanup
Posted by GitBox <gi...@apache.org>.
the-other-tim-brown commented on code in PR #7033:
URL: https://github.com/apache/hudi/pull/7033#discussion_r1002767620
##########
hudi-utilities/src/test/java/org/apache/hudi/utilities/TestHoodieIndexer.java:
##########
@@ -100,6 +101,16 @@ public void init() throws IOException {
initMetaClient();
}
+ @AfterAll
+ public static void cleanup() {
Review Comment:
Yes, I've updated the code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7033: [MINOR] test cleanup
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7033:
URL: https://github.com/apache/hudi/pull/7033#issuecomment-1287925079
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "74c5d9c3ef7845d36a97e27f86399f45050e8dff",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12469",
"triggerID" : "74c5d9c3ef7845d36a97e27f86399f45050e8dff",
"triggerType" : "PUSH"
}, {
"hash" : "94bac61f150805e4ec82f3d5bf44b55699257534",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12471",
"triggerID" : "94bac61f150805e4ec82f3d5bf44b55699257534",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 94bac61f150805e4ec82f3d5bf44b55699257534 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=12471)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] the-other-tim-brown commented on a diff in pull request #7033: [minor] test cleanup
Posted by GitBox <gi...@apache.org>.
the-other-tim-brown commented on code in PR #7033:
URL: https://github.com/apache/hudi/pull/7033#discussion_r1002537771
##########
hudi-utilities/src/test/java/org/apache/hudi/utilities/sources/TestGcsEventsSource.java:
##########
@@ -63,14 +60,8 @@ public static void beforeAll() throws Exception {
UtilitiesTestBase.initTestServices(false, false);
}
- @AfterAll
- public static void afterAll() {
Review Comment:
These parent methods were all being called twice since the test is extending UtilitiesTestBase
##########
hudi-utilities/src/test/java/org/apache/hudi/utilities/deser/TestKafkaAvroSchemaDeserializer.java:
##########
@@ -45,7 +44,7 @@
/**
* Tests {@link KafkaAvroSchemaDeserializer}.
*/
-public class TestKafkaAvroSchemaDeserializer extends UtilitiesTestBase {
+public class TestKafkaAvroSchemaDeserializer {
Review Comment:
The UtilitiesTestBase was just adding unneeded overhead to this test
##########
hudi-utilities/src/test/java/org/apache/hudi/utilities/TestHoodieIndexer.java:
##########
@@ -100,6 +101,16 @@ public void init() throws IOException {
initMetaClient();
}
+ @AfterAll
+ public static void cleanup() {
Review Comment:
The spark context wasn't being closed which could cause other tests to fail that tried to create their own context
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on a diff in pull request #7033: [HUDI-5081] Tests clean up in hudi-utilities
Posted by GitBox <gi...@apache.org>.
xushiyan commented on code in PR #7033:
URL: https://github.com/apache/hudi/pull/7033#discussion_r1002872498
##########
hudi-utilities/src/test/java/org/apache/hudi/utilities/TestHoodieIndexer.java:
##########
@@ -75,46 +68,29 @@
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
-public class TestHoodieIndexer extends HoodieCommonTestHarness implements SparkProvider {
+public class TestHoodieIndexer extends SparkClientFunctionalTestHarness implements SparkProvider {
Review Comment:
/nit SparkProvider already implemented by SparkClientFunctionalTestHarness
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org