You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by GitBox <gi...@apache.org> on 2022/06/29 13:41:13 UTC
[GitHub] [drill] luocooong opened a new pull request, #2582: [Minor Update] Try and reduce the vm crash on fork
luocooong opened a new pull request, #2582:
URL: https://github.com/apache/drill/pull/2582
# [Minor Update] Try and reduce the vm crash on fork
## Description
To-do
## Documentation
To-do
## Testing
Pending CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] luocooong commented on pull request #2582: [Minor Update] Try and reduce the vm crash on fork
Posted by GitBox <gi...@apache.org>.
luocooong commented on PR #2582:
URL: https://github.com/apache/drill/pull/2582#issuecomment-1179470615
_CI-JDK 8 only._
It may be that java-heap plans to continue applying for memory at the 2500MB limit, but the system no longer has available memory.
```
[INFO] Running org.apache.drill.exec.fn.impl.TestAggregateFunctions
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "WorkManager.StatusThread"
Error: The operation was canceled.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] luocooong commented on pull request #2582: [Minor Update] Try and reduce the vm crash on fork
Posted by GitBox <gi...@apache.org>.
luocooong commented on PR #2582:
URL: https://github.com/apache/drill/pull/2582#issuecomment-1178386976
@cgivre Yes, this pull request also try to reduce the frequency of expiry cancellations caused by the Out-Of-Memory.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] jnturton commented on pull request #2582: [Minor Update] Try and reduce the vm crash on fork
Posted by GitBox <gi...@apache.org>.
jnturton commented on PR #2582:
URL: https://github.com/apache/drill/pull/2582#issuecomment-1175030561
Hi @luocooong, please see if the changes in the following patch might allow keep the SNPEGO tests enabled. They correct copy-and-paste errors I found causing conflicting directory use.
diff --git a/exec/java-exec/src/test/java/org/apache/drill/exec/server/rest/spnego/TestDrillSpnegoAuthenticator.java b/exec/java-exec/src/test/java/org/apache/drill/exec/server/rest/spnego/TestDrillSpnegoAuthenticator.java
index 643af83bf..22b574889 100644
--- a/exec/java-exec/src/test/java/org/apache/drill/exec/server/rest/spnego/TestDrillSpnegoAuthenticator.java
+++ b/exec/java-exec/src/test/java/org/apache/drill/exec/server/rest/spnego/TestDrillSpnegoAuthenticator.java
@@ -67,7 +67,6 @@ import static org.mockito.Mockito.verify;
/**
* Test for validating {@link DrillSpnegoAuthenticator}
*/
-@Ignore("See DRILL-5387")
@Category(SecurityTest.class)
public class TestDrillSpnegoAuthenticator extends BaseTest {
@@ -81,7 +80,7 @@ public class TestDrillSpnegoAuthenticator extends BaseTest {
@BeforeClass
public static void setupTest() throws Exception {
- spnegoHelper = new KerberosHelper(TestSpnegoAuthentication.class.getSimpleName(), primaryName);
+ spnegoHelper = new KerberosHelper(TestDrillSpnegoAuthenticator.class.getSimpleName(), primaryName);
spnegoHelper.setupKdc(dirTestWatcher.getTmpDir());
// (1) Refresh Kerberos config.
diff --git a/exec/java-exec/src/test/java/org/apache/drill/exec/server/rest/spnego/TestSpnegoConfig.java b/exec/java-exec/src/test/java/org/apache/drill/exec/server/rest/spnego/TestSpnegoConfig.java
index a91f80231..3d6a8cc21 100644
--- a/exec/java-exec/src/test/java/org/apache/drill/exec/server/rest/spnego/TestSpnegoConfig.java
+++ b/exec/java-exec/src/test/java/org/apache/drill/exec/server/rest/spnego/TestSpnegoConfig.java
@@ -55,7 +55,7 @@ public class TestSpnegoConfig extends BaseTest {
@BeforeClass
public static void setupTest() throws Exception {
- spnegoHelper = new KerberosHelper(TestSpnegoAuthentication.class.getSimpleName(), primaryName);
+ spnegoHelper = new KerberosHelper(TestSpnegoConfig.class.getSimpleName(), primaryName);
spnegoHelper.setupKdc(dirTestWatcher.getTmpDir());
// (1) Refresh Kerberos config.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] cgivre commented on pull request #2582: [Minor Update] Try and reduce the vm crash on fork
Posted by GitBox <gi...@apache.org>.
cgivre commented on PR #2582:
URL: https://github.com/apache/drill/pull/2582#issuecomment-1178146081
@jnturton @luocooong Is this PR still needed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] luocooong commented on pull request #2582: [Minor Update] Try and reduce the vm crash on fork
Posted by GitBox <gi...@apache.org>.
luocooong commented on PR #2582:
URL: https://github.com/apache/drill/pull/2582#issuecomment-1179550719
It is hoped that this pull request will be the last one to resolve the tricky unit test, as it has been released memory from the operating system and has split large tests (easy to hang or crash) and small tests.
In the future, we can label the large tests that are prone to failure as `EasyOutOfMemory`, the CI automatically starts the test with a new (cleanest area) JVM. If everything goes well, we may even move an existing Flaky test to that JVM.
Also, thank to @vdiravka for all his efforts.
Memory status before building and testing :
```
total used free shared buff/cache available
Mem: 6946 522 3342 9 3082 6107
Swap: 4095 0 4095
```
Memory status after buffer release (_drop_caches_) :
```
total used free shared buff/cache available
Mem: 6946 520 5265 9 1161 6109
Swap: 4095 0 4095
```
Memory status after building and testing :
```
total used free shared buff/cache available
Mem: 6946 508 1841 9 4596 6119
Swap: 4095 179 3916
```
Memory status before the specific testing (_drop_caches_) :
```
total used free shared buff/cache available
Mem: 6946 499 6126 9 320 6161
Swap: 4095 179 3916
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] jnturton merged pull request #2582: [Minor Update] Split category unit tests due to memory issues
Posted by GitBox <gi...@apache.org>.
jnturton merged PR #2582:
URL: https://github.com/apache/drill/pull/2582
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] jnturton commented on pull request #2582: [Minor Update] Split category unit tests due to memory issues
Posted by GitBox <gi...@apache.org>.
jnturton commented on PR #2582:
URL: https://github.com/apache/drill/pull/2582#issuecomment-1180152731
Argh, because there was only a single commit here I hit "rebase and merge" instead of squash and then got no chance to ensure a cleaned up commit message.
Thanks for the fixes!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org