You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@auron.apache.org by "XorSum (via GitHub)" <gi...@apache.org> on 2026/04/02 06:12:02 UTC
[PR] [AURON #2118] make native environment initialization retriable after failure [auron]
XorSum opened a new pull request, #2151:
URL: https://github.com/apache/auron/pull/2151
# Which issue does this PR close?
Closes #2118
# Rationale for this change
avoid initialize in static block, make native environment initialization retriable after failure
# What changes are included in this PR?
# Are there any user-facing changes?
# How was this patch tested?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AURON #2118] make native environment initialization retriable after failure [auron]
Posted by "cxzl25 (via GitHub)" <gi...@apache.org>.
cxzl25 merged PR #2151:
URL: https://github.com/apache/auron/pull/2151
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AURON #2118] make native environment initialization retriable after failure [auron]
Posted by "XorSum (via GitHub)" <gi...@apache.org>.
XorSum commented on code in PR #2151:
URL: https://github.com/apache/auron/pull/2151#discussion_r3446757967
##########
auron-core/src/main/java/org/apache/auron/jni/AuronCallNativeWrapper.java:
##########
@@ -90,13 +70,45 @@ public AuronCallNativeWrapper(
this.stageId = stageId;
this.taskId = taskId;
+ init();
LOG.warn("Start executing native plan");
this.nativeRuntimePtr = JniBridge.callNative(
nativeMemory,
AuronAdaptor.getInstance().getAuronConfiguration().get(AuronConfiguration.NATIVE_LOG_LEVEL),
this);
}
+ private static void init() {
+ if (!initialized) {
+ synchronized (AuronCallNativeWrapper.class) {
+ if (!initialized) {
+ // initialize native environment
+ LOG.info("Initializing native environment (batchSize="
+ + AuronAdaptor.getInstance().getAuronConfiguration().get(AuronConfiguration.BATCH_SIZE)
+ + ", "
+ + "memoryFraction="
+ + AuronAdaptor.getInstance().getAuronConfiguration().get(AuronConfiguration.MEMORY_FRACTION)
+ + ")");
+
+ // arrow configuration
+ System.setProperty("arrow.struct.conflict.policy", "CONFLICT_APPEND");
+
+ // preload JNI bridge classes
+ try {
+ Class.forName("org.apache.auron.jni.JniBridge");
+ } catch (ClassNotFoundException e) {
+ throw new RuntimeException("Cannot load JniBridge class", e);
+ }
+
+ AuronAdaptor.getInstance().loadAuronLib();
Review Comment:
Thanks! I have added a comment at the top of `init()` to point out the retry-safety contract.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AURON #2118] make native environment initialization retriable after failure [auron]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #2151: [AURON #2118] make native environment initialization retriable after failure
URL: https://github.com/apache/auron/pull/2151
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AURON #2118] make native environment initialization retriable after failure [auron]
Posted by "weiqingy (via GitHub)" <gi...@apache.org>.
weiqingy commented on code in PR #2151:
URL: https://github.com/apache/auron/pull/2151#discussion_r3416795879
##########
auron-core/src/main/java/org/apache/auron/jni/AuronCallNativeWrapper.java:
##########
@@ -90,13 +70,45 @@ public AuronCallNativeWrapper(
this.stageId = stageId;
this.taskId = taskId;
+ init();
LOG.warn("Start executing native plan");
this.nativeRuntimePtr = JniBridge.callNative(
nativeMemory,
AuronAdaptor.getInstance().getAuronConfiguration().get(AuronConfiguration.NATIVE_LOG_LEVEL),
this);
}
+ private static void init() {
+ if (!initialized) {
+ synchronized (AuronCallNativeWrapper.class) {
+ if (!initialized) {
+ // initialize native environment
+ LOG.info("Initializing native environment (batchSize="
+ + AuronAdaptor.getInstance().getAuronConfiguration().get(AuronConfiguration.BATCH_SIZE)
+ + ", "
+ + "memoryFraction="
+ + AuronAdaptor.getInstance().getAuronConfiguration().get(AuronConfiguration.MEMORY_FRACTION)
+ + ")");
+
+ // arrow configuration
+ System.setProperty("arrow.struct.conflict.policy", "CONFLICT_APPEND");
+
+ // preload JNI bridge classes
+ try {
+ Class.forName("org.apache.auron.jni.JniBridge");
+ } catch (ClassNotFoundException e) {
+ throw new RuntimeException("Cannot load JniBridge class", e);
+ }
+
+ AuronAdaptor.getInstance().loadAuronLib();
Review Comment:
The retry depends on a quiet invariant here: every side effect that runs before `initialized = true` — `System.setProperty`, `Class.forName`, `loadAuronLib`, `addShutdownHook` — has to be either idempotent or guaranteed not to have run on the path that threw. Today that holds, since `loadAuronLib()` throws out of `Files.copy` before `System.load`, so on the failure path the native lib never loads and the shutdown hook never registers — a retry re-runs cleanly. That coupling lives in a different file, so an edit to `loadAuronLib()` that added work after `System.load` wouldn't have any signal at the retry site that it just broke the retry. Is it worth making that invariant visible here — a one-line note, or an idempotency check — so the next edit to the load path can't quietly break it?
##########
auron-core/src/main/java/org/apache/auron/jni/AuronCallNativeWrapper.java:
##########
@@ -90,13 +70,45 @@ public AuronCallNativeWrapper(
this.stageId = stageId;
this.taskId = taskId;
+ init();
LOG.warn("Start executing native plan");
this.nativeRuntimePtr = JniBridge.callNative(
nativeMemory,
AuronAdaptor.getInstance().getAuronConfiguration().get(AuronConfiguration.NATIVE_LOG_LEVEL),
this);
}
+ private static void init() {
Review Comment:
The "How was this patch tested?" box is empty — and I think a faithful regression test is genuinely hard here, since it would need the real native lib plus a thread interrupted mid-`Files.copy` to reproduce the `ClosedByInterruptException`, which is racy and a process-global load. So this is a real question rather than a request: is there any lightweight check worth adding (even just asserting a second construction succeeds after init has run once), or is this effectively integration-only?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AURON #2118] make native environment initialization retriable after failure [auron]
Posted by "XorSum (via GitHub)" <gi...@apache.org>.
XorSum commented on code in PR #2151:
URL: https://github.com/apache/auron/pull/2151#discussion_r3446790545
##########
auron-core/src/main/java/org/apache/auron/jni/AuronCallNativeWrapper.java:
##########
@@ -90,13 +70,45 @@ public AuronCallNativeWrapper(
this.stageId = stageId;
this.taskId = taskId;
+ init();
LOG.warn("Start executing native plan");
this.nativeRuntimePtr = JniBridge.callNative(
nativeMemory,
AuronAdaptor.getInstance().getAuronConfiguration().get(AuronConfiguration.NATIVE_LOG_LEVEL),
this);
}
+ private static void init() {
Review Comment:
Agreed it's effectively integration-only — any unit test would need to either expose `init()`, reflect-reset the static flag, or stub out the static native shutdown hook + `callNative`. Making the production code absorb these concessions for a mere DCL simply isn't worth the trade-off.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AURON #2118] make native environment initialization retriable after failure [auron]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #2151:
URL: https://github.com/apache/auron/pull/2151#discussion_r3026121706
##########
auron-core/src/main/java/org/apache/auron/jni/AuronCallNativeWrapper.java:
##########
@@ -90,13 +70,45 @@ public AuronCallNativeWrapper(
this.stageId = stageId;
this.taskId = taskId;
+ init();
LOG.warn("Start executing native plan");
this.nativeRuntimePtr = JniBridge.callNative(
nativeMemory,
AuronAdaptor.getInstance().getAuronConfiguration().get(AuronConfiguration.NATIVE_LOG_LEVEL),
this);
}
+ private static void init() {
+ if (!initialized) {
+ synchronized (AuronCallNativeWrapper.class) {
+ if (!initialized) {
+ // initialize native environment
+ LOG.info("Initializing native environment (batchSize="
+ + AuronAdaptor.getInstance().getAuronConfiguration().get(AuronConfiguration.BATCH_SIZE)
+ + ", "
+ + "memoryFraction="
+ + AuronAdaptor.getInstance().getAuronConfiguration().get(AuronConfiguration.MEMORY_FRACTION)
+ + ")");
+
+ // arrow configuration
+ System.setProperty("arrow.struct.conflict.policy", "CONFLICT_APPEND");
+
+ // preload JNI bridge classes
+ try {
+ Class.forName("org.apache.auron.jni.JniBridge");
+ } catch (ClassNotFoundException e) {
+ throw new RuntimeException("Cannot load JniBridge class", e);
+ }
+
+ AuronAdaptor.getInstance().loadAuronLib();
+ Runtime.getRuntime().addShutdownHook(new Thread(JniBridge::onExit));
+
+ initialized = true;
Review Comment:
Because initialization is now retried after failures, `AuronAdaptor.getInstance().loadAuronLib()` may be invoked multiple times in the same JVM. In the Spark adaptor, each call extracts to a fresh `createTempFile(...).deleteOnExit()`, so repeated failures can accumulate temp files until executor exit. Consider making `loadAuronLib()` idempotent / cached (or cleaning up temp files on failure) to avoid disk bloat on flaky/interrupt-driven init attempts.
```suggestion
try {
// initialize native environment
LOG.info("Initializing native environment (batchSize="
+ AuronAdaptor.getInstance().getAuronConfiguration().get(AuronConfiguration.BATCH_SIZE)
+ ", "
+ "memoryFraction="
+ AuronAdaptor.getInstance().getAuronConfiguration().get(AuronConfiguration.MEMORY_FRACTION)
+ ")");
// arrow configuration
System.setProperty("arrow.struct.conflict.policy", "CONFLICT_APPEND");
// preload JNI bridge classes
try {
Class.forName("org.apache.auron.jni.JniBridge");
} catch (ClassNotFoundException e) {
throw new RuntimeException("Cannot load JniBridge class", e);
}
AuronAdaptor.getInstance().loadAuronLib();
Runtime.getRuntime().addShutdownHook(new Thread(JniBridge::onExit));
} finally {
// Mark initialization as attempted to avoid repeated native library loading
initialized = true;
}
```
##########
auron-core/src/main/java/org/apache/auron/jni/AuronCallNativeWrapper.java:
##########
@@ -90,13 +70,45 @@ public AuronCallNativeWrapper(
this.stageId = stageId;
this.taskId = taskId;
+ init();
LOG.warn("Start executing native plan");
this.nativeRuntimePtr = JniBridge.callNative(
nativeMemory,
AuronAdaptor.getInstance().getAuronConfiguration().get(AuronConfiguration.NATIVE_LOG_LEVEL),
this);
}
+ private static void init() {
+ if (!initialized) {
+ synchronized (AuronCallNativeWrapper.class) {
+ if (!initialized) {
+ // initialize native environment
+ LOG.info("Initializing native environment (batchSize="
+ + AuronAdaptor.getInstance().getAuronConfiguration().get(AuronConfiguration.BATCH_SIZE)
+ + ", "
+ + "memoryFraction="
+ + AuronAdaptor.getInstance().getAuronConfiguration().get(AuronConfiguration.MEMORY_FRACTION)
Review Comment:
`init()` can run while a Spark task thread is already interrupted/cancelled (the production issue shows `ClosedByInterruptException`). Even though initialization is now retriable, attempting `Files.copy/System.load` from an interrupted task is still likely to fail and generate noisy errors. Consider short-circuiting initialization when the current task is not running (e.g., `!AuronAdaptor.getInstance().isTaskRunning()` or `Thread.currentThread().isInterrupted()`), and fail fast so a later non-cancelled task can perform the one-time init successfully.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AURON #2118] make native environment initialization retriable after failure [auron]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #2151:
URL: https://github.com/apache/auron/pull/2151#discussion_r3450862683
##########
auron-core/src/main/java/org/apache/auron/jni/AuronCallNativeWrapper.java:
##########
@@ -90,13 +70,49 @@ public AuronCallNativeWrapper(
this.stageId = stageId;
this.taskId = taskId;
+ init();
LOG.warn("Start executing native plan");
this.nativeRuntimePtr = JniBridge.callNative(
nativeMemory,
AuronAdaptor.getInstance().getAuronConfiguration().get(AuronConfiguration.NATIVE_LOG_LEVEL),
this);
}
+ private static void init() {
+ if (!initialized) {
+ synchronized (AuronCallNativeWrapper.class) {
+ if (!initialized) {
+ // Retry-safe: on failure `initialized` stays false so the next call retries.
+ // Every step below must be idempotent, or must not run on the failure path.
+ // In particular, loadAuronLib() must throw before System.load — a retried
+ // System.load would load the native library twice.
+ // initialize native environment
+ LOG.info("Initializing native environment (batchSize="
+ + AuronAdaptor.getInstance().getAuronConfiguration().get(AuronConfiguration.BATCH_SIZE)
+ + ", "
+ + "memoryFraction="
+ + AuronAdaptor.getInstance().getAuronConfiguration().get(AuronConfiguration.MEMORY_FRACTION)
+ + ")");
+
+ // arrow configuration
+ System.setProperty("arrow.struct.conflict.policy", "CONFLICT_APPEND");
+
+ // preload JNI bridge classes
+ try {
+ Class.forName("org.apache.auron.jni.JniBridge");
+ } catch (ClassNotFoundException e) {
+ throw new RuntimeException("Cannot load JniBridge class", e);
+ }
+
+ AuronAdaptor.getInstance().loadAuronLib();
+ Runtime.getRuntime().addShutdownHook(new Thread(JniBridge::onExit));
+
+ initialized = true;
Review Comment:
`addShutdownHook(...)` can throw (notably `IllegalStateException` if shutdown is already in progress). In that case, `loadAuronLib()` has already run but `initialized` remains `false`, so a subsequent retry will call `loadAuronLib()` again and can hit the exact double-load scenario the comment warns about. Consider ensuring `initialized` becomes `true` once the native library is successfully loaded (even if shutdown hook registration fails), and/or guard the hook registration separately (e.g., best-effort try/catch or a second “hookRegistered” flag) so retries don’t re-run `loadAuronLib()` after a successful load.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [AURON #2118] make native environment initialization retriable after failure [auron]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #2151:
URL: https://github.com/apache/auron/pull/2151#issuecomment-5923141638
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org