You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@daffodil.apache.org by "olabusayoT (via GitHub)" <gi...@apache.org> on 2026/07/29 14:24:07 UTC
[PR] Inherit sbt's -Xss into saver's forked JVM by default [daffodil-sbt]
olabusayoT opened a new pull request, #210:
URL: https://github.com/apache/daffodil-sbt/pull/210
Schema compilation and saved parser serialization can involve structures that overflow the forked process's default thread stack. Since users already tune sbt's own -Xss (e.g. via .sbtopts) to deal with similar recursion during compilation, reuse that value for the forked process by default instead of requiring it to be configured again. A project can still override it by appending its own -Xss to packageDaffodilBin/javaOptions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Inherit sbt's -Xss into saver's forked JVM by default [daffodil-sbt]
Posted by "olabusayoT (via GitHub)" <gi...@apache.org>.
olabusayoT commented on code in PR #210:
URL: https://github.com/apache/daffodil-sbt/pull/210#discussion_r3807829274
##########
src/main/scala/org/apache/daffodil/DaffodilPlugin.scala:
##########
@@ -518,14 +518,30 @@ object DaffodilPlugin extends AutoPlugin {
/**
* JVM options used for the forked process to build saved parsers
*
- * Defaults to just setting various system properties to configure loggers that might be
- * used by different daffodil versions
+ * Defaults to setting various system properties to configure loggers that might be used by
+ * different daffodil versions, plus the -Xss this sbt JVM was started with, if any. Since
+ * sbt users often already tune sbt's own stack size (e.g. via .sbtopts or .jvmopts),
+ * it's convenient to reuse that value for the forked process rather than requiring it to
+ * be configured again. If a project wants a different stack size just for this forked
+ * process, it can append its own -Xss, e.g.:
+ *
+ * packageDaffodilBin / javaOptions += "-Xss64m"
+ *
+ * Note this relies on the JVM using the last of multiple -Xss arguments it is given, so an
+ * appended -Xss here correctly takes precedence over the inherited one below.
*/
- packageDaffodilBin / javaOptions := Seq(
- s"-Dorg.slf4j.simpleLogger.logFile=System.out",
- s"-Dorg.slf4j.simpleLogger.defaultLogLevel=${(packageDaffodilBin / logLevel).value}",
- s"-Dorg.apache.logging.log4j.level=${(packageDaffodilBin / logLevel).value}"
- ),
+ packageDaffodilBin / javaOptions := {
+ val inheritedXss =
+ java.lang.management.ManagementFactory.getRuntimeMXBean.getInputArguments
+ .toArray(Array.empty[String])
+ .find(_.startsWith("-Xss"))
+
+ Seq(
+ s"-Dorg.slf4j.simpleLogger.logFile=System.out",
+ s"-Dorg.slf4j.simpleLogger.defaultLogLevel=${(packageDaffodilBin / logLevel).value}",
+ s"-Dorg.apache.logging.log4j.level=${(packageDaffodilBin / logLevel).value}"
+ ) ++ inheritedXss
Review Comment:
Hmm, how does a schema project using the daffodil sbt plugin pass it's sbtopts to the plugins forked process? Would seeting `packageDaffodilBin / javaOptions` in the project work and make sense when the project is not the one forking anything?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Inherit sbt's -Xss into saver's forked JVM by default [daffodil-sbt]
Posted by "olabusayoT (via GitHub)" <gi...@apache.org>.
olabusayoT commented on PR #210:
URL: https://github.com/apache/daffodil-sbt/pull/210#issuecomment-5362457747
Setting packageDaffodilBin / javaOptions in the schema project solved this problem
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Inherit sbt's -Xss into saver's forked JVM by default [daffodil-sbt]
Posted by "olabusayoT (via GitHub)" <gi...@apache.org>.
olabusayoT commented on code in PR #210:
URL: https://github.com/apache/daffodil-sbt/pull/210#discussion_r3807829274
##########
src/main/scala/org/apache/daffodil/DaffodilPlugin.scala:
##########
@@ -518,14 +518,30 @@ object DaffodilPlugin extends AutoPlugin {
/**
* JVM options used for the forked process to build saved parsers
*
- * Defaults to just setting various system properties to configure loggers that might be
- * used by different daffodil versions
+ * Defaults to setting various system properties to configure loggers that might be used by
+ * different daffodil versions, plus the -Xss this sbt JVM was started with, if any. Since
+ * sbt users often already tune sbt's own stack size (e.g. via .sbtopts or .jvmopts),
+ * it's convenient to reuse that value for the forked process rather than requiring it to
+ * be configured again. If a project wants a different stack size just for this forked
+ * process, it can append its own -Xss, e.g.:
+ *
+ * packageDaffodilBin / javaOptions += "-Xss64m"
+ *
+ * Note this relies on the JVM using the last of multiple -Xss arguments it is given, so an
+ * appended -Xss here correctly takes precedence over the inherited one below.
*/
- packageDaffodilBin / javaOptions := Seq(
- s"-Dorg.slf4j.simpleLogger.logFile=System.out",
- s"-Dorg.slf4j.simpleLogger.defaultLogLevel=${(packageDaffodilBin / logLevel).value}",
- s"-Dorg.apache.logging.log4j.level=${(packageDaffodilBin / logLevel).value}"
- ),
+ packageDaffodilBin / javaOptions := {
+ val inheritedXss =
+ java.lang.management.ManagementFactory.getRuntimeMXBean.getInputArguments
+ .toArray(Array.empty[String])
+ .find(_.startsWith("-Xss"))
+
+ Seq(
+ s"-Dorg.slf4j.simpleLogger.logFile=System.out",
+ s"-Dorg.slf4j.simpleLogger.defaultLogLevel=${(packageDaffodilBin / logLevel).value}",
+ s"-Dorg.apache.logging.log4j.level=${(packageDaffodilBin / logLevel).value}"
+ ) ++ inheritedXss
Review Comment:
Hmm, how does a schema project using the daffodil sbt plugin pass it's sbtopts to the plugins forked process? Would seeting `packageDaffodilBin / javaOptions` in the project work and make sense when the project is not the one forking anything?
I agree it's not standard, I just wasn't sure of another way to approach this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Inherit sbt's -Xss into saver's forked JVM by default [daffodil-sbt]
Posted by "olabusayoT (via GitHub)" <gi...@apache.org>.
olabusayoT closed pull request #210: Inherit sbt's -Xss into saver's forked JVM by default
URL: https://github.com/apache/daffodil-sbt/pull/210
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Inherit sbt's -Xss into saver's forked JVM by default [daffodil-sbt]
Posted by "stevedlawrence (via GitHub)" <gi...@apache.org>.
stevedlawrence commented on code in PR #210:
URL: https://github.com/apache/daffodil-sbt/pull/210#discussion_r3751241055
##########
src/main/scala/org/apache/daffodil/DaffodilPlugin.scala:
##########
@@ -518,14 +518,30 @@ object DaffodilPlugin extends AutoPlugin {
/**
* JVM options used for the forked process to build saved parsers
*
- * Defaults to just setting various system properties to configure loggers that might be
- * used by different daffodil versions
+ * Defaults to setting various system properties to configure loggers that might be used by
+ * different daffodil versions, plus the -Xss this sbt JVM was started with, if any. Since
+ * sbt users often already tune sbt's own stack size (e.g. via .sbtopts or .jvmopts),
+ * it's convenient to reuse that value for the forked process rather than requiring it to
+ * be configured again. If a project wants a different stack size just for this forked
+ * process, it can append its own -Xss, e.g.:
+ *
+ * packageDaffodilBin / javaOptions += "-Xss64m"
+ *
+ * Note this relies on the JVM using the last of multiple -Xss arguments it is given, so an
+ * appended -Xss here correctly takes precedence over the inherited one below.
*/
- packageDaffodilBin / javaOptions := Seq(
- s"-Dorg.slf4j.simpleLogger.logFile=System.out",
- s"-Dorg.slf4j.simpleLogger.defaultLogLevel=${(packageDaffodilBin / logLevel).value}",
- s"-Dorg.apache.logging.log4j.level=${(packageDaffodilBin / logLevel).value}"
- ),
+ packageDaffodilBin / javaOptions := {
+ val inheritedXss =
+ java.lang.management.ManagementFactory.getRuntimeMXBean.getInputArguments
+ .toArray(Array.empty[String])
+ .find(_.startsWith("-Xss"))
Review Comment:
Should this be `findLast`, since I think JVM's if an option is specified multiple times the JVM uses the last one?
##########
src/main/scala/org/apache/daffodil/DaffodilPlugin.scala:
##########
@@ -518,14 +518,30 @@ object DaffodilPlugin extends AutoPlugin {
/**
* JVM options used for the forked process to build saved parsers
*
- * Defaults to just setting various system properties to configure loggers that might be
- * used by different daffodil versions
+ * Defaults to setting various system properties to configure loggers that might be used by
+ * different daffodil versions, plus the -Xss this sbt JVM was started with, if any. Since
+ * sbt users often already tune sbt's own stack size (e.g. via .sbtopts or .jvmopts),
+ * it's convenient to reuse that value for the forked process rather than requiring it to
+ * be configured again. If a project wants a different stack size just for this forked
+ * process, it can append its own -Xss, e.g.:
+ *
+ * packageDaffodilBin / javaOptions += "-Xss64m"
+ *
+ * Note this relies on the JVM using the last of multiple -Xss arguments it is given, so an
+ * appended -Xss here correctly takes precedence over the inherited one below.
*/
- packageDaffodilBin / javaOptions := Seq(
- s"-Dorg.slf4j.simpleLogger.logFile=System.out",
- s"-Dorg.slf4j.simpleLogger.defaultLogLevel=${(packageDaffodilBin / logLevel).value}",
- s"-Dorg.apache.logging.log4j.level=${(packageDaffodilBin / logLevel).value}"
- ),
+ packageDaffodilBin / javaOptions := {
+ val inheritedXss =
+ java.lang.management.ManagementFactory.getRuntimeMXBean.getInputArguments
+ .toArray(Array.empty[String])
+ .find(_.startsWith("-Xss"))
+
+ Seq(
+ s"-Dorg.slf4j.simpleLogger.logFile=System.out",
+ s"-Dorg.slf4j.simpleLogger.defaultLogLevel=${(packageDaffodilBin / logLevel).value}",
+ s"-Dorg.apache.logging.log4j.level=${(packageDaffodilBin / logLevel).value}"
+ ) ++ inheritedXss
Review Comment:
I'm hesitant about this change.
I would think in the majority of cases a large stack size is really only needed for schema compilation and so it's better that to only apply Xss to `packageDaffodilBin / javaOptions`. This keeps the sbt stack small and also sort of self documents that schema compilation needs a certain stack size. Similarly for options like -Xmx which I think some schemas also need for compilation.
I think this is also isn't really consistent with how SBT works, so it's kind of a non-standard and surprising behavior. In testing I've done, it looks like forking tasks (e.g tests/run) fork using default JVM options, it doesn't assume that options provided to SBT also make sense for other tasks. In the packageDaffodilBin case, thatmight be true, but it isn't necessarily true, and it feels better to be explicit about it by setting `packageDaffodilBin / javaOptions`.
I don't feel super strongly about it, so if this really makes things easier I won't block it.
Note that if this is the convention we do decide to use, we should probably also copy things like -Xmx and maybe some others, since I imagine most cases we need more than just a bigger stack size for
compiling large schemas.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org