You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@daffodil.apache.org by sl...@apache.org on 2023/05/08 19:16:02 UTC
[daffodil] branch main updated: Remove use of the IntegrationTest sbt configuration
This is an automated email from the ASF dual-hosted git repository.
slawrence pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/daffodil.git
The following commit(s) were added to refs/heads/main by this push:
new d524759ea Remove use of the IntegrationTest sbt configuration
d524759ea is described below
commit d524759ea7a7ec8e5eacee1614d3877df3d756ea
Author: Steve Lawrence <sl...@apache.org>
AuthorDate: Mon May 8 13:58:40 2023 -0400
Remove use of the IntegrationTest sbt configuration
Upcoming versions of sbt will deprecate the IntegrationTest
configuration. Using their suggested migration path, this moves the
existing integration tests to a new daffodil-test-integration project.
Because of the extra time needed to stage the CLI and run the tests,
this project is not aggregeated so must be run manually.
Integration test are now run with the command:
sbt daffodil-test-integration/test
DAFFODIL-2811
---
.github/workflows/main.yml | 4 +-
DEVELOP.md | 14 ++++---
README.md | 6 +--
build.sbt | 43 +++++++++-------------
daffodil-cli/build.sbt | 17 ---------
.../org/apache/daffodil/cliTest/TestCLIUdfs.scala | 0
project/Dependencies.scala | 8 ++--
7 files changed, 34 insertions(+), 58 deletions(-)
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
index 9d5ef9904..b07ec3781 100644
--- a/.github/workflows/main.yml
+++ b/.github/workflows/main.yml
@@ -141,7 +141,7 @@ jobs:
############################################################
- name: Compile
- run: $SBT compile Test/compile IntegrationTest/compile
+ run: $SBT compile Test/compile daffodil-test-integration/Test/compile
- name: Build Documentation
run: $SBT unidoc
@@ -161,7 +161,7 @@ jobs:
run: $SBT test
- name: Run Integration Tests
- run: $SBT IntegrationTest/test
+ run: $SBT daffodil-test-integration/test
- name: Run Modified Example Files Check
run: git diff --color --exit-code
diff --git a/DEVELOP.md b/DEVELOP.md
index 40a949f38..37b5711af 100644
--- a/DEVELOP.md
+++ b/DEVELOP.md
@@ -414,7 +414,7 @@ command line:
```text
sbt "daffodil-test/testOnly org.apache.daffodil.section05.facets.TestNulChars"
-sbt "daffodil-cli/IntegrationTest/testOnly org.apache.daffodil.parsing.TestCLIparsing -- --tests=test_XXX_CLI_Parsing_SimpleParse_sax"
+sbt "daffodil-test-integration/testOnly org.apache.daffodil.cliTest.TestCLIUdfs -- --tests=test_noUdfsLoaded_regular_schema"
```
TDML tests are helpful for Daffodil developers because they allow a
@@ -443,14 +443,16 @@ generated C code works successfully:
daffodil test -I daffodilC nums.tdml
```
-Daffodil also has integration tests in the daffodil-cli module which
-test Daffodil's command line interface. If a developer changes any
+Daffodil also has command line interface tests in the daffodil-cli and
+daffodil-test-integration modules. Slower CLI tests that require
+forking should go in the latter module. If a developer changes any
part of Daffodil's command line interface, the developer should test
-that change in an integration test too. You can run integration tests
-with sbt using this command:
+that change in one of these modules. You can run all CLI tests
+using these sbt commands:
```text
-sbt IntegrationTest/test
+sbt daffodil-cli/test
+sbt daffodil-test-integration/test
```
Daffodil also has a [GitHub Actions][] continuous integration workflow
diff --git a/README.md b/README.md
index a84905bd1..7ec007e78 100644
--- a/README.md
+++ b/README.md
@@ -65,9 +65,9 @@ Run unit tests:
sbt test
-Run command line interface tests:
+Run slower integration tests:
- sbt IntegrationTest/test
+ sbt daffodil-test-integration/test
### Command Line Interface
@@ -88,7 +88,7 @@ if any unapproved licenses are found):
Run [sbt-scoverage] (report in `target/scala-ver/scoverage-report/`):
- sbt clean coverage test IntegrationTest/test
+ sbt clean coverage test daffodil-test-integration/test
sbt coverageAggregate
## Getting Help
diff --git a/build.sbt b/build.sbt
index 5b0f7c36f..61958fa32 100644
--- a/build.sbt
+++ b/build.sbt
@@ -36,7 +36,6 @@ lazy val genCExamples = taskKey[Seq[File]]("Generate C example files")
lazy val daffodil = project
.in(file("."))
- .configs(IntegrationTest)
.enablePlugins(JavaUnidocPlugin, ScalaUnidocPlugin)
.aggregate(
cli,
@@ -55,8 +54,9 @@ lazy val daffodil = project
slf4jLogger,
tdmlLib,
tdmlProc,
- test,
+ testDaf,
testIBM1,
+ // testIntegration, // integration tests must be run manually
testStdLayout,
tutorials,
udf,
@@ -64,32 +64,26 @@ lazy val daffodil = project
.settings(commonSettings, nopublish, ratSettings, unidocSettings, genCExamplesSettings)
lazy val macroLib = Project("daffodil-macro-lib", file("daffodil-macro-lib"))
- .configs(IntegrationTest)
.settings(commonSettings, nopublish)
.settings(libraryDependencies += "org.scala-lang" % "scala-reflect" % scalaVersion.value)
.disablePlugins(OsgiCheckPlugin)
lazy val propgen = Project("daffodil-propgen", file("daffodil-propgen"))
- .configs(IntegrationTest)
.settings(commonSettings, nopublish)
lazy val slf4jLogger = Project("daffodil-slf4j-logger", file("daffodil-slf4j-logger"))
- .configs(IntegrationTest)
.settings(commonSettings)
.settings(libraryDependencies ++= Dependencies.slf4jAPI)
lazy val lib = Project("daffodil-lib", file("daffodil-lib"))
- .configs(IntegrationTest)
.dependsOn(macroLib % "compile-internal, test-internal", slf4jLogger % "test")
.settings(commonSettings, libManagedSettings, usesMacros)
lazy val io = Project("daffodil-io", file("daffodil-io"))
- .configs(IntegrationTest)
.dependsOn(lib, macroLib % "compile-internal, test-internal", slf4jLogger % "test")
.settings(commonSettings, usesMacros)
lazy val runtime1 = Project("daffodil-runtime1", file("daffodil-runtime1"))
- .configs(IntegrationTest)
.dependsOn(
io,
lib % "test->test",
@@ -101,7 +95,6 @@ lazy val runtime1 = Project("daffodil-runtime1", file("daffodil-runtime1"))
lazy val runtime1Unparser =
Project("daffodil-runtime1-unparser", file("daffodil-runtime1-unparser"))
- .configs(IntegrationTest)
.dependsOn(
runtime1,
lib % "test->test",
@@ -112,13 +105,11 @@ lazy val runtime1Unparser =
.settings(commonSettings)
lazy val runtime1Layers = Project("daffodil-runtime1-layers", file("daffodil-runtime1-layers"))
- .configs(IntegrationTest)
.dependsOn(runtime1, lib % "test->test", slf4jLogger % "test")
.settings(commonSettings)
val codeGenCLib = Library("libruntime.a")
lazy val codeGenC = Project("daffodil-codegen-c", file("daffodil-codegen-c"))
- .configs(IntegrationTest)
.enablePlugins(CcPlugin)
.dependsOn(core, core % "test->test", slf4jLogger % "test")
.settings(commonSettings)
@@ -141,7 +132,6 @@ lazy val codeGenC = Project("daffodil-codegen-c", file("daffodil-codegen-c"))
)
lazy val core = Project("daffodil-core", file("daffodil-core"))
- .configs(IntegrationTest)
.dependsOn(
runtime1Unparser,
udf,
@@ -153,34 +143,28 @@ lazy val core = Project("daffodil-core", file("daffodil-core"))
.settings(commonSettings)
lazy val japi = Project("daffodil-japi", file("daffodil-japi"))
- .configs(IntegrationTest)
.dependsOn(core, slf4jLogger % "test")
.settings(commonSettings)
lazy val sapi = Project("daffodil-sapi", file("daffodil-sapi"))
- .configs(IntegrationTest)
.dependsOn(core, slf4jLogger % "test")
.settings(commonSettings)
lazy val tdmlLib = Project("daffodil-tdml-lib", file("daffodil-tdml-lib"))
- .configs(IntegrationTest)
.dependsOn(macroLib % "compile-internal", lib, io, io % "test->test", slf4jLogger % "test")
.settings(commonSettings)
lazy val tdmlProc = Project("daffodil-tdml-processor", file("daffodil-tdml-processor"))
- .configs(IntegrationTest)
.dependsOn(tdmlLib, codeGenC, core, slf4jLogger)
.settings(commonSettings)
lazy val cli = Project("daffodil-cli", file("daffodil-cli"))
- .configs(IntegrationTest)
.dependsOn(
tdmlProc,
codeGenC,
sapi,
japi,
schematron % Runtime,
- udf % "it->test",
slf4jLogger,
) // causes codegen-c/sapi/japi to be pulled into the helper zip/tar
.settings(commonSettings, nopublish)
@@ -188,7 +172,6 @@ lazy val cli = Project("daffodil-cli", file("daffodil-cli"))
.settings(libraryDependencies ++= Dependencies.exi)
lazy val udf = Project("daffodil-udf", file("daffodil-udf"))
- .configs(IntegrationTest)
.dependsOn(slf4jLogger % "test")
.settings(commonSettings)
@@ -196,10 +179,8 @@ lazy val schematron = Project("daffodil-schematron", file("daffodil-schematron")
.dependsOn(lib, sapi % Test, slf4jLogger % "test")
.settings(commonSettings)
.settings(libraryDependencies ++= Dependencies.schematron)
- .configs(IntegrationTest)
-lazy val test = Project("daffodil-test", file("daffodil-test"))
- .configs(IntegrationTest)
+lazy val testDaf = Project("daffodil-test", file("daffodil-test"))
.dependsOn(tdmlProc % "test", codeGenC % "test->test", udf % "test->test")
.settings(commonSettings, nopublish)
//
@@ -209,7 +190,6 @@ lazy val test = Project("daffodil-test", file("daffodil-test"))
//.settings(IBMDFDLCrossTesterPlugin.settings)
lazy val testIBM1 = Project("daffodil-test-ibm1", file("daffodil-test-ibm1"))
- .configs(IntegrationTest)
.dependsOn(tdmlProc % "test")
.settings(commonSettings, nopublish)
//
@@ -218,13 +198,25 @@ lazy val testIBM1 = Project("daffodil-test-ibm1", file("daffodil-test-ibm1"))
//
//.settings(IBMDFDLCrossTesterPlugin.settings)
+lazy val testIntegration =
+ Project("daffodil-test-integration", file("daffodil-test-integration"))
+ .dependsOn(cli % "test->test", udf % "test->test")
+ .settings(commonSettings, nopublish)
+ .settings(
+ // CLI integration tests fork a new process which requires extra memory, so these should
+ // only be run sequentially. We also need to stage the CLI script if any of the test
+ // tasks are run
+ Test / parallelExecution := false,
+ Test / test := (Test / test).dependsOn(cli / Compile / stage).value,
+ Test / testOnly := (Test / testOnly).dependsOn(cli / Compile / stage).evaluated,
+ Test / testQuick := (Test / testQuick).dependsOn(cli / Compile / stage).evaluated,
+ )
+
lazy val tutorials = Project("daffodil-tutorials", file("tutorials"))
- .configs(IntegrationTest)
.dependsOn(tdmlProc % "test")
.settings(commonSettings, nopublish)
lazy val testStdLayout = Project("daffodil-test-stdLayout", file("test-stdLayout"))
- .configs(IntegrationTest)
.dependsOn(tdmlProc % "test")
.settings(commonSettings, nopublish)
@@ -256,7 +248,6 @@ lazy val commonSettings = Seq(
sourceManaged := baseDirectory.value / "src_managed",
resourceManaged := baseDirectory.value / "resource_managed",
libraryDependencies ++= Dependencies.common,
- // IntegrationTest / parallelExecution := false, // allow test in parallel
testOptions += Tests.Argument(TestFrameworks.JUnit, "-q", "--verbosity=1"),
) ++ Defaults.itSettings
diff --git a/daffodil-cli/build.sbt b/daffodil-cli/build.sbt
index cf9eb8361..23820583b 100644
--- a/daffodil-cli/build.sbt
+++ b/daffodil-cli/build.sbt
@@ -23,23 +23,6 @@ enablePlugins(JavaAppPackaging)
enablePlugins(RpmPlugin)
enablePlugins(WindowsPlugin)
-// integration tests require forking, which require a lot of extra memory, so
-// these should only be run sequentially
-IntegrationTest / parallelExecution := false
-
-// need 'sbt stage' to build the CLI for cli integration tests
-(IntegrationTest / test) := (IntegrationTest / test).dependsOn(Compile / stage).value
-(IntegrationTest / testOnly) := (IntegrationTest / testOnly)
- .dependsOn(Compile / stage)
- .evaluated
-(IntegrationTest / testQuick) := (IntegrationTest / testQuick)
- .dependsOn(Compile / stage)
- .evaluated
-
-// Add the classes in src/test/ to the IntegrationTest classpath
-// so we can share CLI utliities in Util.scala
-(IntegrationTest / dependencyClasspath) ++= (Test / exportedProducts).value
-
executableScriptName := "daffodil"
Universal / packageName := "apache-daffodil-" + version.value + "-bin" //tarball name
diff --git a/daffodil-cli/src/it/scala/org/apache/daffodil/cliTest/TestCLIUdfs.scala b/daffodil-test-integration/src/test/scala/org/apache/daffodil/cliTest/TestCLIUdfs.scala
similarity index 100%
rename from daffodil-cli/src/it/scala/org/apache/daffodil/cliTest/TestCLIUdfs.scala
rename to daffodil-test-integration/src/test/scala/org/apache/daffodil/cliTest/TestCLIUdfs.scala
diff --git a/project/Dependencies.scala b/project/Dependencies.scala
index badfbc2a0..87509e472 100644
--- a/project/Dependencies.scala
+++ b/project/Dependencies.scala
@@ -47,13 +47,13 @@ object Dependencies {
"org.fusesource.jansi" % "jansi" % "2.4.0",
"org.jline" % "jline" % "3.22.0",
"org.rogach" %% "scallop" % "4.1.0",
- "net.sf.expectit" % "expectit-core" % "0.9.0" % "it,test",
+ "net.sf.expectit" % "expectit-core" % "0.9.0" % "test",
)
lazy val test = Seq(
- "junit" % "junit" % "4.13.2" % "it,test",
- "com.github.sbt" % "junit-interface" % "0.13.3" % "it,test",
- "org.scalacheck" %% "scalacheck" % "1.17.0" % "it,test",
+ "junit" % "junit" % "4.13.2" % "test",
+ "com.github.sbt" % "junit-interface" % "0.13.3" % "test",
+ "org.scalacheck" %% "scalacheck" % "1.17.0" % "test",
)
lazy val schematron = Seq(