You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@daffodil.apache.org by stevedlawrence <gi...@git.apache.org> on 2017/11/30 19:05:01 UTC

[GitHub] incubator-daffodil pull request #14: Update sbt to 1.x, simplifing build con...

GitHub user stevedlawrence opened a pull request:

    https://github.com/apache/incubator-daffodil/pull/14

    Update sbt to 1.x, simplifing build config and replacing deprecated s…

    …yntax
    
    - Remove build.scala. Everything is now in the global build.sbt or
      build.sbt specific to a subproject
    - Fix macrolib so it no longer needs to be published--it now only has a
      compile time dependency and mappings are added to copy the generated
      macros to the correct location
    - Remove the 'sbt cli' tasks. Instead, make the cli tests use the exact
      same directory structure as tests (meaning scala-debug can now work), but
      do not aggregate cli in the root project. This means that to run any
      cli commands, one must specify the daffodil-cli subproject, e.g. 'sbt
      daffodil-cli/test' (also requires modificaations to the Travis CI
      configuration)
    - Move scala-new tests to scala and remove he 'sbt new' commands. The
      'new' workflow added extra overhead and is no longer needed
    - Remove sbt pgp plugin. That should be configured locally in ~/.sbt if
      a user needs to published signed releases
    
    DAFFODIL-1857

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/stevedlawrence/incubator-daffodil daffodil-1857-sbt-1x

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-daffodil/pull/14.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #14
    
----
commit 19f6cc35b124cba970c77263a9f41b6bd83562d1
Author: Steve Lawrence <sl...@apache.org>
Date:   2017-11-30T18:30:52Z

    Update sbt to 1.x, simplifing build config and replacing deprecated syntax
    
    - Remove build.scala. Everything is now in the global build.sbt or
      build.sbt specific to a subproject
    - Fix macrolib so it no longer needs to be published--it now only has a
      compile time dependency and mappings are added to copy the generated
      macros to the correct location
    - Remove the 'sbt cli' tasks. Instead, make the cli tests use the exact
      same directory structure as tests (meaning scala-debug can now work), but
      do not aggregate cli in the root project. This means that to run any
      cli commands, one must specify the daffodil-cli subproject, e.g. 'sbt
      daffodil-cli/test' (also requires modificaations to the Travis CI
      configuration)
    - Move scala-new tests to scala and remove he 'sbt new' commands. The
      'new' workflow added extra overhead and is no longer needed
    - Remove sbt pgp plugin. That should be configured locally in ~/.sbt if
      a user needs to published signed releases
    
    DAFFODIL-1857

----


---

[GitHub] incubator-daffodil pull request #14: Update sbt to 1.x, simplifing build con...

Posted by stevedlawrence <gi...@git.apache.org>.
Github user stevedlawrence commented on a diff in the pull request:

    https://github.com/apache/incubator-daffodil/pull/14#discussion_r154177024
  
    --- Diff: build.sbt ---
    @@ -1,77 +1,193 @@
    -
    -
    -name := "daffodil"
    -
    -organization in ThisBuild := "edu.illinois.ncsa"
    -
    -version in ThisBuild := "2.1.0-SNAPSHOT"
    -
    -scalaVersion in ThisBuild := "2.11.8"
    -
    -// incOptions := incOptions.value.withNameHashing(true) // 2.11 experimental incremental compilation improvements (perhaps not working right?)
    -
    -scalacOptions in ThisBuild ++= Seq("-unchecked", "-deprecation", "-Yinline-warnings", "-Xxml:-coalescing", "-language:experimental.macros", "-Ybackend:GenBCode", "-Yopt-warnings", 
    - "-Ywarn-inaccessible", "-Ywarn-unused-import", "-Ywarn-unused", "-Ywarn-infer-any", "-Ywarn-nullary-override", "-Ydead-code", "-Yopt:inline-global", "-Yinline" , "-Xfatal-warnings")
    -
    -// parallelExecution in ThisBuild := false
    -
    -// concurrentRestrictions in Global += Tags.limit(Tags.Test, 1)
    -
    -logBuffered in ThisBuild := true
    -
    -testOptions in ThisBuild += Tests.Argument(TestFrameworks.JUnit, "-v")
    -
    -transitiveClassifiers := Seq("sources", "javadoc")
    -
    -libraryDependencies in ThisBuild := Seq(
    -  "org.scala-lang.modules" %% "scala-xml" % "1.0.6",
    -  "org.scala-lang.modules" %% "scala-parser-combinators" % "1.0.4",
    -  "junit" % "junit" % "4.11" % "test",
    -  "com.novocode" % "junit-interface" % "0.11" % "test",
    -  "net.sf.expectit" % "expectit-core" % "0.8.1" % "test",
    -  "org.jdom" % "jdom2" % "2.0.6",
    -  "com.ibm.icu" % "icu4j" % "51.1", // new versions avail. 58.1 requires code changes
    -  "xerces" % "xercesImpl" % "2.10.0",
    -  "com.fasterxml.woodstox" % "woodstox-core" % "5.0.3",
    -  "com.fasterxml.jackson.core" % "jackson-core" % "2.8.8",
    -  "xml-resolver" % "xml-resolver" % "1.2",
    -  "jline" % "jline" % "2.12.1", // newer versions avail. 3.0.0-M1 requires code changes
    -  "org.fusesource.jansi" % "jansi" % "1.14",
    -  "org.rogach" %% "scallop" % "0.9.5", // new version avail. 2.0.5 requires code changes
    -  "commons-io" % "commons-io" % "2.5",
    -  "org.scalacheck" %% "scalacheck" % "1.13.4" % "test"
    +lazy val genManaged = taskKey[Unit]("Generate managed sources and resources")
    +lazy val genProps = taskKey[Seq[File]]("Generate properties scala source")
    +lazy val genSchemas = taskKey[Seq[File]]("Generated DFDL schemas")
    +
    +lazy val TestDebug = config("debug") extend(Test)
    +
    +
    +lazy val daffodil         = Project("daffodil", file(".")).configs(TestDebug)
    +                              .aggregate(macroLib, propgen, lib, io, runtime1, runtime1Unparser, core, japi, sapi, tdml, test, testIBM1, tutorials, testStdLayout)
    +                              .settings(commonSettings, nopublish)
    +
    +lazy val macroLib         = Project("daffodil-macro-lib", file("daffodil-macro-lib")).configs(TestDebug)
    +                              .settings(commonSettings, nopublish)
    +                              .settings(libraryDependencies += "org.scala-lang" % "scala-reflect" % scalaVersion.value)
    +
    +lazy val propgen          = Project("daffodil-propgen", file("daffodil-propgen")).configs(TestDebug)
    +                              .settings(commonSettings, nopublish)
    +
    +lazy val lib              = Project("daffodil-lib", file("daffodil-lib")).configs(TestDebug)
    +                              .dependsOn(macroLib % "compile-internal, test-internal")
    +                              .settings(commonSettings, libManagedSettings, usesMacros)
    +
    +lazy val io               = Project("daffodil-io", file("daffodil-io")).configs(TestDebug)
    +                              .dependsOn(lib, macroLib % "compile-internal, test-internal")
    +                              .settings(commonSettings, usesMacros)
    +
    +lazy val runtime1         = Project("daffodil-runtime1", file("daffodil-runtime1")).configs(TestDebug)
    +                              .dependsOn(io, lib % "test->test")
    +                              .settings(commonSettings)
    +
    +lazy val runtime1Unparser = Project("daffodil-runtime1-unparser", file("daffodil-runtime1-unparser")).configs(TestDebug)
    +                              .dependsOn(runtime1, lib % "test->test", runtime1 % "test->test")
    +                              .settings(commonSettings)
    +
    +lazy val core             = Project("daffodil-core", file("daffodil-core")).configs(TestDebug)
    +                              .dependsOn(runtime1Unparser, lib % "test->test", runtime1 % "test->test")
    +                              .settings(commonSettings)
    +
    +lazy val japi             = Project("daffodil-japi", file("daffodil-japi")).configs(TestDebug)
    +                              .dependsOn(core)
    +                              .settings(commonSettings)
    +
    +lazy val sapi             = Project("daffodil-sapi", file("daffodil-sapi")).configs(TestDebug)
    +                              .dependsOn(core)
    +                              .settings(commonSettings)
    +
    +lazy val tdml             = Project("daffodil-tdml", file("daffodil-tdml")).configs(TestDebug)
    +                              .dependsOn(core, core % "test->test")
    +                              .settings(commonSettings)
    +
    +lazy val cli              = Project("daffodil-cli", file("daffodil-cli")).configs(TestDebug)
    +                              .dependsOn(tdml, sapi, japi) // causes sapi/japi to be pulled in to the helper zip/tar
    +                              .settings(commonSettings, nopublish)
    +                              .settings(libraryDependencies ++= Dependencies.cli) 
    +
    +lazy val test             = Project("daffodil-test", file("daffodil-test")).configs(TestDebug)
    +                              .dependsOn(tdml, core % "test->test")
    +                              .settings(commonSettings, nopublish)
    +
    +lazy val testIBM1         = Project("daffodil-test-ibm1", file("daffodil-test-ibm1")).configs(TestDebug)
    +                              .dependsOn(tdml)
    +                              .settings(commonSettings, nopublish)
    +
    +lazy val tutorials        = Project("daffodil-tutorials", file("tutorials")).configs(TestDebug)
    +                              .dependsOn(tdml)
    +                              .settings(commonSettings, nopublish)
    +
    +lazy val testStdLayout    = Project("daffodil-test-stdLayout", file("test-stdLayout")).configs(TestDebug)
    +                              .dependsOn(tdml, core % "test->test")
    +                              .settings(commonSettings, nopublish)
    +
    +
    +
    +lazy val commonSettings = Seq(
    +  organization := "edu.illinois.ncsa",
    +  version := "2.1.0-SNAPSHOT",
    --- End diff --
    
    I think this is the only place where a version needs to be updated. What's the second place you are seeing?


---

[GitHub] incubator-daffodil pull request #14: Update sbt to 1.x, simplifing build con...

Posted by mbeckerle <gi...@git.apache.org>.
Github user mbeckerle commented on a diff in the pull request:

    https://github.com/apache/incubator-daffodil/pull/14#discussion_r154180410
  
    --- Diff: build.sbt ---
    @@ -1,77 +1,193 @@
    -
    -
    -name := "daffodil"
    -
    -organization in ThisBuild := "edu.illinois.ncsa"
    -
    -version in ThisBuild := "2.1.0-SNAPSHOT"
    -
    -scalaVersion in ThisBuild := "2.11.8"
    -
    -// incOptions := incOptions.value.withNameHashing(true) // 2.11 experimental incremental compilation improvements (perhaps not working right?)
    -
    -scalacOptions in ThisBuild ++= Seq("-unchecked", "-deprecation", "-Yinline-warnings", "-Xxml:-coalescing", "-language:experimental.macros", "-Ybackend:GenBCode", "-Yopt-warnings", 
    - "-Ywarn-inaccessible", "-Ywarn-unused-import", "-Ywarn-unused", "-Ywarn-infer-any", "-Ywarn-nullary-override", "-Ydead-code", "-Yopt:inline-global", "-Yinline" , "-Xfatal-warnings")
    -
    -// parallelExecution in ThisBuild := false
    -
    -// concurrentRestrictions in Global += Tags.limit(Tags.Test, 1)
    -
    -logBuffered in ThisBuild := true
    -
    -testOptions in ThisBuild += Tests.Argument(TestFrameworks.JUnit, "-v")
    -
    -transitiveClassifiers := Seq("sources", "javadoc")
    -
    -libraryDependencies in ThisBuild := Seq(
    -  "org.scala-lang.modules" %% "scala-xml" % "1.0.6",
    -  "org.scala-lang.modules" %% "scala-parser-combinators" % "1.0.4",
    -  "junit" % "junit" % "4.11" % "test",
    -  "com.novocode" % "junit-interface" % "0.11" % "test",
    -  "net.sf.expectit" % "expectit-core" % "0.8.1" % "test",
    -  "org.jdom" % "jdom2" % "2.0.6",
    -  "com.ibm.icu" % "icu4j" % "51.1", // new versions avail. 58.1 requires code changes
    -  "xerces" % "xercesImpl" % "2.10.0",
    -  "com.fasterxml.woodstox" % "woodstox-core" % "5.0.3",
    -  "com.fasterxml.jackson.core" % "jackson-core" % "2.8.8",
    -  "xml-resolver" % "xml-resolver" % "1.2",
    -  "jline" % "jline" % "2.12.1", // newer versions avail. 3.0.0-M1 requires code changes
    -  "org.fusesource.jansi" % "jansi" % "1.14",
    -  "org.rogach" %% "scallop" % "0.9.5", // new version avail. 2.0.5 requires code changes
    -  "commons-io" % "commons-io" % "2.5",
    -  "org.scalacheck" %% "scalacheck" % "1.13.4" % "test"
    +lazy val genManaged = taskKey[Unit]("Generate managed sources and resources")
    +lazy val genProps = taskKey[Seq[File]]("Generate properties scala source")
    +lazy val genSchemas = taskKey[Seq[File]]("Generated DFDL schemas")
    +
    +lazy val TestDebug = config("debug") extend(Test)
    +
    +
    +lazy val daffodil         = Project("daffodil", file(".")).configs(TestDebug)
    +                              .aggregate(macroLib, propgen, lib, io, runtime1, runtime1Unparser, core, japi, sapi, tdml, test, testIBM1, tutorials, testStdLayout)
    +                              .settings(commonSettings, nopublish)
    +
    +lazy val macroLib         = Project("daffodil-macro-lib", file("daffodil-macro-lib")).configs(TestDebug)
    +                              .settings(commonSettings, nopublish)
    +                              .settings(libraryDependencies += "org.scala-lang" % "scala-reflect" % scalaVersion.value)
    +
    +lazy val propgen          = Project("daffodil-propgen", file("daffodil-propgen")).configs(TestDebug)
    +                              .settings(commonSettings, nopublish)
    +
    +lazy val lib              = Project("daffodil-lib", file("daffodil-lib")).configs(TestDebug)
    +                              .dependsOn(macroLib % "compile-internal, test-internal")
    +                              .settings(commonSettings, libManagedSettings, usesMacros)
    +
    +lazy val io               = Project("daffodil-io", file("daffodil-io")).configs(TestDebug)
    +                              .dependsOn(lib, macroLib % "compile-internal, test-internal")
    +                              .settings(commonSettings, usesMacros)
    +
    +lazy val runtime1         = Project("daffodil-runtime1", file("daffodil-runtime1")).configs(TestDebug)
    +                              .dependsOn(io, lib % "test->test")
    +                              .settings(commonSettings)
    +
    +lazy val runtime1Unparser = Project("daffodil-runtime1-unparser", file("daffodil-runtime1-unparser")).configs(TestDebug)
    +                              .dependsOn(runtime1, lib % "test->test", runtime1 % "test->test")
    +                              .settings(commonSettings)
    +
    +lazy val core             = Project("daffodil-core", file("daffodil-core")).configs(TestDebug)
    +                              .dependsOn(runtime1Unparser, lib % "test->test", runtime1 % "test->test")
    +                              .settings(commonSettings)
    +
    +lazy val japi             = Project("daffodil-japi", file("daffodil-japi")).configs(TestDebug)
    +                              .dependsOn(core)
    +                              .settings(commonSettings)
    +
    +lazy val sapi             = Project("daffodil-sapi", file("daffodil-sapi")).configs(TestDebug)
    +                              .dependsOn(core)
    +                              .settings(commonSettings)
    +
    +lazy val tdml             = Project("daffodil-tdml", file("daffodil-tdml")).configs(TestDebug)
    +                              .dependsOn(core, core % "test->test")
    +                              .settings(commonSettings)
    +
    +lazy val cli              = Project("daffodil-cli", file("daffodil-cli")).configs(TestDebug)
    +                              .dependsOn(tdml, sapi, japi) // causes sapi/japi to be pulled in to the helper zip/tar
    +                              .settings(commonSettings, nopublish)
    +                              .settings(libraryDependencies ++= Dependencies.cli) 
    +
    +lazy val test             = Project("daffodil-test", file("daffodil-test")).configs(TestDebug)
    +                              .dependsOn(tdml, core % "test->test")
    +                              .settings(commonSettings, nopublish)
    +
    +lazy val testIBM1         = Project("daffodil-test-ibm1", file("daffodil-test-ibm1")).configs(TestDebug)
    +                              .dependsOn(tdml)
    +                              .settings(commonSettings, nopublish)
    +
    +lazy val tutorials        = Project("daffodil-tutorials", file("tutorials")).configs(TestDebug)
    +                              .dependsOn(tdml)
    +                              .settings(commonSettings, nopublish)
    +
    +lazy val testStdLayout    = Project("daffodil-test-stdLayout", file("test-stdLayout")).configs(TestDebug)
    +                              .dependsOn(tdml, core % "test->test")
    +                              .settings(commonSettings, nopublish)
    +
    +
    +
    +lazy val commonSettings = Seq(
    +  organization := "edu.illinois.ncsa",
    +  version := "2.1.0-SNAPSHOT",
    --- End diff --
    
    Pilot error by me. I often am reading quickly, and see something in the pink section, forgetting that pink means deleted.... 
    The side-by-side view is better, except the lines all wrap...
    
    Disregard my comment, or just delete all of these.


---

[GitHub] incubator-daffodil pull request #14: Update sbt to 1.x, simplifing build con...

Posted by mbeckerle <gi...@git.apache.org>.
Github user mbeckerle commented on a diff in the pull request:

    https://github.com/apache/incubator-daffodil/pull/14#discussion_r154172019
  
    --- Diff: build.sbt ---
    @@ -1,77 +1,193 @@
    -
    -
    -name := "daffodil"
    -
    -organization in ThisBuild := "edu.illinois.ncsa"
    -
    -version in ThisBuild := "2.1.0-SNAPSHOT"
    -
    -scalaVersion in ThisBuild := "2.11.8"
    -
    -// incOptions := incOptions.value.withNameHashing(true) // 2.11 experimental incremental compilation improvements (perhaps not working right?)
    -
    -scalacOptions in ThisBuild ++= Seq("-unchecked", "-deprecation", "-Yinline-warnings", "-Xxml:-coalescing", "-language:experimental.macros", "-Ybackend:GenBCode", "-Yopt-warnings", 
    - "-Ywarn-inaccessible", "-Ywarn-unused-import", "-Ywarn-unused", "-Ywarn-infer-any", "-Ywarn-nullary-override", "-Ydead-code", "-Yopt:inline-global", "-Yinline" , "-Xfatal-warnings")
    -
    -// parallelExecution in ThisBuild := false
    -
    -// concurrentRestrictions in Global += Tags.limit(Tags.Test, 1)
    -
    -logBuffered in ThisBuild := true
    -
    -testOptions in ThisBuild += Tests.Argument(TestFrameworks.JUnit, "-v")
    -
    -transitiveClassifiers := Seq("sources", "javadoc")
    -
    -libraryDependencies in ThisBuild := Seq(
    -  "org.scala-lang.modules" %% "scala-xml" % "1.0.6",
    -  "org.scala-lang.modules" %% "scala-parser-combinators" % "1.0.4",
    -  "junit" % "junit" % "4.11" % "test",
    -  "com.novocode" % "junit-interface" % "0.11" % "test",
    -  "net.sf.expectit" % "expectit-core" % "0.8.1" % "test",
    -  "org.jdom" % "jdom2" % "2.0.6",
    -  "com.ibm.icu" % "icu4j" % "51.1", // new versions avail. 58.1 requires code changes
    -  "xerces" % "xercesImpl" % "2.10.0",
    -  "com.fasterxml.woodstox" % "woodstox-core" % "5.0.3",
    -  "com.fasterxml.jackson.core" % "jackson-core" % "2.8.8",
    -  "xml-resolver" % "xml-resolver" % "1.2",
    -  "jline" % "jline" % "2.12.1", // newer versions avail. 3.0.0-M1 requires code changes
    -  "org.fusesource.jansi" % "jansi" % "1.14",
    -  "org.rogach" %% "scallop" % "0.9.5", // new version avail. 2.0.5 requires code changes
    -  "commons-io" % "commons-io" % "2.5",
    -  "org.scalacheck" %% "scalacheck" % "1.13.4" % "test"
    +lazy val genManaged = taskKey[Unit]("Generate managed sources and resources")
    +lazy val genProps = taskKey[Seq[File]]("Generate properties scala source")
    +lazy val genSchemas = taskKey[Seq[File]]("Generated DFDL schemas")
    +
    +lazy val TestDebug = config("debug") extend(Test)
    +
    +
    +lazy val daffodil         = Project("daffodil", file(".")).configs(TestDebug)
    +                              .aggregate(macroLib, propgen, lib, io, runtime1, runtime1Unparser, core, japi, sapi, tdml, test, testIBM1, tutorials, testStdLayout)
    +                              .settings(commonSettings, nopublish)
    +
    +lazy val macroLib         = Project("daffodil-macro-lib", file("daffodil-macro-lib")).configs(TestDebug)
    +                              .settings(commonSettings, nopublish)
    +                              .settings(libraryDependencies += "org.scala-lang" % "scala-reflect" % scalaVersion.value)
    +
    +lazy val propgen          = Project("daffodil-propgen", file("daffodil-propgen")).configs(TestDebug)
    +                              .settings(commonSettings, nopublish)
    +
    +lazy val lib              = Project("daffodil-lib", file("daffodil-lib")).configs(TestDebug)
    +                              .dependsOn(macroLib % "compile-internal, test-internal")
    +                              .settings(commonSettings, libManagedSettings, usesMacros)
    +
    +lazy val io               = Project("daffodil-io", file("daffodil-io")).configs(TestDebug)
    +                              .dependsOn(lib, macroLib % "compile-internal, test-internal")
    +                              .settings(commonSettings, usesMacros)
    +
    +lazy val runtime1         = Project("daffodil-runtime1", file("daffodil-runtime1")).configs(TestDebug)
    +                              .dependsOn(io, lib % "test->test")
    +                              .settings(commonSettings)
    +
    +lazy val runtime1Unparser = Project("daffodil-runtime1-unparser", file("daffodil-runtime1-unparser")).configs(TestDebug)
    +                              .dependsOn(runtime1, lib % "test->test", runtime1 % "test->test")
    +                              .settings(commonSettings)
    +
    +lazy val core             = Project("daffodil-core", file("daffodil-core")).configs(TestDebug)
    +                              .dependsOn(runtime1Unparser, lib % "test->test", runtime1 % "test->test")
    +                              .settings(commonSettings)
    +
    +lazy val japi             = Project("daffodil-japi", file("daffodil-japi")).configs(TestDebug)
    +                              .dependsOn(core)
    +                              .settings(commonSettings)
    +
    +lazy val sapi             = Project("daffodil-sapi", file("daffodil-sapi")).configs(TestDebug)
    +                              .dependsOn(core)
    +                              .settings(commonSettings)
    +
    +lazy val tdml             = Project("daffodil-tdml", file("daffodil-tdml")).configs(TestDebug)
    +                              .dependsOn(core, core % "test->test")
    +                              .settings(commonSettings)
    +
    +lazy val cli              = Project("daffodil-cli", file("daffodil-cli")).configs(TestDebug)
    +                              .dependsOn(tdml, sapi, japi) // causes sapi/japi to be pulled in to the helper zip/tar
    +                              .settings(commonSettings, nopublish)
    +                              .settings(libraryDependencies ++= Dependencies.cli) 
    +
    +lazy val test             = Project("daffodil-test", file("daffodil-test")).configs(TestDebug)
    +                              .dependsOn(tdml, core % "test->test")
    +                              .settings(commonSettings, nopublish)
    +
    +lazy val testIBM1         = Project("daffodil-test-ibm1", file("daffodil-test-ibm1")).configs(TestDebug)
    +                              .dependsOn(tdml)
    +                              .settings(commonSettings, nopublish)
    +
    +lazy val tutorials        = Project("daffodil-tutorials", file("tutorials")).configs(TestDebug)
    +                              .dependsOn(tdml)
    +                              .settings(commonSettings, nopublish)
    +
    +lazy val testStdLayout    = Project("daffodil-test-stdLayout", file("test-stdLayout")).configs(TestDebug)
    +                              .dependsOn(tdml, core % "test->test")
    +                              .settings(commonSettings, nopublish)
    +
    +
    +
    +lazy val commonSettings = Seq(
    +  organization := "edu.illinois.ncsa",
    +  version := "2.1.0-SNAPSHOT",
    --- End diff --
    
    This is annoying. We have to update version number in two places? Below in different build.sbt you access version.value. Can we do that here?


---