You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@daffodil.apache.org by GitBox <gi...@apache.org> on 2022/01/13 00:24:58 UTC
[GitHub] [daffodil-schema.g8] stevedlawrence opened a new pull request #2: Add a template for the new flatted layout
stevedlawrence opened a new pull request #2:
URL: https://github.com/apache/daffodil-schema.g8/pull/2
Two commits, the first moves the existing template to a "namespaced" directory for those that might still want it. The second commit adds a new "simple" template layout for the preferred layout. The README is modified to describe the templates and how to create a new layout using the templates.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-schema.g8] stevedlawrence commented on pull request #2: Add a template for the new flatted layout
Posted by GitBox <gi...@apache.org>.
stevedlawrence commented on pull request #2:
URL: https://github.com/apache/daffodil-schema.g8/pull/2#issuecomment-1015395714
Closing in favor of PR #3
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-schema.g8] stevedlawrence closed pull request #2: Add a template for the new flatted layout
Posted by GitBox <gi...@apache.org>.
stevedlawrence closed pull request #2:
URL: https://github.com/apache/daffodil-schema.g8/pull/2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-schema.g8] stevedlawrence commented on a change in pull request #2: Add a template for the new flatted layout
Posted by GitBox <gi...@apache.org>.
stevedlawrence commented on a change in pull request #2:
URL: https://github.com/apache/daffodil-schema.g8/pull/2#discussion_r783998827
##########
File path: simple/src/main/g8/test/Test$name__Camel$.tdml
##########
@@ -0,0 +1,38 @@
+<?xml version="1.0" encoding="UTF-8"?>
+$!
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+!$
+
+<tdml:testSuite suiteName="$name;format="Camel"$" description="$name;format="Camel"$ tests"
Review comment:
I'm not sure how IBM uses them, but I'm pretty sure they are optional. I have no opinion one way or another about keeping them. I imagine some day it might be useful to have them. For example, maybe one day we'll have a way to autogenerate tests based on just reading the .tdml files rather than also needing a .scala file. In that cas, the suiteName or description might be useful for generating those tests? I"m not really sure. Seems like good practice to have maybe?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-schema.g8] tuxji commented on a change in pull request #2: Add a template for the new flatted layout
Posted by GitBox <gi...@apache.org>.
tuxji commented on a change in pull request #2:
URL: https://github.com/apache/daffodil-schema.g8/pull/2#discussion_r783553041
##########
File path: simple/src/main/g8/build.sbt
##########
@@ -0,0 +1,45 @@
+$!
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+!$
+name := "dfdl-$name;format="norm"$"
+
+organization := "$package$"
+
+version := "0.1.0-SNAPSHOT"
+
+scalaVersion := "2.12.11"
Review comment:
Should we update scalaVersion to 2.12.15 here too?
##########
File path: simple/src/main/g8/test/Test$name__Camel$.scala
##########
@@ -0,0 +1,37 @@
+$!
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+!$
+package $package$.$name;format="camel"$
+
+import org.junit.AfterClass
+import org.junit.Test
+
+import org.apache.daffodil.tdml.Runner
+
+object Test$name;format="Camel"$ {
+ lazy val runner = Runner("/", "Test$name;format="Camel"$.tdml")
+
+ @AfterClass def shutDown {
Review comment:
Should we avoid a warning at this line?
```
@AfterClass def shutDown(): Unit = {
```
##########
File path: build.sbt
##########
@@ -15,15 +15,32 @@
lazy val root = (project in file("."))
.enablePlugins(ScriptedPlugin)
+ .settings(commonSettings)
.settings(
name := "daffodil-schema.g8",
- scalaVersion := "2.12.11",
- crossScalaVersions := Seq("2.12.11"),
- test in Test := {
- val _ = (g8Test in Test).toTask("").value
- },
ratFailBinaries := true,
ratExcludes := Seq(
file(".git"),
),
)
+ .aggregate(namespaced, simple)
+
+lazy val namespaced = (project in file("namespaced"))
+ .settings(commonSettings)
+ .settings(templateSettings)
+
+lazy val simple = (project in file("simple"))
+ .settings(commonSettings)
+ .settings(templateSettings)
+
+lazy val commonSettings = Seq(
+ scalaVersion := "2.12.11",
+ crossScalaVersions := Seq("2.12.11"),
Review comment:
Should we update Scala version to 2.12.15?
##########
File path: simple/src/main/g8/project/build.properties
##########
@@ -0,0 +1,17 @@
+$!
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+!$
+sbt.version=1.3.9
Review comment:
Should we update sbt.version to 1.6.1?
##########
File path: simple/src/main/g8/build.sbt
##########
@@ -0,0 +1,45 @@
+$!
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+!$
+name := "dfdl-$name;format="norm"$"
+
+organization := "$package$"
+
+version := "0.1.0-SNAPSHOT"
+
+scalaVersion := "2.12.11"
+
+libraryDependencies ++= Seq(
+ "org.apache.daffodil" %% "daffodil-tdml-processor" % "$daffodil_version$" % "test",
+ "com.novocode" % "junit-interface" % "0.11" % "test",
+ "junit" % "junit" % "4.12" % "test",
Review comment:
Should we update junit dependencies too? I also found out that adding log4j was needed to avoid a warning too.
```
"com.github.sbt" % "junit-interface" % "0.13.2" % "test",
"junit" % "junit" % "4.13.2" % "test",
"org.apache.daffodil" %% "daffodil-tdml-processor" % "$daffodil_version$" % "test",
"org.apache.logging.log4j" % "log4j-core" % "2.17.1" % "test",
```
##########
File path: simple/src/main/g8/test/Test$name__Camel$.tdml
##########
@@ -0,0 +1,38 @@
+<?xml version="1.0" encoding="UTF-8"?>
+$!
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+!$
+
+<tdml:testSuite suiteName="$name;format="Camel"$" description="$name;format="Camel"$ tests"
+ xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/"
+ xmlns:fn="http://www.w3.org/2005/xpath-functions"
+ xmlns:tdml="http://www.ibm.com/xmlns/dfdl/testData"
+ xmlns:xs="http://www.w3.org/2001/XMLSchema"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ defaultRoundTrip="onePass">
+
+ <tdml:parserTestCase name="test_$name;format="camel"$_01" root="$name;format="Camel"$" model="$name;format="camel"$.dfdl.xsd">
+ <tdml:document>
+ <tdml:documentPart type="file">/data/test_01.$name;format="camel"$</tdml:documentPart>
+ </tdml:document>
+ <tdml:infoset>
+ <tdml:dfdlInfoset type="file">/xml/test_01.$name;format="camel"$.xml</tdml:dfdlInfoset>
Review comment:
Should we use file suffixes ".dat" and ".dat.xml" or continue using $name as a file suffix?
I've found there's no need to start the filenames with /, which makes them look like relative filenames and therefore easier to find the data files if one's reading this TDML file.
For the subdirectory names, "data" is fine but "infosets" may be a better name than "xml".
##########
File path: simple/src/main/g8/test/Test$name__Camel$.tdml
##########
@@ -0,0 +1,38 @@
+<?xml version="1.0" encoding="UTF-8"?>
+$!
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+!$
+
+<tdml:testSuite suiteName="$name;format="Camel"$" description="$name;format="Camel"$ tests"
Review comment:
Does IBM's TDML processor use suiteName and description? Currently I tend to leave them out because Daffodil's TDML processor ignores them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-schema.g8] stevedlawrence commented on pull request #2: Add a template for the new flatted layout
Posted by GitBox <gi...@apache.org>.
stevedlawrence commented on pull request #2:
URL: https://github.com/apache/daffodil-schema.g8/pull/2#issuecomment-1012182536
> I think it's unfortunate that we have two copies of each template file
I've created PR #3 that resolves this. Instead of having two templates, it has a single template and a new "namespaced" property which defaults to "no" for the new flat structure. The directory names get pretty ugly, but there's not really a good way to avoid that without duplication.
That PR also incorporates all your suggested changes, except for the .dat and testsuite/description. I have no strong preference for those--I'd like to hear input from others.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-schema.g8] stevedlawrence commented on pull request #2: Add a template for the new flatted layout
Posted by GitBox <gi...@apache.org>.
stevedlawrence commented on pull request #2:
URL: https://github.com/apache/daffodil-schema.g8/pull/2#issuecomment-1012098041
> Can g8 run a script at generation time? If so, we still could have one set of template files but write a script to move the generated files to the right places depending on the desired layout (simple or namespaced).
Unfortunately I don't think there is a way to run a script.
However, g8 does have basic if/else that can be used to make dynamic file content. I didn't think it could be used dynamic paths, but I double checked and found documentation doing just that:
https://github.com/foundweekends/giter8/blob/develop/docs/03/00.md
Seems like a much better approach to reduce duplication. The templates become more complex, but that's not a big deal. I'll give it a shot and see how it goes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-schema.g8] stevedlawrence commented on a change in pull request #2: Add a template for the new flatted layout
Posted by GitBox <gi...@apache.org>.
stevedlawrence commented on a change in pull request #2:
URL: https://github.com/apache/daffodil-schema.g8/pull/2#discussion_r783996212
##########
File path: simple/src/main/g8/test/Test$name__Camel$.tdml
##########
@@ -0,0 +1,38 @@
+<?xml version="1.0" encoding="UTF-8"?>
+$!
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+!$
+
+<tdml:testSuite suiteName="$name;format="Camel"$" description="$name;format="Camel"$ tests"
+ xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/"
+ xmlns:fn="http://www.w3.org/2005/xpath-functions"
+ xmlns:tdml="http://www.ibm.com/xmlns/dfdl/testData"
+ xmlns:xs="http://www.w3.org/2001/XMLSchema"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ defaultRoundTrip="onePass">
+
+ <tdml:parserTestCase name="test_$name;format="camel"$_01" root="$name;format="Camel"$" model="$name;format="camel"$.dfdl.xsd">
+ <tdml:document>
+ <tdml:documentPart type="file">/data/test_01.$name;format="camel"$</tdml:documentPart>
+ </tdml:document>
+ <tdml:infoset>
+ <tdml:dfdlInfoset type="file">/xml/test_01.$name;format="camel"$.xml</tdml:dfdlInfoset>
Review comment:
> Should we use file suffixes ".dat" and ".dat.xml" or continue using $name as a file suffix?
I usually only prefer to use .dat when there isn't an actual file format, like in Daffodil unit tests. But in this case, the repo should eventually contain real file formats, and I think it's usually easier to deal with those files if they have the expected extension. For example, if someone created a bitmap format repo, it would be much easier to view the bitmaps for correctness if they ended in .bmp rather that .dat. For this template, we make a guess based on the format name, which is probably going to be wrong, but I think it hints to the user that they should replace these test files with actual files with correct extensions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org