You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@daffodil.apache.org by "stevedlawrence (via GitHub)" <gi...@apache.org> on 2024/01/24 12:43:15 UTC
[PR] Support schema projects that build layers and UDFs [daffodil-sbt]
stevedlawrence opened a new pull request, #13:
URL: https://github.com/apache/daffodil-sbt/pull/13
- Add daffodilBuildsLayer and daffodilBuildsUDF settings, which if set to true adds dependencies and modifies crossPaths for projects that builds layers and UDFs, respectively. Defaults to false since most projects will not use these.
Closes #11
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Support schema projects that build layers and UDFs [daffodil-sbt]
Posted by "stevedlawrence (via GitHub)" <gi...@apache.org>.
stevedlawrence commented on code in PR #13:
URL: https://github.com/apache/daffodil-sbt/pull/13#discussion_r1465243707
##########
README.md:
##########
@@ -94,6 +94,23 @@ If used, one may want to use the first value of this setting to configure
daffodilVersion := daffodilPackageBinVersions.value.head
```
+## Layers and User Defined Functions
+
+If your schema project builds a Daffodil layer or user defined function, then
+set the `daffodilBuildsLayer` or `daffodilBuildsUDF` setting to true,
+respectively. For example:
+
+```scala
+daffodilBuildsLayer := true
+
+daffodilBuildsUDF := true
+```
+
+Setting either of these values to true adds additional dependencies needed to
+build the component. Note that this also sets the SBT `crossPaths` setting to
+`true`, which causes the scala version to be included in that jar file name
+since layer and UDF jars are specific to the scala version used to build them.
Review Comment:
> Or add a scalaCrossPaths option which if false shuts off the cross paths, but allows the other settings changes?
>
> Perhaps it's just allowScala which if not present or false scala will not be allowed in the project?
These feel like it might start complicating things too much for users. The goal is to keep things as simple as possible for schema authors and giving them reasonable defaults. I think the new `daffodilBuildsUDF` and `daffodilBuildsLayer` are useful since they provide extra dependencies so users don't have to think about it, but beyond that I think starts making things complicated and causes users to have to remember more and more settings. We're already starting to build up a lot of settings.
If a project wants to disallow scala files, then maybe the suggested way is to scan for it in code reviews or by adding *.scala to .gitignore. And if they don't like our crossPaths defaults, they can manually override it with the existing SBT setting.
I like @tuxji's idea of just documnting that if a project has layers/udfs that don't use any Scala and they don't want the _2.12 then to set crossPaths to false in build.sbt.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Support schema projects that build layers and UDFs [daffodil-sbt]
Posted by "mbeckerle (via GitHub)" <gi...@apache.org>.
mbeckerle commented on code in PR #13:
URL: https://github.com/apache/daffodil-sbt/pull/13#discussion_r1465167442
##########
README.md:
##########
@@ -94,6 +94,23 @@ If used, one may want to use the first value of this setting to configure
daffodilVersion := daffodilPackageBinVersions.value.head
```
+## Layers and User Defined Functions
+
+If your schema project builds a Daffodil layer or user defined function, then
+set the `daffodilBuildsLayer` or `daffodilBuildsUDF` setting to true,
+respectively. For example:
+
+```scala
+daffodilBuildsLayer := true
+
+daffodilBuildsUDF := true
+```
+
+Setting either of these values to true adds additional dependencies needed to
+build the component. Note that this also sets the SBT `crossPaths` setting to
+`true`, which causes the scala version to be included in that jar file name
+since layer and UDF jars are specific to the scala version used to build them.
Review Comment:
Or add a scalaCrossPaths option which if false shuts off the cross paths, but allows the other settings changes?
Perhaps it's just allowScala which if not present or false scala will not be allowed in the project?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Support schema projects that build layers and UDFs [daffodil-sbt]
Posted by "stevedlawrence (via GitHub)" <gi...@apache.org>.
stevedlawrence commented on code in PR #13:
URL: https://github.com/apache/daffodil-sbt/pull/13#discussion_r1465108638
##########
README.md:
##########
@@ -94,6 +94,23 @@ If used, one may want to use the first value of this setting to configure
daffodilVersion := daffodilPackageBinVersions.value.head
```
+## Layers and User Defined Functions
+
+If your schema project builds a Daffodil layer or user defined function, then
+set the `daffodilBuildsLayer` or `daffodilBuildsUDF` setting to true,
+respectively. For example:
+
+```scala
+daffodilBuildsLayer := true
+
+daffodilBuildsUDF := true
+```
+
+Setting either of these values to true adds additional dependencies needed to
+build the component. Note that this also sets the SBT `crossPaths` setting to
+`true`, which causes the scala version to be included in that jar file name
+since layer and UDF jars are specific to the scala version used to build them.
Review Comment:
Yeah, I think you're right.
Maybe instead of using these settings we should just check if `Compile / sources` contains any files that end in *.scala. If any exist then we set `crossPaths` to true. This has the advantage that if a schema project has any scala source files at all, even if they aren't using UDFs or layers, that they get the correct _2.12 since they do use scala.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Support schema projects that build layers and UDFs [daffodil-sbt]
Posted by "mbeckerle (via GitHub)" <gi...@apache.org>.
mbeckerle commented on code in PR #13:
URL: https://github.com/apache/daffodil-sbt/pull/13#discussion_r1465090233
##########
README.md:
##########
@@ -94,6 +94,23 @@ If used, one may want to use the first value of this setting to configure
daffodilVersion := daffodilPackageBinVersions.value.head
```
+## Layers and User Defined Functions
+
+If your schema project builds a Daffodil layer or user defined function, then
+set the `daffodilBuildsLayer` or `daffodilBuildsUDF` setting to true,
+respectively. For example:
+
+```scala
+daffodilBuildsLayer := true
+
+daffodilBuildsUDF := true
+```
+
+Setting either of these values to true adds additional dependencies needed to
+build the component. Note that this also sets the SBT `crossPaths` setting to
+`true`, which causes the scala version to be included in that jar file name
+since layer and UDF jars are specific to the scala version used to build them.
Review Comment:
If the UDF is written in Java then no crossPaths would be needed right?
A goal for the revised layer API should be that one can write a layer in Java I suppose.
Do we need to know if Java or Scala are being used?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Support schema projects that build layers and UDFs [daffodil-sbt]
Posted by "tuxji (via GitHub)" <gi...@apache.org>.
tuxji commented on code in PR #13:
URL: https://github.com/apache/daffodil-sbt/pull/13#discussion_r1465187531
##########
README.md:
##########
@@ -94,6 +94,23 @@ If used, one may want to use the first value of this setting to configure
daffodilVersion := daffodilPackageBinVersions.value.head
```
+## Layers and User Defined Functions
+
+If your schema project builds a Daffodil layer or user defined function, then
+set the `daffodilBuildsLayer` or `daffodilBuildsUDF` setting to true,
+respectively. For example:
+
+```scala
+daffodilBuildsLayer := true
+
+daffodilBuildsUDF := true
+```
+
+Setting either of these values to true adds additional dependencies needed to
+build the component. Note that this also sets the SBT `crossPaths` setting to
+`true`, which causes the scala version to be included in that jar file name
+since layer and UDF jars are specific to the scala version used to build them.
Review Comment:
I'm fine with just adding one or two more sentences to the README explaining that if turning on crossPaths isn't actually desired or needed (say because the UDF or layer is written in Java, not Scala), simply set `crossPaths := false` in build.sbt.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Support schema projects that build layers and UDFs [daffodil-sbt]
Posted by "stevedlawrence (via GitHub)" <gi...@apache.org>.
stevedlawrence commented on code in PR #13:
URL: https://github.com/apache/daffodil-sbt/pull/13#discussion_r1465159252
##########
README.md:
##########
@@ -94,6 +94,23 @@ If used, one may want to use the first value of this setting to configure
daffodilVersion := daffodilPackageBinVersions.value.head
```
+## Layers and User Defined Functions
+
+If your schema project builds a Daffodil layer or user defined function, then
+set the `daffodilBuildsLayer` or `daffodilBuildsUDF` setting to true,
+respectively. For example:
+
+```scala
+daffodilBuildsLayer := true
+
+daffodilBuildsUDF := true
+```
+
+Setting either of these values to true adds additional dependencies needed to
+build the component. Note that this also sets the SBT `crossPaths` setting to
+`true`, which causes the scala version to be included in that jar file name
+since layer and UDF jars are specific to the scala version used to build them.
Review Comment:
Of course it's not as easy as I'd guess. I would expect this to work:
```scala
crossPaths := (Compile / sources).value.exists { _.getName.endsWith(".scala") }
```
But that doesn't compile because `crossPaths` is a "setting" and `Compile / sources` is a "task", and settings can't depend on tasks.
And it's actually relatively difficult if not impossible to accurately figure out the list of sources using just settings. We would have to use the `Compile / unmanagedSourceDirectories)` setting and recursively scan those directories for scala files. And if there are any managedSource (e.g. generated files), we can't know about those at all, since they generating them is a task. I'm not sure if it's worth the effort to figure it out.
Maybe the best option is if there are layers or UDFs then we assume there are scala files like this PR does, and if there aren't and they are all java files, the project can override it by setting `crossPaths := false` in build.sbt. And if a project doesn't do that it doesn't hurt anything if crossPaths is true, it just means there's an unnecessary _2.12 in the jar name.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Support schema projects that build layers and UDFs [daffodil-sbt]
Posted by "stevedlawrence (via GitHub)" <gi...@apache.org>.
stevedlawrence merged PR #13:
URL: https://github.com/apache/daffodil-sbt/pull/13
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org