You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@daffodil.apache.org by "arosien (via GitHub)" <gi...@apache.org> on 2023/03/23 16:56:58 UTC
[GitHub] [daffodil-vscode] arosien opened a new issue, #521: Backend builds sometimes fail with duplicate generated schemas
arosien opened a new issue, #521:
URL: https://github.com/apache/daffodil-vscode/issues/521
Sometimes [jobs fail](https://github.com/apache/daffodil-vscode/actions/runs/4500462875/jobs/7919625386) like:
```
$ sbt test && node ./out/tests/runTest.js
[info] welcome to sbt 1.8.2 (Temurin Java 1.8.0_362)
[info] loading settings for project daffodil-vscode-build from plugins.sbt ...
[info] loading project definition from /Users/runner/work/daffodil-vscode/daffodil-vscode/project
[info] loading settings for project daffodil-debugger from build.sbt ...
[info] set current project to daffodil-debugger (in build file:/Users/runner/work/daffodil-vscode/daffodil-vscode/)
[info] Compiling 1 XSD file(s) to /Users/runner/work/daffodil-vscode/daffodil-vscode/server/core/target/scala-2.12/src_managed_cxf
parsing a schema...
Error: 'DFDLExpression' is already defined
line 38 of file:/Users/runner/work/daffodil-vscode/daffodil-vscode/server/core/target/scala-2.12/resource_managed/xsd/org/apache/daffodil/xsd/DFDL_part1_simpletypes.xsd
...
```
Workaround was committed as #508.
More info:
- XSD schemas are extracted from the daffodil dependencies and used to generate Java sources via JAXB.
- `sbt test` seems to non-deterministically regenerate the Java sources. If you run it 3 times starting from a clean repo, the sources are generated 2 times, so some dependency task is being invalidated and thus rerun.
- I believe CI disk state is persisted across builds.
cc: @scholarsmate @Shanedell
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-vscode] arosien commented on issue #521: Backend builds sometimes fail with duplicate generated schemas
Posted by "arosien (via GitHub)" <gi...@apache.org>.
arosien commented on issue #521:
URL: https://github.com/apache/daffodil-vscode/issues/521#issuecomment-1487815773
> > I feel like these lines
> > https://github.com/apache/daffodil-vscode/blob/main/build.sbt#L180-L182
> > are dubious and are perhaps adding the re-generation of sources to the Test scope, when the Compile scope already defines them, causing the duplication during builds.
>
> Been testing this in a branch and got a bunch of clean builds in a row, had to add Global / lintUnusedKeysOnLoad := false to build.sbt though to get rid of a linting error that happens when I removed those three lines. Not sure if that line is good to keep though. Was also able to get rid of the rm **/target in the CI.yml and still get good builds.
@NolanMatt push a branch and make a (draft) PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-vscode] scholarsmate commented on issue #521: Backend builds sometimes fail with duplicate generated schemas
Posted by "scholarsmate (via GitHub)" <gi...@apache.org>.
scholarsmate commented on issue #521:
URL: https://github.com/apache/daffodil-vscode/issues/521#issuecomment-1490405590
@arosien, @NolanMatt is having a difficult time investigating the problem since it works for him locally and intermittently in CI. He indicated that you are able to reproduce issue locally, so I'll reassign it to you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-vscode] NolanMatt commented on issue #521: Backend builds sometimes fail with duplicate generated schemas
Posted by "NolanMatt (via GitHub)" <gi...@apache.org>.
NolanMatt commented on issue #521:
URL: https://github.com/apache/daffodil-vscode/issues/521#issuecomment-1487513241
> I feel like these lines
>
> https://github.com/apache/daffodil-vscode/blob/main/build.sbt#L180-L182
>
> are dubious and are perhaps adding the re-generation of sources to the Test scope, when the Compile scope already defines them, causing the duplication during builds.
Been testing this in a branch and got a bunch of clean builds in a row, had to add Global / lintUnusedKeysOnLoad := false to build.sbt though to get rid of a linting error that happens when I removed those three lines. Not sure if that line is good to keep though. Was also able to get rid of the rm **/target in the CI.yml and still get good builds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-vscode] arosien closed issue #521: Backend builds sometimes fail with duplicate generated schemas
Posted by "arosien (via GitHub)" <gi...@apache.org>.
arosien closed issue #521: Backend builds sometimes fail with duplicate generated schemas
URL: https://github.com/apache/daffodil-vscode/issues/521
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-vscode] Shanedell commented on issue #521: Backend builds sometimes fail with duplicate generated schemas
Posted by "Shanedell (via GitHub)" <gi...@apache.org>.
Shanedell commented on issue #521:
URL: https://github.com/apache/daffodil-vscode/issues/521#issuecomment-1481944651
@arosien Sometimes jobs fail, eg [failed job](https://github.com/apache/daffodil-vscode/actions/runs/4504410306/jobs/7928848295), something like:
```bash
$ sbt test && node ./out/tests/runTest.js
[info] welcome to sbt 1.8.2 (Temurin Java 1.8.0_362)
[info] loading settings for project daffodil-vscode-build from plugins.sbt ...
[info] loading project definition from /Users/runner/work/daffodil-vscode/daffodil-vscode/project
[info] loading settings for project daffodil-debugger from build.sbt ...
[info] set current project to daffodil-debugger (in build file:/Users/runner/work/daffodil-vscode/daffodil-vscode/)
...
[info] Compiling 1 XSD file(s) to /Users/runner/work/daffodil-vscode/daffodil-vscode/server/core/target/scala-2.12/src_managed_cxf
parsing a schema...
Error: IOException thrown when processing "file:/Users/runner/work/daffodil-vscode/daffodil-vscode/server/core/target/scala-2.12/resource_managed/xsd/org/apache/daffodil/xsd/xml.xsd". Exception: java.io.FileNotFoundException: /Users/runner/work/daffodil-vscode/daffodil-vscode/server/core/target/scala-2.12/resource_managed/xsd/org/apache/daffodil/xsd/xml.xsd (No such file or directory).
unknown location
Error: IOException thrown when processing "file:/Users/runner/work/daffodil-vscode/daffodil-vscode/server/core/target/scala-2.12/resource_managed/xsd/org/apache/daffodil/xsd/tdml.xsd". Exception: java.io.FileNotFoundException: /Users/runner/work/daffodil-vscode/daffodil-vscode/server/core/target/scala-2.12/resource_managed/xsd/org/apache/daffodil/xsd/tdml.xsd (No such file or directory).
unknown location
...
```
Are the possibly related?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-vscode] arosien commented on issue #521: Backend builds sometimes fail with duplicate generated schemas
Posted by "arosien (via GitHub)" <gi...@apache.org>.
arosien commented on issue #521:
URL: https://github.com/apache/daffodil-vscode/issues/521#issuecomment-1485430236
I feel like `yarn test` (which does `sbt test`) should be before the `yarn package` (which does the `universal:packageBin`). This isn't necessarily related to this issue, but it seems better for the build to fail fast with unit tests before any packaging and integration tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-vscode] arosien commented on issue #521: Backend builds sometimes fail with duplicate generated schemas
Posted by "arosien (via GitHub)" <gi...@apache.org>.
arosien commented on issue #521:
URL: https://github.com/apache/daffodil-vscode/issues/521#issuecomment-1485442625
I feel like these lines
https://github.com/apache/daffodil-vscode/blob/main/build.sbt#L180-L182
are dubious and are perhaps adding the re-generation of sources to the Test scope, when the Compile scope already defines them, causing the duplication during builds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [daffodil-vscode] arosien commented on issue #521: Backend builds sometimes fail with duplicate generated schemas
Posted by "arosien (via GitHub)" <gi...@apache.org>.
arosien commented on issue #521:
URL: https://github.com/apache/daffodil-vscode/issues/521#issuecomment-1485482011
> @arosien Sometimes jobs fail, eg [failed job](https://github.com/apache/daffodil-vscode/actions/runs/4504410306/jobs/7928848295), something like:
>
> ```shell
> $ sbt test && node ./out/tests/runTest.js
> [info] welcome to sbt 1.8.2 (Temurin Java 1.8.0_362)
> [info] loading settings for project daffodil-vscode-build from plugins.sbt ...
> [info] loading project definition from /Users/runner/work/daffodil-vscode/daffodil-vscode/project
> [info] loading settings for project daffodil-debugger from build.sbt ...
> [info] set current project to daffodil-debugger (in build file:/Users/runner/work/daffodil-vscode/daffodil-vscode/)
> ...
> [info] Compiling 1 XSD file(s) to /Users/runner/work/daffodil-vscode/daffodil-vscode/server/core/target/scala-2.12/src_managed_cxf
> parsing a schema...
> Error: IOException thrown when processing "file:/Users/runner/work/daffodil-vscode/daffodil-vscode/server/core/target/scala-2.12/resource_managed/xsd/org/apache/daffodil/xsd/xml.xsd". Exception: java.io.FileNotFoundException: /Users/runner/work/daffodil-vscode/daffodil-vscode/server/core/target/scala-2.12/resource_managed/xsd/org/apache/daffodil/xsd/xml.xsd (No such file or directory).
> unknown location
>
> Error: IOException thrown when processing "file:/Users/runner/work/daffodil-vscode/daffodil-vscode/server/core/target/scala-2.12/resource_managed/xsd/org/apache/daffodil/xsd/tdml.xsd". Exception: java.io.FileNotFoundException: /Users/runner/work/daffodil-vscode/daffodil-vscode/server/core/target/scala-2.12/resource_managed/xsd/org/apache/daffodil/xsd/tdml.xsd (No such file or directory).
> unknown location
> ...
> ```
>
> Are the possibly related?
That message looks like there is a "clean" or something removing the files, possibly from another build going on??
I know when launching the extension in VS Code it runs `yarn watch`, which seems to _fork_ the sbt builds, so maybe one of the yarn commands does this concurrent with one of the sbt CI steps.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@daffodil.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org