You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@daffodil.apache.org by "Interrante, John A (GE Research, US)" <Jo...@ge.com> on 2021/11/01 19:42:06 UTC
RE: [DISCUSS] Are we ready to release 3.2.0 Daffodil ?
I checked the Daffodil JIRA to see how many blocker, critical, and major issues we have so far:
Blocker issues: 0
Critical issues: 1
https://issues.apache.org/jira/browse/DAFFODIL-2400 - New SAX API causes performance degradations
Major issues: 132
https://issues.apache.org/jira/browse/DAFFODIL-2574 - Cast error when multiplying two unsignedBytes
and so on - too many to list individually
The critical issue isn't a blocker and in fact hasn't been updated since October 2020 (1 year ago). I see Daffodil's release plan in https://cwiki.apache.org/confluence/display/DAFFODIL/Roadmap+for+Upcoming+Releases already suggests reducing the number of JIRA issues between 3.2.0 and 3.3.0 which sounds like a good plan. Everything planned for 3.2.0 seems to be already checked off.
I looked over the instructions in https://cwiki.apache.org/confluence/display/DAFFODIL/Release+Workflow carefully and preparing a release candidate looks doable but still a time-consuming bit of work. Since we already have a volunteer, I'd rather wait for another release (thanks Mike).
The first step in the Release workflow says to create a [DISCUSS] thread and allow time for discussion before creating the release candidate. I've renamed this email's subject to start that formal thread and I'm fine with waiting less than 72 hours if we get responses from everyone we know who might be working on anything that they want to release in 3.2.0. I plan to make some more changes and improvements to the Daffodil C code generator myself but the changes can go into 3.3.0 since I already build Daffodil from source anyway. Is Steve Lawrence or John Wass working on something that should make it into 3.2.0? Anyone else?
John
-----Original Message-----
From: Mike Beckerle <mb...@apache.org>
Sent: Monday, November 1, 2021 1:44 PM
To: dev@daffodil.apache.org
Subject: EXT: Are we ready to release 3.2.0 Daffodil ?
I believe 3.2.0 is functionally complete.
Should we prepare a release candidate that we can vote on?
We need someone to volunteer to be release manager.
I am willing to do so, as I've not done this for several releases. However, if someone else who hasn't done one would like a chance, I will defer to them.
Re: [DISCUSS] Are we ready to release 3.2.0 Daffodil ?
Posted by Steve Lawrence <sl...@apache.org>.
Regarding the 72 hours, ASF prefers this not be skipped since there's
always a possibility that there are people who might have some input
that we just don't know about yet. All input is welcome for these
discussions, even from first time contributors.
I've started working on DAFFODIL-2574 that was just opened this morning.
It's not too hard to fix, but I've also identified some related cleanup
that takes a bit of effort. And I think I'll want to add a decent number
of tests to ensure we're getting code coverage. That fact that we don't
have tests that catch this error cause by multiplying two unsigned bytes
is a bit concerning.
But there is a clear workaround, so I'm not sure it's should be
considered a blocker. But I also don't think it will take much more than
72 hours to finish.
- Steve
On 11/1/21 3:42 PM, Interrante, John A (GE Research, US) wrote:
> I checked the Daffodil JIRA to see how many blocker, critical, and major issues we have so far:
>
> Blocker issues: 0
> Critical issues: 1
> https://issues.apache.org/jira/browse/DAFFODIL-2400 - New SAX API causes performance degradations
> Major issues: 132
> https://issues.apache.org/jira/browse/DAFFODIL-2574 - Cast error when multiplying two unsignedBytes
> and so on - too many to list individually
>
> The critical issue isn't a blocker and in fact hasn't been updated since October 2020 (1 year ago). I see Daffodil's release plan in https://cwiki.apache.org/confluence/display/DAFFODIL/Roadmap+for+Upcoming+Releases already suggests reducing the number of JIRA issues between 3.2.0 and 3.3.0 which sounds like a good plan. Everything planned for 3.2.0 seems to be already checked off.
>
> I looked over the instructions in https://cwiki.apache.org/confluence/display/DAFFODIL/Release+Workflow carefully and preparing a release candidate looks doable but still a time-consuming bit of work. Since we already have a volunteer, I'd rather wait for another release (thanks Mike).
>
> The first step in the Release workflow says to create a [DISCUSS] thread and allow time for discussion before creating the release candidate. I've renamed this email's subject to start that formal thread and I'm fine with waiting less than 72 hours if we get responses from everyone we know who might be working on anything that they want to release in 3.2.0. I plan to make some more changes and improvements to the Daffodil C code generator myself but the changes can go into 3.3.0 since I already build Daffodil from source anyway. Is Steve Lawrence or John Wass working on something that should make it into 3.2.0? Anyone else?
>
> John
>
> -----Original Message-----
> From: Mike Beckerle <mb...@apache.org>
> Sent: Monday, November 1, 2021 1:44 PM
> To: dev@daffodil.apache.org
> Subject: EXT: Are we ready to release 3.2.0 Daffodil ?
>
> I believe 3.2.0 is functionally complete.
>
> Should we prepare a release candidate that we can vote on?
>
> We need someone to volunteer to be release manager.
>
> I am willing to do so, as I've not done this for several releases. However, if someone else who hasn't done one would like a chance, I will defer to them.
>
Re: [DISCUSS] Are we ready to release 3.2.0 Daffodil ?
Posted by "Adams, Joshua" <ja...@owlcyberdefense.com.INVALID>.
I took a look at the EDIFACT and NACHA schemas, which ar both failing because a predefined variable (encoding) is attempting to reset beyond it's original definition. I attempted to make a simple test case to reproduce the issue by having a choice branch that will backtrack after reading the varible, but was unable to reproduce this double reset that seems to be occurring for these schemas.
I've got a bit of a hacky pull request up that works around this issue, but there may still be a deeper problem here that is causing this double reset to occur.
https://github.com/apache/daffodil/pull/674
Josh
[https://opengraph.githubassets.com/d874d568be0e42159e8e2f2c6d553ed0af1c753e327ba64b30110c5a73cc3f09/apache/daffodil/pull/674]<https://github.com/apache/daffodil/pull/674>
Work around issue resetting predefined vars by jadams-tresys · Pull Request #674 · apache/daffodil<https://github.com/apache/daffodil/pull/674>
This is a quick fix for this bug where a predefined variable is being reset beyond its original definition. This was occurring in the EDIFACT and NACHA schemas. Definitely feels a little hacky and...
github.com
________________________________
From: Mike Beckerle <mb...@apache.org>
Sent: Wednesday, November 3, 2021 3:51 PM
To: dev@daffodil.apache.org <de...@daffodil.apache.org>
Subject: Re: [DISCUSS] Are we ready to release 3.2.0 Daffodil ?
I guess I have found issues indicating we are not quite ready for release.
These may be caused by incompatibilities of the tests, but Owl has a
regression test rig where we have every DFDL schema we can get our hands
on.
I ran this for daffodil 3.1.0 and all tests pass.
For daffodil 3.2.0-SNAPSHOT I get failures for these:
[error] (dfdl-vmf-spock / Test / test) sbt.TestsFailedException: Tests
unsuccessful
[error] (dfdl-edifact / Test / test) sbt.TestsFailedException: Tests
unsuccessful
[error] (udf-geoutil / Test / test) sbt.TestsFailedException: Tests
unsuccessful
[error] (dfdl-nacha / Test / test) sbt.TestsFailedException: Tests
unsuccessful
[error] (dfdl-link16 / Test / test) sbt.TestsFailedException: Tests
unsuccessful
[error] (dfdl-examples-hexwords / Test / test) sbt.TestsFailedException:
Tests unsuccessful
I will investigate why this is. From looking at the diagnostics, one of
them at least is due to xmlns="...." changes in expected output vs. actual,
and since there was a change there in 3.2.0 I expect that may be a matter
of adjusting the test case data.
However, there is at least one bug here. As one of the failures is due to
an Assert.impossible(....) call executing.
On Tue, Nov 2, 2021 at 1:39 PM John Wass <jw...@gmail.com> wrote:
> Nothing here for 3.2.0.
>
> On Mon, Nov 1, 2021 at 3:42 PM Interrante, John A (GE Research, US) <
> John.Interrante@ge.com> wrote:
>
> > I checked the Daffodil JIRA to see how many blocker, critical, and major
> > issues we have so far:
> >
> > Blocker issues: 0
> > Critical issues: 1
> > https://issues.apache.org/jira/browse/DAFFODIL-2400 - New SAX
> API
> > causes performance degradations
> > Major issues: 132
> > https://issues.apache.org/jira/browse/DAFFODIL-2574 - Cast error
> > when multiplying two unsignedBytes
> > and so on - too many to list individually
> >
> > The critical issue isn't a blocker and in fact hasn't been updated since
> > October 2020 (1 year ago). I see Daffodil's release plan in
> >
> https://cwiki.apache.org/confluence/display/DAFFODIL/Roadmap+for+Upcoming+Releases
> > already suggests reducing the number of JIRA issues between 3.2.0 and
> 3.3.0
> > which sounds like a good plan. Everything planned for 3.2.0 seems to be
> > already checked off.
> >
> > I looked over the instructions in
> > https://cwiki.apache.org/confluence/display/DAFFODIL/Release+Workflow
> > carefully and preparing a release candidate looks doable but still a
> > time-consuming bit of work. Since we already have a volunteer, I'd
> rather
> > wait for another release (thanks Mike).
> >
> > The first step in the Release workflow says to create a [DISCUSS] thread
> > and allow time for discussion before creating the release candidate.
> I've
> > renamed this email's subject to start that formal thread and I'm fine
> with
> > waiting less than 72 hours if we get responses from everyone we know who
> > might be working on anything that they want to release in 3.2.0. I plan
> to
> > make some more changes and improvements to the Daffodil C code generator
> > myself but the changes can go into 3.3.0 since I already build Daffodil
> > from source anyway. Is Steve Lawrence or John Wass working on something
> > that should make it into 3.2.0? Anyone else?
> >
> > John
> >
> > -----Original Message-----
> > From: Mike Beckerle <mb...@apache.org>
> > Sent: Monday, November 1, 2021 1:44 PM
> > To: dev@daffodil.apache.org
> > Subject: EXT: Are we ready to release 3.2.0 Daffodil ?
> >
> > I believe 3.2.0 is functionally complete.
> >
> > Should we prepare a release candidate that we can vote on?
> >
> > We need someone to volunteer to be release manager.
> >
> > I am willing to do so, as I've not done this for several releases.
> > However, if someone else who hasn't done one would like a chance, I will
> > defer to them.
> >
>
Re: [DISCUSS] Are we ready to release 3.2.0 Daffodil ?
Posted by Mike Beckerle <mb...@apache.org>.
I guess I have found issues indicating we are not quite ready for release.
These may be caused by incompatibilities of the tests, but Owl has a
regression test rig where we have every DFDL schema we can get our hands
on.
I ran this for daffodil 3.1.0 and all tests pass.
For daffodil 3.2.0-SNAPSHOT I get failures for these:
[error] (dfdl-vmf-spock / Test / test) sbt.TestsFailedException: Tests
unsuccessful
[error] (dfdl-edifact / Test / test) sbt.TestsFailedException: Tests
unsuccessful
[error] (udf-geoutil / Test / test) sbt.TestsFailedException: Tests
unsuccessful
[error] (dfdl-nacha / Test / test) sbt.TestsFailedException: Tests
unsuccessful
[error] (dfdl-link16 / Test / test) sbt.TestsFailedException: Tests
unsuccessful
[error] (dfdl-examples-hexwords / Test / test) sbt.TestsFailedException:
Tests unsuccessful
I will investigate why this is. From looking at the diagnostics, one of
them at least is due to xmlns="...." changes in expected output vs. actual,
and since there was a change there in 3.2.0 I expect that may be a matter
of adjusting the test case data.
However, there is at least one bug here. As one of the failures is due to
an Assert.impossible(....) call executing.
On Tue, Nov 2, 2021 at 1:39 PM John Wass <jw...@gmail.com> wrote:
> Nothing here for 3.2.0.
>
> On Mon, Nov 1, 2021 at 3:42 PM Interrante, John A (GE Research, US) <
> John.Interrante@ge.com> wrote:
>
> > I checked the Daffodil JIRA to see how many blocker, critical, and major
> > issues we have so far:
> >
> > Blocker issues: 0
> > Critical issues: 1
> > https://issues.apache.org/jira/browse/DAFFODIL-2400 - New SAX
> API
> > causes performance degradations
> > Major issues: 132
> > https://issues.apache.org/jira/browse/DAFFODIL-2574 - Cast error
> > when multiplying two unsignedBytes
> > and so on - too many to list individually
> >
> > The critical issue isn't a blocker and in fact hasn't been updated since
> > October 2020 (1 year ago). I see Daffodil's release plan in
> >
> https://cwiki.apache.org/confluence/display/DAFFODIL/Roadmap+for+Upcoming+Releases
> > already suggests reducing the number of JIRA issues between 3.2.0 and
> 3.3.0
> > which sounds like a good plan. Everything planned for 3.2.0 seems to be
> > already checked off.
> >
> > I looked over the instructions in
> > https://cwiki.apache.org/confluence/display/DAFFODIL/Release+Workflow
> > carefully and preparing a release candidate looks doable but still a
> > time-consuming bit of work. Since we already have a volunteer, I'd
> rather
> > wait for another release (thanks Mike).
> >
> > The first step in the Release workflow says to create a [DISCUSS] thread
> > and allow time for discussion before creating the release candidate.
> I've
> > renamed this email's subject to start that formal thread and I'm fine
> with
> > waiting less than 72 hours if we get responses from everyone we know who
> > might be working on anything that they want to release in 3.2.0. I plan
> to
> > make some more changes and improvements to the Daffodil C code generator
> > myself but the changes can go into 3.3.0 since I already build Daffodil
> > from source anyway. Is Steve Lawrence or John Wass working on something
> > that should make it into 3.2.0? Anyone else?
> >
> > John
> >
> > -----Original Message-----
> > From: Mike Beckerle <mb...@apache.org>
> > Sent: Monday, November 1, 2021 1:44 PM
> > To: dev@daffodil.apache.org
> > Subject: EXT: Are we ready to release 3.2.0 Daffodil ?
> >
> > I believe 3.2.0 is functionally complete.
> >
> > Should we prepare a release candidate that we can vote on?
> >
> > We need someone to volunteer to be release manager.
> >
> > I am willing to do so, as I've not done this for several releases.
> > However, if someone else who hasn't done one would like a chance, I will
> > defer to them.
> >
>
Re: [DISCUSS] Are we ready to release 3.2.0 Daffodil ?
Posted by John Wass <jw...@gmail.com>.
Nothing here for 3.2.0.
On Mon, Nov 1, 2021 at 3:42 PM Interrante, John A (GE Research, US) <
John.Interrante@ge.com> wrote:
> I checked the Daffodil JIRA to see how many blocker, critical, and major
> issues we have so far:
>
> Blocker issues: 0
> Critical issues: 1
> https://issues.apache.org/jira/browse/DAFFODIL-2400 - New SAX API
> causes performance degradations
> Major issues: 132
> https://issues.apache.org/jira/browse/DAFFODIL-2574 - Cast error
> when multiplying two unsignedBytes
> and so on - too many to list individually
>
> The critical issue isn't a blocker and in fact hasn't been updated since
> October 2020 (1 year ago). I see Daffodil's release plan in
> https://cwiki.apache.org/confluence/display/DAFFODIL/Roadmap+for+Upcoming+Releases
> already suggests reducing the number of JIRA issues between 3.2.0 and 3.3.0
> which sounds like a good plan. Everything planned for 3.2.0 seems to be
> already checked off.
>
> I looked over the instructions in
> https://cwiki.apache.org/confluence/display/DAFFODIL/Release+Workflow
> carefully and preparing a release candidate looks doable but still a
> time-consuming bit of work. Since we already have a volunteer, I'd rather
> wait for another release (thanks Mike).
>
> The first step in the Release workflow says to create a [DISCUSS] thread
> and allow time for discussion before creating the release candidate. I've
> renamed this email's subject to start that formal thread and I'm fine with
> waiting less than 72 hours if we get responses from everyone we know who
> might be working on anything that they want to release in 3.2.0. I plan to
> make some more changes and improvements to the Daffodil C code generator
> myself but the changes can go into 3.3.0 since I already build Daffodil
> from source anyway. Is Steve Lawrence or John Wass working on something
> that should make it into 3.2.0? Anyone else?
>
> John
>
> -----Original Message-----
> From: Mike Beckerle <mb...@apache.org>
> Sent: Monday, November 1, 2021 1:44 PM
> To: dev@daffodil.apache.org
> Subject: EXT: Are we ready to release 3.2.0 Daffodil ?
>
> I believe 3.2.0 is functionally complete.
>
> Should we prepare a release candidate that we can vote on?
>
> We need someone to volunteer to be release manager.
>
> I am willing to do so, as I've not done this for several releases.
> However, if someone else who hasn't done one would like a chance, I will
> defer to them.
>