You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@daffodil.apache.org by "Beckerle, Mike" <mb...@owlcyberdefense.com> on 2021/05/03 20:11:14 UTC
draft may board report - due on 12th.
Here's what I have planned for the may board report.
Your feedback is welcome.
## Description:
Implementation of Data Format Description Language (DFDL) used to convert data
from native formats into more easily processed forms such as XML, JSON, or the
structures carried by data-processing fabrics.
## Issues:
There are no issues requiring board attention.
## Membership Data:
Apache Daffodil was founded 2021-02-16 (4 months ago)
There are currently 13 committers and 12 PMC members in this project.
The Committer-to-PMC ratio is roughly 7:6.
Community changes, past quarter:
- No new PMC members (project graduated recently).
- No new committers were added.
## Project Activity:
This is our third board report since graduation to a TLP.
The project continues to move towards its first TLP release which will be 3.1.0.
The major features needed are in place including
* the first version of the C-code generator - a new lightweight,
small-footprint C-code generator that we hope will attract new contributors
* raw validation output
* command-line debugger improvements
A small number (5) of critical bugs remain outstanding 2 of which need
developers to work on them.
## Community Health:
We have 3 new contributors who have expressed interest and started work.
Our hope is of course that they will stay engaged and become committers.
Our stats look healthier this month across the board.
Mike Beckerle | Principal Engineer
[cid:efd57f71-bbee-4ca5-b7e5-01ec853c7eef]
mbeckerle@owlcyberdefense.com<ma...@owlcyberdefense.com>
P +1-781-330-0412
Re: release 3.1.0 critical bugs still outstanding
Posted by Steve Lawrence <sl...@apache.org>.
Yeah, ASF doesn't *require* an isolated installation. This link talks
about securing keys:
https://infra.apache.org/release-signing.html#where
ASF only recommends removable media or isolated installation, but
doesn't require it. As long as your laptop is secure and your key is
properly protected with a good passphrase and permissions, you should be
fine.
On 5/12/21 12:27 PM, Beckerle, Mike wrote:
> Hmmm. I think isolation just means not group or world readable.
>
> On Unix/Linux chmod go-rwx.
> The equivalent on Windows I'm not sure of.
> ________________________________
> From: Interrante, John A (GE Research, US) <Jo...@ge.com>
> Sent: Wednesday, May 12, 2021 12:22 PM
> To: dev@daffodil.apache.org <de...@daffodil.apache.org>
> Subject: release 3.1.0 critical bugs still outstanding
>
> I looked at the instructions for generating an Apache code signing key. I believe I can do everything on my work laptop except for one thing - storing my $GPGHOME/.gnupg directory with my code signing key in encrypted and isolated storage. My work laptop's disk is encrypted (per GE security policy) but it doesn't offer isolation unless there's a clever way of enforcing isolation?
>
> I would need to plug a separate USB device into my work laptop and it would have to be a Yubico YubiKey 5 or a Yubico YubiKey 5C since GE's data guardian software doesn't allow most USB devices to work (YubiKey devices are an exception).
>
> Do you know of anyone who uses a YukiKey device to generate and/or store their GPG code signing key and how to use a YubiKey device in the release signing workflow? I found those articles but they don't say much about using the YubiKey in a release signing workflow similar to ours...
>
> https://support.yubico.com/hc/en-us/articles/360016614840-Code-Signing-with-the-YubiKey-on-Windows
> https://ocramius.github.io/blog/yubikey-for-ssh-gpg-git-and-local-login/
> https://eclipsesource.com/blogs/2016/11/25/yubikey-code-signing-with-a-smart-card/
>
> John
>
> -----Original Message-----
> From: Steve Lawrence <sl...@apache.org>
> Sent: Tuesday, May 11, 2021 10:23 AM
> To: dev@daffodil.apache.org
> Subject: EXT: Re: release 3.1.0 critical bugs still outstanding
>
> Agreed, I think the CLI change is worth getting in for 3.1.0. Shouldn't take too long to get it merged.
>
> I'll volunteer to be release manager 3.1.0. Probably for the best since I'm probably the most familiar with the process/script, and it's possible something could be broken with this being the first non-incubator release.
>
> Though, I would recommend that other PMC/committers go through the process to create and publish a gpg key so that when it does come time to do a release, at least that part is out of the way.
>
>
> On 5/11/21 10:10 AM, Interrante, John A (GE Research, US) wrote:
>> Yes, we will be in good shape for the 3.1.0 release after we update the CLI help information, which I'd like to complete today. We've also got at least 4-5 days to add validation.md and anything else we want to the daffodil-site before the earliest possible official release announcement could go out.
>>
>> I'm taking this Thursday and Friday off which is a consideration in volunteering to be the release manager. I'd have to generate a signing key, add its public part to the KEYS file, commit it to the Daffodil release distribution SVN repository, send the fingerprint to the Apache key server, build a release candidate, and start a vote before I go out of town on Thursday. Steve, perhaps I should wait for the following release unless you think I'd be able to do all these steps quickly within a few hours.
>>
>> John
>>
>> -----Original Message-----
>> From: Beckerle, Mike <mb...@owlcyberdefense.com>
>> Sent: Monday, May 10, 2021 2:43 PM
>> To: dev@daffodil.apache.org
>> Subject: EXT: Re: release 3.1.0 critical bugs still outstanding
>>
>> I agree we should do the release.
>>
>> I am in the thick of debugging DAFFODIL-1422, but there's a bunch of refactoring here, 30 files touched, changes in diagnostic behavior, etc. Perhaps best to put it off until after the 3.1.0 release.
>>
>>
>>
>>
>> ________________________________
>> From: Steve Lawrence <sl...@apache.org>
>> Sent: Monday, May 10, 2021 1:59 PM
>> To: dev@daffodil.apache.org <de...@daffodil.apache.org>
>> Subject: Re: release 3.1.0 critical bugs still outstanding
>>
>> We have fixed the later two mentioned issues. The current list of critical issues is now:
>>
>> DAFFODIL-1422: disallow doctype decls in all XML & XSD that we read in
>> DAFFODIL-2400: New SAX API causes performance degredations.
>> DAFFODIL-2473: Need bit-wise AND, OR, NOT, and shift operations
>>
>> I agree the the first two can likely be postponed without issue. The last one doesn't even seem critical to me, unless there are very important formats that require this functions that I'm not aware of. I suggest we also postpone that ticket as well.
>>
>> If others agree, I think we are ready for the 3.1.0 release?
>>
>> Does any want to volunteer to be the release manager? I've done it a handful of times so don't mind, but it might be good to get others experience depending on availability. By this point, the workflow is pretty well documented here:
>>
>> https://cwiki.apache.org/confluence/display/DAFFODIL/Release+Workflow
>>
>>
>> On 5/3/21 5:25 PM, Beckerle, Mike wrote:
>>> Of the 4 remaining "critical bugs or improvements" I think we should postpone and release note these first two:
>>>
>>> *
>>>
>>> * Improvement: https://issues.apache.org/jira/browse/DAFFODIL-2400 - New SAX API causes performance degradations.
>>> * It is a mystery why the SAX API is slower. The whole point of SAX is lighter weight.
>>> * Improvement: https://issues.apache.org/jira/browse/DAFFODIL-1422 - disallow doctype decls in all XML and XSD we read in.
>>> * Assigned to Mike Beckerle. Unlikely to be finished in time for release 3.1.0. Substantial code refactoring to do this right.
>>>
>>> These next two seem rather important to fix:
>>>
>>> * Bug: https://issues.apache.org/jira/browse/DAFFODIL-2183 - Unparse complex nilled element fails
>>> * There are data formats where I advised people a best-practice is to use complex nilled elements to model a specific situation.
>>> * Bug: https://issues.apache.org/jira/browse/DAFFODIL-2399 - error diagnostics output even though there is an infoset
>>> * This one is assigned to Steve Lawrence
>>> * Seems rather important. Was a user-reported issue I believe.
>>>
>>> The 5th critical ticket is for a new feature (bitwise and/or/xor, and shift functions), so we can postpone that one.
>>>
>>> So only DAFFODIL-2183 really needs someone to take it on.
>>>
>>> ________________________________
>>> From: Interrante, John A (GE Research, US) <Jo...@ge.com>
>>> Sent: Monday, May 3, 2021 4:57 PM
>>> To: dev@daffodil.apache.org <de...@daffodil.apache.org>...
>>>
>>> Are any of the 5 critical bugs (2 of which need developers to work on them) going to hold up the 3.1.0 release? The report doesn't say so, but I had the impression you'd added the remaining critical bugs (which were unlikely to be hit by people) to the 3.1.0 release notes so that the 3.1.0 release still could go out. If any critical bugs are holding up 3.1.0, please post links to them so we can help if we have time.
>>>
>>> John
>>>
>>>
>>>
>>
>
>
Re: release 3.1.0 critical bugs still outstanding
Posted by "Beckerle, Mike" <mb...@owlcyberdefense.com>.
Hmmm. I think isolation just means not group or world readable.
On Unix/Linux chmod go-rwx.
The equivalent on Windows I'm not sure of.
________________________________
From: Interrante, John A (GE Research, US) <Jo...@ge.com>
Sent: Wednesday, May 12, 2021 12:22 PM
To: dev@daffodil.apache.org <de...@daffodil.apache.org>
Subject: release 3.1.0 critical bugs still outstanding
I looked at the instructions for generating an Apache code signing key. I believe I can do everything on my work laptop except for one thing - storing my $GPGHOME/.gnupg directory with my code signing key in encrypted and isolated storage. My work laptop's disk is encrypted (per GE security policy) but it doesn't offer isolation unless there's a clever way of enforcing isolation?
I would need to plug a separate USB device into my work laptop and it would have to be a Yubico YubiKey 5 or a Yubico YubiKey 5C since GE's data guardian software doesn't allow most USB devices to work (YubiKey devices are an exception).
Do you know of anyone who uses a YukiKey device to generate and/or store their GPG code signing key and how to use a YubiKey device in the release signing workflow? I found those articles but they don't say much about using the YubiKey in a release signing workflow similar to ours...
https://support.yubico.com/hc/en-us/articles/360016614840-Code-Signing-with-the-YubiKey-on-Windows
https://ocramius.github.io/blog/yubikey-for-ssh-gpg-git-and-local-login/
https://eclipsesource.com/blogs/2016/11/25/yubikey-code-signing-with-a-smart-card/
John
-----Original Message-----
From: Steve Lawrence <sl...@apache.org>
Sent: Tuesday, May 11, 2021 10:23 AM
To: dev@daffodil.apache.org
Subject: EXT: Re: release 3.1.0 critical bugs still outstanding
Agreed, I think the CLI change is worth getting in for 3.1.0. Shouldn't take too long to get it merged.
I'll volunteer to be release manager 3.1.0. Probably for the best since I'm probably the most familiar with the process/script, and it's possible something could be broken with this being the first non-incubator release.
Though, I would recommend that other PMC/committers go through the process to create and publish a gpg key so that when it does come time to do a release, at least that part is out of the way.
On 5/11/21 10:10 AM, Interrante, John A (GE Research, US) wrote:
> Yes, we will be in good shape for the 3.1.0 release after we update the CLI help information, which I'd like to complete today. We've also got at least 4-5 days to add validation.md and anything else we want to the daffodil-site before the earliest possible official release announcement could go out.
>
> I'm taking this Thursday and Friday off which is a consideration in volunteering to be the release manager. I'd have to generate a signing key, add its public part to the KEYS file, commit it to the Daffodil release distribution SVN repository, send the fingerprint to the Apache key server, build a release candidate, and start a vote before I go out of town on Thursday. Steve, perhaps I should wait for the following release unless you think I'd be able to do all these steps quickly within a few hours.
>
> John
>
> -----Original Message-----
> From: Beckerle, Mike <mb...@owlcyberdefense.com>
> Sent: Monday, May 10, 2021 2:43 PM
> To: dev@daffodil.apache.org
> Subject: EXT: Re: release 3.1.0 critical bugs still outstanding
>
> I agree we should do the release.
>
> I am in the thick of debugging DAFFODIL-1422, but there's a bunch of refactoring here, 30 files touched, changes in diagnostic behavior, etc. Perhaps best to put it off until after the 3.1.0 release.
>
>
>
>
> ________________________________
> From: Steve Lawrence <sl...@apache.org>
> Sent: Monday, May 10, 2021 1:59 PM
> To: dev@daffodil.apache.org <de...@daffodil.apache.org>
> Subject: Re: release 3.1.0 critical bugs still outstanding
>
> We have fixed the later two mentioned issues. The current list of critical issues is now:
>
> DAFFODIL-1422: disallow doctype decls in all XML & XSD that we read in
> DAFFODIL-2400: New SAX API causes performance degredations.
> DAFFODIL-2473: Need bit-wise AND, OR, NOT, and shift operations
>
> I agree the the first two can likely be postponed without issue. The last one doesn't even seem critical to me, unless there are very important formats that require this functions that I'm not aware of. I suggest we also postpone that ticket as well.
>
> If others agree, I think we are ready for the 3.1.0 release?
>
> Does any want to volunteer to be the release manager? I've done it a handful of times so don't mind, but it might be good to get others experience depending on availability. By this point, the workflow is pretty well documented here:
>
> https://cwiki.apache.org/confluence/display/DAFFODIL/Release+Workflow
>
>
> On 5/3/21 5:25 PM, Beckerle, Mike wrote:
>> Of the 4 remaining "critical bugs or improvements" I think we should postpone and release note these first two:
>>
>> *
>>
>> * Improvement: https://issues.apache.org/jira/browse/DAFFODIL-2400 - New SAX API causes performance degradations.
>> * It is a mystery why the SAX API is slower. The whole point of SAX is lighter weight.
>> * Improvement: https://issues.apache.org/jira/browse/DAFFODIL-1422 - disallow doctype decls in all XML and XSD we read in.
>> * Assigned to Mike Beckerle. Unlikely to be finished in time for release 3.1.0. Substantial code refactoring to do this right.
>>
>> These next two seem rather important to fix:
>>
>> * Bug: https://issues.apache.org/jira/browse/DAFFODIL-2183 - Unparse complex nilled element fails
>> * There are data formats where I advised people a best-practice is to use complex nilled elements to model a specific situation.
>> * Bug: https://issues.apache.org/jira/browse/DAFFODIL-2399 - error diagnostics output even though there is an infoset
>> * This one is assigned to Steve Lawrence
>> * Seems rather important. Was a user-reported issue I believe.
>>
>> The 5th critical ticket is for a new feature (bitwise and/or/xor, and shift functions), so we can postpone that one.
>>
>> So only DAFFODIL-2183 really needs someone to take it on.
>>
>> ________________________________
>> From: Interrante, John A (GE Research, US) <Jo...@ge.com>
>> Sent: Monday, May 3, 2021 4:57 PM
>> To: dev@daffodil.apache.org <de...@daffodil.apache.org>...
>>
>> Are any of the 5 critical bugs (2 of which need developers to work on them) going to hold up the 3.1.0 release? The report doesn't say so, but I had the impression you'd added the remaining critical bugs (which were unlikely to be hit by people) to the 3.1.0 release notes so that the 3.1.0 release still could go out. If any critical bugs are holding up 3.1.0, please post links to them so we can help if we have time.
>>
>> John
>>
>>
>>
>
release 3.1.0 critical bugs still outstanding
Posted by "Interrante, John A (GE Research, US)" <Jo...@ge.com>.
I looked at the instructions for generating an Apache code signing key. I believe I can do everything on my work laptop except for one thing - storing my $GPGHOME/.gnupg directory with my code signing key in encrypted and isolated storage. My work laptop's disk is encrypted (per GE security policy) but it doesn't offer isolation unless there's a clever way of enforcing isolation?
I would need to plug a separate USB device into my work laptop and it would have to be a Yubico YubiKey 5 or a Yubico YubiKey 5C since GE's data guardian software doesn't allow most USB devices to work (YubiKey devices are an exception).
Do you know of anyone who uses a YukiKey device to generate and/or store their GPG code signing key and how to use a YubiKey device in the release signing workflow? I found those articles but they don't say much about using the YubiKey in a release signing workflow similar to ours...
https://support.yubico.com/hc/en-us/articles/360016614840-Code-Signing-with-the-YubiKey-on-Windows
https://ocramius.github.io/blog/yubikey-for-ssh-gpg-git-and-local-login/
https://eclipsesource.com/blogs/2016/11/25/yubikey-code-signing-with-a-smart-card/
John
-----Original Message-----
From: Steve Lawrence <sl...@apache.org>
Sent: Tuesday, May 11, 2021 10:23 AM
To: dev@daffodil.apache.org
Subject: EXT: Re: release 3.1.0 critical bugs still outstanding
Agreed, I think the CLI change is worth getting in for 3.1.0. Shouldn't take too long to get it merged.
I'll volunteer to be release manager 3.1.0. Probably for the best since I'm probably the most familiar with the process/script, and it's possible something could be broken with this being the first non-incubator release.
Though, I would recommend that other PMC/committers go through the process to create and publish a gpg key so that when it does come time to do a release, at least that part is out of the way.
On 5/11/21 10:10 AM, Interrante, John A (GE Research, US) wrote:
> Yes, we will be in good shape for the 3.1.0 release after we update the CLI help information, which I'd like to complete today. We've also got at least 4-5 days to add validation.md and anything else we want to the daffodil-site before the earliest possible official release announcement could go out.
>
> I'm taking this Thursday and Friday off which is a consideration in volunteering to be the release manager. I'd have to generate a signing key, add its public part to the KEYS file, commit it to the Daffodil release distribution SVN repository, send the fingerprint to the Apache key server, build a release candidate, and start a vote before I go out of town on Thursday. Steve, perhaps I should wait for the following release unless you think I'd be able to do all these steps quickly within a few hours.
>
> John
>
> -----Original Message-----
> From: Beckerle, Mike <mb...@owlcyberdefense.com>
> Sent: Monday, May 10, 2021 2:43 PM
> To: dev@daffodil.apache.org
> Subject: EXT: Re: release 3.1.0 critical bugs still outstanding
>
> I agree we should do the release.
>
> I am in the thick of debugging DAFFODIL-1422, but there's a bunch of refactoring here, 30 files touched, changes in diagnostic behavior, etc. Perhaps best to put it off until after the 3.1.0 release.
>
>
>
>
> ________________________________
> From: Steve Lawrence <sl...@apache.org>
> Sent: Monday, May 10, 2021 1:59 PM
> To: dev@daffodil.apache.org <de...@daffodil.apache.org>
> Subject: Re: release 3.1.0 critical bugs still outstanding
>
> We have fixed the later two mentioned issues. The current list of critical issues is now:
>
> DAFFODIL-1422: disallow doctype decls in all XML & XSD that we read in
> DAFFODIL-2400: New SAX API causes performance degredations.
> DAFFODIL-2473: Need bit-wise AND, OR, NOT, and shift operations
>
> I agree the the first two can likely be postponed without issue. The last one doesn't even seem critical to me, unless there are very important formats that require this functions that I'm not aware of. I suggest we also postpone that ticket as well.
>
> If others agree, I think we are ready for the 3.1.0 release?
>
> Does any want to volunteer to be the release manager? I've done it a handful of times so don't mind, but it might be good to get others experience depending on availability. By this point, the workflow is pretty well documented here:
>
> https://cwiki.apache.org/confluence/display/DAFFODIL/Release+Workflow
>
>
> On 5/3/21 5:25 PM, Beckerle, Mike wrote:
>> Of the 4 remaining "critical bugs or improvements" I think we should postpone and release note these first two:
>>
>> *
>>
>> * Improvement: https://issues.apache.org/jira/browse/DAFFODIL-2400 - New SAX API causes performance degradations.
>> * It is a mystery why the SAX API is slower. The whole point of SAX is lighter weight.
>> * Improvement: https://issues.apache.org/jira/browse/DAFFODIL-1422 - disallow doctype decls in all XML and XSD we read in.
>> * Assigned to Mike Beckerle. Unlikely to be finished in time for release 3.1.0. Substantial code refactoring to do this right.
>>
>> These next two seem rather important to fix:
>>
>> * Bug: https://issues.apache.org/jira/browse/DAFFODIL-2183 - Unparse complex nilled element fails
>> * There are data formats where I advised people a best-practice is to use complex nilled elements to model a specific situation.
>> * Bug: https://issues.apache.org/jira/browse/DAFFODIL-2399 - error diagnostics output even though there is an infoset
>> * This one is assigned to Steve Lawrence
>> * Seems rather important. Was a user-reported issue I believe.
>>
>> The 5th critical ticket is for a new feature (bitwise and/or/xor, and shift functions), so we can postpone that one.
>>
>> So only DAFFODIL-2183 really needs someone to take it on.
>>
>> ________________________________
>> From: Interrante, John A (GE Research, US) <Jo...@ge.com>
>> Sent: Monday, May 3, 2021 4:57 PM
>> To: dev@daffodil.apache.org <de...@daffodil.apache.org>...
>>
>> Are any of the 5 critical bugs (2 of which need developers to work on them) going to hold up the 3.1.0 release? The report doesn't say so, but I had the impression you'd added the remaining critical bugs (which were unlikely to be hit by people) to the 3.1.0 release notes so that the 3.1.0 release still could go out. If any critical bugs are holding up 3.1.0, please post links to them so we can help if we have time.
>>
>> John
>>
>>
>>
>
Re: release 3.1.0 critical bugs still outstanding
Posted by Steve Lawrence <sl...@apache.org>.
Agreed, I think the CLI change is worth getting in for 3.1.0. Shouldn't
take too long to get it merged.
I'll volunteer to be release manager 3.1.0. Probably for the best since
I'm probably the most familiar with the process/script, and it's
possible something could be broken with this being the first
non-incubator release.
Though, I would recommend that other PMC/committers go through the
process to create and publish a gpg key so that when it does come time
to do a release, at least that part is out of the way.
On 5/11/21 10:10 AM, Interrante, John A (GE Research, US) wrote:
> Yes, we will be in good shape for the 3.1.0 release after we update the CLI help information, which I'd like to complete today. We've also got at least 4-5 days to add validation.md and anything else we want to the daffodil-site before the earliest possible official release announcement could go out.
>
> I'm taking this Thursday and Friday off which is a consideration in volunteering to be the release manager. I'd have to generate a signing key, add its public part to the KEYS file, commit it to the Daffodil release distribution SVN repository, send the fingerprint to the Apache key server, build a release candidate, and start a vote before I go out of town on Thursday. Steve, perhaps I should wait for the following release unless you think I'd be able to do all these steps quickly within a few hours.
>
> John
>
> -----Original Message-----
> From: Beckerle, Mike <mb...@owlcyberdefense.com>
> Sent: Monday, May 10, 2021 2:43 PM
> To: dev@daffodil.apache.org
> Subject: EXT: Re: release 3.1.0 critical bugs still outstanding
>
> I agree we should do the release.
>
> I am in the thick of debugging DAFFODIL-1422, but there's a bunch of refactoring here, 30 files touched, changes in diagnostic behavior, etc. Perhaps best to put it off until after the 3.1.0 release.
>
>
>
>
> ________________________________
> From: Steve Lawrence <sl...@apache.org>
> Sent: Monday, May 10, 2021 1:59 PM
> To: dev@daffodil.apache.org <de...@daffodil.apache.org>
> Subject: Re: release 3.1.0 critical bugs still outstanding
>
> We have fixed the later two mentioned issues. The current list of critical issues is now:
>
> DAFFODIL-1422: disallow doctype decls in all XML & XSD that we read in
> DAFFODIL-2400: New SAX API causes performance degredations.
> DAFFODIL-2473: Need bit-wise AND, OR, NOT, and shift operations
>
> I agree the the first two can likely be postponed without issue. The last one doesn't even seem critical to me, unless there are very important formats that require this functions that I'm not aware of. I suggest we also postpone that ticket as well.
>
> If others agree, I think we are ready for the 3.1.0 release?
>
> Does any want to volunteer to be the release manager? I've done it a handful of times so don't mind, but it might be good to get others experience depending on availability. By this point, the workflow is pretty well documented here:
>
> https://cwiki.apache.org/confluence/display/DAFFODIL/Release+Workflow
>
>
> On 5/3/21 5:25 PM, Beckerle, Mike wrote:
>> Of the 4 remaining "critical bugs or improvements" I think we should postpone and release note these first two:
>>
>> *
>>
>> * Improvement: https://issues.apache.org/jira/browse/DAFFODIL-2400 - New SAX API causes performance degradations.
>> * It is a mystery why the SAX API is slower. The whole point of SAX is lighter weight.
>> * Improvement: https://issues.apache.org/jira/browse/DAFFODIL-1422 - disallow doctype decls in all XML and XSD we read in.
>> * Assigned to Mike Beckerle. Unlikely to be finished in time for release 3.1.0. Substantial code refactoring to do this right.
>>
>> These next two seem rather important to fix:
>>
>> * Bug: https://issues.apache.org/jira/browse/DAFFODIL-2183 - Unparse complex nilled element fails
>> * There are data formats where I advised people a best-practice is to use complex nilled elements to model a specific situation.
>> * Bug: https://issues.apache.org/jira/browse/DAFFODIL-2399 - error diagnostics output even though there is an infoset
>> * This one is assigned to Steve Lawrence
>> * Seems rather important. Was a user-reported issue I believe.
>>
>> The 5th critical ticket is for a new feature (bitwise and/or/xor, and shift functions), so we can postpone that one.
>>
>> So only DAFFODIL-2183 really needs someone to take it on.
>>
>> ________________________________
>> From: Interrante, John A (GE Research, US) <Jo...@ge.com>
>> Sent: Monday, May 3, 2021 4:57 PM
>> To: dev@daffodil.apache.org <de...@daffodil.apache.org>...
>>
>> Are any of the 5 critical bugs (2 of which need developers to work on them) going to hold up the 3.1.0 release? The report doesn't say so, but I had the impression you'd added the remaining critical bugs (which were unlikely to be hit by people) to the 3.1.0 release notes so that the 3.1.0 release still could go out. If any critical bugs are holding up 3.1.0, please post links to them so we can help if we have time.
>>
>> John
>>
>>
>>
>
RE: release 3.1.0 critical bugs still outstanding
Posted by "Interrante, John A (GE Research, US)" <Jo...@ge.com>.
Yes, we will be in good shape for the 3.1.0 release after we update the CLI help information, which I'd like to complete today. We've also got at least 4-5 days to add validation.md and anything else we want to the daffodil-site before the earliest possible official release announcement could go out.
I'm taking this Thursday and Friday off which is a consideration in volunteering to be the release manager. I'd have to generate a signing key, add its public part to the KEYS file, commit it to the Daffodil release distribution SVN repository, send the fingerprint to the Apache key server, build a release candidate, and start a vote before I go out of town on Thursday. Steve, perhaps I should wait for the following release unless you think I'd be able to do all these steps quickly within a few hours.
John
-----Original Message-----
From: Beckerle, Mike <mb...@owlcyberdefense.com>
Sent: Monday, May 10, 2021 2:43 PM
To: dev@daffodil.apache.org
Subject: EXT: Re: release 3.1.0 critical bugs still outstanding
I agree we should do the release.
I am in the thick of debugging DAFFODIL-1422, but there's a bunch of refactoring here, 30 files touched, changes in diagnostic behavior, etc. Perhaps best to put it off until after the 3.1.0 release.
________________________________
From: Steve Lawrence <sl...@apache.org>
Sent: Monday, May 10, 2021 1:59 PM
To: dev@daffodil.apache.org <de...@daffodil.apache.org>
Subject: Re: release 3.1.0 critical bugs still outstanding
We have fixed the later two mentioned issues. The current list of critical issues is now:
DAFFODIL-1422: disallow doctype decls in all XML & XSD that we read in
DAFFODIL-2400: New SAX API causes performance degredations.
DAFFODIL-2473: Need bit-wise AND, OR, NOT, and shift operations
I agree the the first two can likely be postponed without issue. The last one doesn't even seem critical to me, unless there are very important formats that require this functions that I'm not aware of. I suggest we also postpone that ticket as well.
If others agree, I think we are ready for the 3.1.0 release?
Does any want to volunteer to be the release manager? I've done it a handful of times so don't mind, but it might be good to get others experience depending on availability. By this point, the workflow is pretty well documented here:
https://cwiki.apache.org/confluence/display/DAFFODIL/Release+Workflow
On 5/3/21 5:25 PM, Beckerle, Mike wrote:
> Of the 4 remaining "critical bugs or improvements" I think we should postpone and release note these first two:
>
> *
>
> * Improvement: https://issues.apache.org/jira/browse/DAFFODIL-2400 - New SAX API causes performance degradations.
> * It is a mystery why the SAX API is slower. The whole point of SAX is lighter weight.
> * Improvement: https://issues.apache.org/jira/browse/DAFFODIL-1422 - disallow doctype decls in all XML and XSD we read in.
> * Assigned to Mike Beckerle. Unlikely to be finished in time for release 3.1.0. Substantial code refactoring to do this right.
>
> These next two seem rather important to fix:
>
> * Bug: https://issues.apache.org/jira/browse/DAFFODIL-2183 - Unparse complex nilled element fails
> * There are data formats where I advised people a best-practice is to use complex nilled elements to model a specific situation.
> * Bug: https://issues.apache.org/jira/browse/DAFFODIL-2399 - error diagnostics output even though there is an infoset
> * This one is assigned to Steve Lawrence
> * Seems rather important. Was a user-reported issue I believe.
>
> The 5th critical ticket is for a new feature (bitwise and/or/xor, and shift functions), so we can postpone that one.
>
> So only DAFFODIL-2183 really needs someone to take it on.
>
> ________________________________
> From: Interrante, John A (GE Research, US) <Jo...@ge.com>
> Sent: Monday, May 3, 2021 4:57 PM
> To: dev@daffodil.apache.org <de...@daffodil.apache.org>...
>
> Are any of the 5 critical bugs (2 of which need developers to work on them) going to hold up the 3.1.0 release? The report doesn't say so, but I had the impression you'd added the remaining critical bugs (which were unlikely to be hit by people) to the 3.1.0 release notes so that the 3.1.0 release still could go out. If any critical bugs are holding up 3.1.0, please post links to them so we can help if we have time.
>
> John
>
>
>
Re: release 3.1.0 critical bugs still outstanding
Posted by "Beckerle, Mike" <mb...@owlcyberdefense.com>.
I agree we should do the release.
I am in the thick of debugging DAFFODIL-1422, but there's a bunch of refactoring here, 30 files touched, changes in diagnostic behavior, etc. Perhaps best to put it off until after the 3.1.0 release.
________________________________
From: Steve Lawrence <sl...@apache.org>
Sent: Monday, May 10, 2021 1:59 PM
To: dev@daffodil.apache.org <de...@daffodil.apache.org>
Subject: Re: release 3.1.0 critical bugs still outstanding
We have fixed the later two mentioned issues. The current list of
critical issues is now:
DAFFODIL-1422: disallow doctype decls in all XML & XSD that we read in
DAFFODIL-2400: New SAX API causes performance degredations.
DAFFODIL-2473: Need bit-wise AND, OR, NOT, and shift operations
I agree the the first two can likely be postponed without issue. The
last one doesn't even seem critical to me, unless there are very
important formats that require this functions that I'm not aware of. I
suggest we also postpone that ticket as well.
If others agree, I think we are ready for the 3.1.0 release?
Does any want to volunteer to be the release manager? I've done it a
handful of times so don't mind, but it might be good to get others
experience depending on availability. By this point, the workflow is
pretty well documented here:
https://cwiki.apache.org/confluence/display/DAFFODIL/Release+Workflow
On 5/3/21 5:25 PM, Beckerle, Mike wrote:
> Of the 4 remaining "critical bugs or improvements" I think we should postpone and release note these first two:
>
> *
>
> * Improvement: https://issues.apache.org/jira/browse/DAFFODIL-2400 - New SAX API causes performance degradations.
> * It is a mystery why the SAX API is slower. The whole point of SAX is lighter weight.
> * Improvement: https://issues.apache.org/jira/browse/DAFFODIL-1422 - disallow doctype decls in all XML and XSD we read in.
> * Assigned to Mike Beckerle. Unlikely to be finished in time for release 3.1.0. Substantial code refactoring to do this right.
>
> These next two seem rather important to fix:
>
> * Bug: https://issues.apache.org/jira/browse/DAFFODIL-2183 - Unparse complex nilled element fails
> * There are data formats where I advised people a best-practice is to use complex nilled elements to model a specific situation.
> * Bug: https://issues.apache.org/jira/browse/DAFFODIL-2399 - error diagnostics output even though there is an infoset
> * This one is assigned to Steve Lawrence
> * Seems rather important. Was a user-reported issue I believe.
>
> The 5th critical ticket is for a new feature (bitwise and/or/xor, and shift functions), so we can postpone that one.
>
> So only DAFFODIL-2183 really needs someone to take it on.
>
> ________________________________
> From: Interrante, John A (GE Research, US) <Jo...@ge.com>
> Sent: Monday, May 3, 2021 4:57 PM
> To: dev@daffodil.apache.org <de...@daffodil.apache.org>...
>
> Are any of the 5 critical bugs (2 of which need developers to work on them) going to hold up the 3.1.0 release? The report doesn't say so, but I had the impression you'd added the remaining critical bugs (which were unlikely to be hit by people) to the 3.1.0 release notes so that the 3.1.0 release still could go out. If any critical bugs are holding up 3.1.0, please post links to them so we can help if we have time.
>
> John
>
>
>
Re: release 3.1.0 critical bugs still outstanding
Posted by Steve Lawrence <sl...@apache.org>.
We have fixed the later two mentioned issues. The current list of
critical issues is now:
DAFFODIL-1422: disallow doctype decls in all XML & XSD that we read in
DAFFODIL-2400: New SAX API causes performance degredations.
DAFFODIL-2473: Need bit-wise AND, OR, NOT, and shift operations
I agree the the first two can likely be postponed without issue. The
last one doesn't even seem critical to me, unless there are very
important formats that require this functions that I'm not aware of. I
suggest we also postpone that ticket as well.
If others agree, I think we are ready for the 3.1.0 release?
Does any want to volunteer to be the release manager? I've done it a
handful of times so don't mind, but it might be good to get others
experience depending on availability. By this point, the workflow is
pretty well documented here:
https://cwiki.apache.org/confluence/display/DAFFODIL/Release+Workflow
On 5/3/21 5:25 PM, Beckerle, Mike wrote:
> Of the 4 remaining "critical bugs or improvements" I think we should postpone and release note these first two:
>
> *
>
> * Improvement: https://issues.apache.org/jira/browse/DAFFODIL-2400 - New SAX API causes performance degradations.
> * It is a mystery why the SAX API is slower. The whole point of SAX is lighter weight.
> * Improvement: https://issues.apache.org/jira/browse/DAFFODIL-1422 - disallow doctype decls in all XML and XSD we read in.
> * Assigned to Mike Beckerle. Unlikely to be finished in time for release 3.1.0. Substantial code refactoring to do this right.
>
> These next two seem rather important to fix:
>
> * Bug: https://issues.apache.org/jira/browse/DAFFODIL-2183 - Unparse complex nilled element fails
> * There are data formats where I advised people a best-practice is to use complex nilled elements to model a specific situation.
> * Bug: https://issues.apache.org/jira/browse/DAFFODIL-2399 - error diagnostics output even though there is an infoset
> * This one is assigned to Steve Lawrence
> * Seems rather important. Was a user-reported issue I believe.
>
> The 5th critical ticket is for a new feature (bitwise and/or/xor, and shift functions), so we can postpone that one.
>
> So only DAFFODIL-2183 really needs someone to take it on.
>
> ________________________________
> From: Interrante, John A (GE Research, US) <Jo...@ge.com>
> Sent: Monday, May 3, 2021 4:57 PM
> To: dev@daffodil.apache.org <de...@daffodil.apache.org>...
>
> Are any of the 5 critical bugs (2 of which need developers to work on them) going to hold up the 3.1.0 release? The report doesn't say so, but I had the impression you'd added the remaining critical bugs (which were unlikely to be hit by people) to the 3.1.0 release notes so that the 3.1.0 release still could go out. If any critical bugs are holding up 3.1.0, please post links to them so we can help if we have time.
>
> John
>
>
>
release 3.1.0 critical bugs still outstanding
Posted by "Beckerle, Mike" <mb...@owlcyberdefense.com>.
Of the 4 remaining "critical bugs or improvements" I think we should postpone and release note these first two:
*
* Improvement: https://issues.apache.org/jira/browse/DAFFODIL-2400 - New SAX API causes performance degradations.
* It is a mystery why the SAX API is slower. The whole point of SAX is lighter weight.
* Improvement: https://issues.apache.org/jira/browse/DAFFODIL-1422 - disallow doctype decls in all XML and XSD we read in.
* Assigned to Mike Beckerle. Unlikely to be finished in time for release 3.1.0. Substantial code refactoring to do this right.
These next two seem rather important to fix:
* Bug: https://issues.apache.org/jira/browse/DAFFODIL-2183 - Unparse complex nilled element fails
* There are data formats where I advised people a best-practice is to use complex nilled elements to model a specific situation.
* Bug: https://issues.apache.org/jira/browse/DAFFODIL-2399 - error diagnostics output even though there is an infoset
* This one is assigned to Steve Lawrence
* Seems rather important. Was a user-reported issue I believe.
The 5th critical ticket is for a new feature (bitwise and/or/xor, and shift functions), so we can postpone that one.
So only DAFFODIL-2183 really needs someone to take it on.
________________________________
From: Interrante, John A (GE Research, US) <Jo...@ge.com>
Sent: Monday, May 3, 2021 4:57 PM
To: dev@daffodil.apache.org <de...@daffodil.apache.org>...
Are any of the 5 critical bugs (2 of which need developers to work on them) going to hold up the 3.1.0 release? The report doesn't say so, but I had the impression you'd added the remaining critical bugs (which were unlikely to be hit by people) to the 3.1.0 release notes so that the 3.1.0 release still could go out. If any critical bugs are holding up 3.1.0, please post links to them so we can help if we have time.
John
RE: draft may board report - due on 12th.
Posted by "Interrante, John A (GE Research, US)" <Jo...@ge.com>.
Report looks good.
Are any of the 5 critical bugs (2 of which need developers to work on them) going to hold up the 3.1.0 release? The report doesn't say so, but I had the impression you'd added the remaining critical bugs (which were unlikely to be hit by people) to the 3.1.0 release notes so that the 3.1.0 release still could go out. If any critical bugs are holding up 3.1.0, please post links to them so we can help if we have time.
John
From: Beckerle, Mike <mb...@owlcyberdefense.com>
Sent: Monday, May 3, 2021 4:11 PM
To: dev@daffodil.apache.org
Subject: EXT: draft may board report - due on 12th.
Here's what I have planned for the may board report.
Your feedback is welcome.
## Description:
Implementation of Data Format Description Language (DFDL) used to convert data
from native formats into more easily processed forms such as XML, JSON, or the
structures carried by data-processing fabrics.
## Issues:
There are no issues requiring board attention.
## Membership Data:
Apache Daffodil was founded 2021-02-16 (4 months ago)
There are currently 13 committers and 12 PMC members in this project.
The Committer-to-PMC ratio is roughly 7:6.
Community changes, past quarter:
- No new PMC members (project graduated recently).
- No new committers were added.
## Project Activity:
This is our third board report since graduation to a TLP.
The project continues to move towards its first TLP release which will be 3.1.0.
The major features needed are in place including
* the first version of the C-code generator - a new lightweight,
small-footprint C-code generator that we hope will attract new contributors
* raw validation output
* command-line debugger improvements
A small number (5) of critical bugs remain outstanding 2 of which need
developers to work on them.
## Community Health:
We have 3 new contributors who have expressed interest and started work.
Our hope is of course that they will stay engaged and become committers.
Our stats look healthier this month across the board.
Mike Beckerle | Principal Engineer
[cid:efd57f71-bbee-4ca5-b7e5-01ec853c7eef]
mbeckerle@owlcyberdefense.com<ma...@owlcyberdefense.com>
P +1-781-330-0412