You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@daffodil.apache.org by "Beckerle, Mike" <mb...@tresys.com> on 2019/12/02 18:29:48 UTC
2.5.0 branch performance looks problematic currently (2019-12-02)
Tresys ran some performance tests on 2.5.0 branch recently.
Nacha is faster relative to baseline (baseline is release 2.0.0)
Pretty much everything else looks substantially slower.
Compared to the last prior performance report I have (from April 2019), things then were more mixed relative to baseline, with some formats faster, some slower. That was a more reasonable result. I don't have a 2.4.0 point of comparison however.
Anyway I'm concerned that while 2.5.0 is functionally complete, the performance is not sufficient.
Now I happened to notice that a bunch of important optimizations are simply not occurring in the unparser a while back. I don't know if those have been not-happening for a long time, or this recently crept in. I don't know that this is the source of the slowdown that was measured either. It's just a suspect. There is a JIRA ticket for it.
https://issues.apache.org/jira/browse/DAFFODIL-2238
I suspect the lack of those optimizations would cause a significant slow-down for unparse.
In general I think we need to take a look at performance before we can create a release candidate.
Re: 2.5.0 branch performance looks problematic currently (2019-12-02)
Posted by "Beckerle, Mike" <mb...@tresys.com>.
I agree we need at least a simple thing built into the basic daffodil source tree that's easy to run and gives
a basic hunch about single-thread performance on any given developer's system.
I created DAFFODIL-2246 for this and put some ideas there.
________________________________
From: Sloane, Brandon <bs...@tresys.com>
Sent: Monday, December 2, 2019 2:50 PM
To: dev@daffodil.apache.org <de...@daffodil.apache.org>
Subject: Re: 2.5.0 branch performance looks problematic currently (2019-12-02)
Do we have an easy to run performance test suite? Going forward, it would be nice to integrate performance testing in our normal workflow.
Particularly if we are going to be focusing optimizing. It would be much easier to do so with the quick feedback on an integrated performance test-suite.
________________________________
From: Beckerle, Mike <mb...@tresys.com>
Sent: Monday, December 2, 2019 1:29 PM
To: dev@daffodil.apache.org <de...@daffodil.apache.org>
Subject: 2.5.0 branch performance looks problematic currently (2019-12-02)
Tresys ran some performance tests on 2.5.0 branch recently.
Nacha is faster relative to baseline (baseline is release 2.0.0)
Pretty much everything else looks substantially slower.
Compared to the last prior performance report I have (from April 2019), things then were more mixed relative to baseline, with some formats faster, some slower. That was a more reasonable result. I don't have a 2.4.0 point of comparison however.
Anyway I'm concerned that while 2.5.0 is functionally complete, the performance is not sufficient.
Now I happened to notice that a bunch of important optimizations are simply not occurring in the unparser a while back. I don't know if those have been not-happening for a long time, or this recently crept in. I don't know that this is the source of the slowdown that was measured either. It's just a suspect. There is a JIRA ticket for it.
https://issues.apache.org/jira/browse/DAFFODIL-2238
I suspect the lack of those optimizations would cause a significant slow-down for unparse.
In general I think we need to take a look at performance before we can create a release candidate.
Re: 2.5.0 branch performance looks problematic currently (2019-12-02)
Posted by "Sloane, Brandon" <bs...@tresys.com>.
Do we have an easy to run performance test suite? Going forward, it would be nice to integrate performance testing in our normal workflow.
Particularly if we are going to be focusing optimizing. It would be much easier to do so with the quick feedback on an integrated performance test-suite.
________________________________
From: Beckerle, Mike <mb...@tresys.com>
Sent: Monday, December 2, 2019 1:29 PM
To: dev@daffodil.apache.org <de...@daffodil.apache.org>
Subject: 2.5.0 branch performance looks problematic currently (2019-12-02)
Tresys ran some performance tests on 2.5.0 branch recently.
Nacha is faster relative to baseline (baseline is release 2.0.0)
Pretty much everything else looks substantially slower.
Compared to the last prior performance report I have (from April 2019), things then were more mixed relative to baseline, with some formats faster, some slower. That was a more reasonable result. I don't have a 2.4.0 point of comparison however.
Anyway I'm concerned that while 2.5.0 is functionally complete, the performance is not sufficient.
Now I happened to notice that a bunch of important optimizations are simply not occurring in the unparser a while back. I don't know if those have been not-happening for a long time, or this recently crept in. I don't know that this is the source of the slowdown that was measured either. It's just a suspect. There is a JIRA ticket for it.
https://issues.apache.org/jira/browse/DAFFODIL-2238
I suspect the lack of those optimizations would cause a significant slow-down for unparse.
In general I think we need to take a look at performance before we can create a release candidate.