You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@santuario.apache.org by Markus Kilås <ma...@primekey.se> on 2014/06/12 14:41:10 UTC
Patch performance regression as signature data is not buffered
Hi,
After upgrading from xmlsec (java) 1.4 to 1.5 we saw a significant drop
in signature generation performance especially when using a network
based HSM.
After some investigation it turns out that the problem is that the
hashing is done with one byte at a time which with network latencies
gives the bad performance.
Looking in the code of DOMSignedInfo.java it looks like the code intends
to use an UnsyncBufferedOutputStream however only its close method is
actually called, which as far as I can see won't have any side affect at
all when operated on a ByteArrayOutputStream.
The attached patch resolves the performance issue by actually using the
UnsyncBufferedOutputStream and that way perform the digests on a
possibly full buffer instead of byte by byte. The patch has been tested
on version 1.5.5 but also applies on 1.5.6.
Downstream ticket in SignServer:
https://jira.primekey.se/browse/DSS-698
--
Kind regards,
Markus Kilås
PrimeKey Solutions
Re: Patch performance regression as signature data is not buffered
Posted by Colm O hEigeartaigh <co...@apache.org>.
Excellent, thanks!
Colm.
On Mon, Jun 16, 2014 at 8:47 AM, Markus Kilås <ma...@primekey.se> wrote:
> I can confirm it is fixed in 1.5.x-fixes.
>
> We put back the 1.5.5 JAR and were again able to reproduce the proble
> and then verified that after switching to the 1.5.7-SNAPSHOT JAR (built
> from r1602385) it works fine.
>
>
> Cheers,
> Markus
> PrimeKey Solutions
>
> On 2014-06-13 12:07, Colm O hEigeartaigh wrote:
> >
> > Thanks. If you could verify that the issue is fixed with the latest
> > 1.5.x-fixes code that'd be great.
> >
> > Colm.
> >
> >
> > On Thu, Jun 12, 2014 at 2:47 PM, Markus Kilås <markus@primekey.se
> > <ma...@primekey.se>> wrote:
> >
> > No problem, here you go:
> > https://issues.apache.org/jira/browse/SANTUARIO-393
> >
> > Cheers,
> > Markus
> >
> > On 2014-06-12 15:38, Colm O hEigeartaigh wrote:
> > >
> > > Could you create a new JIRA here + attach the patch to it?
> > >
> > > https://issues.apache.org/jira/browse/SANTUARIO
> > >
> > > Colm.
> > >
> > >
> > > On Thu, Jun 12, 2014 at 1:41 PM, Markus Kilås <markus@primekey.se
> > <ma...@primekey.se>
> > > <mailto:markus@primekey.se <ma...@primekey.se>>> wrote:
> > >
> > > Hi,
> > >
> > > After upgrading from xmlsec (java) 1.4 to 1.5 we saw a
> > significant drop
> > > in signature generation performance especially when using a
> > network
> > > based HSM.
> > >
> > > After some investigation it turns out that the problem is that
> the
> > > hashing is done with one byte at a time which with network
> > latencies
> > > gives the bad performance.
> > >
> > > Looking in the code of DOMSignedInfo.java it looks like the
> > code intends
> > > to use an UnsyncBufferedOutputStream however only its close
> > method is
> > > actually called, which as far as I can see won't have any side
> > affect at
> > > all when operated on a ByteArrayOutputStream.
> > >
> > > The attached patch resolves the performance issue by actually
> > using the
> > > UnsyncBufferedOutputStream and that way perform the digests on
> a
> > > possibly full buffer instead of byte by byte. The patch has
> > been tested
> > > on version 1.5.5 but also applies on 1.5.6.
> > >
> > >
> > > Downstream ticket in SignServer:
> > > https://jira.primekey.se/browse/DSS-698
> > >
> > > --
> > > Kind regards,
> > > Markus Kilås
> > > PrimeKey Solutions
> > >
> > >
> > >
> > >
> > > --
> > > Colm O hEigeartaigh
> > >
> > > Talend Community Coder
> > > http://coders.talend.com
> >
> >
> >
> > --
> > Kind regards,
> > Markus Kilås
> > PKI Specialist
> >
> > PrimeKey Solutions AB
> >
> > Anderstorpsv. 16
> > 171 54 Solna
> > Sweden
> >
> > Phone: +46 70 424 94 85 <tel:%2B46%2070%20424%2094%2085>
> > Skype: markusatskype
> > Email: markus.kilas@primekey.se <ma...@primekey.se>
> >
> > www.primekey.se <http://www.primekey.se>
> >
> >
> >
> >
> >
> > --
> > Colm O hEigeartaigh
> >
> > Talend Community Coder
> > http://coders.talend.com
>
>
>
> --
> Kind regards,
> Markus Kilås
> PKI Specialist
>
> PrimeKey Solutions AB
>
> Anderstorpsv. 16
> 171 54 Solna
> Sweden
>
> Phone: +46 70 424 94 85
> Skype: markusatskype
> Email: markus.kilas@primekey.se
>
> www.primekey.se
>
>
>
--
Colm O hEigeartaigh
Talend Community Coder
http://coders.talend.com
Re: Patch performance regression as signature data is not buffered
Posted by Markus Kilås <ma...@primekey.se>.
I can confirm it is fixed in 1.5.x-fixes.
We put back the 1.5.5 JAR and were again able to reproduce the proble
and then verified that after switching to the 1.5.7-SNAPSHOT JAR (built
from r1602385) it works fine.
Cheers,
Markus
PrimeKey Solutions
On 2014-06-13 12:07, Colm O hEigeartaigh wrote:
>
> Thanks. If you could verify that the issue is fixed with the latest
> 1.5.x-fixes code that'd be great.
>
> Colm.
>
>
> On Thu, Jun 12, 2014 at 2:47 PM, Markus Kilås <markus@primekey.se
> <ma...@primekey.se>> wrote:
>
> No problem, here you go:
> https://issues.apache.org/jira/browse/SANTUARIO-393
>
> Cheers,
> Markus
>
> On 2014-06-12 15:38, Colm O hEigeartaigh wrote:
> >
> > Could you create a new JIRA here + attach the patch to it?
> >
> > https://issues.apache.org/jira/browse/SANTUARIO
> >
> > Colm.
> >
> >
> > On Thu, Jun 12, 2014 at 1:41 PM, Markus Kilås <markus@primekey.se
> <ma...@primekey.se>
> > <mailto:markus@primekey.se <ma...@primekey.se>>> wrote:
> >
> > Hi,
> >
> > After upgrading from xmlsec (java) 1.4 to 1.5 we saw a
> significant drop
> > in signature generation performance especially when using a
> network
> > based HSM.
> >
> > After some investigation it turns out that the problem is that the
> > hashing is done with one byte at a time which with network
> latencies
> > gives the bad performance.
> >
> > Looking in the code of DOMSignedInfo.java it looks like the
> code intends
> > to use an UnsyncBufferedOutputStream however only its close
> method is
> > actually called, which as far as I can see won't have any side
> affect at
> > all when operated on a ByteArrayOutputStream.
> >
> > The attached patch resolves the performance issue by actually
> using the
> > UnsyncBufferedOutputStream and that way perform the digests on a
> > possibly full buffer instead of byte by byte. The patch has
> been tested
> > on version 1.5.5 but also applies on 1.5.6.
> >
> >
> > Downstream ticket in SignServer:
> > https://jira.primekey.se/browse/DSS-698
> >
> > --
> > Kind regards,
> > Markus Kilås
> > PrimeKey Solutions
> >
> >
> >
> >
> > --
> > Colm O hEigeartaigh
> >
> > Talend Community Coder
> > http://coders.talend.com
>
>
>
> --
> Kind regards,
> Markus Kilås
> PKI Specialist
>
> PrimeKey Solutions AB
>
> Anderstorpsv. 16
> 171 54 Solna
> Sweden
>
> Phone: +46 70 424 94 85 <tel:%2B46%2070%20424%2094%2085>
> Skype: markusatskype
> Email: markus.kilas@primekey.se <ma...@primekey.se>
>
> www.primekey.se <http://www.primekey.se>
>
>
>
>
>
> --
> Colm O hEigeartaigh
>
> Talend Community Coder
> http://coders.talend.com
--
Kind regards,
Markus Kilås
PKI Specialist
PrimeKey Solutions AB
Anderstorpsv. 16
171 54 Solna
Sweden
Phone: +46 70 424 94 85
Skype: markusatskype
Email: markus.kilas@primekey.se
www.primekey.se
Re: Patch performance regression as signature data is not buffered
Posted by Colm O hEigeartaigh <co...@apache.org>.
Thanks. If you could verify that the issue is fixed with the latest
1.5.x-fixes code that'd be great.
Colm.
On Thu, Jun 12, 2014 at 2:47 PM, Markus Kilås <ma...@primekey.se> wrote:
> No problem, here you go:
> https://issues.apache.org/jira/browse/SANTUARIO-393
>
> Cheers,
> Markus
>
> On 2014-06-12 15:38, Colm O hEigeartaigh wrote:
> >
> > Could you create a new JIRA here + attach the patch to it?
> >
> > https://issues.apache.org/jira/browse/SANTUARIO
> >
> > Colm.
> >
> >
> > On Thu, Jun 12, 2014 at 1:41 PM, Markus Kilås <markus@primekey.se
> > <ma...@primekey.se>> wrote:
> >
> > Hi,
> >
> > After upgrading from xmlsec (java) 1.4 to 1.5 we saw a significant
> drop
> > in signature generation performance especially when using a network
> > based HSM.
> >
> > After some investigation it turns out that the problem is that the
> > hashing is done with one byte at a time which with network latencies
> > gives the bad performance.
> >
> > Looking in the code of DOMSignedInfo.java it looks like the code
> intends
> > to use an UnsyncBufferedOutputStream however only its close method is
> > actually called, which as far as I can see won't have any side
> affect at
> > all when operated on a ByteArrayOutputStream.
> >
> > The attached patch resolves the performance issue by actually using
> the
> > UnsyncBufferedOutputStream and that way perform the digests on a
> > possibly full buffer instead of byte by byte. The patch has been
> tested
> > on version 1.5.5 but also applies on 1.5.6.
> >
> >
> > Downstream ticket in SignServer:
> > https://jira.primekey.se/browse/DSS-698
> >
> > --
> > Kind regards,
> > Markus Kilås
> > PrimeKey Solutions
> >
> >
> >
> >
> > --
> > Colm O hEigeartaigh
> >
> > Talend Community Coder
> > http://coders.talend.com
>
>
>
> --
> Kind regards,
> Markus Kilås
> PKI Specialist
>
> PrimeKey Solutions AB
>
> Anderstorpsv. 16
> 171 54 Solna
> Sweden
>
> Phone: +46 70 424 94 85
> Skype: markusatskype
> Email: markus.kilas@primekey.se
>
> www.primekey.se
>
>
>
--
Colm O hEigeartaigh
Talend Community Coder
http://coders.talend.com
Re: Patch performance regression as signature data is not buffered
Posted by Markus Kilås <ma...@primekey.se>.
No problem, here you go:
https://issues.apache.org/jira/browse/SANTUARIO-393
Cheers,
Markus
On 2014-06-12 15:38, Colm O hEigeartaigh wrote:
>
> Could you create a new JIRA here + attach the patch to it?
>
> https://issues.apache.org/jira/browse/SANTUARIO
>
> Colm.
>
>
> On Thu, Jun 12, 2014 at 1:41 PM, Markus Kilås <markus@primekey.se
> <ma...@primekey.se>> wrote:
>
> Hi,
>
> After upgrading from xmlsec (java) 1.4 to 1.5 we saw a significant drop
> in signature generation performance especially when using a network
> based HSM.
>
> After some investigation it turns out that the problem is that the
> hashing is done with one byte at a time which with network latencies
> gives the bad performance.
>
> Looking in the code of DOMSignedInfo.java it looks like the code intends
> to use an UnsyncBufferedOutputStream however only its close method is
> actually called, which as far as I can see won't have any side affect at
> all when operated on a ByteArrayOutputStream.
>
> The attached patch resolves the performance issue by actually using the
> UnsyncBufferedOutputStream and that way perform the digests on a
> possibly full buffer instead of byte by byte. The patch has been tested
> on version 1.5.5 but also applies on 1.5.6.
>
>
> Downstream ticket in SignServer:
> https://jira.primekey.se/browse/DSS-698
>
> --
> Kind regards,
> Markus Kilås
> PrimeKey Solutions
>
>
>
>
> --
> Colm O hEigeartaigh
>
> Talend Community Coder
> http://coders.talend.com
--
Kind regards,
Markus Kilås
PKI Specialist
PrimeKey Solutions AB
Anderstorpsv. 16
171 54 Solna
Sweden
Phone: +46 70 424 94 85
Skype: markusatskype
Email: markus.kilas@primekey.se
www.primekey.se
Re: Patch performance regression as signature data is not buffered
Posted by Colm O hEigeartaigh <co...@apache.org>.
Could you create a new JIRA here + attach the patch to it?
https://issues.apache.org/jira/browse/SANTUARIO
Colm.
On Thu, Jun 12, 2014 at 1:41 PM, Markus Kilås <ma...@primekey.se> wrote:
> Hi,
>
> After upgrading from xmlsec (java) 1.4 to 1.5 we saw a significant drop
> in signature generation performance especially when using a network
> based HSM.
>
> After some investigation it turns out that the problem is that the
> hashing is done with one byte at a time which with network latencies
> gives the bad performance.
>
> Looking in the code of DOMSignedInfo.java it looks like the code intends
> to use an UnsyncBufferedOutputStream however only its close method is
> actually called, which as far as I can see won't have any side affect at
> all when operated on a ByteArrayOutputStream.
>
> The attached patch resolves the performance issue by actually using the
> UnsyncBufferedOutputStream and that way perform the digests on a
> possibly full buffer instead of byte by byte. The patch has been tested
> on version 1.5.5 but also applies on 1.5.6.
>
>
> Downstream ticket in SignServer:
> https://jira.primekey.se/browse/DSS-698
>
> --
> Kind regards,
> Markus Kilås
> PrimeKey Solutions
>
>
--
Colm O hEigeartaigh
Talend Community Coder
http://coders.talend.com