Memory usage possible improvement

[Charles Bosquet <bo...@gmail.com>]

hey all,

I'am new to Santuario, but there is may be an improvement about memory usage
in XMLSignatureInput file to reduce memory usage :
I can see in source XMLSignatureInput code :
"if (_inputOctetStreamProxy instanceof FileInputStream) {"
But il may be replaced by :
"if (_inputOctetStreamProxy.markSupported()) {"
This modification avoids ByteArrayInputStream usuage in some cases : if we
verify a signature on a non FileInputStream but "resatable" actual code use
ByteArrayInputStream...
I can give you real world cases if needed.

Best regards

Charles BOSQUET (FRANCE...sorry for my english)

Re: Memory usage possible improvement

[Charles Bosquet <bo...@gmail.com>]

Hi Colm,

Verify signature on a large data file inside zip file causes out of memory
exception.
Mullan has introcuded a fix to bug 36526 (out of memory for big files) but
this fix seams only working for FileInputStream @see
org.apache.xml.security.signature.XMLSignatureInput (instanceof
FileInputStream usage at line 265 and 519).
I think (but not sure) that Mullan idea was : "IF stream instanceof
FileInputStream THEN don't put it in memory because it is potentially big".
If right "instanceof FileInputStream" is to restrictive, we can have
"zipStream" for exemple.
if right this logic could be rewritted but it is not very easy to decide if
a stream has a lot of data to handle it like FileInputStream...
I have an other difficuty to help you : i don't if Mullan fix cause or not a
loose of functionality when using FileInputStream because FileInputStream
are not resetables.

I join a patch that contain a testcase that run into out of memory
exception.

Merry Christmas

Charles

2010/12/23 Colm O hEigeartaigh <co...@apache.org>

> Hi Charles,
>
> We're definately interested in memory usage improvements and
> optimisations for the 1.5 release next year.
>
> One problem I can see with your change is that FileInputStream does
> not necessarily support "markSupported()", so you're changing the
> semantics of the code, as previously all FileInputStream instances
> would have entered the if statement.
>
> Could you take a look at how the actual InputStream is used and point
> out specific areas of code where using the "markSupported" input
> stream is more efficient than the existing code?
>
> Colm.
>
> On Wed, Dec 22, 2010 at 6:56 PM, Charles Bosquet
> <bo...@gmail.com> wrote:
> > hey all,
> > I'am new to Santuario, but there is may be an improvement about memory
> usage
> > in XMLSignatureInput file to reduce memory usage :
> > I can see in source XMLSignatureInput code :
> > "if (_inputOctetStreamProxy instanceof FileInputStream) {"
> > But il may be replaced by :
> > "if (_inputOctetStreamProxy.markSupported()) {"
> > This modification avoids ByteArrayInputStream usuage in some cases : if
> we
> > verify a signature on a non FileInputStream but "resatable" actual code
> use
> > ByteArrayInputStream...
> > I can give you real world cases if needed.
> > Best regards
> > Charles BOSQUET (FRANCE...sorry for my english)
>

Re: Memory usage possible improvement

[Colm O hEigeartaigh <co...@apache.org>]

Hi Charles,

We're definately interested in memory usage improvements and
optimisations for the 1.5 release next year.

One problem I can see with your change is that FileInputStream does
not necessarily support "markSupported()", so you're changing the
semantics of the code, as previously all FileInputStream instances
would have entered the if statement.

Could you take a look at how the actual InputStream is used and point
out specific areas of code where using the "markSupported" input
stream is more efficient than the existing code?

Colm.

On Wed, Dec 22, 2010 at 6:56 PM, Charles Bosquet
<bo...@gmail.com> wrote:
> hey all,
> I'am new to Santuario, but there is may be an improvement about memory usage
> in XMLSignatureInput file to reduce memory usage :
> I can see in source XMLSignatureInput code :
> "if (_inputOctetStreamProxy instanceof FileInputStream) {"
> But il may be replaced by :
> "if (_inputOctetStreamProxy.markSupported()) {"
> This modification avoids ByteArrayInputStream usuage in some cases : if we
> verify a signature on a non FileInputStream but "resatable" actual code use
> ByteArrayInputStream...
> I can give you real world cases if needed.
> Best regards
> Charles BOSQUET (FRANCE...sorry for my english)