Performance Testing

[mujahedsyed <ms...@gmail.com>]

Hi All,

Finally my app is all ready, so I have started to do some performance
testing. Firstly let me tell you the functionality end-to-end. The
application i had to write based on our user requirements was to put a
Processing instruction tag before any element that needed to be encrypted,
so i wrote some helper class to identify processing instruction tag and then
take the qname of next element and assume that this is suppose to be
encrypted. The user that will be using this application will have no
knowledge of java/xml so thats the reason i had to take this approach.
Further I had to select StAX with encryption because sometimes files that
are passed are very big. So below is the end to end flow.

1. XML message arrives at JMS Queue
2. Message gets picked up
3. timer start to calculate the performance of encryption.
4. encryptwithstax method is called
5. timer stops to check start and end time of encryption.
6. Encrypted message is put on JMS Queue for downstream system.
7. downstream system picks up the message
8. timer starts to calculate the decryption time.
9. decryptwithStax method is called.
10. timer stops.

When i ran about 1000 messages for xml file of size 439 bytes both
encryption and decryption were very fast, to be precise the average
encryption time to encrypt one element was 0.0037187870847 secs, and to
decrypt it was 0.0268766103982.

But the system I am writing this app for occasionally gets messages of about
size 1mb. So when i ran this again with 1mb file, although encryption was
still reasonable (avg time was 0.409718753283 sec) but decryption was very
slow on an average it took every file 79.184482710915 secs to decrypt!

I am not sure why decryption is taking so much time for 1mb file when
compared to encryption, i have realized few things:
1. it does matter which tag we are encrypting i mean if it is a first tag of
the big xml than things were relatively faster.

It would be helpful if you can provide some pointers or let me know if you
think that the time taken for decryption 79 secs was still good given the
file size as 1mb.

Thanks a lot!
Regards,
Mujahed



--
View this message in context: http://apache-xml-project.6118.n7.nabble.com/Performance-Testing-tp41389.html
Sent from the Apache XML - Security - Dev mailing list archive at Nabble.com.

Re: Performance Testing

[mujahedsyed <ms...@gmail.com>]

Hi Marc,

Thanks for the response. 

Actually, in order for our expectations to meet my applications works in two
steps firstly it calculates which element needs to be encrypted and it does
that my making a copy of data and once it found the processing instruction
it than remembers that the next element is the *only* element to be
encrypted .. here is the code:



So, basically if there is a PI before <color_swatch> tag than only that
particular element will be encrypted and not all the elements that are
<color_swatch>'s. 

I know that xmlsec library works different than this.

I buried myself in this code in last few days and I think this performance
cannot be improved that's because of following reasons:
1. The more complicated XML structure the more time it will take for
processing. 
2. It's not encrypted or decryption that takes significant time but its the
stream that is being handled after decryption that takes time, if the number
of elements are significantly in XML than the stream takes a lot of time.
3. XMLStreamReader that is returned by processInMessage is taking time, I
was wrong in thinking that next method of XMLSecurityStreamReader
(http://grepcode.com/file/repo1.maven.org/maven2/org.apache.santuario/xmlsec/2.0.0-beta/org/apache/xml/security/stax/impl/XMLSecurityStreamReader.java#XMLSecurityStreamReader.next%28%29)
takes lot of time .... but we cannot blame it either as I tried to change
the implementation as you suggested (thanks for that) :

this also takes almost same amount of time.
4. I think what i can take from this exercise is that we cannot optmize this
that's because the way the other implemenations of transfomers
"com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl." are
also taking same time.. may be i am wrong but this is what i believe at the
moment.

Please let me know if you have any comments.
Thanks a lot.
Mujahed



--
View this message in context: http://apache-xml-project.6118.n7.nabble.com/Performance-Testing-tp41389p41418.html
Sent from the Apache XML - Security - Dev mailing list archive at Nabble.com.

Re: Performance Testing

[Marc Giger <gi...@apache.org>]

On Wed, 20 Aug 2014 13:34:05 -0700 (PDT)
mujahedsyed <ms...@gmail.com> wrote:

> Hi Marc,
> 
> Here is the sample project:
> https://github.com/mujahedsyed/xml-security.git due to data
> protection I had to create a sample xml that is little bit complex,
> but this still simulates the issue. The processing instruction tag
> that indicates the qname to be encrypted is located on line 6987 of
> https://github.com/mujahedsyed/xml-security/blob/master/src/main/resources/data-file.xml
> the file size is 326KB when you run this sample app you will notice
> that decryption time is around 38 secs. If you remove the encrypt PI
> tag from line 6987 and place it on line 5 than decryption time is
> around 6 secs!
> 
> My investigation suggests that the most time consuming operation is
> next method as previously mention - i have a doubt if this can be
> optimized.

Thanks for the sample.

I think the behavior of the framework does not match with
your expectations. If I'm not wrong you expect to have the
immediate element after the PI to be encrypted, nothing more
and nothing less, right?
Unfortunately, that's not the way how the StAX implementation works.
We don't have anything else as the QName's to identify the elements
that should be encrypted. So what now happens is the following:
If you place the PI on the <color_swatch> element it instructs the
framework to encrypt _all_ elements that matches that QName.
For the sample document "data-file.xml" that resulted in about
2775 encrypted <color_swatch> elements. This also explains why it
is much faster when you place the PI on the <product> element because
there are much less of them in the doc.

Apart from that it is known/expected that the current StAX impl.
performs not very well compared to the DOM impl. for small
documents or a lot small encrypted parts. The overhead is
simply to big for such documents. The advantage over
the DOM approach is mainly the smaller memory footprint.


Marc



> 
> Please let me know your valuable suggestions. Thanks.
> 
> Regards,
> Mujahed
> 
> 
> 
> --
> View this message in context:
> http://apache-xml-project.6118.n7.nabble.com/Performance-Testing-tp41389p41396.html
> Sent from the Apache XML - Security - Dev mailing list archive at
> Nabble.com.

Re: Performance Testing

[mujahedsyed <ms...@gmail.com>]

Hi Marc,

Here is the sample project: https://github.com/mujahedsyed/xml-security.git
due to data protection I had to create a sample xml that is little bit
complex, but this still simulates the issue. The processing instruction tag
that indicates the qname to be encrypted is located on line 6987 of
https://github.com/mujahedsyed/xml-security/blob/master/src/main/resources/data-file.xml
the file size is 326KB when you run this sample app you will notice that
decryption time is around 38 secs. If you remove the encrypt PI tag from
line 6987 and place it on line 5 than decryption time is around 6 secs!

My investigation suggests that the most time consuming operation is next
method as previously mention - i have a doubt if this can be optimized.

Please let me know your valuable suggestions. Thanks.

Regards,
Mujahed



--
View this message in context: http://apache-xml-project.6118.n7.nabble.com/Performance-Testing-tp41389p41396.html
Sent from the Apache XML - Security - Dev mailing list archive at Nabble.com.

Re: Performance Testing

[mujahedsyed <ms...@gmail.com>]

I found something that is worth noting. It depends a lot on which element is
getting encrypted, in my xml of size 1mb I have 98,174 elements if i encrypt
the very first element every thing happens fast both encryption and
decryption and it doesn't matter the size of the file .... 

If i encrypt something which is in the middle than after decryption this
call:

XmlReaderToWriter.writeAll(securityStreamReader, writer)

takes significant amount of time, what i am unable to answer myself is why
does this happens only on after decryption, the same call is their in
encrypt and that runs smooth. My sample app is almost done will attach soon,
thanks for the patience.




--
View this message in context: http://apache-xml-project.6118.n7.nabble.com/Performance-Testing-tp41389p41393.html
Sent from the Apache XML - Security - Dev mailing list archive at Nabble.com.

Re: Performance Testing

[Marc Giger <gi...@apache.org>]

On Wed, 20 Aug 2014 04:15:29 -0700 (PDT)
mujahedsyed <ms...@gmail.com> wrote:

> UPDATE:
> 
> After reviewing code this morning I found the most time consuming
> operation and it is this statement:
> 
> 
> 
> It takes around 20 secs, decryption is fast. so, I am investigating
> how I can get ByteArrayOutputStream with XMLStreamReader, not sure if
> this is possible but I referred to API's and there doesn't seems to
> be any helper method. is there a way around this - i mean how can i
> return the decrypted xmlstreamreader as a string? appreciate your
> help! Thanks
> Mujahed


As mentioned in the previous email:

transformer.tranform(new StAXSource(xmlSecurityStreamReader), new
StreamResult(...));

for example...

> 
> 
> 
> --
> View this message in context:
> http://apache-xml-project.6118.n7.nabble.com/Performance-Testing-tp41389p41392.html
> Sent from the Apache XML - Security - Dev mailing list archive at
> Nabble.com.

Re: Performance Testing

[mujahedsyed <ms...@gmail.com>]

UPDATE:

After reviewing code this morning I found the most time consuming operation
and it is this statement:



It takes around 20 secs, decryption is fast. so, I am investigating how I
can get ByteArrayOutputStream with XMLStreamReader, not sure if this is
possible but I referred to API's and there doesn't seems to be any helper
method. is there a way around this - i mean how can i return the decrypted
xmlstreamreader as a string? appreciate your help!
Thanks
Mujahed



--
View this message in context: http://apache-xml-project.6118.n7.nabble.com/Performance-Testing-tp41389p41392.html
Sent from the Apache XML - Security - Dev mailing list archive at Nabble.com.

Re: Performance Testing

[Marc Giger <gi...@apache.org>]

Hi Mujahed,

On Wed, 20 Aug 2014 02:56:19 -0700 (PDT)
mujahedsyed <ms...@gmail.com> wrote:
> 
> Can I ask:
> 1. in my decrypt method I tried to use OutboundXMLSec classes and
> passed the encrypted stream to processInMessage but it didn't like
> it, I think the purpose of OutboundXMLSec is to encrypt only. is this
> true? 

Yes, OutboundXMLSec is used to sign/encrypt and InboundXMLSec is used
to verify sig/decrypt.

Apart from that there is no such method
OutboundXMLSec#processInMessage() !?

> 2. Can I use OutboundXMLSec in decryption and return decrypted
> xml as baos? 

@see 1)

> 3. If I have to use only InboundXMLSec for decryption
> than how can i achieve streaming?

Regarding an OutputStream as in OutboundXMLSec#processOutMessage() but
for decryption?

Whats with 
new StAXSource(xmlSecurityStreamReader) ?

> 4. does my explanation for decryptwithstax method regarding document
> object creation and it's cause to impact performance make sense?

Yes building a w3c document gives some additional overhead of course.

@see my next emails for additional infos and answers.


Marc


> 
> I am preparing sample app will post here, thanks very much for your
> help, much appreciated.
> Regards
> Mujahed
> 
> 
> 
> --
> View this message in context:
> http://apache-xml-project.6118.n7.nabble.com/Performance-Testing-tp41389p41391.html
> Sent from the Apache XML - Security - Dev mailing list archive at
> Nabble.com.

Re: Performance Testing

[mujahedsyed <ms...@gmail.com>]

Hi Marc,

Thank you very much for the reply, I am really surprised with the processing
speed you are able to achieve. I am preparing a small app and will post it
here soon. 

regarding your questions: "/Which JRE, App-Container and StAX parser are you
using? /"
1. I used both JDK 6, 7. 7 was slightly faster but still the minimum speed i
was able to achieve was around 60 secs.
2. App-Container: I ran the application in Mule ESB Container and as a
standalone application in spring. Running on spring was slightly faster.
3. StAX parser: I referred to the test case decryptUsingStAX
(https://github.com/coheigea/testcases/blob/master/apache/santuario/santuario-xml-encryption/src/test/java/org/apache/coheigea/santuario/xmlencryption/EncryptionUtils.java)
and tried to write something similar below is a detailed explanation, i
think the problem could be that I am creating a dom object in decrypton:

My encryption was able to return a bytearrayoutputstream (baos) because the
class OutboundXMLSec had a processOutMessage which takes baos parameter here
is the code for encrypt which works superbly fast:

Now, decrypting with stax wont be able to return baos and it is returning
Document object, it's not able to do that because the InboundXMLSec
classes's processInMessage is not taking baos or there is no such overloaded
method available as we have in OutboundXMLSec. the code for my decrypt looks
like this:


I may be wrong but I think because this document is getting created in
memory and we know that dom is usually memory intensive operation it might
have been cause of the issue. 

Can I ask:
1. in my decrypt method I tried to use OutboundXMLSec classes and passed the
encrypted stream to processInMessage but it didn't like it, I think the
purpose of OutboundXMLSec is to encrypt only. is this true?
2. Can I use OutboundXMLSec in decryption and return decrypted xml as baos?
3. If I have to use only InboundXMLSec for decryption than how can i achieve
streaming?
4. does my explanation for decryptwithstax method regarding document object
creation and it's cause to impact performance make sense?

I am preparing sample app will post here, thanks very much for your help,
much appreciated.
Regards
Mujahed



--
View this message in context: http://apache-xml-project.6118.n7.nabble.com/Performance-Testing-tp41389p41391.html
Sent from the Apache XML - Security - Dev mailing list archive at Nabble.com.

Re: Performance Testing

[Marc Giger <gi...@apache.org>]

Hi Mujahed,

On Tue, 19 Aug 2014 03:31:53 -0700 (PDT)
mujahedsyed <ms...@gmail.com> wrote:

> Hi All,
> 
> Finally my app is all ready, so I have started to do some performance
> testing. Firstly let me tell you the functionality end-to-end. The
> application i had to write based on our user requirements was to put a
> Processing instruction tag before any element that needed to be
> encrypted, so i wrote some helper class to identify processing
> instruction tag and then take the qname of next element and assume
> that this is suppose to be encrypted. The user that will be using
> this application will have no knowledge of java/xml so thats the
> reason i had to take this approach. Further I had to select StAX with
> encryption because sometimes files that are passed are very big. So
> below is the end to end flow.
> 
> 1. XML message arrives at JMS Queue
> 2. Message gets picked up
> 3. timer start to calculate the performance of encryption.
> 4. encryptwithstax method is called
> 5. timer stops to check start and end time of encryption.
> 6. Encrypted message is put on JMS Queue for downstream system.
> 7. downstream system picks up the message
> 8. timer starts to calculate the decryption time.
> 9. decryptwithStax method is called.
> 10. timer stops.
> 
> When i ran about 1000 messages for xml file of size 439 bytes both
> encryption and decryption were very fast, to be precise the average
> encryption time to encrypt one element was 0.0037187870847 secs, and
> to decrypt it was 0.0268766103982.
> 
> But the system I am writing this app for occasionally gets messages
> of about size 1mb. So when i ran this again with 1mb file, although
> encryption was still reasonable (avg time was 0.409718753283 sec) but
> decryption was very slow on an average it took every file
> 79.184482710915 secs to decrypt!
> 
> I am not sure why decryption is taking so much time for 1mb file when
> compared to encryption, i have realized few things:
> 1. it does matter which tag we are encrypting i mean if it is a first
> tag of the big xml than things were relatively faster.
> 
> It would be helpful if you can provide some pointers or let me know
> if you think that the time taken for decryption 79 secs was still
> good given the file size as 1mb.

79 secs? I just decrypted a 30MB document in 0.726 secs ...

I don't have an explanation for it yet. Can you provide
a small sample project that shows this behavior?

Which JRE, App-Container and StAX parser are you using?


Marc



> 
> Thanks a lot!
> Regards,
> Mujahed
> 
> 
> 
> --
> View this message in context:
> http://apache-xml-project.6118.n7.nabble.com/Performance-Testing-tp41389.html
> Sent from the Apache XML - Security - Dev mailing list archive at
> Nabble.com.