You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@xml.apache.org by Pa...@wdr.com on 2000/02/07 11:52:44 UTC
Performance, XML messages compression
Does anyone know or any working groups that are looking at a standard
for XML compression.
I'm specifically looking at the B2B senario, where a high volume of
messages will be flowing between companies. This is obviously a
performance issue.
I think there was some standards surounding the canonicaliser started
by IBM. I would have thought compression might have been added to this
??
thanks in advance.
Paul
This message contains confidential information and is intended only
for the individual named. If you are not the named addressee you
should not disseminate, distribute or copy this e-mail. Please
notify the sender immediately by e-mail if you have received this
e-mail by mistake and delete this e-mail from your system.
E-mail transmission cannot be guaranteed to be secure or error-free
as information could be intercepted, corrupted, lost, destroyed,
arrive late or incomplete, or contain viruses. The sender therefore
does not accept liability for any errors or omissions in the contents
of this message which arise as a result of e-mail transmission. If
verification is required please request a hard-copy version. This
message is provided for informational purposes and should not be
construed as a solicitation or offer to buy or sell any securities or
related financial instruments.
Re: Performance, XML messages compression
Posted by Tim Becker <tb...@pironet.com>.
Paul.Waugh@wdr.com wrote:
>
> Does anyone know or any working groups that are looking at a standard
> for XML compression.
>> This email from the XML-DEV list is right on point for xml compression:
>> http://www.lists.ic.ac.uk/hypermail/xml-dev/xml-dev-Dec-1999/0718.html
>>
>> It references a web site:
>> http://www.research.att.com/sw/tools/xmill
You also might want to look at WBXML (WAP Binary XML Content Format)
which:
"...defines a compact binary representation of the Extensible Markup
Language [XML]. The binary XML content format is designed to reduce the
transmission size of XML documents, allowing more effective use of XML
data on narrowband communication channels. Refer to the [WML]
specification for one example use of the binary XML content format. ..."
It's not specific to WML, though and it's pretty simple to implement. I
don't know how the compression compares to LZW, unfortunately.
The spec is available at wap-forum
(http://www.wapforum.com/what/technical.htm)
-Tim Becker
--
PIRONET AG
Tim Becker (SCJP/CCNA) - Junior Consultant - SBU Software
Im Mediapark 5 - 50670 Cologne - Germany
Tel.: +49 (0)221 454 3976 - Fax: +49 (0)221 454 3710
mailto:tbecker@piro.net - http://www.pironet.com
Besuchen Sie PIRONET auf der CeBIT in Hannover
in Halle 6, Stand E 05 vom 24. Februar bis 1. März 2000
Re: Performance, XML messages compression
Posted by James Duncan Davidson <ja...@eng.sun.com>.
> One thing is for sure: XML is normally much more compressible than text.
> Exactly like PostScript. Since the "system information" is very verbose
> and doesn't add to the real information and amount of entropy of the
> stream.
I've had extremely good luck simply wrapping my socket in/out streams with
Gzip streams.
.duncan
Re: Performance, XML messages compression
Posted by Stefano Mazzocchi <st...@apache.org>.
Thomas Yip wrote:
>
> My compression knowledge limited to one course in univeristy.
Being data compression algorithms one of my favorite hobbies, I think I
can add more on this.
> But, what I understand is that compression base on LZW like WinZip,
> gz etc and will do well on file lot of repetitions, which is just XML.
You're right, but XML is not text, is XML so, creating a
specifically-tuned-for-xml compression algorithm would improve
compression. In fact, XML verbosity doesn't add any more information to
the stream.
> And, the compression which used in modem, base on LZ77 do exceptionally
> well on text compression.
True, but they work creating an adapting dictionary. If you think about
it, your schema tells you _a_lot_ about how the document can be. By
using a schema as a good °seed° to your token dictionary, you can, say,
express a tag with a few bits instead of using all the letters.
> So, in both base, you can except good compression rate. So, I don't
> really think a new compression is neccessary.
To be honest, I really can't tell. Lempel and Ziv created the LZ
algorithms to °measure° the amount of information of a stream, they
didn't want to compress it. But of course, information is present only
if you can recreate the original message. So they achieve compression.
The best thing is to invent a new algorithm based on specific XML
properties. Then put it in practice and compare it with other non-lossy
text based algorithms.... Hmmmm, if I just had the time....
> I think it is realistic to except compressed xml file will be 20% to 35%
> of original size. Does it enough in your case?
> For comparsion, average text file is about 50%.
One thing is for sure: XML is normally much more compressible than text.
Exactly like PostScript. Since the "system information" is very verbose
and doesn't add to the real information and amount of entropy of the
stream.
> > Does anyone know or any working groups that are looking at a standard
> > for XML compression.
AFAIK, no.
> > I'm specifically looking at the B2B senario, where a high volume of
> > messages will be flowing between companies. This is obviously a
> > performance issue.
Yep. I'd welcome such an effort in xml.apache.org.
--
Stefano Mazzocchi One must still have chaos in oneself to be
able to give birth to a dancing star.
<st...@apache.org> Friedrich Nietzsche
--------------------------------------------------------------------
Come to the first official Apache Software Foundation Conference!
------------------------- http://ApacheCon.Com ---------------------
Re: Performance, XML messages compression
Posted by Thomas Yip <ty...@yahoo.com>.
My compression knowledge limited to one course in univeristy.
But, what I understand is that compression base on LZW like WinZip,
gz etc and will do well on file lot of repetitions, which is just XML.
And, the compression which used in modem, base on LZ77 do exceptionally
well on text compression.
So, in both base, you can except good compression rate. So, I don't
really think a new compression is neccessary.
I think it is realistic to except compressed xml file will be 20% to 35%
of original size. Does it enough in your case?
For comparsion, average text file is about 50%.
> Does anyone know or any working groups that are looking at a standard
> for XML compression.
>
> I'm specifically looking at the B2B senario, where a high volume of
> messages will be flowing between companies. This is obviously a
> performance issue.
>
> I think there was some standards surounding the canonicaliser started
> by IBM. I would have thought compression might have been added to
this
> ??
>
> thanks in advance.
>
> Paul
>
>
> This message contains confidential information and is intended only
> for the individual named. If you are not the named addressee you
> should not disseminate, distribute or copy this e-mail. Please
> notify the sender immediately by e-mail if you have received this
> e-mail by mistake and delete this e-mail from your system.
>
> E-mail transmission cannot be guaranteed to be secure or error-free
> as information could be intercepted, corrupted, lost, destroyed,
> arrive late or incomplete, or contain viruses. The sender therefore
> does not accept liability for any errors or omissions in the contents
> of this message which arise as a result of e-mail transmission. If
> verification is required please request a hard-copy version. This
> message is provided for informational purposes and should not be
> construed as a solicitation or offer to buy or sell any securities or
> related financial instruments.
>
__________________________________________________
Do You Yahoo!?
Talk to your friends online with Yahoo! Messenger.
http://im.yahoo.com