You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Maxime Petazzoni <ma...@bulix.org> on 2005/07/13 21:51:02 UTC
mod_mbox: js speed and mbox size
Hi,
As I already said, the AJAX browser in mod_mbox relies on a hidden
XmlHttpRequest and on Javascript processing of the replied XML
document to generate the message list.
After setting up a browser stub, I'm not going to concentrate on
compatibility and speed problems before continuing. In fact, there are
two main problems implied by the current mechanism :
- On large mailboxes, the /ajax/msglist query reply can get *very*
large. As an example, this mailing list's 2001-08 archives contains
1708 messages, and the corresponding XML is 452492 bytes big. This
is too much. As a comparison, the old mod_mbox resulting page for
the same month is "only" 311 KB big.
As a solution to this, Ian advised the use of LiveGrid
(http://openrico.org/livegrid.page). It fetches only the piece of
data that need to be displayed (ten, twenty, ... rows instead of the
whole 1700+). I don't really like this solution because the user
must scroll the table, and then wait for the data to appear,
etc. And that we don't get the whole list (no in-page search for
example).
Another solution would be to use a page system as in the Debian
mailing list archives, displaying 100 or 200 messages per page
only. This would reduce bandwidth usage, but it will also make things
harder when updating the context box. For the moment, the message
list is fetched once, and all operations are made on this stored
list. When updating the context box, we just take -3 to +3 messages
from the list. With the page system, we'll experience problems while
approaching the end of a page ...
- The second problem is brought by the first one. On large mailboxes,
the Javascript processing takes too much time. Building all elements
each time the message list is displayed (at loading time or when you
close a message box) does not make the application usable.
Of course, the paging system would solve this problem, too. In
addition to this, my Javascript code may not be the most optimized
we could find.
- Concerning compatibility problems, the whole thing does not work
nor with IE, neither with Opera, although both of these browsers
does not comply against any javascript error.
Any JS compatibility expert in the house ?
Well, if you have any ideas or suggestions, they're really welcome.
- Sam
--
Maxime Petazzoni (http://www.bulix.org)
-- gone crazy, back soon. leave message.
Re: mod_mbox: js speed and mbox size
Posted by Maxime Petazzoni <ma...@bulix.org>.
> would preprocessing (and caching) some of this stuff on the server
> help? it might make your 450k XML file larger, but you could
> provide some of the thread info in the XML file itself.
The XML answer already contains thread information.
> this is why I suggested openrico .. they have done all of this for
> you, and have their code used by a wider range of people. the code
> has a ASL2 licence.. it is perfect for httpd.
Yep, I have to take a closer look to it. Anyway, I think the page
system is a good alternative. I won't work much tonight because I'm
really exhausted. Living GMT+8 in France is not really
strength-keeping :)
> >This is a cool project, glad it's on the httpd dev list.
And I'm glad to be part of it :)
- Sam
--
Maxime Petazzoni (http://www.bulix.org)
-- gone crazy, back soon. leave message.
Re: mod_mbox: js speed and mbox size
Posted by Ian Holsman <li...@holsman.net>.
Bill Stoddard wrote:
> Maxime Petazzoni wrote:
>
>> Hi,
>>
>> As I already said, the AJAX browser in mod_mbox relies on a hidden
>> XmlHttpRequest and on Javascript processing of the replied XML
>> document to generate the message list.
>>
>> After setting up a browser stub, I'm not going to concentrate on
>> compatibility and speed problems before continuing. In fact, there are
>> two main problems implied by the current mechanism :
>>
>> - On large mailboxes, the /ajax/msglist query reply can get *very*
>> large. As an example, this mailing list's 2001-08 archives contains
>> 1708 messages, and the corresponding XML is 452492 bytes big. This
>> is too much. As a comparison, the old mod_mbox resulting page for
>> the same month is "only" 311 KB big.
mod_deflate should be able to help with that. XML (or any text)
compresses real well.
>>
>> As a solution to this, Ian advised the use of LiveGrid
>> (http://openrico.org/livegrid.page). It fetches only the piece of
>> data that need to be displayed (ten, twenty, ... rows instead of the
>> whole 1700+). I don't really like this solution because the user
>> must scroll the table, and then wait for the data to appear,
>> etc.
>
> Isn't this where XmlHttpRequest comes in handy? Asynchronously fetch X
> (a hundred?) rows forward from where the user is currently looking in
> anticipation that that user will scroll forward. So you don't slurp down
> the whole mailbox at once, but you try to stay ahead of what the user
> might be expected to do and always have some data ready to display.
>
That is what openrico do I'm guessing. they just provide a set of higher
level API's to manage that.
>> And that we don't get the whole list (no in-page search for
>> example).
>>
>> Another solution would be to use a page system as in the Debian
>> mailing list archives, displaying 100 or 200 messages per page
>> only. This would reduce bandwidth usage, but it will also make things
>> harder when updating the context box. For the moment, the message
>> list is fetched once, and all operations are made on this stored
>> list. When updating the context box, we just take -3 to +3 messages
>> from the list. With the page system, we'll experience problems while
>> approaching the end of a page ...
>>
>> - The second problem is brought by the first one. On large mailboxes,
>> the Javascript processing takes too much time. Building all elements
>> each time the message list is displayed (at loading time or when you
>> close a message box) does not make the application usable.
>>
would preprocessing (and caching) some of this stuff on the server help?
it might make your 450k XML file larger, but you could provide some of
the thread info in the XML file itself.
>> Of course, the paging system would solve this problem, too. In
>> addition to this, my Javascript code may not be the most optimized
>> we could find.
>>
>> - Concerning compatibility problems, the whole thing does not work
>> nor with IE, neither with Opera, although both of these browsers
>> does not comply against any javascript error.
>>
>> Any JS compatibility expert in the house ?
yes. I've done a bit, and it is yuk
this is why I suggested openrico .. they have done all of this for you,
and have their code used by a wider range of people. the code has a ASL2
licence.. it is perfect for httpd.
>
> Heh, you wish :0
>
> This is a cool project, glad it's on the httpd dev list.
>
> Bill
>
Re: mod_mbox: js speed and mbox size
Posted by Jeff White <jl...@earthlink.net>.
From: "Bill Stoddard"
> Maxime Petazzoni wrote:
>> As I already said, the AJAX browser
>> in mod_mbox relies on a hidden
>> XmlHttpRequest and on Javascript
>> processing of the replied XML
>> document to generate the message list.
>>
Snip
>> Of course, the paging system would
>> solve this problem, too. In addition to this,
>> my Javascript code may not be the most
>> optimized we could find.
>>
>> - Concerning compatibility problems, the
>> whole thing does not work nor with IE,
>> neither with Opera, although both of these
>> browsers does not comply against any
>> javascript error.
>>
>> Any JS compatibility expert in the house ?
>
> Heh, you wish :0
>
> This is a cool project, glad it's on the httpd
> dev list.
>
> Bill
>
Hasn't someone been suggesting,
for years now, to deaf ears at ASF,
for ASF to use the very helpful
Windows Scripting (even XmlHttp)?
Now ASF's developers and open source
developers have discovered XmlHttp!
Atlas = Ajax = asp.net 2.0
script callbacks and more
http://blogs.msdn.com/nickmalik/archive/2005/07/08/436823.aspx
Perhaps (once again):
Learn to Script
http://www.microsoft.com/technet/scriptcenter/learnit.mspx
Jeff
Re: mod_mbox: js speed and mbox size
Posted by Bill Stoddard <bi...@wstoddard.com>.
Maxime Petazzoni wrote:
> Hi,
>
> As I already said, the AJAX browser in mod_mbox relies on a hidden
> XmlHttpRequest and on Javascript processing of the replied XML
> document to generate the message list.
>
> After setting up a browser stub, I'm not going to concentrate on
> compatibility and speed problems before continuing. In fact, there are
> two main problems implied by the current mechanism :
>
> - On large mailboxes, the /ajax/msglist query reply can get *very*
> large. As an example, this mailing list's 2001-08 archives contains
> 1708 messages, and the corresponding XML is 452492 bytes big. This
> is too much. As a comparison, the old mod_mbox resulting page for
> the same month is "only" 311 KB big.
>
> As a solution to this, Ian advised the use of LiveGrid
> (http://openrico.org/livegrid.page). It fetches only the piece of
> data that need to be displayed (ten, twenty, ... rows instead of the
> whole 1700+). I don't really like this solution because the user
> must scroll the table, and then wait for the data to appear,
> etc.
Isn't this where XmlHttpRequest comes in handy? Asynchronously fetch X (a hundred?) rows forward from where
the user is currently looking in anticipation that that user will scroll forward. So you don't slurp down the
whole mailbox at once, but you try to stay ahead of what the user might be expected to do and always have some
data ready to display.
> And that we don't get the whole list (no in-page search for
> example).
>
> Another solution would be to use a page system as in the Debian
> mailing list archives, displaying 100 or 200 messages per page
> only. This would reduce bandwidth usage, but it will also make things
> harder when updating the context box. For the moment, the message
> list is fetched once, and all operations are made on this stored
> list. When updating the context box, we just take -3 to +3 messages
> from the list. With the page system, we'll experience problems while
> approaching the end of a page ...
>
> - The second problem is brought by the first one. On large mailboxes,
> the Javascript processing takes too much time. Building all elements
> each time the message list is displayed (at loading time or when you
> close a message box) does not make the application usable.
>
> Of course, the paging system would solve this problem, too. In
> addition to this, my Javascript code may not be the most optimized
> we could find.
>
> - Concerning compatibility problems, the whole thing does not work
> nor with IE, neither with Opera, although both of these browsers
> does not comply against any javascript error.
>
> Any JS compatibility expert in the house ?
Heh, you wish :0
This is a cool project, glad it's on the httpd dev list.
Bill