You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by "Robert Burrell Donkin (JIRA)" <se...@james.apache.org> on 2006/12/02 15:07:22 UTC
[jira] Created: (JAMES-720) STATUS command performance
STATUS command performance
--------------------------
Key: JAMES-720
URL: http://issues.apache.org/jira/browse/JAMES-720
Project: James
Issue Type: Improvement
Components: IMAPServer
Affects Versions: Trunk
Environment: evolution email client
Reporter: Robert Burrell Donkin
I've been running my server for a little while with extra performance logging to try to work out why it's sluggish.
The command parsing is orders of magnitudes faster than the data access.
The reasons seems to be the performance of the STATUS command. Since STATUS is typically called every couple of minutes for every mail box visible on screen, it's quite a significant issue for this email client. (some other clients use SEARCH instead but this is no currently supported.)
This performance seems to worsen quickly as the number of messages in the box increases.
On my box:
STATUS on an empty mailbox with three empty sub mailboxes takes ~45 ms
STATUS on a mailbox with ~10 read messages takes ~100ms
STATUS on a mailbox with ~ 50 messages with 40 unread takes ~450ms
STATUS on a mailbox with ~ 150 with 40 unread takes ~1800ms
STATUS on a mailbox with ~ 570 unread messages takes ~6600ms
Any clues about where to start looking?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[jira] Assigned: (JAMES-720) STATUS command performance
Posted by "Joachim Draeger (JIRA)" <se...@james.apache.org>.
[ http://issues.apache.org/jira/browse/JAMES-720?page=all ]
Joachim Draeger reassigned JAMES-720:
-------------------------------------
Assignee: Joachim Draeger
> STATUS command performance
> --------------------------
>
> Key: JAMES-720
> URL: http://issues.apache.org/jira/browse/JAMES-720
> Project: James
> Issue Type: Improvement
> Components: IMAPServer
> Affects Versions: Trunk
> Environment: evolution email client
> Reporter: Robert Burrell Donkin
> Assigned To: Joachim Draeger
> Attachments: STATUS-perf.patch
>
>
> I've been running my server for a little while with extra performance logging to try to work out why it's sluggish.
> The command parsing is orders of magnitudes faster than the data access.
> The reasons seems to be the performance of the STATUS command. Since STATUS is typically called every couple of minutes for every mail box visible on screen, it's quite a significant issue for this email client. (some other clients use SEARCH instead but this is no currently supported.)
> This performance seems to worsen quickly as the number of messages in the box increases.
> On my box:
> STATUS on an empty mailbox with three empty sub mailboxes takes ~45 ms
> STATUS on a mailbox with ~10 read messages takes ~100ms
> STATUS on a mailbox with ~ 50 messages with 40 unread takes ~450ms
> STATUS on a mailbox with ~ 150 with 40 unread takes ~1800ms
> STATUS on a mailbox with ~ 570 unread messages takes ~6600ms
> Any clues about where to start looking?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[jira] Commented: (JAMES-720) STATUS command performance
Posted by "Joachim Draeger (JIRA)" <se...@james.apache.org>.
[ http://issues.apache.org/jira/browse/JAMES-720?page=comments#action_12456598 ]
Joachim Draeger commented on JAMES-720:
---------------------------------------
Which revision is your performance measurement based on?
I could improve peformance significantly in r481944 and again in r482635 .
BTW I would prefer moving discussion to mailinglist.
> STATUS command performance
> --------------------------
>
> Key: JAMES-720
> URL: http://issues.apache.org/jira/browse/JAMES-720
> Project: James
> Issue Type: Improvement
> Components: IMAPServer
> Affects Versions: Trunk
> Environment: evolution email client
> Reporter: Robert Burrell Donkin
> Assigned To: Joachim Draeger
> Attachments: STATUS-perf.patch
>
>
> I've been running my server for a little while with extra performance logging to try to work out why it's sluggish.
> The command parsing is orders of magnitudes faster than the data access.
> The reasons seems to be the performance of the STATUS command. Since STATUS is typically called every couple of minutes for every mail box visible on screen, it's quite a significant issue for this email client. (some other clients use SEARCH instead but this is no currently supported.)
> This performance seems to worsen quickly as the number of messages in the box increases.
> On my box:
> STATUS on an empty mailbox with three empty sub mailboxes takes ~45 ms
> STATUS on a mailbox with ~10 read messages takes ~100ms
> STATUS on a mailbox with ~ 50 messages with 40 unread takes ~450ms
> STATUS on a mailbox with ~ 150 with 40 unread takes ~1800ms
> STATUS on a mailbox with ~ 570 unread messages takes ~6600ms
> Any clues about where to start looking?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[jira] Commented: (JAMES-720) STATUS command performance
Posted by "Robert Burrell Donkin (JIRA)" <se...@james.apache.org>.
[ http://issues.apache.org/jira/browse/JAMES-720?page=comments#action_12455212 ]
Robert Burrell Donkin commented on JAMES-720:
---------------------------------------------
I wonder whether caching UIDs against message IDs is actually worthwhile unless the caching data access can be optimised.
> STATUS command performance
> --------------------------
>
> Key: JAMES-720
> URL: http://issues.apache.org/jira/browse/JAMES-720
> Project: James
> Issue Type: Improvement
> Components: IMAPServer
> Affects Versions: Trunk
> Environment: evolution email client
> Reporter: Robert Burrell Donkin
> Attachments: STATUS-perf.patch
>
>
> I've been running my server for a little while with extra performance logging to try to work out why it's sluggish.
> The command parsing is orders of magnitudes faster than the data access.
> The reasons seems to be the performance of the STATUS command. Since STATUS is typically called every couple of minutes for every mail box visible on screen, it's quite a significant issue for this email client. (some other clients use SEARCH instead but this is no currently supported.)
> This performance seems to worsen quickly as the number of messages in the box increases.
> On my box:
> STATUS on an empty mailbox with three empty sub mailboxes takes ~45 ms
> STATUS on a mailbox with ~10 read messages takes ~100ms
> STATUS on a mailbox with ~ 50 messages with 40 unread takes ~450ms
> STATUS on a mailbox with ~ 150 with 40 unread takes ~1800ms
> STATUS on a mailbox with ~ 570 unread messages takes ~6600ms
> Any clues about where to start looking?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[jira] Commented: (JAMES-720) STATUS command performance
Posted by "Robert Burrell Donkin (JIRA)" <se...@james.apache.org>.
[ http://issues.apache.org/jira/browse/JAMES-720?page=comments#action_12455182 ]
Robert Burrell Donkin commented on JAMES-720:
---------------------------------------------
SQL can be logged through Derby by altering a line in the derby properties file
But most of the time appears to being taken in the creating of the mailbox session. Seems to be a lot of work required to open a new read-write mailbox session just to execute a few read only queries. Is there any other reasonable way to get status information without having to open a new session?
> STATUS command performance
> --------------------------
>
> Key: JAMES-720
> URL: http://issues.apache.org/jira/browse/JAMES-720
> Project: James
> Issue Type: Improvement
> Components: IMAPServer
> Affects Versions: Trunk
> Environment: evolution email client
> Reporter: Robert Burrell Donkin
>
> I've been running my server for a little while with extra performance logging to try to work out why it's sluggish.
> The command parsing is orders of magnitudes faster than the data access.
> The reasons seems to be the performance of the STATUS command. Since STATUS is typically called every couple of minutes for every mail box visible on screen, it's quite a significant issue for this email client. (some other clients use SEARCH instead but this is no currently supported.)
> This performance seems to worsen quickly as the number of messages in the box increases.
> On my box:
> STATUS on an empty mailbox with three empty sub mailboxes takes ~45 ms
> STATUS on a mailbox with ~10 read messages takes ~100ms
> STATUS on a mailbox with ~ 50 messages with 40 unread takes ~450ms
> STATUS on a mailbox with ~ 150 with 40 unread takes ~1800ms
> STATUS on a mailbox with ~ 570 unread messages takes ~6600ms
> Any clues about where to start looking?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[jira] Commented: (JAMES-720) STATUS command performance
Posted by "Robert Burrell Donkin (JIRA)" <se...@james.apache.org>.
[ http://issues.apache.org/jira/browse/JAMES-720?page=comments#action_12456517 ]
Robert Burrell Donkin commented on JAMES-720:
---------------------------------------------
I've been studying the my performance logs in normal operation over the last few days. getImapMailboxSession seems to be the principal factor behind the poor performance for operations involving non-selected mailboxes. for clients such as evolution, these are the majority of operations performed and so this has a major impact on usability.
getImapMailboxSession is a heavy method. i can understand why this is needed for selected mailboxes but wonder whether it might be possible to use something lighter for methods which act on unselected mailboxes.
> STATUS command performance
> --------------------------
>
> Key: JAMES-720
> URL: http://issues.apache.org/jira/browse/JAMES-720
> Project: James
> Issue Type: Improvement
> Components: IMAPServer
> Affects Versions: Trunk
> Environment: evolution email client
> Reporter: Robert Burrell Donkin
> Assigned To: Joachim Draeger
> Attachments: STATUS-perf.patch
>
>
> I've been running my server for a little while with extra performance logging to try to work out why it's sluggish.
> The command parsing is orders of magnitudes faster than the data access.
> The reasons seems to be the performance of the STATUS command. Since STATUS is typically called every couple of minutes for every mail box visible on screen, it's quite a significant issue for this email client. (some other clients use SEARCH instead but this is no currently supported.)
> This performance seems to worsen quickly as the number of messages in the box increases.
> On my box:
> STATUS on an empty mailbox with three empty sub mailboxes takes ~45 ms
> STATUS on a mailbox with ~10 read messages takes ~100ms
> STATUS on a mailbox with ~ 50 messages with 40 unread takes ~450ms
> STATUS on a mailbox with ~ 150 with 40 unread takes ~1800ms
> STATUS on a mailbox with ~ 570 unread messages takes ~6600ms
> Any clues about where to start looking?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[jira] Commented: (JAMES-720) STATUS command performance
Posted by "Robert Burrell Donkin (JIRA)" <se...@james.apache.org>.
[ http://issues.apache.org/jira/browse/JAMES-720?page=comments#action_12455439 ]
Robert Burrell Donkin commented on JAMES-720:
---------------------------------------------
My local fork has wandered quite a way away from the JAMES base: I gave up running the test suite on my personal fork quite a while ago. The code wasn't really intended to be a serious proposal. just what works for me.
The STATUS issue is so large when using evolution that it renders JAMES IMAP unusable for anyone other than developers. it basically takes 10 minutes to open the client even with the relatively small volumes of data I have currently (probably < 5000 messages).
Any speed up would be great but i fear that something more radical may be needed. Evolution sends a STATUS every couple of minutes for every visible mailbox.
If the hack above sounds risky then perhaps it would be wise to consider altering the MailboxManager API. Perhaps pass through a parameter object (MailboxConfiguration, say) with mailboxName as a property something like:
public interface MailboxManager {
...
ImapMailboxSession getImapMailboxSession(MailboxConfiguration configuration) throws MailboxManagerException;
...
}
Then a 'readOnly' property could be added to the configuration object. Given that the mailbox is not used for writing then it should be safe enough to postpone loading the cache until needed. STATUS command (and any other similar ones that happen to have performance issue) could set this property to true. This would isolate the potential impact of not caching on init().
> STATUS command performance
> --------------------------
>
> Key: JAMES-720
> URL: http://issues.apache.org/jira/browse/JAMES-720
> Project: James
> Issue Type: Improvement
> Components: IMAPServer
> Affects Versions: Trunk
> Environment: evolution email client
> Reporter: Robert Burrell Donkin
> Assigned To: Joachim Draeger
> Attachments: STATUS-perf.patch
>
>
> I've been running my server for a little while with extra performance logging to try to work out why it's sluggish.
> The command parsing is orders of magnitudes faster than the data access.
> The reasons seems to be the performance of the STATUS command. Since STATUS is typically called every couple of minutes for every mail box visible on screen, it's quite a significant issue for this email client. (some other clients use SEARCH instead but this is no currently supported.)
> This performance seems to worsen quickly as the number of messages in the box increases.
> On my box:
> STATUS on an empty mailbox with three empty sub mailboxes takes ~45 ms
> STATUS on a mailbox with ~10 read messages takes ~100ms
> STATUS on a mailbox with ~ 50 messages with 40 unread takes ~450ms
> STATUS on a mailbox with ~ 150 with 40 unread takes ~1800ms
> STATUS on a mailbox with ~ 570 unread messages takes ~6600ms
> Any clues about where to start looking?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[jira] Updated: (JAMES-720) STATUS command performance
Posted by "Robert Burrell Donkin (JIRA)" <se...@james.apache.org>.
[ http://issues.apache.org/jira/browse/JAMES-720?page=all ]
Robert Burrell Donkin updated JAMES-720:
----------------------------------------
Attachment: STATUS-perf.patch
Illustrative patch (hand edited so most likely wont apply). Basically, fit a boolean into the hierarchy to allow instances to opt out of expensive preemptive caching
> STATUS command performance
> --------------------------
>
> Key: JAMES-720
> URL: http://issues.apache.org/jira/browse/JAMES-720
> Project: James
> Issue Type: Improvement
> Components: IMAPServer
> Affects Versions: Trunk
> Environment: evolution email client
> Reporter: Robert Burrell Donkin
> Attachments: STATUS-perf.patch
>
>
> I've been running my server for a little while with extra performance logging to try to work out why it's sluggish.
> The command parsing is orders of magnitudes faster than the data access.
> The reasons seems to be the performance of the STATUS command. Since STATUS is typically called every couple of minutes for every mail box visible on screen, it's quite a significant issue for this email client. (some other clients use SEARCH instead but this is no currently supported.)
> This performance seems to worsen quickly as the number of messages in the box increases.
> On my box:
> STATUS on an empty mailbox with three empty sub mailboxes takes ~45 ms
> STATUS on a mailbox with ~10 read messages takes ~100ms
> STATUS on a mailbox with ~ 50 messages with 40 unread takes ~450ms
> STATUS on a mailbox with ~ 150 with 40 unread takes ~1800ms
> STATUS on a mailbox with ~ 570 unread messages takes ~6600ms
> Any clues about where to start looking?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[jira] Commented: (JAMES-720) STATUS command performance
Posted by "Robert Burrell Donkin (JIRA)" <se...@james.apache.org>.
[ http://issues.apache.org/jira/browse/JAMES-720?page=comments#action_12455209 ]
Robert Burrell Donkin commented on JAMES-720:
---------------------------------------------
Switching ImapMailboxSessionWrapper so that it's super class does not pre-emptively cache UIDs solves the STATUS performance issue. (Attachment to follow demonstrates what I mean). Not sure whether this is the right solution.
> STATUS command performance
> --------------------------
>
> Key: JAMES-720
> URL: http://issues.apache.org/jira/browse/JAMES-720
> Project: James
> Issue Type: Improvement
> Components: IMAPServer
> Affects Versions: Trunk
> Environment: evolution email client
> Reporter: Robert Burrell Donkin
>
> I've been running my server for a little while with extra performance logging to try to work out why it's sluggish.
> The command parsing is orders of magnitudes faster than the data access.
> The reasons seems to be the performance of the STATUS command. Since STATUS is typically called every couple of minutes for every mail box visible on screen, it's quite a significant issue for this email client. (some other clients use SEARCH instead but this is no currently supported.)
> This performance seems to worsen quickly as the number of messages in the box increases.
> On my box:
> STATUS on an empty mailbox with three empty sub mailboxes takes ~45 ms
> STATUS on a mailbox with ~10 read messages takes ~100ms
> STATUS on a mailbox with ~ 50 messages with 40 unread takes ~450ms
> STATUS on a mailbox with ~ 150 with 40 unread takes ~1800ms
> STATUS on a mailbox with ~ 570 unread messages takes ~6600ms
> Any clues about where to start looking?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[jira] Closed: (IMAP-13) STATUS command performance
Posted by "Robert Burrell Donkin (JIRA)" <se...@james.apache.org>.
[ https://issues.apache.org/jira/browse/IMAP-13?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Burrell Donkin closed IMAP-13.
-------------------------------------
Resolution: Fixed
Fix Version/s: 0.1
Assignee: Robert Burrell Donkin (was: Joachim Draeger)
Status performance now seems reasonable. Expect more improvements once mailbox API is restructured.
> STATUS command performance
> --------------------------
>
> Key: IMAP-13
> URL: https://issues.apache.org/jira/browse/IMAP-13
> Project: JAMES Imap
> Issue Type: Improvement
> Environment: evolution email client
> Reporter: Robert Burrell Donkin
> Assignee: Robert Burrell Donkin
> Fix For: 0.1
>
> Attachments: STATUS-perf.patch
>
>
> I've been running my server for a little while with extra performance logging to try to work out why it's sluggish.
> The command parsing is orders of magnitudes faster than the data access.
> The reasons seems to be the performance of the STATUS command. Since STATUS is typically called every couple of minutes for every mail box visible on screen, it's quite a significant issue for this email client. (some other clients use SEARCH instead but this is no currently supported.)
> This performance seems to worsen quickly as the number of messages in the box increases.
> On my box:
> STATUS on an empty mailbox with three empty sub mailboxes takes ~45 ms
> STATUS on a mailbox with ~10 read messages takes ~100ms
> STATUS on a mailbox with ~ 50 messages with 40 unread takes ~450ms
> STATUS on a mailbox with ~ 150 with 40 unread takes ~1800ms
> STATUS on a mailbox with ~ 570 unread messages takes ~6600ms
> Any clues about where to start looking?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[jira] Commented: (JAMES-720) STATUS command performance
Posted by "Robert Burrell Donkin (JIRA)" <se...@james.apache.org>.
[ http://issues.apache.org/jira/browse/JAMES-720?page=comments#action_12455207 ]
Robert Burrell Donkin commented on JAMES-720:
---------------------------------------------
I've found the cause of the problem. when a ImapMailboxSessionWrapper is created, the init code is called in this superclass:
public abstract class NumberStableSessionWrapper extends AbstractGeneralMailbox implements EventQueueingSessionMailbox,
MailboxListener {
...
public NumberStableSessionWrapper(GeneralMailbox generalMailbox) throws MailboxManagerException {
setMailbox(generalMailbox);
init();
}
...
public void init() throws MailboxManagerException {
mailbox.addListener(eventDispatcher, MessageResult.UID);
getNumberCache();
eventDispatcher.addMailboxListener(this);
}
protected UidToMsnBidiMap getNumberCache() throws MailboxManagerException {
if (numberCache == null) {
MessageResult[] mr = mailbox.getMessages(GeneralMessageSetImpl
.all(), MessageResult.UID);
numberCache = new UidToMsnBidiMap();
for (int i = 0; i < mr.length; i++) {
numberCache.add(mr[i].getUid());
}
}
return numberCache;
}
as you can see, this calls getNumberCache.
This calls:
public class TorqueMailbox extends AbstractGeneralMailbox implements ImapMailbox {
...
public MessageResult[] getMessages(GeneralMessageSet set, int result)
throws MailboxManagerException {
checkAccess();
if (!set.isValid()) {
return new MessageResult[0];
}
Criteria c = criteriaForMessageSet(set);
UidRange range = uidRangeForMessageSet(set);
try {
List l = getMailboxRow().getMessageRows(c);
MessageResult[] messageResults = fillMessageResult(l, result
| MessageResult.UID | MessageResult.FLAGS);
checkForScanGap(range.getFromUid());
getUidChangeTracker().found(range, messageResults, null);
return messageResults;
} catch (TorqueException e) {
throw new MailboxManagerException(e);
} catch (MessagingException e) {
throw new MailboxManagerException(e);
}
}
Nearly all the time is spent in this method.
Seems like a lot of time to spend when opening a sesson in any case (maybe lazy caching would be a better plan) but it's a killer when requesting a STATUS.
> STATUS command performance
> --------------------------
>
> Key: JAMES-720
> URL: http://issues.apache.org/jira/browse/JAMES-720
> Project: James
> Issue Type: Improvement
> Components: IMAPServer
> Affects Versions: Trunk
> Environment: evolution email client
> Reporter: Robert Burrell Donkin
>
> I've been running my server for a little while with extra performance logging to try to work out why it's sluggish.
> The command parsing is orders of magnitudes faster than the data access.
> The reasons seems to be the performance of the STATUS command. Since STATUS is typically called every couple of minutes for every mail box visible on screen, it's quite a significant issue for this email client. (some other clients use SEARCH instead but this is no currently supported.)
> This performance seems to worsen quickly as the number of messages in the box increases.
> On my box:
> STATUS on an empty mailbox with three empty sub mailboxes takes ~45 ms
> STATUS on a mailbox with ~10 read messages takes ~100ms
> STATUS on a mailbox with ~ 50 messages with 40 unread takes ~450ms
> STATUS on a mailbox with ~ 150 with 40 unread takes ~1800ms
> STATUS on a mailbox with ~ 570 unread messages takes ~6600ms
> Any clues about where to start looking?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[jira] Commented: (JAMES-720) STATUS command performance
Posted by "Joachim Draeger (JIRA)" <se...@james.apache.org>.
[ http://issues.apache.org/jira/browse/JAMES-720?page=comments#action_12455106 ]
Joachim Draeger commented on JAMES-720:
---------------------------------------
Well, some backend operations are not optimized in any way.
And yes I don't consider parsing as significant.
But that case seems strange to me, because from a quick review I can say that
every information is retrieved by a single count() query. That should take only about 100ms even for 1000 messages!
Unfortunately torque doesn't support counting rows out-of-the-box at the moment,
Maybe this query does take long because it is misconstructed or indexes are missing.
Maybe there is an issue with derby.
At first we need to know where exactly the time is consumed.
You could instrument the code and put timemarks around the queries.
Just around all that mailbox.get*count() calls.
Is there a way to make derby log queries?
AFAIK Torque can be configured to log it's queries to log4j...
You could try out ImapServerLauncher. it is much easier to investigate such things when starting it from an IDE than starting whole James everytime.
SQuirreL SQL Client can be useful running statements on derby by hand.
> STATUS command performance
> --------------------------
>
> Key: JAMES-720
> URL: http://issues.apache.org/jira/browse/JAMES-720
> Project: James
> Issue Type: Improvement
> Components: IMAPServer
> Affects Versions: Trunk
> Environment: evolution email client
> Reporter: Robert Burrell Donkin
>
> I've been running my server for a little while with extra performance logging to try to work out why it's sluggish.
> The command parsing is orders of magnitudes faster than the data access.
> The reasons seems to be the performance of the STATUS command. Since STATUS is typically called every couple of minutes for every mail box visible on screen, it's quite a significant issue for this email client. (some other clients use SEARCH instead but this is no currently supported.)
> This performance seems to worsen quickly as the number of messages in the box increases.
> On my box:
> STATUS on an empty mailbox with three empty sub mailboxes takes ~45 ms
> STATUS on a mailbox with ~10 read messages takes ~100ms
> STATUS on a mailbox with ~ 50 messages with 40 unread takes ~450ms
> STATUS on a mailbox with ~ 150 with 40 unread takes ~1800ms
> STATUS on a mailbox with ~ 570 unread messages takes ~6600ms
> Any clues about where to start looking?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[jira] Commented: (JAMES-720) STATUS command performance
Posted by "Joachim Draeger (JIRA)" <se...@james.apache.org>.
[ http://issues.apache.org/jira/browse/JAMES-720?page=comments#action_12455216 ]
Joachim Draeger commented on JAMES-720:
---------------------------------------
I could improve the performance by a join between message_row and message_flags in TorqueMailbox.getMessages().
BTW Torque is quite limited and sometimes cumbersome doing this.
The cache is not used for optimizing data access but for MSN stability on external access to the db by a different host.
Another goal is to deliver appropriate events if Flags are changed externally.
(I realized some ideas I had for cluster use)
There was a reason, I don't remember right now, why I forced to load the uid cache on startup. Did you run the tests?
Of course loading them on first need would be better. But before I don't know, if the reason still exists, I prefer loading them on init().
> STATUS command performance
> --------------------------
>
> Key: JAMES-720
> URL: http://issues.apache.org/jira/browse/JAMES-720
> Project: James
> Issue Type: Improvement
> Components: IMAPServer
> Affects Versions: Trunk
> Environment: evolution email client
> Reporter: Robert Burrell Donkin
> Assigned To: Joachim Draeger
> Attachments: STATUS-perf.patch
>
>
> I've been running my server for a little while with extra performance logging to try to work out why it's sluggish.
> The command parsing is orders of magnitudes faster than the data access.
> The reasons seems to be the performance of the STATUS command. Since STATUS is typically called every couple of minutes for every mail box visible on screen, it's quite a significant issue for this email client. (some other clients use SEARCH instead but this is no currently supported.)
> This performance seems to worsen quickly as the number of messages in the box increases.
> On my box:
> STATUS on an empty mailbox with three empty sub mailboxes takes ~45 ms
> STATUS on a mailbox with ~10 read messages takes ~100ms
> STATUS on a mailbox with ~ 50 messages with 40 unread takes ~450ms
> STATUS on a mailbox with ~ 150 with 40 unread takes ~1800ms
> STATUS on a mailbox with ~ 570 unread messages takes ~6600ms
> Any clues about where to start looking?
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org