RE: diff files - writing mails to the spool store in between proc esso rs

["Chaudhuri, Hiran" <Hi...@softwareag.com>]

Hi, Shilpa.

How about stripping the caching functionality of your filememrepository to a
cacherepository instead? This cacherepository could then cache the messages
and send them to another real persistent repository, thus making the cache
functionality pluggable to all available repositories.

Hiran

-----------------------------------------
Hiran Chaudhuri
SAG Systemhaus GmbH
Elsenheimerstraße 11
80687 München
Germany
Hiran.Chaudhuri@softwareag.com
Phone +49-89-54742-134
Fax   +49-6151-9234-5134

> -----Original Message-----
> From: Shilpa Dalmia [mailto:Shilpa@PostX.com]
> Sent: Monday, May 13, 2002 10:47 PM
> To: 'James Developers List'
> Subject: diff files - writing mails to the spool store in between
> processo rs
> 
> 
> Attached are the files added/changed to provide the following 
> functionality.
> 
> 1. There are foll. 3 new files added.
> mailrepository/FileMemMailRepository.java
> mailrepository/FileMemSpoolRepository.java
> mailrepository/MimeMessageMemorySource.java
> Implement a new mailstore "filemem". This is based on the 
> AvalonMail store
> (disk store) that first stores the mail messages on the disk 
> & caches in
> memory. All subsequent reads & writes are done from the cache.
> 
> 2. 
> transport/LinearProcessor.java (diff file attached)
> Avoid renaming the mail messages while writing back to the 
> spool after a
> processor finishes its processing. 
> 
> 3.
> transport/JamesSpoolManager.java (diff file attached)
> Remove the message from the spool only if the message is in 
> GHOST state or
> there are no more recipients attached to the message.
> 
>   
> 
> 
> Shilpa Dalmia
> PostX Corporation
> 
> -----Original Message-----
> From: Serge Knystautas [mailto:sergek@lokitech.com]
> Sent: Thursday, May 09, 2002 3:51 PM
> To: James Developers List
> Subject: Re: writing mails to the spool store in between processors
> 
> 
> Shilpa,
> 
> Both sound like good ideas... we're about to do an update to 
> the latest 
> Avalon, so if you could wait until that's complete before you 
> send any 
> diffs, that would be great.
> 
> Also, I'd be just as interested in what tests you've setup... 
> we could 
> really use those as well.
> -- 
> Serge Knystautas
> Loki Technologies - Unstoppable Websites
> http://www.lokitech.com/
> 
> Shilpa Dalmia wrote:
> > I'm putting down a step by step listing of the proposed 
> solution with the
> > effect it will have on current processing.
> > 
> > Problem we're trying to solve
> > 1. Avoid having to rename the mail message everytime the 
> message is being
> > written back to the spool to be transferred to the next processor. 
> > 2. Improve performance by minimizing disk I/O for disk spooling.
> > 
> > Solution 
> > 
> > Part 1:
> > 
> > Do not rename the message before writing to the spool. 
> Remove the message
> > only if it has finished processing (check the state is GHOST or no
> > recipients to be processed).
> > 
> > Advantage:
> > 1. Reduce disk i/o by not having to rewrite the same 
> message again to the
> > disk, just writing the header is enough.
> > 2. Does not affect too many files. The change has to be made only in
> > LinearProcessor.java (remove the renaming of mail message).
> > I've tested this solution & it gives a 40% boost in the performance.
> > 
> > snippet of service method() in LinearProcessor (changes 
> highlighted in
> > bold.)
> > 
> >                 //mail.setName(newName(mail));
> >                 spool.store(mail);
> >                 mail = null;
> >                 continue;
> >             } else {
> >                 //Ok, we made it through with the same 
> state... move it to
> > the next
> >                 //  spot in the array
> >                 unprocessed[i + 1].add(mail);
> >             }
> > 
> > 
> > and
> > JamesSpoolManager.java (check if the mail is in GHOST state 
> or has no more
> > recipients, before removing it)
> > 
> >                    if (mail.getState().equals(mail.GHOST) ||
> >                     mail.getRecipients() == null ||
> > mail.getRecipients().size() == 0)
> >                     {
> >                       spool.remove(key);
> >                       getLogger().info("==== Removed from 
> spool mail "
> >                                        + mail.getName() + " ====");
> >                     }
> >                     mail = null;
> >  
> > Part 2:
> > 
> > We could further improve the performance (minimize disk 
> i/o) by caching
> the
> > message when it is first written to the spool. I've created 
> a new protocol
> > "filemem" based on the existing "file" protocol which uses both disk
> > spooling and memory spoling. Create a memory spool 
> repository & write the
> > spool message to this spool after it has been written to 
> the disk spool.
> All
> > subsequent spool accesses, by different processors, would 
> be through the
> > memory spool. Once processing is done, message will be 
> removed from both
> the
> > spools. Disk spool will be used for retries.
> > 
> > I have implemented & tested these solutions. If you think 
> it is worth
> > incorporating it in the James code base, I'd be happy to do that.
> > 
> > Shilpa Dalmia
> > PostX Corporation
> 
> 
> --
> To unsubscribe, e-mail:   
> <ma...@jakarta.apache.org>
> For additional commands, e-mail: 
> <ma...@jakarta.apache.org>
> 
> 

--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>