You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-user@james.apache.org by Michael Weissenbacher <mw...@dermichi.com> on 2007/03/07 00:58:46 UTC
Custom Mailets, Threads and (Hibernate-) Sessions
Hi List,
I have developed some custom mailets that connect to a backend database
via Hibernate, which is a ORM mapper.
Usually it's safe to use a ThreadLocal variable to store the current
Hibernate Session (which is, in essence, nothing else than the current
database transaction). I've written a TransactionStarterMailet that
starts the Transaction at the start of the root chain, a
TransactionCommiterMailet that runs as the last Mailet of the root chain
and a RollbackMailet that is invoked in the error chain. So far this is
working flawlessly, but I haven't put the server under serious stress yet.
My question is pretty simple: Does James use a single thread for the
processing of a single Mail instance, working through all the matchers &
mailets? Will this work in a way similar to a servlet filter? If it does
I think my current design is safe. If it doesn't I guess I will have to
re-think it.
Are there any other/better ideas how to carry a single database
transaction from mailet to mailet?
thanks in advance,
Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org
Re: Custom Mailets, Threads and (Hibernate-) Sessions
Posted by Michael Weissenbacher <mw...@dermichi.com>.
Hello Stefano,
> Once the mail is store into the error processor the current thread has
> no more tasks to do. Another thread will pick up the mail from the error
> state and run it through the error processor.
Then it is as I feared ;)
> I suggest you to read this too:
> http://wiki.apache.org/james/HandlingExceptions
I already read it before, but I think it will not help me with the
problem of doing a rollback on the current transaction. All I can do to
hande exceptions is send the mail to another processor, which will spool
the mail in between. I guess the best way will be to surround all my
mailets with a try/catch block that does the rollback.
> From one processor to another a Mail is persisted and read from the
> persistence. From a Java point of view a new instance of the MailImpl
> object is created every time you retrieve a mail from the spool.
Ok, no problem for me here because I don't rely on Mail identity via
Java Object identity. I also see that this is the reason why I can't put
the Transaction into the Mail's Attributes (they have to be Serializable).
> Use a transaction for a processor and not for the whole spooling or it
> won't work.
Sounds resonable, will do.
> Yes, it would be nice, but it is harder than it seems: we don't have
> only JDBC operations, so we would need to support XA and require the use
> of only transactional resources.
I see.
> I know very well hibernate, but we don't plan to ever use it because of
> licensing issues (LGPL is bad) and because it would probably be overkill
> for our simple SpoolRepository scenario. We instead should better have
> fine grained control on how we store and retrieve data (e.g: we can read
> large streams in a better way than what hibernate could allow us).
Are there any plans to support other databases? Does it suffice to add
the correct statements in sqlResources.xml? In particular I would like
to see firebird support. Maybe I can do that myself and contribute it.
> BTW, please keep in mind that SMTP by specification does not ensure you
> ...
> This is really important to know when you start being worried of
> transactionality.
Does the Message-ID does help in such cases? Unfortunately not all mails
have one, but that's another story...
thanks a lot for your insights,
Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org
Re: Custom Mailets, Threads and (Hibernate-) Sessions
Posted by Norman Maurer <nm...@byteaction.de>.
Hi Michael,
</snip>
>
>
>> I know very well hibernate, but we don't plan to ever use it because
>> of licensing issues (LGPL is bad) and because it would probably be
>> overkill for our simple SpoolRepository scenario. We instead should
>> better have fine grained control on how we store and retrieve data
>> (e.g: we can read large streams in a better way than what hibernate
>> could allow us).
> Are there any plans to support other databases? Does it suffice to add
> the correct statements in sqlResources.xml? In particular I would like
> to see firebird support. Maybe I can do that myself and contribute it.
Yes thats all. For sure you need to add the library to the classpath.
But I asspect you allready know that ;-)
>
>> BTW, please keep in mind that SMTP by specification does not ensure
>> you ...
>> This is really important to know when you start being worried of
>> transactionality.
> Does the Message-ID does help in such cases? Unfortunately not all
> mails have one, but that's another story...
Well the message-id should be generated by the mailclient and should be
unique.. But like you said , not all mail contains a message-id. Maybe a
checksum generated "on-the-fly" could help.
>
> thanks a lot for your insights,
> Michael
>
bye
Norman
--
Mit freundlichen Grüßen
i.A. Norman Maurer
Systemadministrator
ByteAction GmbH
Auf der Beune 83-85
64839 Münster
Phone: +49 (0) 60 71 92 16 - 21
Fax: +49 (0) 60 71 92 16 - 20
E-mail: nm@byteaction.de
Internet: www.byteaction.de
AG Darmstadt, HRB 33271
Ust-Id: DE206997247
GF: Thomas Volkert
------------------------------------------------------
Diese E-Mail enthält vertrauliche Informationen und ist nur für den in der E-Mail genannten Adressaten bestimmt. Für den Fall, dass der Empfänger dieser E-Mail nicht der in der E-Mail benannte Adressat ist, weisen wir darauf hin, dass das Lesen, Kopieren, die Wiedergabe, Verbreitung, Vervielfältigung, Bekanntmachung, Veränderung, Verteilung und/oder Veröffentlichung der E-Mail strengstens untersagt ist. Bitte verständigen Sie den Absender dieser E-Mail unter folgender Rufnummer +49 (0) 6071 / 9216-0, falls Sie irrtümlich diese E-Mail erhalten haben und löschen Sie diese E-Mail. Der Inhalt dieser E-Mail ist nur rechtsverbindlich, wenn er von unserer Seite schriftlich durch Brief oder Telefax bestätigt wird. Die Versendung von E-Mails an uns hat keine fristwahrende Wirkung.
This e-mail contains information which is privileged and is intended only for the Addressee named in the e-mail. In case that the recipient of this e-mail is not the named addressee, we would like to inform you that it is strictly prohibited to read, to reproduce, to disseminate, to copy, to disclose, to modify, to distribute and/or to publish this e-mail. If you have received this e-mail in error, please call the sender under following telephone number +49 (0) 6071 / 9216-0 and delete this e-mail. The content of this e-mail is not legally binding unless confirmed by letter or telefax. E-mails which are sent to us do not constitute compliance with any time limits or deadlines.
------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org
Re: Custom Mailets, Threads and (Hibernate-) Sessions
Posted by Stefano Bagnara <ap...@bago.org>.
Michael Weissenbacher ha scritto:
> What happens if an error occurs and the mail is sent to the error
> processor? If I understand correctly the error processor may be handled
> by a different thread. In that case it's not OK to rollback the current
> ThreadLocal transaction there: It might be a different one because James
> may have picked another thread from the pool. Am I right here?
Once the mail is store into the error processor the current thread has
no more tasks to do. Another thread will pick up the mail from the error
state and run it through the error processor.
I suggest you to read this too:
http://wiki.apache.org/james/HandlingExceptions
>> When the mail is read from the spool aa new Java instance is created,
>> so it is instead probably true that a single java instance of a Mail
>> object will be processed by a single thread, but James will use
>> multiple java instances for a single mail processing.
> What exactly do you mean with multiple Java instances? Multiple
> Instances of MailImpl or multiple instances of Thread or both?
From one processor to another a Mail is persisted and read from the
persistence. From a Java point of view a new instance of the MailImpl
object is created every time you retrieve a mail from the spool.
Mail m1;
spool.store(m1);
Mail m2 = spool.retrieve(m1.key);
m1 and m2 are 2 different java instances of the same logical mail.
>> The spool processing is not a short running operation and IMHO it is
>> not suggested to have transactions running from the beginning to the end.
> My current processing takes <300ms, which isn't too bad for the life
> span of a DB transaction IMO. I don't like the idea of starting and
> commiting 10+ transactions for the processing of a single mail for both
> performance and safety.
Use a transaction for a processor and not for the whole spooling or it
won't work.
>> Keep in mind that unfortunately our spoolmanager is not (yet)
>> transactional: there are no atomic processing. This could lead to
>> duplicated processing: this is not a BIG issue because SMTP
>> specifications already ensure that we receive at least 1 copy of each
>> sent message but it does not ensure we'll receive only 1 copy.
> It would be really nice if it could be made transactional (IMHO) at
> least inside a single processor. It could be a major selling point vs.
> other mailers. I understand that this can only work with a DB backend
> and would be a major effort. Have you ever looked into Hibernate, it
> supports virtually all major DBMS's.
Yes, it would be nice, but it is harder than it seems: we don't have
only JDBC operations, so we would need to support XA and require the use
of only transactional resources.
I know very well hibernate, but we don't plan to ever use it because of
licensing issues (LGPL is bad) and because it would probably be overkill
for our simple SpoolRepository scenario. We instead should better have
fine grained control on how we store and retrieve data (e.g: we can read
large streams in a better way than what hibernate could allow us).
BTW, please keep in mind that SMTP by specification does not ensure you
that you will receive a single copy of a message: this is a known issue
of SMTP (a specification) and no SMTP server will be able to save you
from this.
If the connection is interrupted after an SMTP server sent the reply to
the "." of the DATA stream and before the client is able to read the
reply, then the client will send it again, and the server will not be
able to know what happened and will reprocess the same message again.
This is really important to know when you start being worried of
transactionality.
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org
Re: Custom Mailets, Threads and (Hibernate-) Sessions
Posted by Michael Weissenbacher <mw...@dermichi.com>.
Hi,
> There is a thread pool for the spoolmanager, every thread simply take a
> mail from the spool and let it run through a single processor (a
> sequence of mailets). At the end of the processor the mail (if it still
> exists) is stored to another processor or ghosted (or duplicated).
> Another thread will take care of it. This thread instead will start
> another mail (if present).
Thanks for your insights. If I understand correctly, a single thread
handles a single mail through a single processor. That would be exactly
what I need.
What happens if an error occurs and the mail is sent to the error
processor? If I understand correctly the error processor may be handled
by a different thread. In that case it's not OK to rollback the current
ThreadLocal transaction there: It might be a different one because James
may have picked another thread from the pool. Am I right here?
> When the mail is read from the spool aa new Java instance is created, so
> it is instead probably true that a single java instance of a Mail object
> will be processed by a single thread, but James will use multiple java
> instances for a single mail processing.
What exactly do you mean with multiple Java instances? Multiple
Instances of MailImpl or multiple instances of Thread or both?
> The spool processing is not a short running operation and IMHO it is not
> suggested to have transactions running from the beginning to the end.
My current processing takes <300ms, which isn't too bad for the life
span of a DB transaction IMO. I don't like the idea of starting and
commiting 10+ transactions for the processing of a single mail for both
performance and safety.
> Keep in mind that unfortunately our spoolmanager is not (yet)
> transactional: there are no atomic processing. This could lead to
> duplicated processing: this is not a BIG issue because SMTP
> specifications already ensure that we receive at least 1 copy of each
> sent message but it does not ensure we'll receive only 1 copy.
It would be really nice if it could be made transactional (IMHO) at
least inside a single processor. It could be a major selling point vs.
other mailers. I understand that this can only work with a DB backend
and would be a major effort. Have you ever looked into Hibernate, it
supports virtually all major DBMS's.
thanks,
Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org
Re: Custom Mailets, Threads and (Hibernate-) Sessions
Posted by Stefano Bagnara <ap...@bago.org>.
Michael Weissenbacher ha scritto:
> My question is pretty simple: Does James use a single thread for the
> processing of a single Mail instance, working through all the matchers &
> mailets? Will this work in a way similar to a servlet filter? If it does
> I think my current design is safe. If it doesn't I guess I will have to
> re-think it.
No. but it depends on your "Mail instance" definition.
There is a thread pool for the spoolmanager, every thread simply take a
mail from the spool and let it run through a single processor (a
sequence of mailets). At the end of the processor the mail (if it still
exists) is stored to another processor or ghosted (or duplicated).
Another thread will take care of it. This thread instead will start
another mail (if present).
When the mail is read from the spool aa new Java instance is created, so
it is instead probably true that a single java instance of a Mail object
will be processed by a single thread, but James will use multiple java
instances for a single mail processing.
The spool processing is not a short running operation and IMHO it is not
suggested to have transactions running from the beginning to the end.
Keep in mind that unfortunately our spoolmanager is not (yet)
transactional: there are no atomic processing. This could lead to
duplicated processing: this is not a BIG issue because SMTP
specifications already ensure that we receive at least 1 copy of each
sent message but it does not ensure we'll receive only 1 copy.
Stefano
> Are there any other/better ideas how to carry a single database
> transaction from mailet to mailet?
>
> thanks in advance,
> Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org