You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-user@james.apache.org by Ajay Chitre <aj...@google.com> on 2008/08/16 03:03:59 UTC
Retrying
Hello,
In our application, under certain error conditions, we would like to retry X
number of times after say every Y number of minutes and then if the error
still persists, ghost the message. I searched the Mail archives, and
noticed that some people are writing special code to handle this, but most
of these messages are more than a year old.
Thought by now this functionality might be built into James. I looked at
the docs, but couldn't find anything obvious.
Can someone please point me in the right direction?
Thanks.
- Ajay
Re: Retrying
Posted by Stefano Bagnara <ap...@bago.org>.
Ajay Chitre ha scritto:
> I am able to get this working.. I think.. need to do some more testing.. but
> here's what I had to do:
>
> I noticed that, once I send a message to Spool, all the Mail attributes get
> lost. MailImpl constructor re-initializes all of them.
> MailImpl.setAttributesRaw() is not exposed via Mail interface, so I figured
> using Mail.setAttribute isn't going to help me.
>
> When I looked at the 'store' method of AvalonMailRepository, it looks like
> the only object that gets persisted is the MimeMessage, so decided to save
> the 'Retry Count' in that object as a header value - something like...
AFAIKT mail attributes are persisted in repository.
AvalonMailRepository.store() calls or.put(key, mc);
mc is a MailImpl and has a "private void
writeObject(java.io.ObjectOutputStream out) throws IOException" method.
> mail.getMessage().addHeader(RETRY_COUNT, retries + "");
>
> You get the idea..! How's this approach... good/bad/ugly? Please let me
> know. Thanks.
It must work with attributes.
In fact we use attributes to store delivery exception in RemoteDelivery
and the attributes are used by DSNBounce to create an appropriate
bounce, so they simply work.
Make sure your attributes are Serializable.
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org
Re: Retrying
Posted by Ajay Chitre <aj...@google.com>.
I am able to get this working.. I think.. need to do some more testing.. but
here's what I had to do:
I noticed that, once I send a message to Spool, all the Mail attributes get
lost. MailImpl constructor re-initializes all of them.
MailImpl.setAttributesRaw() is not exposed via Mail interface, so I figured
using Mail.setAttribute isn't going to help me.
When I looked at the 'store' method of AvalonMailRepository, it looks like
the only object that gets persisted is the MimeMessage, so decided to save
the 'Retry Count' in that object as a header value - something like...
mail.getMessage().addHeader(RETRY_COUNT, retries + "");
You get the idea..! How's this approach... good/bad/ugly? Please let me
know. Thanks.
On Wed, Aug 27, 2008 at 2:19 PM, Ajay Chitre <aj...@google.com> wrote:
> I got busy with other issues, so I didn't get time to look at this for a
> while.
>
> Anyway, as per Stefano's suggestion I started from RemoteDelivery and I am
> able to use most of the relevant code; but our 'process' logic is a bit
> different. This is what we would like to do:
>
> 1) We have a special processor "xyz" that uses 3 Mailets to do some
> business logic. The "root" processor sends the message "ToProcessor" xyz.
> 2) These 3 Mailets 'onMailetException' forward the mail to the new "retry"
> processor.
> 3) The RetryMailet of the "retry" processor, writes the message to a
> "retry" spool (similar to 'outgoing').
> 4) When the Thread wakes up, it calls 'process(Mail)'. Everything works
> well upto this point. But here's what we want to do in the 'process'
> method:
>
> If (Max Retry count hasn't reached)
> Send the Mail back to the "xyz" processor for re-processing
> Else
> Send the Mail to the "error" processor, which will "Bounce"
>
>
> To implement this, I am trying this:
>
> if (retries < maxRetries) {
> mail.setState("xyz");
> } else {
> mail.setState(Mail.ERROR);
> }
>
> // re-insert the mail into the spool
> MailetContext mc = getMailetContext();
> try {
> mc.sendMail(mail);
> } catch (MessagingException e) {
> // we shouldn't get an exception, because the mail was already
> processed
> log("Exception re-inserting failed mail: ", e);
> return false;
> }
>
> Thinking that the Mail object will have all the attributes intact when the
> "xyz" processor is called. Unfortunately, nothing (apparent) happens. When
> I tried mail.setState(Mail.DEFAULT), the control goes to the "root"
> processor, which forwards to our "xyz" - but all the Mail attributes are
> gone, resulting in a loop.
>
> Anyway, I am debugging thru the James code, but if someone can tell me if
> there's a better way to do this, that would be appreciated. Thanks.
>
> PS: Obviously, the other approach is to refactor our code such that we
> don't have to send it back to "xyz" processor and instead do what
> RemoteDelivery is doing, but that's our last resort - cause that requires a
> lot of refactoring of combining 3 Mailets into one.
>
>
>
> On Tue, Aug 19, 2008 at 8:41 AM, Tom Brown <to...@gmail.com> wrote:
>
>> Ajay,
>>
>> Of course, given the number of people who have asked for this over the
>> years, if you add this feature (or simplify this procedure) and
>> contribute it back to the project, it would be greatly appreciated.
>>
>> Tom Brown
>>
>> On Sat, Aug 16, 2008 at 6:01 AM, Stefano Bagnara <ap...@bago.org> wrote:
>> > Ajay Chitre ha scritto:
>> >>
>> >> Hello,
>> >>
>> >> In our application, under certain error conditions, we would like to
>> retry
>> >> X
>> >> number of times after say every Y number of minutes and then if the
>> error
>> >> still persists, ghost the message. I searched the Mail archives, and
>> >> noticed that some people are writing special code to handle this, but
>> most
>> >> of these messages are more than a year old.
>> >>
>> >> Thought by now this functionality might be built into James. I looked
>> at
>> >> the docs, but couldn't find anything obvious.
>> >>
>> >> Can someone please point me in the right direction?
>> >
>> > No built in way, yet.
>> >
>> > The easiest way is to start from RemoteDelivery code and write your own
>> > logic.
>> >
>> > Half of RemoteDelivery code deal with placing the mail to be processed
>> in
>> > another spool (named outgoing in this case) and then start a pool of
>> threads
>> > dealing with delivery for this.
>> >
>> > public void service(Mail mail) is the method where you take the mail
>> from
>> > the main spool and put it in your "retrying" spool. RemoteDelivery also
>> spit
>> > each Mail in multiple mails based on the recipients domains, but maybe
>> you
>> > don't need this and just need to store the message to your retrying
>> spool.
>> >
>> > public void run() is the code executed by the worker threads. You can
>> see
>> > that at some point (after delivery specific code) is starts a loop and
>> > inside the loop it calls the "deliver(mail, session)".
>> >
>> > So, you can simply change it to "process(mail)" and then write your
>> process
>> > code.
>> >
>> > That code will retry the message as long as process(mail) will return
>> false,
>> > and will delete the message from the spool when process(mail) returns
>> true.
>> >
>> > HTH,
>> > Stefano
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
>> > For additional commands, e-mail: server-user-help@james.apache.org
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
>> For additional commands, e-mail: server-user-help@james.apache.org
>>
>>
>
Re: Retrying
Posted by Ajay Chitre <aj...@google.com>.
Stefano,
You are *absolutely* correct! The Mail attributes get persisted and remain
intact when the message is re-delivered. There was some other exception I
was getting that was wiping them out... long story :)
Anyway, I have changed my process to use the Mail attributes now, and it
seems to work. I need to do some more testing & clean up the code. After
that, as per your suggestion, I will create a JIRA and attach the code for
your review.
Thanks (a lot) for your help with this.
- Ajay
On Thu, Aug 28, 2008 at 2:13 AM, Stefano Bagnara <ap...@bago.org> wrote:
> Ajay Chitre ha scritto:
>
> I got busy with other issues, so I didn't get time to look at this for a
>> while.
>>
>> Anyway, as per Stefano's suggestion I started from RemoteDelivery and I am
>> able to use most of the relevant code; but our 'process' logic is a bit
>> different. This is what we would like to do:
>>
>> 1) We have a special processor "xyz" that uses 3 Mailets to do some
>> business logic. The "root" processor sends the message "ToProcessor" xyz.
>> 2) These 3 Mailets 'onMailetException' forward the mail to the new
>> "retry"
>> processor.
>> 3) The RetryMailet of the "retry" processor, writes the message to a
>> "retry" spool (similar to 'outgoing').
>> 4) When the Thread wakes up, it calls 'process(Mail)'. Everything works
>> well upto this point. But here's what we want to do in the 'process'
>> method:
>>
>> If (Max Retry count hasn't reached)
>> Send the Mail back to the "xyz" processor for re-processing
>> Else
>> Send the Mail to the "error" processor, which will "Bounce"
>>
>>
>> To implement this, I am trying this:
>>
>> if (retries < maxRetries) {
>> mail.setState("xyz");
>> } else {
>> mail.setState(Mail.ERROR);
>> }
>>
>> // re-insert the mail into the spool
>> MailetContext mc = getMailetContext();
>> try {
>> mc.sendMail(mail);
>> } catch (MessagingException e) {
>> // we shouldn't get an exception, because the mail was already
>> processed
>> log("Exception re-inserting failed mail: ", e);
>> return false;
>> }
>>
>> Thinking that the Mail object will have all the attributes intact when the
>> "xyz" processor is called. Unfortunately, nothing (apparent) happens.
>> When
>> I tried mail.setState(Mail.DEFAULT), the control goes to the "root"
>> processor, which forwards to our "xyz" - but all the Mail attributes are
>> gone, resulting in a loop.
>>
>
> What does exactly 'forwards to our "xyz"' means? Maybe the way you use to
> move the message simply loose mail attributes.
>
> What is the attribute being lost?
> Can you check if the attribute is there in the root processor?
>
> Anyway, I am debugging thru the James code, but if someone can tell me if
>> there's a better way to do this, that would be appreciated. Thanks.
>>
>
> The way you do things seems fine.
> Maybe you want to open a JIRA issue for this feature and attach your code
> so that I can better review the code instead of trying to understand from
> descriptions ;-)
>
> PS: Obviously, the other approach is to refactor our code such that we
>> don't
>> have to send it back to "xyz" processor and instead do what RemoteDelivery
>> is doing, but that's our last resort - cause that requires a lot of
>> refactoring of combining 3 Mailets into one.
>>
>
> There are many ways to duplicate a Mail object.
> The only one that should duplicate attributes is new MailImpl(oldmail,
> "newname") or oldmail.duplicate().
>
>
> Stefano
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
> For additional commands, e-mail: server-user-help@james.apache.org
>
>
Re: Retrying
Posted by Stefano Bagnara <ap...@bago.org>.
Ajay Chitre ha scritto:
> I got busy with other issues, so I didn't get time to look at this for a
> while.
>
> Anyway, as per Stefano's suggestion I started from RemoteDelivery and I am
> able to use most of the relevant code; but our 'process' logic is a bit
> different. This is what we would like to do:
>
> 1) We have a special processor "xyz" that uses 3 Mailets to do some
> business logic. The "root" processor sends the message "ToProcessor" xyz.
> 2) These 3 Mailets 'onMailetException' forward the mail to the new "retry"
> processor.
> 3) The RetryMailet of the "retry" processor, writes the message to a
> "retry" spool (similar to 'outgoing').
> 4) When the Thread wakes up, it calls 'process(Mail)'. Everything works
> well upto this point. But here's what we want to do in the 'process'
> method:
>
> If (Max Retry count hasn't reached)
> Send the Mail back to the "xyz" processor for re-processing
> Else
> Send the Mail to the "error" processor, which will "Bounce"
>
>
> To implement this, I am trying this:
>
> if (retries < maxRetries) {
> mail.setState("xyz");
> } else {
> mail.setState(Mail.ERROR);
> }
>
> // re-insert the mail into the spool
> MailetContext mc = getMailetContext();
> try {
> mc.sendMail(mail);
> } catch (MessagingException e) {
> // we shouldn't get an exception, because the mail was already
> processed
> log("Exception re-inserting failed mail: ", e);
> return false;
> }
>
> Thinking that the Mail object will have all the attributes intact when the
> "xyz" processor is called. Unfortunately, nothing (apparent) happens. When
> I tried mail.setState(Mail.DEFAULT), the control goes to the "root"
> processor, which forwards to our "xyz" - but all the Mail attributes are
> gone, resulting in a loop.
What does exactly 'forwards to our "xyz"' means? Maybe the way you use
to move the message simply loose mail attributes.
What is the attribute being lost?
Can you check if the attribute is there in the root processor?
> Anyway, I am debugging thru the James code, but if someone can tell me if
> there's a better way to do this, that would be appreciated. Thanks.
The way you do things seems fine.
Maybe you want to open a JIRA issue for this feature and attach your
code so that I can better review the code instead of trying to
understand from descriptions ;-)
> PS: Obviously, the other approach is to refactor our code such that we don't
> have to send it back to "xyz" processor and instead do what RemoteDelivery
> is doing, but that's our last resort - cause that requires a lot of
> refactoring of combining 3 Mailets into one.
There are many ways to duplicate a Mail object.
The only one that should duplicate attributes is new MailImpl(oldmail,
"newname") or oldmail.duplicate().
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org
Re: Retrying
Posted by Ajay Chitre <aj...@google.com>.
I got busy with other issues, so I didn't get time to look at this for a
while.
Anyway, as per Stefano's suggestion I started from RemoteDelivery and I am
able to use most of the relevant code; but our 'process' logic is a bit
different. This is what we would like to do:
1) We have a special processor "xyz" that uses 3 Mailets to do some
business logic. The "root" processor sends the message "ToProcessor" xyz.
2) These 3 Mailets 'onMailetException' forward the mail to the new "retry"
processor.
3) The RetryMailet of the "retry" processor, writes the message to a
"retry" spool (similar to 'outgoing').
4) When the Thread wakes up, it calls 'process(Mail)'. Everything works
well upto this point. But here's what we want to do in the 'process'
method:
If (Max Retry count hasn't reached)
Send the Mail back to the "xyz" processor for re-processing
Else
Send the Mail to the "error" processor, which will "Bounce"
To implement this, I am trying this:
if (retries < maxRetries) {
mail.setState("xyz");
} else {
mail.setState(Mail.ERROR);
}
// re-insert the mail into the spool
MailetContext mc = getMailetContext();
try {
mc.sendMail(mail);
} catch (MessagingException e) {
// we shouldn't get an exception, because the mail was already
processed
log("Exception re-inserting failed mail: ", e);
return false;
}
Thinking that the Mail object will have all the attributes intact when the
"xyz" processor is called. Unfortunately, nothing (apparent) happens. When
I tried mail.setState(Mail.DEFAULT), the control goes to the "root"
processor, which forwards to our "xyz" - but all the Mail attributes are
gone, resulting in a loop.
Anyway, I am debugging thru the James code, but if someone can tell me if
there's a better way to do this, that would be appreciated. Thanks.
PS: Obviously, the other approach is to refactor our code such that we don't
have to send it back to "xyz" processor and instead do what RemoteDelivery
is doing, but that's our last resort - cause that requires a lot of
refactoring of combining 3 Mailets into one.
On Tue, Aug 19, 2008 at 8:41 AM, Tom Brown <to...@gmail.com> wrote:
> Ajay,
>
> Of course, given the number of people who have asked for this over the
> years, if you add this feature (or simplify this procedure) and
> contribute it back to the project, it would be greatly appreciated.
>
> Tom Brown
>
> On Sat, Aug 16, 2008 at 6:01 AM, Stefano Bagnara <ap...@bago.org> wrote:
> > Ajay Chitre ha scritto:
> >>
> >> Hello,
> >>
> >> In our application, under certain error conditions, we would like to
> retry
> >> X
> >> number of times after say every Y number of minutes and then if the
> error
> >> still persists, ghost the message. I searched the Mail archives, and
> >> noticed that some people are writing special code to handle this, but
> most
> >> of these messages are more than a year old.
> >>
> >> Thought by now this functionality might be built into James. I looked
> at
> >> the docs, but couldn't find anything obvious.
> >>
> >> Can someone please point me in the right direction?
> >
> > No built in way, yet.
> >
> > The easiest way is to start from RemoteDelivery code and write your own
> > logic.
> >
> > Half of RemoteDelivery code deal with placing the mail to be processed in
> > another spool (named outgoing in this case) and then start a pool of
> threads
> > dealing with delivery for this.
> >
> > public void service(Mail mail) is the method where you take the mail
> from
> > the main spool and put it in your "retrying" spool. RemoteDelivery also
> spit
> > each Mail in multiple mails based on the recipients domains, but maybe
> you
> > don't need this and just need to store the message to your retrying
> spool.
> >
> > public void run() is the code executed by the worker threads. You can
> see
> > that at some point (after delivery specific code) is starts a loop and
> > inside the loop it calls the "deliver(mail, session)".
> >
> > So, you can simply change it to "process(mail)" and then write your
> process
> > code.
> >
> > That code will retry the message as long as process(mail) will return
> false,
> > and will delete the message from the spool when process(mail) returns
> true.
> >
> > HTH,
> > Stefano
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
> > For additional commands, e-mail: server-user-help@james.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
> For additional commands, e-mail: server-user-help@james.apache.org
>
>
Re: Retrying
Posted by Tom Brown <to...@gmail.com>.
Ajay,
Of course, given the number of people who have asked for this over the
years, if you add this feature (or simplify this procedure) and
contribute it back to the project, it would be greatly appreciated.
Tom Brown
On Sat, Aug 16, 2008 at 6:01 AM, Stefano Bagnara <ap...@bago.org> wrote:
> Ajay Chitre ha scritto:
>>
>> Hello,
>>
>> In our application, under certain error conditions, we would like to retry
>> X
>> number of times after say every Y number of minutes and then if the error
>> still persists, ghost the message. I searched the Mail archives, and
>> noticed that some people are writing special code to handle this, but most
>> of these messages are more than a year old.
>>
>> Thought by now this functionality might be built into James. I looked at
>> the docs, but couldn't find anything obvious.
>>
>> Can someone please point me in the right direction?
>
> No built in way, yet.
>
> The easiest way is to start from RemoteDelivery code and write your own
> logic.
>
> Half of RemoteDelivery code deal with placing the mail to be processed in
> another spool (named outgoing in this case) and then start a pool of threads
> dealing with delivery for this.
>
> public void service(Mail mail) is the method where you take the mail from
> the main spool and put it in your "retrying" spool. RemoteDelivery also spit
> each Mail in multiple mails based on the recipients domains, but maybe you
> don't need this and just need to store the message to your retrying spool.
>
> public void run() is the code executed by the worker threads. You can see
> that at some point (after delivery specific code) is starts a loop and
> inside the loop it calls the "deliver(mail, session)".
>
> So, you can simply change it to "process(mail)" and then write your process
> code.
>
> That code will retry the message as long as process(mail) will return false,
> and will delete the message from the spool when process(mail) returns true.
>
> HTH,
> Stefano
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
> For additional commands, e-mail: server-user-help@james.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org
Re: Retrying
Posted by Stefano Bagnara <ap...@bago.org>.
Ajay Chitre ha scritto:
> Hello,
>
> In our application, under certain error conditions, we would like to retry X
> number of times after say every Y number of minutes and then if the error
> still persists, ghost the message. I searched the Mail archives, and
> noticed that some people are writing special code to handle this, but most
> of these messages are more than a year old.
>
> Thought by now this functionality might be built into James. I looked at
> the docs, but couldn't find anything obvious.
>
> Can someone please point me in the right direction?
No built in way, yet.
The easiest way is to start from RemoteDelivery code and write your own
logic.
Half of RemoteDelivery code deal with placing the mail to be processed
in another spool (named outgoing in this case) and then start a pool of
threads dealing with delivery for this.
public void service(Mail mail) is the method where you take the mail
from the main spool and put it in your "retrying" spool. RemoteDelivery
also spit each Mail in multiple mails based on the recipients domains,
but maybe you don't need this and just need to store the message to your
retrying spool.
public void run() is the code executed by the worker threads. You can
see that at some point (after delivery specific code) is starts a loop
and inside the loop it calls the "deliver(mail, session)".
So, you can simply change it to "process(mail)" and then write your
process code.
That code will retry the message as long as process(mail) will return
false, and will delete the message from the spool when process(mail)
returns true.
HTH,
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: server-user-unsubscribe@james.apache.org
For additional commands, e-mail: server-user-help@james.apache.org