You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Donald Whytock <dw...@gmail.com> on 2011/04/19 21:40:22 UTC
ROUTE_STOP and mail
Hi all...
Does using ROUTE_STOP on an email exchange prevent the message from
being deleted from the POP3 server?
Don
Re: ROUTE_STOP and mail
Posted by Donald Whytock <dw...@gmail.com>.
>> - For whatever reason there are mail servers that demand the whole
>> email address as the login ID. java.net.URI is incapable of handling
>> things like
>>
>> imap://bot@dummy.com@dummy.com?password=test
>> imap://dummy.com?password=test&username=bot@dummy.com
>>
>> You can probably direct-parse the URI using getAuthority() to get the
>> part before the question mark and getQuery() for the rest, looking for
>> embedded @. I did a workaround in my application instead:
>>
>> final Endpoint inmail = camelcontext.getEndpoint("imap://" + server +
>> "?delete=true");
>> MailEndpoint minmail = (MailEndpoint) inmail;
>> MailConfiguration mconfig = minmail.getConfiguration();
>> mconfig.setUsername(username);
>> mconfig.setPassword(password);
>>
>
> Care to explain a bit more? Ah the username has @ in it? Is that the problem?
Yes, MailConfiguration uses java.net.URI to parse the endpoint string.
When you have
imap://name@mailserver.com@mailserver.com
both getHost() and getUserInfo() come back null. When you have
imap://mailserver.com?username=name@mailserver.com
getHost() works, but getUserInfo() still comes back null.
Either way it doesn't connect to the server. Using
MailConfiguration's setUsername() works even with the @ in the user
name.
Re: ROUTE_STOP and mail
Posted by Claus Ibsen <cl...@gmail.com>.
Hi Donald
On Wed, Apr 27, 2011 at 12:37 AM, Donald Whytock <dw...@gmail.com> wrote:
> Trying with a different mail server, it consistently deletes with IMAP
> and consistently fails to delete with POP3. Long as it's consistent,
> I'm good. Thanks.
>
Thanks for reporting back.
Yeah that's in line what people have seen before as well with the POP3 vs IMAP.
> A couple issues that came up...
>
> - I got a FolderNotOpen error when the component was trying to set the
> DELETED flag. So to the beginning of MailConsumer.processCommit, in
> the try block, I added:
>
> Folder folder=mail.getFolder();
> if (!folder.isOpen())
> folder.open(Folder.READ_WRITE);
>
Please create a JIRA ticket and attach a patch if possible. We are
happy to get that improvement in the code base.
> - For whatever reason there are mail servers that demand the whole
> email address as the login ID. java.net.URI is incapable of handling
> things like
>
> imap://bot@dummy.com@dummy.com?password=test
> imap://dummy.com?password=test&username=bot@dummy.com
>
> You can probably direct-parse the URI using getAuthority() to get the
> part before the question mark and getQuery() for the rest, looking for
> embedded @. I did a workaround in my application instead:
>
> final Endpoint inmail = camelcontext.getEndpoint("imap://" + server +
> "?delete=true");
> MailEndpoint minmail = (MailEndpoint) inmail;
> MailConfiguration mconfig = minmail.getConfiguration();
> mconfig.setUsername(username);
> mconfig.setPassword(password);
>
Care to explain a bit more? Ah the username has @ in it? Is that the problem?
> - The mail component is dependent on Spring's JavaMailSender. Since I
> don't use Spring I'm putting together a very lean substitute. Let me
> know if you're interested.
>
Yeah there are 2 components which uses the spring framework as a
library (eg only its JARs).
- camel-jms
- camel-mail
And we would like to add light-weight alternatives in the future. I
have my eyes on the JMS to build a simpler component with less bells
and whistles. Optimized for AMQ and using pure JMS API. My gut felling
says its possible to do a faster and lighter version than the Spring
DMLC which is a bit complicated and heavy. And being able to easier
support async TX and batch acknowledgement.
> Thanks again for the help...
>
> Don
>
> On Mon, Apr 25, 2011 at 12:50 PM, Donald Whytock <dw...@gmail.com> wrote:
>> Using Apache James at the moment. The current stable version of that
>> (2.3.2) doesn't support IMAP. ("A backport to James Server 2.3.2 would
>> require a volunteer.")
>>
>> I have another localhost server I can try. Might work better anyway.
>> Ideally, the production app won't be running on a localhost email
>> server...
>>
>> On Sat, Apr 23, 2011 at 3:22 AM, Claus Ibsen <cl...@gmail.com> wrote:
>>> Hi
>>>
>>> Ah try using imap instead of pop3. pop3 is very limited and it may not
>>> work setting those flags. I think we have seen this in the past
>>> setting the flags to SEEN or DELETED etc.
>>>
>>> We have a little tip about use imap instead of pop3 at the wiki page
>>> http://camel.apache.org/mail.html
>>>
>>>
>>>
>>> On Fri, Apr 22, 2011 at 9:27 PM, Donald Whytock <dw...@gmail.com> wrote:
>>>> At the moment my route is very short:
>>>>
>>>> from("pop3://bot@localhost?password=test")
>>>> .process(message1)
>>>> .process(peeker)
>>>> .choice()
>>>> .when(property("chatterbot.email").isNotNull())
>>>> .to("seda:mailtester");
>>>>
>>>> from("seda:mailtester")
>>>> .routeId("listener.test")
>>>> .process(message3);
>>>>
>>>> Peeker verifies the "from" header against a table; if found, it
>>>> changes the message body to String and sets chatterbot.email to
>>>> "true". Message1 and message3 kick out static messages to System.out.
>>>> The seda:mailtester consumer is in a different Felix bundle from the
>>>> mail consumer. Very little processing, very little time.
>>>>
>>>> I can send it an email, which it will pick up. Message 1 fires,
>>>> peeker shows the from address and body, message3 fires, and then I see
>>>> the debug log say the DELETED flag was set on the message ("Exchange
>>>> processed, so flagging message as DELETED"). A bit later I see the
>>>> same message being picked up again, and again the DELETED flag is set.
>>>> Occasionally I'll see the message picked up a third time. After that
>>>> the message doesn't get picked up again.
>>>>
>>>> Oddly, if I remove the routeId(), it stops happening. If I put the
>>>> routeId() back in, the problem happens again.
>>>>
>>>> On Fri, Apr 22, 2011 at 5:00 AM, Claus Ibsen <cl...@gmail.com> wrote:
>>>>> On Thu, Apr 21, 2011 at 8:50 PM, Donald Whytock <dw...@gmail.com> wrote:
>>>>>> Okay, it's not ROUTE_STOP, or not necessarily. I've been getting
>>>>>> mixed results with email deletion. In all cases I see the message
>>>>>> "Exchange processed, so flagging message as DELETED" coming from
>>>>>> MailConsumer.processCommit() as the exchange's onCompletion is run,
>>>>>> but in some cases the email doesn't actually get deleted, and shows up
>>>>>> again in the queues with the next polling.
>>>>>>
>>>>>
>>>>> How long time do it take to process the exchange, when you notice the
>>>>> email isn't deleted?
>>>>> Maybe the mail session times out, and it wont re-connect to set the
>>>>> DELETED flag.
>>>>> There should be something in the logs as camel-mail log at DEBUG level
>>>>> when it attempts to delete the mail.
>>>>>
>>>>>
>>>>>
>>>>>> I've removed and restored things I'm doing in my routes, and nothing
>>>>>> consistently causes the email to not be deleted. I'm at the point now
>>>>>> where, without changing the route, I see a message appear once, twice
>>>>>> or three times (varying) before it gets deleted.
>>>>>>
>>>>>> Is there a timeout on message deletion, such that if my route takes
>>>>>> too long the message won't get deleted? Do I need to copy the
>>>>>> exchange as soon as I get it, finish with the original and run with
>>>>>> the copy?
>>>>>>
>>>>>> On Wed, Apr 20, 2011 at 11:35 AM, Donald Whytock <dw...@gmail.com> wrote:
>>>>>>> But if I do filter on the pipeline it'll determine whether the entire
>>>>>>> pipeline is or isn't performed, right? So to break out of the
>>>>>>> pipeline I'd have to put the filter on each pipelined item, such as
>>>>>>>
>>>>>>> from X
>>>>>>> filter when Y
>>>>>>> to A
>>>>>>> filter when Y
>>>>>>> to B
>>>>>>> end // filter
>>>>>>> to C // do something after filter for all messages
>>>>>>>
>>>>>>> so as to pick up when Y became false.
>>>>>>>
>>>>>>> Since I use an instance of Pipeline with an ArrayList of Processors,
>>>>>>> this would become an ArrayList of FilterProcessor(predicate,
>>>>>>> processor)?
>>>>>>>
>>>>>>> On Wed, Apr 20, 2011 at 4:59 AM, Claus Ibsen <cl...@gmail.com> wrote:
>>>>>>>> On Wed, Apr 20, 2011 at 12:42 AM, Donald Whytock <dw...@gmail.com> wrote:
>>>>>>>>> I ask because it appears to, in my application where I'm using
>>>>>>>>> ROUTE_STOP to break out of a pipeline.
>>>>>>>>>
>>>>>>>>> Perhaps there should be some mechanism for breaking out of a pipeline
>>>>>>>>> without killing the whole route? A PIPELINE_STOP property, or maybe
>>>>>>>>> ROUTE_STOP could contain the ID of the route to be broken out of,
>>>>>>>>> allowing containing routes to continue?
>>>>>>>>>
>>>>>>>>
>>>>>>>> You can use the filter EIP to only process messages based on a predicate.
>>>>>>>>
>>>>>>>> from X
>>>>>>>> filter when Y
>>>>>>>> to A
>>>>>>>> to B
>>>>>>>> end // filter
>>>>>>>> to C // do something after filter for all messages
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Apr 19, 2011 at 3:40 PM, Donald Whytock <dw...@gmail.com> wrote:
>>>>>>>>>> Hi all...
>>>>>>>>>>
>>>>>>>>>> Does using ROUTE_STOP on an email exchange prevent the message from
>>>>>>>>>> being deleted from the POP3 server?
>>>>>>>>>>
>>>>>>>>>> Don
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Claus Ibsen
>>>>>>>> -----------------
>>>>>>>> FuseSource
>>>>>>>> Email: cibsen@fusesource.com
>>>>>>>> Web: http://fusesource.com
>>>>>>>> CamelOne 2011: http://fusesource.com/camelone2011/
>>>>>>>> Twitter: davsclaus
>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> -----------------
>>>>> FuseSource
>>>>> Email: cibsen@fusesource.com
>>>>> Web: http://fusesource.com
>>>>> CamelOne 2011: http://fusesource.com/camelone2011/
>>>>> Twitter: davsclaus
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> -----------------
>>> FuseSource
>>> Email: cibsen@fusesource.com
>>> Web: http://fusesource.com
>>> CamelOne 2011: http://fusesource.com/camelone2011/
>>> Twitter: davsclaus
>>> Blog: http://davsclaus.blogspot.com/
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>
>>
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
CamelOne 2011: http://fusesource.com/camelone2011/
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: ROUTE_STOP and mail
Posted by Donald Whytock <dw...@gmail.com>.
Trying with a different mail server, it consistently deletes with IMAP
and consistently fails to delete with POP3. Long as it's consistent,
I'm good. Thanks.
A couple issues that came up...
- I got a FolderNotOpen error when the component was trying to set the
DELETED flag. So to the beginning of MailConsumer.processCommit, in
the try block, I added:
Folder folder=mail.getFolder();
if (!folder.isOpen())
folder.open(Folder.READ_WRITE);
- For whatever reason there are mail servers that demand the whole
email address as the login ID. java.net.URI is incapable of handling
things like
imap://bot@dummy.com@dummy.com?password=test
imap://dummy.com?password=test&username=bot@dummy.com
You can probably direct-parse the URI using getAuthority() to get the
part before the question mark and getQuery() for the rest, looking for
embedded @. I did a workaround in my application instead:
final Endpoint inmail = camelcontext.getEndpoint("imap://" + server +
"?delete=true");
MailEndpoint minmail = (MailEndpoint) inmail;
MailConfiguration mconfig = minmail.getConfiguration();
mconfig.setUsername(username);
mconfig.setPassword(password);
- The mail component is dependent on Spring's JavaMailSender. Since I
don't use Spring I'm putting together a very lean substitute. Let me
know if you're interested.
Thanks again for the help...
Don
On Mon, Apr 25, 2011 at 12:50 PM, Donald Whytock <dw...@gmail.com> wrote:
> Using Apache James at the moment. The current stable version of that
> (2.3.2) doesn't support IMAP. ("A backport to James Server 2.3.2 would
> require a volunteer.")
>
> I have another localhost server I can try. Might work better anyway.
> Ideally, the production app won't be running on a localhost email
> server...
>
> On Sat, Apr 23, 2011 at 3:22 AM, Claus Ibsen <cl...@gmail.com> wrote:
>> Hi
>>
>> Ah try using imap instead of pop3. pop3 is very limited and it may not
>> work setting those flags. I think we have seen this in the past
>> setting the flags to SEEN or DELETED etc.
>>
>> We have a little tip about use imap instead of pop3 at the wiki page
>> http://camel.apache.org/mail.html
>>
>>
>>
>> On Fri, Apr 22, 2011 at 9:27 PM, Donald Whytock <dw...@gmail.com> wrote:
>>> At the moment my route is very short:
>>>
>>> from("pop3://bot@localhost?password=test")
>>> .process(message1)
>>> .process(peeker)
>>> .choice()
>>> .when(property("chatterbot.email").isNotNull())
>>> .to("seda:mailtester");
>>>
>>> from("seda:mailtester")
>>> .routeId("listener.test")
>>> .process(message3);
>>>
>>> Peeker verifies the "from" header against a table; if found, it
>>> changes the message body to String and sets chatterbot.email to
>>> "true". Message1 and message3 kick out static messages to System.out.
>>> The seda:mailtester consumer is in a different Felix bundle from the
>>> mail consumer. Very little processing, very little time.
>>>
>>> I can send it an email, which it will pick up. Message 1 fires,
>>> peeker shows the from address and body, message3 fires, and then I see
>>> the debug log say the DELETED flag was set on the message ("Exchange
>>> processed, so flagging message as DELETED"). A bit later I see the
>>> same message being picked up again, and again the DELETED flag is set.
>>> Occasionally I'll see the message picked up a third time. After that
>>> the message doesn't get picked up again.
>>>
>>> Oddly, if I remove the routeId(), it stops happening. If I put the
>>> routeId() back in, the problem happens again.
>>>
>>> On Fri, Apr 22, 2011 at 5:00 AM, Claus Ibsen <cl...@gmail.com> wrote:
>>>> On Thu, Apr 21, 2011 at 8:50 PM, Donald Whytock <dw...@gmail.com> wrote:
>>>>> Okay, it's not ROUTE_STOP, or not necessarily. I've been getting
>>>>> mixed results with email deletion. In all cases I see the message
>>>>> "Exchange processed, so flagging message as DELETED" coming from
>>>>> MailConsumer.processCommit() as the exchange's onCompletion is run,
>>>>> but in some cases the email doesn't actually get deleted, and shows up
>>>>> again in the queues with the next polling.
>>>>>
>>>>
>>>> How long time do it take to process the exchange, when you notice the
>>>> email isn't deleted?
>>>> Maybe the mail session times out, and it wont re-connect to set the
>>>> DELETED flag.
>>>> There should be something in the logs as camel-mail log at DEBUG level
>>>> when it attempts to delete the mail.
>>>>
>>>>
>>>>
>>>>> I've removed and restored things I'm doing in my routes, and nothing
>>>>> consistently causes the email to not be deleted. I'm at the point now
>>>>> where, without changing the route, I see a message appear once, twice
>>>>> or three times (varying) before it gets deleted.
>>>>>
>>>>> Is there a timeout on message deletion, such that if my route takes
>>>>> too long the message won't get deleted? Do I need to copy the
>>>>> exchange as soon as I get it, finish with the original and run with
>>>>> the copy?
>>>>>
>>>>> On Wed, Apr 20, 2011 at 11:35 AM, Donald Whytock <dw...@gmail.com> wrote:
>>>>>> But if I do filter on the pipeline it'll determine whether the entire
>>>>>> pipeline is or isn't performed, right? So to break out of the
>>>>>> pipeline I'd have to put the filter on each pipelined item, such as
>>>>>>
>>>>>> from X
>>>>>> filter when Y
>>>>>> to A
>>>>>> filter when Y
>>>>>> to B
>>>>>> end // filter
>>>>>> to C // do something after filter for all messages
>>>>>>
>>>>>> so as to pick up when Y became false.
>>>>>>
>>>>>> Since I use an instance of Pipeline with an ArrayList of Processors,
>>>>>> this would become an ArrayList of FilterProcessor(predicate,
>>>>>> processor)?
>>>>>>
>>>>>> On Wed, Apr 20, 2011 at 4:59 AM, Claus Ibsen <cl...@gmail.com> wrote:
>>>>>>> On Wed, Apr 20, 2011 at 12:42 AM, Donald Whytock <dw...@gmail.com> wrote:
>>>>>>>> I ask because it appears to, in my application where I'm using
>>>>>>>> ROUTE_STOP to break out of a pipeline.
>>>>>>>>
>>>>>>>> Perhaps there should be some mechanism for breaking out of a pipeline
>>>>>>>> without killing the whole route? A PIPELINE_STOP property, or maybe
>>>>>>>> ROUTE_STOP could contain the ID of the route to be broken out of,
>>>>>>>> allowing containing routes to continue?
>>>>>>>>
>>>>>>>
>>>>>>> You can use the filter EIP to only process messages based on a predicate.
>>>>>>>
>>>>>>> from X
>>>>>>> filter when Y
>>>>>>> to A
>>>>>>> to B
>>>>>>> end // filter
>>>>>>> to C // do something after filter for all messages
>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Apr 19, 2011 at 3:40 PM, Donald Whytock <dw...@gmail.com> wrote:
>>>>>>>>> Hi all...
>>>>>>>>>
>>>>>>>>> Does using ROUTE_STOP on an email exchange prevent the message from
>>>>>>>>> being deleted from the POP3 server?
>>>>>>>>>
>>>>>>>>> Don
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Claus Ibsen
>>>>>>> -----------------
>>>>>>> FuseSource
>>>>>>> Email: cibsen@fusesource.com
>>>>>>> Web: http://fusesource.com
>>>>>>> CamelOne 2011: http://fusesource.com/camelone2011/
>>>>>>> Twitter: davsclaus
>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> -----------------
>>>> FuseSource
>>>> Email: cibsen@fusesource.com
>>>> Web: http://fusesource.com
>>>> CamelOne 2011: http://fusesource.com/camelone2011/
>>>> Twitter: davsclaus
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> -----------------
>> FuseSource
>> Email: cibsen@fusesource.com
>> Web: http://fusesource.com
>> CamelOne 2011: http://fusesource.com/camelone2011/
>> Twitter: davsclaus
>> Blog: http://davsclaus.blogspot.com/
>> Author of Camel in Action: http://www.manning.com/ibsen/
>>
>
Re: ROUTE_STOP and mail
Posted by Donald Whytock <dw...@gmail.com>.
Using Apache James at the moment. The current stable version of that
(2.3.2) doesn't support IMAP. ("A backport to James Server 2.3.2 would
require a volunteer.")
I have another localhost server I can try. Might work better anyway.
Ideally, the production app won't be running on a localhost email
server...
On Sat, Apr 23, 2011 at 3:22 AM, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> Ah try using imap instead of pop3. pop3 is very limited and it may not
> work setting those flags. I think we have seen this in the past
> setting the flags to SEEN or DELETED etc.
>
> We have a little tip about use imap instead of pop3 at the wiki page
> http://camel.apache.org/mail.html
>
>
>
> On Fri, Apr 22, 2011 at 9:27 PM, Donald Whytock <dw...@gmail.com> wrote:
>> At the moment my route is very short:
>>
>> from("pop3://bot@localhost?password=test")
>> .process(message1)
>> .process(peeker)
>> .choice()
>> .when(property("chatterbot.email").isNotNull())
>> .to("seda:mailtester");
>>
>> from("seda:mailtester")
>> .routeId("listener.test")
>> .process(message3);
>>
>> Peeker verifies the "from" header against a table; if found, it
>> changes the message body to String and sets chatterbot.email to
>> "true". Message1 and message3 kick out static messages to System.out.
>> The seda:mailtester consumer is in a different Felix bundle from the
>> mail consumer. Very little processing, very little time.
>>
>> I can send it an email, which it will pick up. Message 1 fires,
>> peeker shows the from address and body, message3 fires, and then I see
>> the debug log say the DELETED flag was set on the message ("Exchange
>> processed, so flagging message as DELETED"). A bit later I see the
>> same message being picked up again, and again the DELETED flag is set.
>> Occasionally I'll see the message picked up a third time. After that
>> the message doesn't get picked up again.
>>
>> Oddly, if I remove the routeId(), it stops happening. If I put the
>> routeId() back in, the problem happens again.
>>
>> On Fri, Apr 22, 2011 at 5:00 AM, Claus Ibsen <cl...@gmail.com> wrote:
>>> On Thu, Apr 21, 2011 at 8:50 PM, Donald Whytock <dw...@gmail.com> wrote:
>>>> Okay, it's not ROUTE_STOP, or not necessarily. I've been getting
>>>> mixed results with email deletion. In all cases I see the message
>>>> "Exchange processed, so flagging message as DELETED" coming from
>>>> MailConsumer.processCommit() as the exchange's onCompletion is run,
>>>> but in some cases the email doesn't actually get deleted, and shows up
>>>> again in the queues with the next polling.
>>>>
>>>
>>> How long time do it take to process the exchange, when you notice the
>>> email isn't deleted?
>>> Maybe the mail session times out, and it wont re-connect to set the
>>> DELETED flag.
>>> There should be something in the logs as camel-mail log at DEBUG level
>>> when it attempts to delete the mail.
>>>
>>>
>>>
>>>> I've removed and restored things I'm doing in my routes, and nothing
>>>> consistently causes the email to not be deleted. I'm at the point now
>>>> where, without changing the route, I see a message appear once, twice
>>>> or three times (varying) before it gets deleted.
>>>>
>>>> Is there a timeout on message deletion, such that if my route takes
>>>> too long the message won't get deleted? Do I need to copy the
>>>> exchange as soon as I get it, finish with the original and run with
>>>> the copy?
>>>>
>>>> On Wed, Apr 20, 2011 at 11:35 AM, Donald Whytock <dw...@gmail.com> wrote:
>>>>> But if I do filter on the pipeline it'll determine whether the entire
>>>>> pipeline is or isn't performed, right? So to break out of the
>>>>> pipeline I'd have to put the filter on each pipelined item, such as
>>>>>
>>>>> from X
>>>>> filter when Y
>>>>> to A
>>>>> filter when Y
>>>>> to B
>>>>> end // filter
>>>>> to C // do something after filter for all messages
>>>>>
>>>>> so as to pick up when Y became false.
>>>>>
>>>>> Since I use an instance of Pipeline with an ArrayList of Processors,
>>>>> this would become an ArrayList of FilterProcessor(predicate,
>>>>> processor)?
>>>>>
>>>>> On Wed, Apr 20, 2011 at 4:59 AM, Claus Ibsen <cl...@gmail.com> wrote:
>>>>>> On Wed, Apr 20, 2011 at 12:42 AM, Donald Whytock <dw...@gmail.com> wrote:
>>>>>>> I ask because it appears to, in my application where I'm using
>>>>>>> ROUTE_STOP to break out of a pipeline.
>>>>>>>
>>>>>>> Perhaps there should be some mechanism for breaking out of a pipeline
>>>>>>> without killing the whole route? A PIPELINE_STOP property, or maybe
>>>>>>> ROUTE_STOP could contain the ID of the route to be broken out of,
>>>>>>> allowing containing routes to continue?
>>>>>>>
>>>>>>
>>>>>> You can use the filter EIP to only process messages based on a predicate.
>>>>>>
>>>>>> from X
>>>>>> filter when Y
>>>>>> to A
>>>>>> to B
>>>>>> end // filter
>>>>>> to C // do something after filter for all messages
>>>>>>
>>>>>>>
>>>>>>> On Tue, Apr 19, 2011 at 3:40 PM, Donald Whytock <dw...@gmail.com> wrote:
>>>>>>>> Hi all...
>>>>>>>>
>>>>>>>> Does using ROUTE_STOP on an email exchange prevent the message from
>>>>>>>> being deleted from the POP3 server?
>>>>>>>>
>>>>>>>> Don
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Claus Ibsen
>>>>>> -----------------
>>>>>> FuseSource
>>>>>> Email: cibsen@fusesource.com
>>>>>> Web: http://fusesource.com
>>>>>> CamelOne 2011: http://fusesource.com/camelone2011/
>>>>>> Twitter: davsclaus
>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> -----------------
>>> FuseSource
>>> Email: cibsen@fusesource.com
>>> Web: http://fusesource.com
>>> CamelOne 2011: http://fusesource.com/camelone2011/
>>> Twitter: davsclaus
>>> Blog: http://davsclaus.blogspot.com/
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>
>>
>
>
>
> --
> Claus Ibsen
> -----------------
> FuseSource
> Email: cibsen@fusesource.com
> Web: http://fusesource.com
> CamelOne 2011: http://fusesource.com/camelone2011/
> Twitter: davsclaus
> Blog: http://davsclaus.blogspot.com/
> Author of Camel in Action: http://www.manning.com/ibsen/
>
Re: ROUTE_STOP and mail
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Ah try using imap instead of pop3. pop3 is very limited and it may not
work setting those flags. I think we have seen this in the past
setting the flags to SEEN or DELETED etc.
We have a little tip about use imap instead of pop3 at the wiki page
http://camel.apache.org/mail.html
On Fri, Apr 22, 2011 at 9:27 PM, Donald Whytock <dw...@gmail.com> wrote:
> At the moment my route is very short:
>
> from("pop3://bot@localhost?password=test")
> .process(message1)
> .process(peeker)
> .choice()
> .when(property("chatterbot.email").isNotNull())
> .to("seda:mailtester");
>
> from("seda:mailtester")
> .routeId("listener.test")
> .process(message3);
>
> Peeker verifies the "from" header against a table; if found, it
> changes the message body to String and sets chatterbot.email to
> "true". Message1 and message3 kick out static messages to System.out.
> The seda:mailtester consumer is in a different Felix bundle from the
> mail consumer. Very little processing, very little time.
>
> I can send it an email, which it will pick up. Message 1 fires,
> peeker shows the from address and body, message3 fires, and then I see
> the debug log say the DELETED flag was set on the message ("Exchange
> processed, so flagging message as DELETED"). A bit later I see the
> same message being picked up again, and again the DELETED flag is set.
> Occasionally I'll see the message picked up a third time. After that
> the message doesn't get picked up again.
>
> Oddly, if I remove the routeId(), it stops happening. If I put the
> routeId() back in, the problem happens again.
>
> On Fri, Apr 22, 2011 at 5:00 AM, Claus Ibsen <cl...@gmail.com> wrote:
>> On Thu, Apr 21, 2011 at 8:50 PM, Donald Whytock <dw...@gmail.com> wrote:
>>> Okay, it's not ROUTE_STOP, or not necessarily. I've been getting
>>> mixed results with email deletion. In all cases I see the message
>>> "Exchange processed, so flagging message as DELETED" coming from
>>> MailConsumer.processCommit() as the exchange's onCompletion is run,
>>> but in some cases the email doesn't actually get deleted, and shows up
>>> again in the queues with the next polling.
>>>
>>
>> How long time do it take to process the exchange, when you notice the
>> email isn't deleted?
>> Maybe the mail session times out, and it wont re-connect to set the
>> DELETED flag.
>> There should be something in the logs as camel-mail log at DEBUG level
>> when it attempts to delete the mail.
>>
>>
>>
>>> I've removed and restored things I'm doing in my routes, and nothing
>>> consistently causes the email to not be deleted. I'm at the point now
>>> where, without changing the route, I see a message appear once, twice
>>> or three times (varying) before it gets deleted.
>>>
>>> Is there a timeout on message deletion, such that if my route takes
>>> too long the message won't get deleted? Do I need to copy the
>>> exchange as soon as I get it, finish with the original and run with
>>> the copy?
>>>
>>> On Wed, Apr 20, 2011 at 11:35 AM, Donald Whytock <dw...@gmail.com> wrote:
>>>> But if I do filter on the pipeline it'll determine whether the entire
>>>> pipeline is or isn't performed, right? So to break out of the
>>>> pipeline I'd have to put the filter on each pipelined item, such as
>>>>
>>>> from X
>>>> filter when Y
>>>> to A
>>>> filter when Y
>>>> to B
>>>> end // filter
>>>> to C // do something after filter for all messages
>>>>
>>>> so as to pick up when Y became false.
>>>>
>>>> Since I use an instance of Pipeline with an ArrayList of Processors,
>>>> this would become an ArrayList of FilterProcessor(predicate,
>>>> processor)?
>>>>
>>>> On Wed, Apr 20, 2011 at 4:59 AM, Claus Ibsen <cl...@gmail.com> wrote:
>>>>> On Wed, Apr 20, 2011 at 12:42 AM, Donald Whytock <dw...@gmail.com> wrote:
>>>>>> I ask because it appears to, in my application where I'm using
>>>>>> ROUTE_STOP to break out of a pipeline.
>>>>>>
>>>>>> Perhaps there should be some mechanism for breaking out of a pipeline
>>>>>> without killing the whole route? A PIPELINE_STOP property, or maybe
>>>>>> ROUTE_STOP could contain the ID of the route to be broken out of,
>>>>>> allowing containing routes to continue?
>>>>>>
>>>>>
>>>>> You can use the filter EIP to only process messages based on a predicate.
>>>>>
>>>>> from X
>>>>> filter when Y
>>>>> to A
>>>>> to B
>>>>> end // filter
>>>>> to C // do something after filter for all messages
>>>>>
>>>>>>
>>>>>> On Tue, Apr 19, 2011 at 3:40 PM, Donald Whytock <dw...@gmail.com> wrote:
>>>>>>> Hi all...
>>>>>>>
>>>>>>> Does using ROUTE_STOP on an email exchange prevent the message from
>>>>>>> being deleted from the POP3 server?
>>>>>>>
>>>>>>> Don
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> -----------------
>>>>> FuseSource
>>>>> Email: cibsen@fusesource.com
>>>>> Web: http://fusesource.com
>>>>> CamelOne 2011: http://fusesource.com/camelone2011/
>>>>> Twitter: davsclaus
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>
>>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> -----------------
>> FuseSource
>> Email: cibsen@fusesource.com
>> Web: http://fusesource.com
>> CamelOne 2011: http://fusesource.com/camelone2011/
>> Twitter: davsclaus
>> Blog: http://davsclaus.blogspot.com/
>> Author of Camel in Action: http://www.manning.com/ibsen/
>>
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
CamelOne 2011: http://fusesource.com/camelone2011/
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: ROUTE_STOP and mail
Posted by Donald Whytock <dw...@gmail.com>.
At the moment my route is very short:
from("pop3://bot@localhost?password=test")
.process(message1)
.process(peeker)
.choice()
.when(property("chatterbot.email").isNotNull())
.to("seda:mailtester");
from("seda:mailtester")
.routeId("listener.test")
.process(message3);
Peeker verifies the "from" header against a table; if found, it
changes the message body to String and sets chatterbot.email to
"true". Message1 and message3 kick out static messages to System.out.
The seda:mailtester consumer is in a different Felix bundle from the
mail consumer. Very little processing, very little time.
I can send it an email, which it will pick up. Message 1 fires,
peeker shows the from address and body, message3 fires, and then I see
the debug log say the DELETED flag was set on the message ("Exchange
processed, so flagging message as DELETED"). A bit later I see the
same message being picked up again, and again the DELETED flag is set.
Occasionally I'll see the message picked up a third time. After that
the message doesn't get picked up again.
Oddly, if I remove the routeId(), it stops happening. If I put the
routeId() back in, the problem happens again.
On Fri, Apr 22, 2011 at 5:00 AM, Claus Ibsen <cl...@gmail.com> wrote:
> On Thu, Apr 21, 2011 at 8:50 PM, Donald Whytock <dw...@gmail.com> wrote:
>> Okay, it's not ROUTE_STOP, or not necessarily. I've been getting
>> mixed results with email deletion. In all cases I see the message
>> "Exchange processed, so flagging message as DELETED" coming from
>> MailConsumer.processCommit() as the exchange's onCompletion is run,
>> but in some cases the email doesn't actually get deleted, and shows up
>> again in the queues with the next polling.
>>
>
> How long time do it take to process the exchange, when you notice the
> email isn't deleted?
> Maybe the mail session times out, and it wont re-connect to set the
> DELETED flag.
> There should be something in the logs as camel-mail log at DEBUG level
> when it attempts to delete the mail.
>
>
>
>> I've removed and restored things I'm doing in my routes, and nothing
>> consistently causes the email to not be deleted. I'm at the point now
>> where, without changing the route, I see a message appear once, twice
>> or three times (varying) before it gets deleted.
>>
>> Is there a timeout on message deletion, such that if my route takes
>> too long the message won't get deleted? Do I need to copy the
>> exchange as soon as I get it, finish with the original and run with
>> the copy?
>>
>> On Wed, Apr 20, 2011 at 11:35 AM, Donald Whytock <dw...@gmail.com> wrote:
>>> But if I do filter on the pipeline it'll determine whether the entire
>>> pipeline is or isn't performed, right? So to break out of the
>>> pipeline I'd have to put the filter on each pipelined item, such as
>>>
>>> from X
>>> filter when Y
>>> to A
>>> filter when Y
>>> to B
>>> end // filter
>>> to C // do something after filter for all messages
>>>
>>> so as to pick up when Y became false.
>>>
>>> Since I use an instance of Pipeline with an ArrayList of Processors,
>>> this would become an ArrayList of FilterProcessor(predicate,
>>> processor)?
>>>
>>> On Wed, Apr 20, 2011 at 4:59 AM, Claus Ibsen <cl...@gmail.com> wrote:
>>>> On Wed, Apr 20, 2011 at 12:42 AM, Donald Whytock <dw...@gmail.com> wrote:
>>>>> I ask because it appears to, in my application where I'm using
>>>>> ROUTE_STOP to break out of a pipeline.
>>>>>
>>>>> Perhaps there should be some mechanism for breaking out of a pipeline
>>>>> without killing the whole route? A PIPELINE_STOP property, or maybe
>>>>> ROUTE_STOP could contain the ID of the route to be broken out of,
>>>>> allowing containing routes to continue?
>>>>>
>>>>
>>>> You can use the filter EIP to only process messages based on a predicate.
>>>>
>>>> from X
>>>> filter when Y
>>>> to A
>>>> to B
>>>> end // filter
>>>> to C // do something after filter for all messages
>>>>
>>>>>
>>>>> On Tue, Apr 19, 2011 at 3:40 PM, Donald Whytock <dw...@gmail.com> wrote:
>>>>>> Hi all...
>>>>>>
>>>>>> Does using ROUTE_STOP on an email exchange prevent the message from
>>>>>> being deleted from the POP3 server?
>>>>>>
>>>>>> Don
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> -----------------
>>>> FuseSource
>>>> Email: cibsen@fusesource.com
>>>> Web: http://fusesource.com
>>>> CamelOne 2011: http://fusesource.com/camelone2011/
>>>> Twitter: davsclaus
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>
>>>
>>
>
>
>
> --
> Claus Ibsen
> -----------------
> FuseSource
> Email: cibsen@fusesource.com
> Web: http://fusesource.com
> CamelOne 2011: http://fusesource.com/camelone2011/
> Twitter: davsclaus
> Blog: http://davsclaus.blogspot.com/
> Author of Camel in Action: http://www.manning.com/ibsen/
>
Re: ROUTE_STOP and mail
Posted by Claus Ibsen <cl...@gmail.com>.
On Thu, Apr 21, 2011 at 8:50 PM, Donald Whytock <dw...@gmail.com> wrote:
> Okay, it's not ROUTE_STOP, or not necessarily. I've been getting
> mixed results with email deletion. In all cases I see the message
> "Exchange processed, so flagging message as DELETED" coming from
> MailConsumer.processCommit() as the exchange's onCompletion is run,
> but in some cases the email doesn't actually get deleted, and shows up
> again in the queues with the next polling.
>
How long time do it take to process the exchange, when you notice the
email isn't deleted?
Maybe the mail session times out, and it wont re-connect to set the
DELETED flag.
There should be something in the logs as camel-mail log at DEBUG level
when it attempts to delete the mail.
> I've removed and restored things I'm doing in my routes, and nothing
> consistently causes the email to not be deleted. I'm at the point now
> where, without changing the route, I see a message appear once, twice
> or three times (varying) before it gets deleted.
>
> Is there a timeout on message deletion, such that if my route takes
> too long the message won't get deleted? Do I need to copy the
> exchange as soon as I get it, finish with the original and run with
> the copy?
>
> On Wed, Apr 20, 2011 at 11:35 AM, Donald Whytock <dw...@gmail.com> wrote:
>> But if I do filter on the pipeline it'll determine whether the entire
>> pipeline is or isn't performed, right? So to break out of the
>> pipeline I'd have to put the filter on each pipelined item, such as
>>
>> from X
>> filter when Y
>> to A
>> filter when Y
>> to B
>> end // filter
>> to C // do something after filter for all messages
>>
>> so as to pick up when Y became false.
>>
>> Since I use an instance of Pipeline with an ArrayList of Processors,
>> this would become an ArrayList of FilterProcessor(predicate,
>> processor)?
>>
>> On Wed, Apr 20, 2011 at 4:59 AM, Claus Ibsen <cl...@gmail.com> wrote:
>>> On Wed, Apr 20, 2011 at 12:42 AM, Donald Whytock <dw...@gmail.com> wrote:
>>>> I ask because it appears to, in my application where I'm using
>>>> ROUTE_STOP to break out of a pipeline.
>>>>
>>>> Perhaps there should be some mechanism for breaking out of a pipeline
>>>> without killing the whole route? A PIPELINE_STOP property, or maybe
>>>> ROUTE_STOP could contain the ID of the route to be broken out of,
>>>> allowing containing routes to continue?
>>>>
>>>
>>> You can use the filter EIP to only process messages based on a predicate.
>>>
>>> from X
>>> filter when Y
>>> to A
>>> to B
>>> end // filter
>>> to C // do something after filter for all messages
>>>
>>>>
>>>> On Tue, Apr 19, 2011 at 3:40 PM, Donald Whytock <dw...@gmail.com> wrote:
>>>>> Hi all...
>>>>>
>>>>> Does using ROUTE_STOP on an email exchange prevent the message from
>>>>> being deleted from the POP3 server?
>>>>>
>>>>> Don
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> -----------------
>>> FuseSource
>>> Email: cibsen@fusesource.com
>>> Web: http://fusesource.com
>>> CamelOne 2011: http://fusesource.com/camelone2011/
>>> Twitter: davsclaus
>>> Blog: http://davsclaus.blogspot.com/
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>
>>
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
CamelOne 2011: http://fusesource.com/camelone2011/
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: ROUTE_STOP and mail
Posted by Donald Whytock <dw...@gmail.com>.
Okay, it's not ROUTE_STOP, or not necessarily. I've been getting
mixed results with email deletion. In all cases I see the message
"Exchange processed, so flagging message as DELETED" coming from
MailConsumer.processCommit() as the exchange's onCompletion is run,
but in some cases the email doesn't actually get deleted, and shows up
again in the queues with the next polling.
I've removed and restored things I'm doing in my routes, and nothing
consistently causes the email to not be deleted. I'm at the point now
where, without changing the route, I see a message appear once, twice
or three times (varying) before it gets deleted.
Is there a timeout on message deletion, such that if my route takes
too long the message won't get deleted? Do I need to copy the
exchange as soon as I get it, finish with the original and run with
the copy?
On Wed, Apr 20, 2011 at 11:35 AM, Donald Whytock <dw...@gmail.com> wrote:
> But if I do filter on the pipeline it'll determine whether the entire
> pipeline is or isn't performed, right? So to break out of the
> pipeline I'd have to put the filter on each pipelined item, such as
>
> from X
> filter when Y
> to A
> filter when Y
> to B
> end // filter
> to C // do something after filter for all messages
>
> so as to pick up when Y became false.
>
> Since I use an instance of Pipeline with an ArrayList of Processors,
> this would become an ArrayList of FilterProcessor(predicate,
> processor)?
>
> On Wed, Apr 20, 2011 at 4:59 AM, Claus Ibsen <cl...@gmail.com> wrote:
>> On Wed, Apr 20, 2011 at 12:42 AM, Donald Whytock <dw...@gmail.com> wrote:
>>> I ask because it appears to, in my application where I'm using
>>> ROUTE_STOP to break out of a pipeline.
>>>
>>> Perhaps there should be some mechanism for breaking out of a pipeline
>>> without killing the whole route? A PIPELINE_STOP property, or maybe
>>> ROUTE_STOP could contain the ID of the route to be broken out of,
>>> allowing containing routes to continue?
>>>
>>
>> You can use the filter EIP to only process messages based on a predicate.
>>
>> from X
>> filter when Y
>> to A
>> to B
>> end // filter
>> to C // do something after filter for all messages
>>
>>>
>>> On Tue, Apr 19, 2011 at 3:40 PM, Donald Whytock <dw...@gmail.com> wrote:
>>>> Hi all...
>>>>
>>>> Does using ROUTE_STOP on an email exchange prevent the message from
>>>> being deleted from the POP3 server?
>>>>
>>>> Don
>>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> -----------------
>> FuseSource
>> Email: cibsen@fusesource.com
>> Web: http://fusesource.com
>> CamelOne 2011: http://fusesource.com/camelone2011/
>> Twitter: davsclaus
>> Blog: http://davsclaus.blogspot.com/
>> Author of Camel in Action: http://www.manning.com/ibsen/
>>
>
Re: ROUTE_STOP and mail
Posted by Donald Whytock <dw...@gmail.com>.
But if I do filter on the pipeline it'll determine whether the entire
pipeline is or isn't performed, right? So to break out of the
pipeline I'd have to put the filter on each pipelined item, such as
from X
filter when Y
to A
filter when Y
to B
end // filter
to C // do something after filter for all messages
so as to pick up when Y became false.
Since I use an instance of Pipeline with an ArrayList of Processors,
this would become an ArrayList of FilterProcessor(predicate,
processor)?
On Wed, Apr 20, 2011 at 4:59 AM, Claus Ibsen <cl...@gmail.com> wrote:
> On Wed, Apr 20, 2011 at 12:42 AM, Donald Whytock <dw...@gmail.com> wrote:
>> I ask because it appears to, in my application where I'm using
>> ROUTE_STOP to break out of a pipeline.
>>
>> Perhaps there should be some mechanism for breaking out of a pipeline
>> without killing the whole route? A PIPELINE_STOP property, or maybe
>> ROUTE_STOP could contain the ID of the route to be broken out of,
>> allowing containing routes to continue?
>>
>
> You can use the filter EIP to only process messages based on a predicate.
>
> from X
> filter when Y
> to A
> to B
> end // filter
> to C // do something after filter for all messages
>
>>
>> On Tue, Apr 19, 2011 at 3:40 PM, Donald Whytock <dw...@gmail.com> wrote:
>>> Hi all...
>>>
>>> Does using ROUTE_STOP on an email exchange prevent the message from
>>> being deleted from the POP3 server?
>>>
>>> Don
>>>
>>
>
>
>
> --
> Claus Ibsen
> -----------------
> FuseSource
> Email: cibsen@fusesource.com
> Web: http://fusesource.com
> CamelOne 2011: http://fusesource.com/camelone2011/
> Twitter: davsclaus
> Blog: http://davsclaus.blogspot.com/
> Author of Camel in Action: http://www.manning.com/ibsen/
>
Re: ROUTE_STOP and mail
Posted by Claus Ibsen <cl...@gmail.com>.
On Wed, Apr 20, 2011 at 12:42 AM, Donald Whytock <dw...@gmail.com> wrote:
> I ask because it appears to, in my application where I'm using
> ROUTE_STOP to break out of a pipeline.
>
> Perhaps there should be some mechanism for breaking out of a pipeline
> without killing the whole route? A PIPELINE_STOP property, or maybe
> ROUTE_STOP could contain the ID of the route to be broken out of,
> allowing containing routes to continue?
>
You can use the filter EIP to only process messages based on a predicate.
from X
filter when Y
to A
to B
end // filter
to C // do something after filter for all messages
>
> On Tue, Apr 19, 2011 at 3:40 PM, Donald Whytock <dw...@gmail.com> wrote:
>> Hi all...
>>
>> Does using ROUTE_STOP on an email exchange prevent the message from
>> being deleted from the POP3 server?
>>
>> Don
>>
>
--
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
CamelOne 2011: http://fusesource.com/camelone2011/
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Re: ROUTE_STOP and mail
Posted by Donald Whytock <dw...@gmail.com>.
I ask because it appears to, in my application where I'm using
ROUTE_STOP to break out of a pipeline.
Perhaps there should be some mechanism for breaking out of a pipeline
without killing the whole route? A PIPELINE_STOP property, or maybe
ROUTE_STOP could contain the ID of the route to be broken out of,
allowing containing routes to continue?
On Tue, Apr 19, 2011 at 3:40 PM, Donald Whytock <dw...@gmail.com> wrote:
> Hi all...
>
> Does using ROUTE_STOP on an email exchange prevent the message from
> being deleted from the POP3 server?
>
> Don
>