Re: Capture IMAP events for use in our CRM

[Eric Charles <er...@apache.org>]

Hi,
Glad to see it's working and congrats for your imagination.

For your suggestion on the system session, would you mind opening a JIRA 
for this ? (https://issues.apache.org/jira/browse/IMAP)

Thx,
Eric

On 31/01/12 09:53, lclhst wrote:
> Thank you for your helpful posts, Eric. It really helped me figure
> some things out.
>
> So here's what I'm going to do:
>
> Create 2 beans:
> - SyncToCrmBean
> - SyncFromCrmBean
>
> SyncToCrmBean:
> - Registers a global mailbox listener, so it receives all the events
> related to the modification of any message
> - Opens a JMS channel (we're probably going to use ActiveMQ, since
> that's already integrated with JAMES, too)
> - It only handles events with the session type "user" (so we only
> react to events actually done by  the user)
> - Whenever we get a notification, we create the appropriate JMS
> messages and send them to the JMS channel, including the message ID of
> the corresponding message (so we can find the correct messages in our
> CRM)
>
> SyncFromCrmBean:
> - We start a JMS listener, listening for messages that we send to this
> queue. They come from our CRM and contain status information about
> messages
> - We use the mailbox manager to create a system session
> - Through this system session we access the correct mailbox (using
> e-mail address, namespace and mailbox name)
> - We find the messages using the Message-ID supplied from our CRM
> - We do the appropriate actions on the messages
> - JAMES triggers the message events, but they are triggered from a
> system session, so we can ignore these in the SyncToCrmBean.
>
> Then we have some programming to do in our CRM/database/ESB, but this
> doesn't really regard JAMES. It basically takes care of sending and
> receiving messages to and from the appropriate JMS channels to do
> status updates in the database.
>
> But as far as I can see, JAMES is ready to do everything we want for now. :)
>
> PS: Is it possible to create a system session through a normal
> IMAP-connection? It would be so much easier to just push the messages
> to our IMAP-server using standard tools, rather than receiving
> everything through JMS or something else and programmatically storing
> everything etc.
>
> On Mon, Jan 30, 2012 at 7:39 PM, Eric Charles<er...@apache.org>  wrote:
>> Thx a lot for sharing this.
>> (forget my last post, it has being crossing the more detailed information
>> you give here).
>>
>> I will further reply tomorrow (short on time here), but smooth migration to
>> james sounds indeed very exciting :)
>>
>> Eric
>>
>>
>> On 30/01/12 17:31, lclhst wrote:
>>>
>>> Hi, yes and no. :)
>>>
>>> For now, we have our CRM as an e-mail client. And we have a custom
>>> e-mail server, which is FAR from RFC-compliant. :)
>>>
>>> That's why we want to switch to a real mailserver and we're going to
>>> use Apache JAMES for this.
>>>
>>> However, we cannot make such a big step at once, so what we want to do
>>> now is have our mailserver (JAMES) and our custommailserver
>>> synchronise.
>>>
>>> So the basic idea is:
>>> When a user marks a message as read in our CRM, we send a signal
>>> (probably through JMS or something), then pick up that signal in a
>>> custom Apache JAMES component, then mark the message read on the
>>> IMAP-server. This way we can synchronize.
>>> And also the other way around: When a user marks a message as read in
>>> the IMAP-server (JAMES), we send a signal to some to-be-defined
>>> server, which will mark the message as read in our CRM. And we're
>>> synchronized.
>>>
>>> However, when I mark a message as read through the JAMES-API (or
>>> through an IMAP-session), an event "MailboxListener.event(FlagsUpdated
>>> event)" is triggered and we also respond to that, so we will update
>>> our CRM again. This is something I want to prevent. So somehow, I
>>> should be able to mark a message read in JAMES, without triggering an
>>> event. OR I should be able to detect the currently connected host
>>> (which will return the IP address of the server that created the
>>> IMAP-session), so I can filter out events coming from that host. OR I
>>> should be able to access all mailboxes and accounts with a certain
>>> "system" user. I believe the latter is possible, but I wouldn't know
>>> what username to specify.
>>>
>>> I hope I'm making this clear. I actually think this discussion would
>>> be quite useful for others too, since I can't believe I'm going to be
>>> the only one trying to do this. :)
>>>
>>> And yes, eventually our CRM will be a real IMAP-client. But for now,
>>> we'll have to do this step by step. :) But it's quite fun to dig into
>>> JAMES! :) It's lovely.
>>>
>>> Thanks for the input.
>>>
>>> On Mon, Jan 30, 2012 at 5:21 PM, Eric Charles<er...@apache.org>    wrote:
>>>>
>>>>
>>>> So your CRM acts as an IMAP client, right and ?
>>>> If such, you could assign a username/password to your CRM.
>>>>
>>>> The event contains the MailboSession which has the
>>>> getUser().getUsername(), so you will know if the event has been triggered by
>>>> the CRM or by a real user.
>>>>
>>>> Did I get your point?
>>>> Eric
>>>>
>>>>
>>>> On 30/01/12 14:25, lclhst wrote:
>>>>>
>>>>>
>>>>> Thank you for the useful replies.
>>>>>
>>>>> Ok, creating a reference to the mailboxmanager using @Resource is a lot
>>>>> better. So that's nice.
>>>>>
>>>>> Regarding the hostname, I was looking for a way to detect wether this
>>>>> session is a user-session (i.e. an actual user that reads a message in
>>>>> his
>>>>> IMAP-inbox) or a system-session (our sync-program that marks a message
>>>>> as
>>>>> read, because a message was marked as read in our CRM). When I create
>>>>> that
>>>>> IMAP-session and mark the message as read, it's a user-session, so I
>>>>> can't
>>>>> distinguish that.
>>>>>
>>>>> On Mon, Jan 30, 2012 at 1:39 PM, Eric Charles<er...@apache.org>      wrote:
>>>>>
>>>>>> I don't know much from spring integration, but it will probably not fit
>>>>>> our existing event system.
>>>>>>
>>>>>>
>>>>>> On 30/01/12 10:14, Dhrubo wrote:
>>>>>>
>>>>>>> Can we use something like an integration pack for example Spring
>>>>>>> integration which has spring imap support? But offcourse event is
>>>>>>> required.
>>>>>>>
>>>>>>> On Mon, Jan 30, 2012 at 2:08 PM,
>>>>>>> lclhst<l0...@gmail.com>>
>>>>>>>
>>>>>>>   wrote:
>>>>>>>
>>>>>>>   Does anybody have any input to this?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> I'm now creating my CustomMailboxListener with a 1 parameter
>>>>>>>> constructor, which is the MailboxManager, so I can access the
>>>>>>>> mailboxes.
>>>>>>>>
>>>>>>>> Not sure on the questions I asked before yet, though.
>>>>>>>>
>>>>>>>> On Fri, Jan 27, 2012 at 6:05 PM,
>>>>>>>> lclhst<l0...@gmail.com>>
>>>>>>>>   wrote:
>>>>>>>>
>>>>>>>>> Ok, I have been playing around a bit with this.
>>>>>>>>>
>>>>>>>>> I added the listener through the Spring XML files and now I receive
>>>>>>>>> the events in my listener. It's working great.
>>>>>>>>>
>>>>>>>>> When event.getSession().getType().**equals(SessionType.System), the
>>>>>>>>>
>>>>>>>>> e-mail is being handled/received/modified through SMTP, right?
>>>>>>>>> Otherwise it's user, which is the user doing some actions himself,
>>>>>>>>> right?
>>>>>>>>>
>>>>>>>>> So this way I could check if a user is dragging an e-mail inside his
>>>>>>>>> INBOX, or it's a message received by SMTP.
>>>>>>>>>
>>>>>>>>> Because when a user is dragging an e-mail inside his INBOX, I need
>>>>>>>>> to
>>>>>>>>> sync this back to our CRM, so I can make the message there, too.
>>>>>>>>>
>>>>>>>>> Also, I was wondering what would be the best way to query the
>>>>>>>>> mailbox.
>>>>>>>>> For example, when I receive a message in the listener, could I use
>>>>>>>>> the
>>>>>>>>> event.getSession().**getAttributes().get("**MESSAGEMAPPER") safely?
>>>>>>>>>
>>>>>>>>> It's
>>>>>>>>> mentioned this class isn't thread safe. But I need to be able to
>>>>>>>>> fetch
>>>>>>>>> the full message including headers and everything, so I can sync it
>>>>>>>>> to
>>>>>>>>> our CRM.
>>>>>>>>>
>>>>>>>>> Also, I was wondering what would be the best way to do the status
>>>>>>>>> updates from our CRM. Because when I make an IMAP session and mark a
>>>>>>>>> message read, the listener receives it and syncs it back to our CRM.
>>>>>>>>> That way I'm doing double syncing, which isn't what we want of
>>>>>>>>> course.
>>>>>>>>>
>>>>>>>>> Could I check for example the host name inside the public void
>>>>>>>>> event(Event event)? I cannot seem to find any methods or properties
>>>>>>>>> that would give me a context containing these details.
>>>>>>>>>
>>>>>>>>> I also saw a createSystemSession()-method on the MailboxManager. I'm
>>>>>>>>> not sure how to access this MailboxManager though. I could register
>>>>>>>>> it
>>>>>>>>> to some class using the spring beans (or even my listener class),
>>>>>>>>> but
>>>>>>>>> I don't think that would be a good idea. I could set up something
>>>>>>>>> using JMS messaging so a class inside JAMES would be receiving
>>>>>>>>> messages through JMS and then update the mailbox. Though I think
>>>>>>>>> there
>>>>>>>>> must be a better way, but I cannot seem to find it yet.
>>>>>>>>>
>>>>>>>>> I hope you guys could give me a bit more insight on how to do this.
>>>>>>>>> Not asking for complete solutions. :) I just don't know where to
>>>>>>>>> look
>>>>>>>>> yet. :)
>>>>>>>>>
>>>>>>>>> Thanks for the help so far, much appreciated.
>>>>>>>>>
>>>>>>>>> On Thu, Jan 26, 2012 at 4:56 PM, Norman Maurer
>>>>>>>>> <no...@googlemail.com>       wrote:
>>>>>>>>>
>>>>>>>>>> Hi there,
>>>>>>>>>>
>>>>>>>>>> you can register a MailboxListener in the MailboxManager instance
>>>>>>>>>> and
>>>>>>>>>> so get notifications about mailbox events. This should do the job.
>>>>>>>>>>
>>>>>>>>>> Have a look here:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>   http://svn.apache.org/viewvc/**james/mailbox/trunk/api/src/**
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> main/java/org/apache/james/**mailbox/MailboxManager.java?**view=markup<http://svn.apache.org/viewvc/james/mailbox/trunk/api/src/main/java/org/apache/james/mailbox/MailboxManager.java?view=markup>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>   http://svn.apache.org/viewvc/**james/mailbox/trunk/api/src/**
>>>>>>>>
>>>>>>>>
>>>>>>>> main/java/org/apache/james/**mailbox/**MailboxListenerSupport.java?**
>>>>>>>>
>>>>>>>> view=markup<http://svn.apache.org/viewvc/james/mailbox/trunk/api/src/main/java/org/apache/james/mailbox/MailboxListenerSupport.java?view=markup>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>   http://svn.apache.org/viewvc/**james/mailbox/trunk/api/src/**
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> main/java/org/apache/james/**mailbox/MailboxListener.java?**view=markup<http://svn.apache.org/viewvc/james/mailbox/trunk/api/src/main/java/org/apache/james/mailbox/MailboxListener.java?view=markup>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Let me know if you need more info..
>>>>>>>>>>
>>>>>>>>>> Bye,
>>>>>>>>>> Norman
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2012/1/26 l0c4l h0st<l0...@gmail.com>**:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Hello,
>>>>>>>>>>>
>>>>>>>>>>> What I'm trying to do is capture events that the IMAP-server
>>>>>>>>>>> should
>>>>>>>>>>> (or is supposed to) raise.
>>>>>>>>>>>
>>>>>>>>>>> Whenever a message gets marked as read, I want to get a
>>>>>>>>>>> notification
>>>>>>>>>>> of this event.
>>>>>>>>>>> Whenever a message gets marked as unread, I want to get a
>>>>>>>>>>> notification
>>>>>>>>>>> of this event.
>>>>>>>>>>> Whenever a message gets flagged, I want to get a notification of
>>>>>>>>>>> this
>>>>>>>>>>>
>>>>>>>>>> event.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Etc. etc.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Whenever a user moves a message to a different IMAP folder, I want
>>>>>>>>>>> to
>>>>>>>>>>> get a notification of this event.
>>>>>>>>>>> Whenever a user deletes a message to a different IMAP folder, I
>>>>>>>>>>> want
>>>>>>>>>>> to get a notification of this event.
>>>>>>>>>>> Whenever a user copies a message to a different IMAP folder, I
>>>>>>>>>>> want to
>>>>>>>>>>> get a notification of this event.
>>>>>>>>>>>
>>>>>>>>>>> Right now, I don't see any way of doing this. I thought this would
>>>>>>>>>>> be
>>>>>>>>>>> possible with the Matchers and Mailets, but I don't see how this
>>>>>>>>>>> would
>>>>>>>>>>> be possible.
>>>>>>>>>>>
>>>>>>>>>>> I can define a custom mailet, that matches on ALL, when I add this
>>>>>>>>>>> mailet in the root- and transport-processors, I get notifications
>>>>>>>>>>> only
>>>>>>>>>>> of mail that arrives through SMTP and mail that is sent through
>>>>>>>>>>> SMTP.
>>>>>>>>>>>
>>>>>>>>>>> Am I missing the IMAP way of doing this? Or is there currently no
>>>>>>>>>>> way
>>>>>>>>>>> of doing this with IMAP?
>>>>>>>>>>>
>>>>>>>>>>> I've tried searching through the code, but I don't see any event
>>>>>>>>>>> listeners or anything that would be triggered. I did find
>>>>>>>>>>> ImapStateAwareMailboxListener, but I cannot find how to subscribe
>>>>>>>>>>> to
>>>>>>>>>>> this event or anything.
>>>>>>>>>>>
>>>>>>>>>>> Also, I found the UpdatedFlags class, but there is no notification
>>>>>>>>>>> sent anywhere.
>>>>>>>>>>>
>>>>>>>>>>> SO:
>>>>>>>>>>> How can I get a notification of every single change that is made
>>>>>>>>>>> to a
>>>>>>>>>>> message in IMAP? IMAP is currently most important for us.
>>>>>>>>>>>
>>>>>>>>>>> The reason I want to do this is we have a custom CRM tool that has
>>>>>>>>>>> a
>>>>>>>>>>> custom mail database and we want to slowly migrate to a real
>>>>>>>>>>> mailserver, we thought of using Apache JAMES to do
>>>>>>>>>>> synchronisation,
>>>>>>>>>>> because at this time we cannot yet realize a full migration. We
>>>>>>>>>>> want
>>>>>>>>>>> to do this in steps.
>>>>>>>>>>>
>>>>>>>>>>> My ultimate goal, regarding synchronisation, is to receive all
>>>>>>>>>>> events
>>>>>>>>>>> that are occurring in the IMAP part of JAMES, so we can update the
>>>>>>>>>>> status of the messages in our CRM-tool. My goal is also to
>>>>>>>>>>> synchronize
>>>>>>>>>>> changes in the CRM-tool back to IMAP-JAMES. But this can easily be
>>>>>>>>>>> done through an IMAP-session. That shouldn't be a problem.
>>>>>>>>>>>
>>>>>>>>>>> I hope anyone can help me out on this, because I find a lack of
>>>>>>>>>>> documentation regarding this subject.
>>>>>>>>>>>
>>>>>>>>>>> Right now, the only way of doing this seems to me creating
>>>>>>>>>>> database
>>>>>>>>>>> triggers (we're using this with PostgreSQL).
>>>>>>>>>>>
>>>>>>>>>>> Thank you in advance.
>>>>>>>>>>>
>>>>>>>>>>> ------------------------------**------------------------------**
>>>>>>>>>>> ---------
>>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>>> server-dev-unsubscribe@james.**apache.org<se...@james.apache.org>
>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>> server-dev-help@james.apache.**org<se...@james.apache.org>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> ------------------------------**------------------------------**
>>>>>>>>>> ---------
>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>> server-dev-unsubscribe@james.**apache.org<se...@james.apache.org>
>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>> server-dev-help@james.apache.**org<se...@james.apache.org>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> ------------------------------**------------------------------**
>>>>>>>> ---------
>>>>>>>> To unsubscribe, e-mail:
>>>>>>>> server-dev-unsubscribe@james.**apache.org<se...@james.apache.org>
>>>>>>>> For additional commands, e-mail:
>>>>>>>> server-dev-help@james.apache.**org<se...@james.apache.org>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> --
>>>>>> eric | http://about.echarles.net | @echarles
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------**------------------------------**---------
>>>>>> To unsubscribe, e-mail:
>>>>>> server-dev-unsubscribe@james.**apache.org<se...@james.apache.org>
>>>>>> For additional commands, e-mail:
>>>>>> server-dev-help@james.apache.**org<se...@james.apache.org>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>> --
>>>> eric | http://about.echarles.net | @echarles
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>
>>
>> --
>> eric | http://about.echarles.net | @echarles
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>> For additional commands, e-mail: server-dev-help@james.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>

-- 
eric | http://about.echarles.net | @echarles

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org