You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by Patrick Ruhkopf <pa...@gmail.com> on 2008/11/25 07:54:10 UTC
JCR and IMAP
Hi,
Being a big fan of the JCR API I'm thrilled by the idea to archive emails in
such a repository. You could easily access, search, backup mails and most
importantly link them with other content, attach meta data, etc.
However, I still want to be able to connect to that repository with my
favorite email client so I can read, delete, flag new mails as usual. So, I
guess what I want is an imap server that uses JCR.
Searching through the mailinglist I figure JAMES 3 is going to support both
IMAP and JCR. Yah! But..
It seems like it's currently an either or choice, isn't it? You either
specify JCR or IMAP as the repository. In the latter case Torque is used to
persist data. Is it possible to use the JCR store instead?
Thanks
Patrick
Re: JCR and IMAP
Posted by Robert Burrell Donkin <ro...@gmail.com>.
On 11/28/08, Steven Mak <tc...@yahoo.com> wrote:
> In my experience, the QUOTA function is kind of tricky.
QUOTA is nasty but thankfully it's an extension RFC
I'm most interested in personal and corporate solutions where the
margin cost of disk space is orders of magnitude less than the
benefit. So - for example - I'm keen on solutions - such as garbage
collection for unreferenced documents - which give the adminstrator
much control over entropy.
There have also been talk of aggressively trading storage space for
performance by using exploding MIME models to avoid excessive
reparsing.
So I'm glad that there are people who are interested in areas like QUOTA.
> There are at least
> two challenges:
>
> 1) Return the answer quickly, that depends on the data structure that stores
> the mail
+1
For IMAP, the raw octet mail count probably needs to be stored as
meta-data on the way
> 2) Overflow, I have encountered a bug with another IMAP server where it
> turned into negative number when the user got more than 4GB of mails. It's
> later found that it was the datatype storing such number is not large enough
> to cater the value.
+1
One of the interesting aspects about mail is the range of document sizes
Fancy volunteering to help add QUOTAs?
- Robert
>
> --- On Fri, 11/28/08, Robert Burrell Donkin <ro...@gmail.com>
> wrote:
> From: Robert Burrell Donkin <ro...@gmail.com>
> Subject: Re: JCR and IMAP
> To: "James Developers List" <se...@james.apache.org>
> Date: Friday, November 28, 2008, 6:27 AM
>
> On Wed, Nov 26, 2008 at 10:36 PM, Patrick Ruhkopf
> <pa...@gmail.com> wrote:
>> On Wed, Nov 26, 2008 at 3:40 AM, Robert Burrell Donkin
>> <ro...@gmail.com> wrote:
>>> It took me about a year to learn enough about IMAP to create a working
>>> implementation but the good news is that the hard leg work's over
> now
>>> - a comprehensive functional test suite exists together with working
>>> IMAP commands and codecs.
>>
>> Sweet! I've heard that imap is quite complex. I guess I don't want
> to
>> know all the reasons.
>
> +1
>
>> If only we could "reinvent Email using REST" [1]. ;-)
>
> that particular windmill is one i've been tilting at obliquely for a
> couple of years now ;-)
>
>>> I'd much rather start from first principals by designing a good
> model.
>>> I'm very familiar with the actual requirements (rather than the
> API's
>>> requirements). Second, create a fully working and tested module just
>>> containing the JCR stuff. Third, create an adapter module (probably by
>>> cloning the torque module) between the API and the JCR module.
>>
>> Sounds like plan. We just have to keep an eye on 3) along the way so
>> that writing an adapter doesn't become a complex time consuming task
>> spoiling the work done in 1) and 2).
>
> i'm now very familiar with IMAP's actual requirements and the
> inadequacies of the current API so 3) shouldn't be too much of a
> problem
>
> let me by start by giving a whirlwind tour of essential IMAP. please
> dive in an ask questions, comments and ideas...
>
> IMAP has three primary concepts: user, mail, mailbox.
>
> as well as authentication and authorisation, users have subscriptions
> (links to mailboxes).
>
> each mail is a MIME-typed read only document with meta-data
>
> a mailbox is a container for mail. additional mutable per-mailbox
> meta-data is set for each mail. probably more elegant to use links
> than actual containment.
>
> - robert
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: JCR and IMAP
Posted by Steven Mak <tc...@yahoo.com>.
In my experience, the QUOTA function is kind of tricky. There are at least two challenges:
1) Return the answer quickly, that depends on the data structure that stores the mail
2) Overflow, I have encountered a bug with another IMAP server where it turned into negative number when the user got more than 4GB of mails. It's later found that it was the datatype storing such number is not large enough to cater the value.
--- On Fri, 11/28/08, Robert Burrell Donkin <ro...@gmail.com> wrote:
From: Robert Burrell Donkin <ro...@gmail.com>
Subject: Re: JCR and IMAP
To: "James Developers List" <se...@james.apache.org>
Date: Friday, November 28, 2008, 6:27 AM
On Wed, Nov 26, 2008 at 10:36 PM, Patrick Ruhkopf
<pa...@gmail.com> wrote:
> On Wed, Nov 26, 2008 at 3:40 AM, Robert Burrell Donkin
> <ro...@gmail.com> wrote:
>> It took me about a year to learn enough about IMAP to create a working
>> implementation but the good news is that the hard leg work's over
now
>> - a comprehensive functional test suite exists together with working
>> IMAP commands and codecs.
>
> Sweet! I've heard that imap is quite complex. I guess I don't want
to
> know all the reasons.
+1
> If only we could "reinvent Email using REST" [1]. ;-)
that particular windmill is one i've been tilting at obliquely for a
couple of years now ;-)
>> I'd much rather start from first principals by designing a good
model.
>> I'm very familiar with the actual requirements (rather than the
API's
>> requirements). Second, create a fully working and tested module just
>> containing the JCR stuff. Third, create an adapter module (probably by
>> cloning the torque module) between the API and the JCR module.
>
> Sounds like plan. We just have to keep an eye on 3) along the way so
> that writing an adapter doesn't become a complex time consuming task
> spoiling the work done in 1) and 2).
i'm now very familiar with IMAP's actual requirements and the
inadequacies of the current API so 3) shouldn't be too much of a
problem
let me by start by giving a whirlwind tour of essential IMAP. please
dive in an ask questions, comments and ideas...
IMAP has three primary concepts: user, mail, mailbox.
as well as authentication and authorisation, users have subscriptions
(links to mailboxes).
each mail is a MIME-typed read only document with meta-data
a mailbox is a container for mail. additional mutable per-mailbox
meta-data is set for each mail. probably more elegant to use links
than actual containment.
- robert
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: JCR and IMAP
Posted by Robert Burrell Donkin <ro...@gmail.com>.
On Wed, Nov 26, 2008 at 10:36 PM, Patrick Ruhkopf
<pa...@gmail.com> wrote:
> On Wed, Nov 26, 2008 at 3:40 AM, Robert Burrell Donkin
> <ro...@gmail.com> wrote:
>> It took me about a year to learn enough about IMAP to create a working
>> implementation but the good news is that the hard leg work's over now
>> - a comprehensive functional test suite exists together with working
>> IMAP commands and codecs.
>
> Sweet! I've heard that imap is quite complex. I guess I don't want to
> know all the reasons.
+1
> If only we could "reinvent Email using REST" [1]. ;-)
that particular windmill is one i've been tilting at obliquely for a
couple of years now ;-)
>> I'd much rather start from first principals by designing a good model.
>> I'm very familiar with the actual requirements (rather than the API's
>> requirements). Second, create a fully working and tested module just
>> containing the JCR stuff. Third, create an adapter module (probably by
>> cloning the torque module) between the API and the JCR module.
>
> Sounds like plan. We just have to keep an eye on 3) along the way so
> that writing an adapter doesn't become a complex time consuming task
> spoiling the work done in 1) and 2).
i'm now very familiar with IMAP's actual requirements and the
inadequacies of the current API so 3) shouldn't be too much of a
problem
let me by start by giving a whirlwind tour of essential IMAP. please
dive in an ask questions, comments and ideas...
IMAP has three primary concepts: user, mail, mailbox.
as well as authentication and authorisation, users have subscriptions
(links to mailboxes).
each mail is a MIME-typed read only document with meta-data
a mailbox is a container for mail. additional mutable per-mailbox
meta-data is set for each mail. probably more elegant to use links
than actual containment.
- robert
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: JCR and IMAP
Posted by Patrick Ruhkopf <pa...@gmail.com>.
On Wed, Nov 26, 2008 at 3:40 AM, Robert Burrell Donkin
<ro...@gmail.com> wrote:
> It took me about a year to learn enough about IMAP to create a working
> implementation but the good news is that the hard leg work's over now
> - a comprehensive functional test suite exists together with working
> IMAP commands and codecs.
Sweet! I've heard that imap is quite complex. I guess I don't want to
know all the reasons. If only we could "reinvent Email using REST"
[1]. ;-)
> I'd much rather start from first principals by designing a good model.
> I'm very familiar with the actual requirements (rather than the API's
> requirements). Second, create a fully working and tested module just
> containing the JCR stuff. Third, create an adapter module (probably by
> cloning the torque module) between the API and the JCR module.
Sounds like plan. We just have to keep an eye on 3) along the way so
that writing an adapter doesn't become a complex time consuming task
spoiling the work done in 1) and 2).
Patrick
---
[1] http://prescod.net/rest/restmail/
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: JCR and IMAP
Posted by Robert Burrell Donkin <ro...@gmail.com>.
On 11/26/08, Patrick Ruhkopf <pa...@gmail.com> wrote:
> Thanks for the prompt response, Robert! Glad to hear that you are
> enthusiastic about the idea like me.
:-)
> On Tue, Nov 25, 2008 at 3:19 AM, Robert Burrell Donkin
> <ro...@gmail.com> wrote:
>> But if you're interested in helping out with IMAP development then we
>> can take a branch and start working.
>
> Sounds like a plan. Frankly I don't know much about the imap
> protocol.
You're timing's good :-)
It took me about a year to learn enough about IMAP to create a working
implementation but the good news is that the hard leg work's over now
- a comprehensive functional test suite exists together with working
IMAP commands and codecs. The torque implementation sucks but I run
around 1G of mail on it ATM and it's usable.
> So if "revising data access" means interface changes I could probably
> only make suggestions from the JCR point of view and not much of the
> other side of the interface. You know what I mean? If on the other
> hand it's only about replacing the legacy implementation then I guess
> the task would be easier for me to tackle.
The torque implementation makes some very poor design choices. The
interface between the data access and processor layers is too complex
and lacks elegance.
> If you think it makes sense let me get a little bit more familiar with
> the code base to get some idea of how much effort is involved and how
> well the current data access api would play along with JCR.
I'd much rather start from first principals by designing a good model.
I'm very familiar with the actual requirements (rather than the API's
requirements). Second, create a fully working and tested module just
containing the JCR stuff. Third, create an adapter module (probably by
cloning the torque module) between the API and the JCR module.
The good news is that it should be possible to get something that runs
(about as slow as Torque) without too much effort since a lot of the
complex stuff is implemented programatically.
Sound like a plan?
Robert
>
> Patrick
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: JCR and IMAP
Posted by Patrick Ruhkopf <pa...@gmail.com>.
Thanks for the prompt response, Robert! Glad to hear that you are
enthusiastic about the idea like me.
On Tue, Nov 25, 2008 at 3:19 AM, Robert Burrell Donkin
<ro...@gmail.com> wrote:
> But if you're interested in helping out with IMAP development then we
> can take a branch and start working.
Sounds like a plan. Frankly I don't know much about the imap protocol.
So if "revising data access" means interface changes I could probably
only make suggestions from the JCR point of view and not much of the
other side of the interface. You know what I mean? If on the other
hand it's only about replacing the legacy implementation then I guess
the task would be easier for me to tackle.
If you think it makes sense let me get a little bit more familiar with
the code base to get some idea of how much effort is involved and how
well the current data access api would play along with JCR.
Patrick
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: JCR and IMAP
Posted by Robert Burrell Donkin <ro...@gmail.com>.
On 11/25/08, Patrick Ruhkopf <pa...@gmail.com> wrote:
> Hi,
>
> Being a big fan of the JCR API I'm thrilled by the idea to archive emails in
> such a repository. You could easily access, search, backup mails and most
> importantly link them with other content, attach meta data, etc.
I'm a big fan too: the document+meta-data paradigm is powerful and and
there are lots of very cool possibilities for mail once you switch to
it.
For example, I'm very interested in total information management:
aggregating communications into a single feed accessible through
mutliple protocols eg combined mail store with POP3 and Atom feeds
accessible over IMAP.
These mashups start to get really cool when you start running unified
analytics on the combined store...
> However, I still want to be able to connect to that repository with my
> favorite email client so I can read, delete, flag new mails as usual. So, I
> guess what I want is an imap server that uses JCR.
Yeh - this is one of my aims too
> Searching through the mailinglist I figure JAMES 3 is going to support both
> IMAP and JCR. Yah! But..
>
> It seems like it's currently an either or choice, isn't it? You either
> specify JCR or IMAP as the repository. In the latter case Torque is used to
> persist data. Is it possible to use the JCR store instead?
The data access in IMAP works but the legacy design is rubbish. Full
and satusfactory integration with James requires changes in both. ATM
looks like we'll probably completely revising the data access in IMAP
then rolling over to James.
IMAP development has been on hold so that a M1 can be cut for James.
This push has stalled since too many James developers are too busy
right now.
But if you're interested in helping out with IMAP development then we
can take a branch and start working.
Robert
>
> Thanks
> Patrick
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org