You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by Stephan Wiesner <st...@stephan-wiesner.de> on 2003/04/20 15:13:24 UTC
Re: Serializing Users in FilesForwarding mail to NNTP
I would like to forward some of my mails to my local newsserver. This question was rised before
(http://marc.theaimsgroup.com/?l=james-user&m=104336487319378&w=2) and Harmeet answered:
"One could write a mailet and post messages from james-user mailing list to
an nntp server group called org.apache.james.user.
It would be simple but not too elegant. [...]"
If this is so simple, can anybody give me some more hints?
I mean, I can't just send the mail as a mail to, well to whom?
Stephan
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users in FilesForwarding mail to NNTP
Posted by Stephan Wiesner <st...@stephan-wiesner.de>.
Good morning Harmeet,
thanks again, works perfectly.
I will turn it into a Mailet and post that one to the list later today.
Stephan
Harmeet Bedi wrote:
>The attached file would do the trick. It is a utility to post msg to either
>authenticated or non-authenticated nntp server.
>
>If you write to spool directory in your local james server, you won't need
>to deal with authentication. If you do this, first create a temp file and
>then move the temp file after write is finished to spool directory,
>otherwise spooler may try to act on a partially written file.
>
>Harmeet
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: Serializing Users in FilesForwarding mail to NNTP
Posted by "Noel J. Bergman" <no...@devtech.com>.
> If you write to spool directory in your local james server, you won't need
> to deal with authentication.
If you get a chance, a mailet that took a message and posted it to a
newsgroup would be a good thing. We could try pairing that with a change to
NNTP that allowed a moderator address for a newsgroup.
The workflow would be:
if newsgroup not moderated, post locally
if newsgroup moderated, send message to e-mail address
Then on the mail side we can use the mailet to post approved messages into
the NNTP spool.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users in FilesForwarding mail to NNTP
Posted by Harmeet Bedi <ha...@kodemuse.com>.
The attached file would do the trick. It is a utility to post msg to either
authenticated or non-authenticated nntp server.
If you write to spool directory in your local james server, you won't need
to deal with authentication. If you do this, first create a temp file and
then move the temp file after write is finished to spool directory,
otherwise spooler may try to act on a partially written file.
Harmeet
----- Original Message -----
From: "Stephan Wiesner" <st...@stephan-wiesner.de>
To: "James Developers List" <ja...@jakarta.apache.org>
Sent: Monday, April 21, 2003 2:49 AM
Subject: Re: Serializing Users in FilesForwarding mail to NNTP
> Hi Harmeet,
> thanks for your code, you were right, it is very simple :-)
> However, I did not manage to connect to a newsserver that requires
> authentication and did not find anything to do this, neither in the
> common.net.apidocs, nor in RFC 822. Can you help me there too?
>
> Stephan
>
> >From: "Harmeet Bedi" <ha...@kodemuse.com>
> >
> >
> >>Mailet could identify interesting mail by looking to, cc or from header
of
> >>mail message and post using nntp client API to james NNTP server.
> >>Mimemessage in mailet is a perfect article source.
> >>
> >>James currently packages commons-net library. Attached is an example
that
> >>uses this already packaged library to post messages to nntp server.
> >>
> >>
> >
> >
> >
> >
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users in FilesForwarding mail to NNTP
Posted by Stephan Wiesner <st...@stephan-wiesner.de>.
Hi Harmeet,
thanks for your code, you were right, it is very simple :-)
However, I did not manage to connect to a newsserver that requires
authentication and did not find anything to do this, neither in the
common.net.apidocs, nor in RFC 822. Can you help me there too?
Stephan
>From: "Harmeet Bedi" <ha...@kodemuse.com>
>
>
>>Mailet could identify interesting mail by looking to, cc or from header of
>>mail message and post using nntp client API to james NNTP server.
>>Mimemessage in mailet is a perfect article source.
>>
>>James currently packages commons-net library. Attached is an example that
>>uses this already packaged library to post messages to nntp server.
>>
>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users in FilesForwarding mail to NNTP
Posted by Harmeet Bedi <ha...@kodemuse.com>.
> From: "Stephan Wiesner" <st...@stephan-wiesner.de>
> > If this is so simple, can anybody give me some more hints?
From: "Harmeet Bedi" <ha...@kodemuse.com>
> Mailet could identify interesting mail by looking to, cc or from header of
> mail message and post using nntp client API to james NNTP server.
> Mimemessage in mailet is a perfect article source.
>
> James currently packages commons-net library. Attached is an example that
> uses this already packaged library to post messages to nntp server.
Stephan there was a mistake in the attachment file posted earlier. Here is
the fixed example file. It should give you more than a few 'hints' to send
mail message to nntp server. :-)
You can also directly write messages to nntp spool directly and have it
picked up. Remember to add the right newsgroup header.Another thing to know
is that the message id is maintained so you can only create one newsgroup
article with one mime message. Do send mail if you hit issues.
Harmeet
Re: Serializing Users in FilesForwarding mail to NNTP
Posted by Harmeet Bedi <ha...@kodemuse.com>.
From: "Stephan Wiesner" <st...@stephan-wiesner.de>
> If this is so simple, can anybody give me some more hints?
Mailet could identify interesting mail by looking to, cc or from header of
mail message and post using nntp client API to james NNTP server.
Mimemessage in mailet is a perfect article source.
James currently packages commons-net library. Attached is an example that
uses this already packaged library to post messages to nntp server.
There is also some old code in cvs that uses jdk 1.3 internal classes that
one could and to writes files to nntp spool directory. I had used this to
bootstrap nntp repository for testing in the past.
http://cvs.apache.org/viewcvs.cgi/jakarta-james/src/java/org/apache/james/te
sting/Attic/NNTPClient.java?rev=HEAD&hideattic=0&content-type=text/vnd.viewc
vs-markup
You may want to take look at NNTPSpooler <run> method to see how message is
picked up.
Ref:
http://cvs.apache.org/viewcvs.cgi/jakarta-james/src/java/org/apache/james/nn
tpserver/repository/NNTPSpooler.java?rev=1.19&content-type=text/vnd.viewcvs-
markup
MimeMessage from Mailet is perfect source for NNTP Article.
Harmeet
----- Original Message -----
From: "Stephan Wiesner" <st...@stephan-wiesner.de>
To: "James Developers List" <ja...@jakarta.apache.org>
Sent: Sunday, April 20, 2003 9:13 AM
Subject: Re: Serializing Users in FilesForwarding mail to NNTP
> I would like to forward some of my mails to my local newsserver. This
question was rised before
> (http://marc.theaimsgroup.com/?l=james-user&m=104336487319378&w=2) and
Harmeet answered:
>
> "One could write a mailet and post messages from james-user mailing list
to
> an nntp server group called org.apache.james.user.
>
> It would be simple but not too elegant. [...]"
>
> If this is so simple, can anybody give me some more hints?
> I mean, I can't just send the mail as a mail to, well to whom?
>
> Stephan
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: Serializing Users in FilesForwarding mail to NNTP
Posted by Vincent de Lau <vi...@delau.nl>.
> ... In my (incredibly arrogant tonight) opinion, the
> only thing the filesystem structure is good for is storing message
> files. Any other semantic information should be held in a database where
> it is easy to search for.
Storing messages in a database can be usefull as well, since you could do a
fulltext search on them for instance. With filesystems you can also get into
trouble, for example waste of space because of fs block size and number of
files per directory.
> I also think the 1.2 option of storing the message files in a database
> is (currently) a bad idea. Unless one is running the database on an OS
> with a 64 bit filesystem (which leaves out Win32 and Linux), creating
> large database storage capacity is a real pain because the datafiles are
> limited to 2 GB's and one must aggregate multiple datafiles to provide a
> reasonable amount of storage.
This isn't a problem for MySQL 4 (MyISAM/RAID or InnoDB) or MSSQL.
Vincent de Lau
vincent@delau.nl
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users... (Now message repository implementations)
Posted by Sid Stuart <si...@weaselworks.com>.
Stephan Wiesner wrote:
> Reading the messages about using files or databases for storing the
> messages and considering my need for an archieving system, I decided
> to write a PHP based interface to the MySQL-stored mails. Including
> full text search and all (read only functionallity). This would have
> been much more work with the file base approach.
It's my assertion that you should be able to do this using the mailet
interface. I would like to see James 3.0 support this requirment instead
of forcing you to delve into an implementation detail.
> I hesitated long, however, as I keep to get infinity loops when
> playing around with Mailets and it is much more easy to just rm * in
> the spool directory then to log into the database and do it there.
> Convenience is iportant to me, I guess .-)
Yes! May I quote you about the inifinite loops? I was just arguing for
changes to the mailet definition to reduce their possibility.
Sid
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: Serializing Users... (Now message repository implementations)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> it is much more easy to just rm * in the spool
> directory then to log into the database and do
> it there.
Try mysql --exec="drop from maildb.spool;"
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users... (Now message repository implementations)
Posted by Stephan Wiesner <st...@stephan-wiesner.de>.
Reading the messages about using files or databases for storing the
messages and considering my need for an archieving system, I decided to
write a PHP based interface to the MySQL-stored mails. Including full
text search and all (read only functionallity). This would have been
much more work with the file base approach.
I hesitated long, however, as I keep to get infinity loops when playing
around with Mailets and it is much more easy to just rm * in the spool
directory then to log into the database and do it there. Convenience is
iportant to me, I guess .-)
Stephan
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users... (Now message repository implementations)
Posted by Serge Knystautas <se...@lokitech.com>.
Sid Stuart wrote:
> On the one hand I see a group of talented OO architects/developers
> designing a system using good programming paradigms like developing an
> interface and then writing classes to that interface to hide the
> implementation detail. On the other hand, I see these same programmers
> opening up the implementation details of the storage repository to the
> world and saying, have at it. How is it that what is considered good
> programming practise for implementing the system is not good programming
> practise for designing the system?
I think James does a good job isolating most users from the underlying
implementations. The management tools are admitedly, woefully
inadequate. It's all open source, so if people need to contribute in
those areas, then they need to see all the details understandably.
The repository interface is also not 100% solid, and as you can see,
currently evolving. This will be the first time we're evolve it, and
this will be to improve the API separation as well as give everyone more
options. Hopefully we'll get it close enough to right to not need to do
it again so the management tools and mailet authors don't need to
understand the guts.
> The contract with the end user should be the protocols implemented on
> top of James, SMTP, POP3, IMAP and the programming interface provided by
> the mailet package. If the requirements of the users are not met with
> these specifications, the specifications should be reviewed/enhanced or
> the users should be told to find another system. All implementation
> details should be hidden behind protocol servers or interface
> specifications.
Yes, the repository API is getting moved into the mailet package so
those who need access to them don't have to worry about this. We're
still somewhat debating requirements, but some movement has already
begun towards those improvements. And as Noel said, for those
improvements that can work against the current design, they're being
applied to the 2.1 branch.
Your thoughts and contributions towards this are of course welcome. :)
--
Serge Knystautas
President
Lokitech >> software . strategy . design >> http://www.lokitech.com
p. 301.656.5501
e. sergek@lokitech.com
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: Serializing Users... (Now message repository implementations)
Posted by Danny Angus <da...@apache.org>.
> The contract with the end user should be the protocols implemented on
> top of James, SMTP, POP3, IMAP and the programming interface provided by
> the mailet package. If the requirements of the users are not met with
> these specifications, the specifications should be reviewed/enhanced or
> the users should be told to find another system. All implementation
> details should be hidden behind protocol servers or interface
> specifications.
I completely disagree.
The popularity of httpd is in no small part owed to the fact that it can be extended by thirdparties for their own purposes.
What we are aiming for is that we make some implementation decisions about what ships with James, but do not prevent you, or anyone else, from making alternative implementations of *many* of James internal components to better suit James to your specific needs.
The Mailet API is one thing James is something else.
d.
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users... (Now message repository implementations)
Posted by Sid Stuart <si...@weaselworks.com>.
Serge Knystautas wrote:
>
> Can you explain how the "db" implementation is slowing down the
> delivery schedule? The repository changes in the works affect dbfile
> in every way they affect db. In fact, db and dbfile are going to be
> easier to migrate than plain file because of the ability to do the
> schema change scripts, which as you say is why it's nice to use a
> database for metadata.
>
> And also, we are actually looking to add MANY more repository types,
> namely mbox, maildir, remote IMAP/POP3, as well as maybe other types
> as implementations are available. We may also be separating the
> SMTP/spooling logic from the underlying repository, allowing us to use
> any JavaMail store implementation for repositories serving as user
> mailboxes.
I didn't say the "db" implementation is slowing dow the delivery
schedule. I said "From the mail I have seen over the last week, the
design and implementation of the message repository is impacting the
delivery schedule." I was referencing two notes from Mr. Bedi:
Excerpted from "Re: preprocessing after smtp DATA", dated 4/20/2003 4:27 AM
Is there any idea when will 3.0 released? 1 month, 1 year?
I think it is far away. The main change is moving to a
different repository implementation and there has been
limited activity on that front. I would guess release is
6 months or so away.
Excerpted from "Re: Serializing Users in FilesForwarding mail to NNTP",
dated 4/20/2003 6:32 PM.
Everybody agrees we should converge to a repository and
format asap. At the moment it is not too far apart at file
system level but everything else is parallel.
The key is moving code to standard repository API. The
sooner we do this the better. If javamail proposal takes
too long to build maybe we should just let it go away and
use the current repository API/implementations as it is
and move nntp to it. thoughts.. ?
I now understand the James 3.0 proposal includes adding several more
repository implementations such as mbox and maildir. The idea was so
foreign to me that it took long time to grasp. Quite frankly, when I
read that the rational for doing so is that external tools can access
the implementations, I had to wonder what planet I was on.
On the one hand I see a group of talented OO architects/developers
designing a system using good programming paradigms like developing an
interface and then writing classes to that interface to hide the
implementation detail. On the other hand, I see these same programmers
opening up the implementation details of the storage repository to the
world and saying, have at it. How is it that what is considered good
programming practise for implementing the system is not good programming
practise for designing the system?
The contract with the end user should be the protocols implemented on
top of James, SMTP, POP3, IMAP and the programming interface provided by
the mailet package. If the requirements of the users are not met with
these specifications, the specifications should be reviewed/enhanced or
the users should be told to find another system. All implementation
details should be hidden behind protocol servers or interface
specifications.
Sid
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users... (Now message repository implementations)
Posted by Serge Knystautas <se...@lokitech.com>.
Sid Stuart wrote:
> 3. James design/implementation simplicity: the fewer repository
> implementations that need
> to be supported, the easier it will be to get James 3.0 out the
> door. From the mail I have
> seen over the last week, the design and implementation of the
> message repository is impacting
> the delivery schedule. Shouldn't that trigger a design review?
>
> These are my arguments against the James "db" storage format. What are
> the arguments for it? What benefits does the "db" implementation offer
> over "dbfile"? I would really like to know.
If you object to the "db" storage format, then don't use it. Other
people want to use it, and won't agree with all your points. As for my
reasons otherwise, but basically I would just repeat your points one and
two and throw in the word "not". Just a difference of opinion, and
that's why it's very nice to have pluggable repositories so different
views can coexist.
Can you explain how the "db" implementation is slowing down the delivery
schedule? The repository changes in the works affect dbfile in every
way they affect db. In fact, db and dbfile are going to be easier to
migrate than plain file because of the ability to do the schema change
scripts, which as you say is why it's nice to use a database for metadata.
And also, we are actually looking to add MANY more repository types,
namely mbox, maildir, remote IMAP/POP3, as well as maybe other types as
implementations are available. We may also be separating the
SMTP/spooling logic from the underlying repository, allowing us to use
any JavaMail store implementation for repositories serving as user
mailboxes.
Nobody has quite cracked all the issues outstanding with the changes
underway. We just know...
- we want to integrate with other mail products by supporting the other
standards.
- we want to continue to improve performance
- we want to get NNTP using the same repositories
- we want to allow IMAP to have repositories it can use
- possibly we want to separate spooling logic so multiple James servers
could use the same repository and use a distributed spool implementation.
--
Serge Knystautas
President
Lokitech >> software . strategy . design >> http://www.lokitech.com
p. 301.656.5501
e. sergek@lokitech.com
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users... (Now message repository implementations)
Posted by Sid Stuart <si...@weaselworks.com>.
Noel J. Bergman wrote:
>>I've supported developers working on real-time replication applications.
>>There was a lot of overhead involved in performing the replication
>>
>>
>
>Different needs. For example, we have heavily database-oriented web sites,
>and we use replication to provide real-time backup of transactions.
>
>
Yes, but how large are the rows in the tables that support those
transactions? Are there any BLOB's or CLOB's involved? My concern is
that adding the message content to the database will introduce a
noticble latency even before it is configured for replication. Also, do
you really do mean replication instead of hot-standby? I haven't seen
too many websites that need the smaller failover time provided by
replication. Hot-standby is simpler to implement and has a lower runtime
overhead.
>>If I were implementing a robust system, I would reduce the
>>load on the database by storing the content on a clustered
>>NFS server.
>>
>>
>
>You would make the tradeoff by going from a centrally managed DB server to a
>cluster of servers using file locking? That is NOT a tradeoff I would make.
>I'm not sure which databases support transactions in such an environment.
>Or are you just refering to the dbfile example, and putting the content on
>an NFS mount? The latter is certainly an option that can be employed.
>
>
I was referring to the dbfile example. I agree that file locking is not
an acceptable solution in a distributed environment.
Sid
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: Serializing Users... (Now message repository implementations)
Posted by "Noel J. Bergman" <no...@devtech.com>.
>For James v3, I hope to have hsql embedded so that all James distributions
>can make use of JDBC, although we'll have to talk about whether or not we
>can rely upon it.
> I have to confess, my rants about only providing dbfile as the message
> storage implementation are based on the conviction that hsql is
> reliable.
Big assumption. I hope that it is reliable, too.
> Have you considered using that package for persistance instead of
> writing custom JDBC?
What custom JDBC? We're talking about http://hsqldb.sourceforge.net/, which
has a JDBC driver and is supported by the Avalon folks as a component. And
James already has a SQL Resources construct that factors out differences
between SQL language implementations.
> > One benefit of db storage is the provision for real-time replication
> I've supported developers working on real-time replication applications.
> There was a lot of overhead involved in performing the replication
Different needs. For example, we have heavily database-oriented web sites,
and we use replication to provide real-time backup of transactions.
> If I were implementing a robust system, I would reduce the
> load on the database by storing the content on a clustered
> NFS server.
You would make the tradeoff by going from a centrally managed DB server to a
cluster of servers using file locking? That is NOT a tradeoff I would make.
I'm not sure which databases support transactions in such an environment.
Or are you just refering to the dbfile example, and putting the content on
an NFS mount? The latter is certainly an option that can be employed.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users... (Now message repository implementations)
Posted by Sid Stuart <si...@weaselworks.com>.
Noel J. Bergman wrote:
>>In fact, I love the database concept so much (dbfile that is), I
>>think it should be the only mechanism used for storing metadata
>>associated with the messages in the repository.
>>
>>
>
>For James v3, I hope to have hsql embedded so that all James distributions
>can make use of JDBC, although we'll have to talk about whether or not we
>can rely upon it.
>
>
I have to confess, my rants about only providing dbfile as the message
storage implementation are based on the conviction that hsql is
reliable. I have not used it before. I googled it, but could not find
any good reports on it. The only bad report came from PointBase, but
that was a generalized screed against open source Java database
implementations. I'm not sure it would carry much weight here. :)
One note in hsql's favour is the Hibernate folks have support for it.
Have you considered using that package for persistance instead of
writing custom JDBC? A friend who does high transaction-rate java
commercial implementations (banking) likes it a lot.
>>storing the message content in the database is a bad
>>idea for several reasons,
>>
>>
>
>One benefit of db storage is the provision for real-time replication,
>providing the most robust storage available in a mail server. Yes, there is
>a tradeoff in terms of bandwidth and overhead. Again, flexibility. The
>mail admin can choose the desired mail storage to fit the needs of their
>environment.
>
I've supported developers working on real-time replication applications.
There was a lot of overhead involved in performing the replication and
that slowed the DB down. If I were implementing a robust system, I would
reduce the load on the database by storing the content on a clustered
NFS server.
Even Linux supports that feature these days.
Sid
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: Serializing Users... (Now message repository implementations)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> My comfort level for the size of a message repository in these days of
> high bandwidth connections and multimedia attachments is at least 100 GB.
Needs vary. James is flexible. At an average of 10K per message (a high
average based upon real-life traffic), your requirement is for 10 million
messages in storage. That is over 10,000 messages in storage for a 1000
user installation. Reasonable for IMAP, highly unusual for POP3. It is
roughly 100 messages per user for a 100,000 user installation. [Assuming I
calculated the results properly; I'm a bit under the weather today.]
> In fact, I love the database concept so much (dbfile that is), I
> think it should be the only mechanism used for storing metadata
> associated with the messages in the repository.
For James v3, I hope to have hsql embedded so that all James distributions
can make use of JDBC, although we'll have to talk about whether or not we
can rely upon it.
> storing the message content in the database is a bad
> idea for several reasons,
One benefit of db storage is the provision for real-time replication,
providing the most robust storage available in a mail server. Yes, there is
a tradeoff in terms of bandwidth and overhead. Again, flexibility. The
mail admin can choose the desired mail storage to fit the needs of their
environment.
> From the mail I have seen over the last week, the design and
> implementation of the message repository is impacting the
> delivery schedule. Shouldn't that trigger a design review?
Actually, the availability of developer cycles is the gating factor. Not
the design. Things that don't require design change fit perfectly well into
James v2.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users... (Now message repository implementations)
Posted by Sid Stuart <si...@weaselworks.com>.
Noel J. Bergman wrote:
>>I've spent the couple of months on a MUA design that I
>>would like to implement. I hoped that James would provide
>>me with the tools to do so easily. Unfortunately, I've
>>come to the realization that James doesn't provide the
>>flexibility I need
>>
>>
>
>What do you believe to be missing?
>
A facility to support access to and modification of the metadata
associated with the message content stored in the message repository and
the metadata associated with the users stored in the user repository. My
design depends on the ability to define custom attributes for users and
messages and then perform operations on those attributes, as well as
standard attributes, using something similar to the directed graphs of
matchers and mailets. The facility would need to provide event driven
and calendar driven initiation of the processors defined by the directed
graphs.
Examples of how it would be used,
a) Rerun the dead letter queue on a nightly basis
b) Generate message list digests (C.3 from Mr. Behlendorf's wish list)
c) Provide the access definitions needed for C.5 on Mr.
Behlendorf's wish list.
d) Provide pre-filtering of incoming and outgoing email for
guardians of minors.
e) Provide pass/block list filtering of incoming messages.
e) Provide a facility for post-delivery removal of spam and virus's
>By the way, you do realize that James is an MTA, right? I can refer you to
>Open Source MUA projects.
>
> --- Noel
>
>
Late night slip of the fingers. I meant to say MDA. If I understand the
definitions correctly, James is actually an MTA and a MDA (just had to
delete MUA again...) It's the MDA part I am interested in.
Sid
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: Serializing Users... (Now message repository implementations)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> I've spent the couple of months on a MUA design that I
> would like to implement. I hoped that James would provide
> me with the tools to do so easily. Unfortunately, I've
> come to the realization that James doesn't provide the
> flexibility I need
What do you believe to be missing?
By the way, you do realize that James is an MTA, right? I can refer you to
Open Source MUA projects.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users... (Now message repository implementations)
Posted by Sid Stuart <si...@weaselworks.com>.
Serge Knystautas wrote:
> Sid Stuart wrote:
>
>> I started looking at James because I also want to write tools that
>> manipulate data in the repositories. I want to do it through a
>> standard api though so that the tool I develop will work for all
>> implementations of the mail system, not just those using a particular
>> implementation of the message repository. Instead of writing multiple
>> storage implementations, your time would be better spent
>> understanding the operations that developers want to perform on the
>> repository and providing a facility that allows them to implement
>> those operations.
>
>
> Sid,
>
> I don't think that came out as helpful as you had intended.
> Criticizing Danny's use of time isn't going to convince him to work on
> what you want him to.
>
> This is an open source project where requirements gathering is not
> formal, with decisions largely based on people's persuasive skills as
> well as their willingness to do something about it. The mailing list
> archives are an excellent source of information about what people have
> requested as well as suggested solutions to these requests. A lot of
> people have weighed in on this already, and we have all done our best
> to recognize people's interests.
>
I certainly didn't mean to criticize Danny or anyone else. Everyone on
this mailing list has been quite helpful and patient. I've spent the
couple of months on a MUA design that I would like to implement. I hoped
that James would provide me with the tools to do so easily.
Unfortunately, I've come to the realization that James doesn't provide
the flexibility I need and I see no hope of it doing so in the future.
My disappointment at the realization was poorly expressed. My apologies
for that to all of you.
Sid
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users... (Now message repository implementations)
Posted by Serge Knystautas <se...@lokitech.com>.
Sid Stuart wrote:
> I started looking at James because I also want to write tools that
> manipulate data in the repositories. I want to do it through a standard
> api though so that the tool I develop will work for all implementations
> of the mail system, not just those using a particular implementation of
> the message repository. Instead of writing multiple storage
> implementations, your time would be better spent understanding the
> operations that developers want to perform on the repository and
> providing a facility that allows them to implement those operations.
Sid,
I don't think that came out as helpful as you had intended. Criticizing
Danny's use of time isn't going to convince him to work on what you want
him to.
This is an open source project where requirements gathering is not
formal, with decisions largely based on people's persuasive skills as
well as their willingness to do something about it. The mailing list
archives are an excellent source of information about what people have
requested as well as suggested solutions to these requests. A lot of
people have weighed in on this already, and we have all done our best to
recognize people's interests.
--
Serge Knystautas
President
Lokitech >> software . strategy . design >> http://www.lokitech.com/
p. 1.301.656.5501
e. sergek@lokitech.com
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: Serializing Users... (Now message repository implementations)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> I think the JavaMail interface is an interesting place to start. Is
> there a requirements document that lists the needs of the various
> servers and verifies that the interface will provide the functionality
> require?
Ask Bill Shannon.
> (a) I have concerns that the metadata required by the JavaMail
> interface cannot be expressed by the mbox implementation.
You might want to look at one of the mbox, maildir or mh implementations.
Where do you find metadata requirements? As an aside, consider the dbfile
representation; there is nothing to prevent metadata from being stored
orthogonally to the standard data representation.
> How is it a standard interface if an underlaying implementation
> does not support all it's semantics?
Considering that JavaMail was designed for this purpose, if there is
something that you feel is lacking, you might want to raise it with Bill
Shannon. I've done so. The next version of JavaMail is at least a year
away, but there are at least workarounds for the things James needs.
> (b) The thousands administrators who have chosen of Cyrus, UofW IMAP/POP
> and Exchange. None of these systems offer message storage options.
If we adopt JavaMail as the storage interface, we can provide our own
storage service provider optimized for what we feel is best for performance
and reliability, and have the option of allowing other storage formats for
free.
> One quick note on the request for mbox support, I saw the suggestion to
> use eyebrowse in the email from Brian Behlendorf in the wiki. I also
> took a quick look at the eyebrowse page. I'm not sure just providing the
> mbox format will be sufficient to support that tool as it also requires
> metadata stored in a mySQL database. Has anyone spec'ed out a design of
> how eyebrowse would interface to James?
You seem to be confused. eyebrowse (http://eyebrowse.tigris.org/) generates
the metadata, which it stores either via Lucene or in MySQL. eyebrowse
doesn't interface to the mail server. It processes mbox directly. Please
note that it also stores its metadata independently of the archive content,
which I would consider to bolster the case, rather than undermine it.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users... (Now message repository implementations)
Posted by Sid Stuart <si...@weaselworks.com>.
Noel J. Bergman wrote:
>>I also want to write tools that manipulate data in the repositories.
>>I want to do it through a standard api though so that the tool I
>>develop will work for all implementations of the mail system
>>
>>
>
>Thank you for yet another justification to adopt the JavaMail interface.
>
I think the JavaMail interface is an interesting place to start. Is
there a requirements document that lists the needs of the various
servers and verifies that the interface will provide the functionality
require?
>>Instead of writing multiple storage implementations, your time
>>would be better spent understanding the operations that
>>developers want to perform on the repository and providing
>>a facility that allows them to implement those operations.
>>
>>
>
>What makes you think that (a) a standard interface isn't the idea, or (b)
>that end users don't care about the storage implementation? Remember: there
>are other users who may feel that they want to use a particular format
>because of other tools that they have, such as tools that take mbox format
>and produce web archives.
>
> --- Noel
>
(a) I have concerns that the metadata required by the JavaMail
interface cannot be expressed by the mbox implementation. How is it a
standard interface if an underlaying implementation does not support all
it's semantics?
(b) The thousands administrators who have chosen of Cyrus, UofW IMAP/POP
and Exchange. None of these systems offer message storage options. The
question is who is your customer base? a limited set of sophisticated
programmers/administrators or the general administrator population. The
general population cares about what protocols the server supports, how
easy it is to install and how reliable it is. (I quote from Brian
Behlendorf's email listed in the wiki, "reliable, reliable, reliable".)
(Okay, the Exchange administrators don't care about it being reliable or
easy to install but they have been brainwashed to accept a lower set of
standards.)
One quick note on the request for mbox support, I saw the suggestion to
use eyebrowse in the email from Brian Behlendorf in the wiki. I also
took a quick look at the eyebrowse page. I'm not sure just providing the
mbox format will be sufficient to support that tool as it also requires
metadata stored in a mySQL database. Has anyone spec'ed out a design of
how eyebrowse would interface to James?
Sid
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users... (Now message repository implementations)
Posted by Sid Stuart <si...@weaselworks.com>.
Noel J. Bergman wrote:
>>I also want to write tools that manipulate data in the repositories.
>>I want to do it through a standard api though so that the tool I
>>develop will work for all implementations of the mail system
>>
>>
>
>Thank you for yet another justification to adopt the JavaMail interface.
>
>
>
Hi Noel, what do you mean by adopt the JavaMail interface? In
particular, which class or interface would a repository implementation
provide for access to the data?
Sid
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: Serializing Users... (Now message repository implementations)
Posted by "Noel J. Bergman" <no...@devtech.com>.
> I also want to write tools that manipulate data in the repositories.
> I want to do it through a standard api though so that the tool I
> develop will work for all implementations of the mail system
Thank you for yet another justification to adopt the JavaMail interface.
> Instead of writing multiple storage implementations, your time
> would be better spent understanding the operations that
> developers want to perform on the repository and providing
> a facility that allows them to implement those operations.
What makes you think that (a) a standard interface isn't the idea, or (b)
that end users don't care about the storage implementation? Remember: there
are other users who may feel that they want to use a particular format
because of other tools that they have, such as tools that take mbox format
and produce web archives.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users... (Now message repository implementations)
Posted by Sid Stuart <si...@weaselworks.com>.
Danny Angus wrote:
>>These are my arguments against the James "db" storage format. What are
>>the arguments for it? What benefits does the "db" implementation offer
>>over "dbfile"? I would really like to know.
>>
>>
>
>briefly I use db message format because..
>I use db replication for data integrity
>I also use a number of other db aware tools to manipulate the repositories. If/when we implement plain text file storage I could write scripts etc to maipulate the files, 'till then I would have to use Java for everything which is not a serious option.
>
>
I started looking at James because I also want to write tools that
manipulate data in the repositories. I want to do it through a standard
api though so that the tool I develop will work for all implementations
of the mail system, not just those using a particular implementation of
the message repository. Instead of writing multiple storage
implementations, your time would be better spent understanding the
operations that developers want to perform on the repository and
providing a facility that allows them to implement those operations.
Sid
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: Serializing Users... (Now message repository implementations)
Posted by Danny Angus <da...@apache.org>.
> If the data stored in the database is limited to the metadata required
> to support the repository interface and the content associated with the
> message in stored in a filesystem (essentially the James dbfile
> implementation), then the size and complexity of the database is reduced
> significantly.
Well we provide you with that option, go use it :-)
> I need to go back to school, as my communication skills seem to have
> deteriorated. I have been trying to say that using a database is a great
> idea. In fact, I love the database concept so much (dbfile that is), I
> think it should be the only mechanism used for storing metadata
> associated with the messages in the repository. However, storing the
> message content in the database is a bad idea for several reasons,
>
> 1. The implementation complexity discussed above.
> 2. Performance considerations, for James to operate well, the access
> to metadata stored in the
> database should be as low latency as possible. Moving message
> content data through the
> database will increase the latency of operations accessing
> metadata. No, I have not run tests
> on this assertion, but it is common sense.
There is little complexity in configuring MySQL to use large tables, but I accept the principle.
I also agree that in pure IO byte rate terms file IO should outperform db, but with cacheing and read ahead in rdbms this is not guarenteed.
.
> From the mail I have
> seen over the last week, the design and implementation of the
> message repository is impacting
> the delivery schedule. Shouldn't that trigger a design review?
Not really true, its more like lack of developer resources, and a whole load of less interesting stuff waiting to be done.
Plus AFAIK there is no immediate plan to release v3, v3 represents some major overhaul and is likely to be some time coming.
>
> These are my arguments against the James "db" storage format. What are
> the arguments for it? What benefits does the "db" implementation offer
> over "dbfile"? I would really like to know.
briefly I use db message format because..
I use db replication for data integrity
I also use a number of other db aware tools to manipulate the repositories. If/when we implement plain text file storage I could write scripts etc to maipulate the files, 'till then I would have to use Java for everything which is not a serious option.
d.
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users... (Now message repository implementations)
Posted by Sid Stuart <si...@weaselworks.com>.
Danny Angus wrote:
>Sid,
>
>
>And second.. Well, from what I can see on the MySQL link you sent MySQL provide a lot of information and support to help people overcome the 4gb limit, and achieve "8 million terabytes" for each table, not 2gb per database as you suggest.
>
I did not say one could not create a large database, I said it was
painful to do so as one needed to create multiple 2 or 4 gigabyte
datafiles to support it. As an example, say the database server is a
version of Linux that supports a 4 GB file size limit. My comfort level
for the size of a message repository in these days of high bandwidth
connections and multimedia attachments is at least 100 GB. That means an
installation using the database to support the James "db" storage
implementation will need 25 datafiles to support the repository. In
addition, growing the repository now requires the intervention of
someone schooled in the database and downtime for the system as the
database is reconfigured.
If the data stored in the database is limited to the metadata required
to support the repository interface and the content associated with the
message in stored in a filesystem (essentially the James dbfile
implementation), then the size and complexity of the database is reduced
significantly. It also makes growing the space allocated to content
storage as simple as adding a filesystem and adding a parameter to a
James configuration file.
>In addition I still can't see why you think using a db is not a good idea, surely a db is ideal for uses such as this where a number of identically structured data items are to be stored and retrieved by criteria possibly several times during their lifetime? What is your complaint about this?
>
>
I need to go back to school, as my communication skills seem to have
deteriorated. I have been trying to say that using a database is a great
idea. In fact, I love the database concept so much (dbfile that is), I
think it should be the only mechanism used for storing metadata
associated with the messages in the repository. However, storing the
message content in the database is a bad idea for several reasons,
1. The implementation complexity discussed above.
2. Performance considerations, for James to operate well, the access
to metadata stored in the
database should be as low latency as possible. Moving message
content data through the
database will increase the latency of operations accessing
metadata. No, I have not run tests
on this assertion, but it is common sense.
3. James design/implementation simplicity: the fewer repository
implementations that need
to be supported, the easier it will be to get James 3.0 out the
door. From the mail I have
seen over the last week, the design and implementation of the
message repository is impacting
the delivery schedule. Shouldn't that trigger a design review?
These are my arguments against the James "db" storage format. What are
the arguments for it? What benefits does the "db" implementation offer
over "dbfile"? I would really like to know.
Sid
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: Serializing Users in FilesForwarding mail to NNTP
Posted by Danny Angus <da...@apache.org>.
Sid,
1st of all please use plain text for mail to these lists.
And second.. Well, from what I can see on the MySQL link you sent MySQL provide a lot of information and support to help people overcome the 4gb limit, and achieve "8 million terabytes" for each table, not 2gb per database as you suggest.
In addition I still can't see why you think using a db is not a good idea, surely a db is ideal for uses such as this where a number of identically structured data items are to be stored and retrieved by criteria possibly several times during their lifetime? What is your complaint about this?
d.
-----Original Message-----
From: Sid Stuart [mailto:sid@weaselworks.com]
Sent: 21 April 2003 12:24
To: James Developers List
Subject: Re: Serializing Users in FilesForwarding mail to NNTP
Danny Angus wrote:
Unless one is running the database on an OS
with a 64 bit filesystem (which leaves out Win32 and Linux), creating
large database storage capacity is a real pain because the datafiles are
limited to 2 GB's and one must aggregate multiple datafiles to provide a
reasonable amount of storage.
I never heard this *ever* before, and I don't believe it either, what do you base this on?
d.
>From this, http://www.mysql.com/doc/en/Table_size.html
Note that if one uses Oracle and implements uses their feature to store data in raw partitions, this problem is overcome. Most people I know dislike that solution though for supportability reasons.
Sid
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users in FilesForwarding mail to NNTP
Posted by Sid Stuart <si...@weaselworks.com>.
Danny Angus wrote:
>>Unless one is running the database on an OS
>>with a 64 bit filesystem (which leaves out Win32 and Linux), creating
>>large database storage capacity is a real pain because the datafiles are
>>limited to 2 GB's and one must aggregate multiple datafiles to provide a
>>reasonable amount of storage.
>>
>>
>
>I never heard this *ever* before, and I don't believe it either, what do you base this on?
>
>d.
>
>
>
From this, http://www.mysql.com/doc/en/Table_size.html
Note that if one uses Oracle and implements uses their feature to store
data in raw partitions, this problem is overcome. Most people I know
dislike that solution though for supportability reasons.
Sid
RE: Serializing Users in FilesForwarding mail to NNTP
Posted by Danny Angus <da...@apache.org>.
> I also think the 1.2 option of storing the message files in a database
> is (currently) a bad idea.
Well thats a matter of personal opinion.
> Unless one is running the database on an OS
> with a 64 bit filesystem (which leaves out Win32 and Linux), creating
> large database storage capacity is a real pain because the datafiles are
> limited to 2 GB's and one must aggregate multiple datafiles to provide a
> reasonable amount of storage.
I never heard this *ever* before, and I don't believe it either, what do you base this on?
d.
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: Serializing Users in FilesForwarding mail to NNTP
Posted by Danny Angus <da...@apache.org>.
Sid Stuart wrote:
> James 1.3
Do you mean James v3?
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users in FilesForwarding mail to NNTP
Posted by Sid Stuart <si...@weaselworks.com>.
Harmeet Bedi wrote:
>Everybody agrees we should converge to a repository and format asap. At the
>moment it is not too far apart at file system level but everything else is
>parallel.
>The key is moving code to standard repository API. The sooner we do this the
>better. If javamail proposal takes too long to build maybe we should just
>let it go away and use the current repository API/implementations as it is
>and move nntp to it. thoughts.. ?
>
>Harmeet
>
>
I think the path to convergence lies in a clear definition of the goals
of James 1.3 and the requirements of the various protocol servers it
will support. I've looked around, but have not seen any such
documentation. What operations/data do fetchmail, pop3server,
imapserver, smtpserver, remotemanager and the spoolmanager need to
provide their service? The data respository specification should be
defined to support their requirements.
Quite frankly, nothing I can envision (have not read the code) about the
needs of those protocols would require maildir or mailbox. I would
really like to understand the need for these and the myriad of storage
options in James 1.2. I don't see other MUA's such as Cyrus or UofW or
Exchange providing this flexibility and yet they are well recieved in
the user community. In my (incredibly arrogant tonight) opinion, the
only thing the filesystem structure is good for is storing message
files. Any other semantic information should be held in a database where
it is easy to search for.
I also think the 1.2 option of storing the message files in a database
is (currently) a bad idea. Unless one is running the database on an OS
with a 64 bit filesystem (which leaves out Win32 and Linux), creating
large database storage capacity is a real pain because the datafiles are
limited to 2 GB's and one must aggregate multiple datafiles to provide a
reasonable amount of storage.
Sid
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Storage architecture (was: Serializing Users in FilesForwarding mail to NNTP)
Posted by Vincent de Lau <vi...@delau.nl>.
First of all, let me introduce myself. I'm lurking the list for a few weeks
now, since the James project was mentioned in the OOogw (OpenOffice.org
groupware) project mailinglist. (http://groupware.openoffice.org/) I have
experience with Microsoft Exchange as administrator and posted some thoughts
about a groupware architecture for OOogw.
I wouldn't call myself a Java programmer. I can read complex source and
write a HelloWorld app :)
> The idea was to have something valuable out and move to a more standard
> repository when available and make switch over as clean as possible.
>
> > I have heard rumours of work being done in this area for James 1.3. Is
> > there any documentation on the plans for it? (Other than what is in the
> > wiki.)
>
> Not sure about the documentation, but the consensus was to use Java Mail
> abstraction and standard formats like maildir, mailbox under it. To some
> extent repository ideas for SMTP and NNTP are similar to maildir. SMTP and
> NNTP both store raw messages in single files.
> IMAP was tested by Darrell with inmemory(non persistent) repository but
> hasn't moved from there as far as I know.
>
> Everybody agrees we should converge to a repository and format
> asap. At the
> moment it is not too far apart at file system level but everything else is
> parallel.
> The key is moving code to standard repository API. The sooner we
> do this the
> better. If javamail proposal takes too long to build maybe we should just
> let it go away and use the current repository API/implementations as it is
> and move nntp to it. thoughts.. ?
I have some thoughts, but they would impose a big structural overhaul of
James:
(I'm thinking more RDBMS centered then OO design/Java here...)
First off all we start with four different concepts:
- objects(files): These could store anything from an email, newsposting or
any other type of file. They could be stored with a MIME type
(message/rfc822, text/plain, iCalendar etc...)
- folders(store): These would be containers for objects. They could
represent a user inbox or other folder, a public folder/NNTP group. They are
ordered in a hierarchy, starting at a root object (parent set to itself)
- users: These are user accounts that are allowed to log in on any of the
servers services.
- directory(recipients): This would represent any type of recipient: users,
nntp groups, remote contacts, mailinglists. There would be a mechanisme to
allow for custom attributes to be stored.
I think that working with this abstraction, you can implement most mail,
new, calendering and other type of groupware services:
- NNTP: Can serve (part of) the folder hierarchy.
- IMAP4: Can theoretically serve the whole folder hierarchy, where the users
home directory is specified in a directory entry.
- POP3: Could serve the users (IMAP4) inbox or another folder, specified in
a directory entry.
- SMTP: Any incoming mail is matched against the directory for a valid
recipient and stores the email in the specified folder or sends it to the
mailinglist members defined in the directory.
- iCAP: Could use a set of folders to store the user calendars.
- LDAP: Should be able to serve the directory.
Pro's:
- flexible storage abstraction, allowing easy expansion for new services
- email addresses are not required to be bound to a user and vice versa
- objects and folders could be 'linked' using a many-to-many relation,
alling single instance storage accros the whole server
Con's:
- major James overhaul since almost the whole storage architecture has te be
rewritten, along with the configuration and probably the other services
- this architecture might create some extra overhead (extra database
lookups), degrading the overall performance
- somebody has to implement it :)
If there are questions left (there probably are...), I'll be happy to answer
them. It is very hard to describe the thing I have in my head in words. If
there is interest in this idea, I'll be happy to send a (My)SQL script that
would implement something like this.
Vincent de Lau
vincent@delau.nl
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users in FilesForwarding mail to NNTP
Posted by Harmeet Bedi <ha...@kodemuse.com>.
----- Original Message -----
From: "Sid Stuart" <si...@weaselworks.com>
> ... different repositories. The differences
> between them seem to be artificial boundries. I believe there should be
> a single message repository that provides a unique ID for a message, a
> URL to it's storage location and a set of attributes. All the protocols
> should be able to build on that simple structure to provide their
services.
>
Would java mail abstraction satisfy this ?
> I don't understand why there are different repositories. The differences
When NNTP Started, the mail repositories were slow and seemed more heavy
weight than need be. I did a some testing with standard mail repository and
built
- An NNTP repository on raw mime messages.
- Hid it behind a set of interfaces so that the core protocol code did not
ever need to interact with repository directly. The intent was to to
implement alternate nntp repository block when the standard repository
stabilized. This has got a more coupled over time.
The idea was to have something valuable out and move to a more standard
repository when available and make switch over as clean as possible.
> I have heard rumours of work being done in this area for James 1.3. Is
> there any documentation on the plans for it? (Other than what is in the
> wiki.)
Not sure about the documentation, but the consensus was to use Java Mail
abstraction and standard formats like maildir, mailbox under it. To some
extent repository ideas for SMTP and NNTP are similar to maildir. SMTP and
NNTP both store raw messages in single files.
IMAP was tested by Darrell with inmemory(non persistent) repository but
hasn't moved from there as far as I know.
Everybody agrees we should converge to a repository and format asap. At the
moment it is not too far apart at file system level but everything else is
parallel.
The key is moving code to standard repository API. The sooner we do this the
better. If javamail proposal takes too long to build maybe we should just
let it go away and use the current repository API/implementations as it is
and move nntp to it. thoughts.. ?
Harmeet
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: Serializing Users in FilesForwarding mail to NNTP
Posted by "Noel J. Bergman" <no...@devtech.com>.
> My understanding is that maildir and mbox are formats for storing mail
messages on a disk.
Correct.
> How does Javamail provide a standard storage interface to this?
Your inquiry can be rephrased as a statement: JavaMail provides a standard
storage interface to message repositories.
> Do you mean that the system will physically store the mail
> in mbox or maildir format and use the JavaMail Store class
> to hide the implementation details to the higher levels of
> code?
Essentially correct. But the storage format isn't limited to those two.
The current formats can be implemented underneath JavaMail.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users in FilesForwarding mail to NNTP
Posted by Sid Stuart <si...@weaselworks.com>.
Noel J. Bergman wrote:
>>Quite frankly, nothing I can envision (have not read the code) about the
>>needs of those protocols would require maildir or mailbox.
>>
>>
>
>Some USERS have preferences on the storage format because of others tools
>that they might use. We have had specific requests for things like maildir,
>mbox, etc. And others could not care less. JavaMail provides a standard
>storage interface suitable for all of these needs.
>
> --- Noel
>
Okay, you lost me here. My understanding is that maildir and mbox are
formats for storing mail messages on a disk. How does Javamail provide a
standard storage interface to this? Do you mean that the system will
physically store the mail in mbox or maildir format and use the JavaMail
/Store/ class to hide the implementation details to the higher levels of
code?
Sid
RE: Serializing Users in FilesForwarding mail to NNTP
Posted by "Noel J. Bergman" <no...@devtech.com>.
> I don't understand why there are different repositories.
Because that was what Harmeet chose to do when he coded NNTP. It is planned
to change in James v3 when we use JavaMail storage interfaces.
> Quite frankly, nothing I can envision (have not read the code) about the
> needs of those protocols would require maildir or mailbox.
Some USERS have preferences on the storage format because of others tools
that they might use. We have had specific requests for things like maildir,
mbox, etc. And others could not care less. JavaMail provides a standard
storage interface suitable for all of these needs.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Serializing Users in FilesForwarding mail to NNTP
Posted by Sid Stuart <si...@weaselworks.com>.
Noel J. Bergman wrote:
>This is something that a lot of us want: the ability to combine mail and
>newsgroups seamlessly. There are missing tools to do this at the moment.
>We need mailets that understand the NNTP repositories, which are different
>from the mail repositories in James v2.
>
>
>
>
I don't understand why there are different repositories. The differences
between them seem to be artificial boundries. I believe there should be
a single message repository that provides a unique ID for a message, a
URL to it's storage location and a set of attributes. All the protocols
should be able to build on that simple structure to provide their services.
I have heard rumours of work being done in this area for James 1.3. Is
there any documentation on the plans for it? (Other than what is in the
wiki.)
Sid
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: Serializing Users in FilesForwarding mail to NNTP
Posted by "Noel J. Bergman" <no...@devtech.com>.
This is something that a lot of us want: the ability to combine mail and
newsgroups seamlessly. There are missing tools to do this at the moment.
We need mailets that understand the NNTP repositories, which are different
from the mail repositories in James v2.
For example, we want to be able to create both a mailing list and an NNTP
newsgroup, and have messages intended for the newsgroup be posted internally
through an SMTP processor as if intended for a mailing list, then if they
make it through the filtering process, be posted to the newsgroup and the
mailing list.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org