James architecture: core components, sub-components and utility components

[Tellier Benoit <bt...@apache.org>]

Hi David,

I gave a shot at defining:

 - Core components, that offers services at heart of James
 - Utility components, that offers services core-components
implementations can rely on.
 - Mailbox sub-components.

Here is the result. I encourage other project members to review it.

 - Did I forgot some components?
 - Is the way I split them relevant?
 - Is the way to describe a component (descriptions, entities, key
interfaces, plugin) relevant? Would you add/remove something?
 - Of course all the component level details...

I hope it helps your documentation effort.

Best regards,

Benoit

-------------------------------------

# Core components

## MailetContainer

Enables mail processing.

Entity:
 - **Mails**: Message transiting between a sender and recipients

Key interfaces:
 - **Mailet**: Enables to act upon an email: modify it, trigger side effects
 - **Matcher**: Condition to trigger a mailet action

Plugin: A user can register their own mailets / matchers

Here is a list of mailets providing central services, and bridges other
core components defined here after :

 - **RemoteDelivery** enables sending emails to other mail servers,
using the SMTP protocol.
 - **LocalDelivery** enables delivering emails to local user mailboxes.
 - **RecipientRewriteTable** queries and applied recipient rewritting rules.
 - **ToMailRepository** stores emails in a given mail repository.
 - **Sieve** executes stored sieve scripts against incoming emails.

## MailQueue

Enables asynchronous mail processing. Enables review and administration
of mail traffic awaiting processing.

Entity: **Mails**

Key interface: **MailQueue**, **ManageableMailQueue**

Implementations:
 - Memory
 - (embedded) ActiveMQ
 - Distributed (RabbitMQ + Cassandra)
 - Files

## MailRepositories

Enables storing emails, along with their processing context. Enables
traffic review.

Can be used to recover error, stores spam, etc...

Entity: **Mails**

Key interfaces:
 - **MailRepository**
 - **MailRepositoryStore**: enables instanciation of MailRepositories

Plugins:
 - A user can register new classes of MailRepository, that he can then
configure within it's Mailet Container

Bundled implementation:
 - Memory
 - Cassandra
 - JDBC
 - File

## Mailbox

The mailbox stores user messages for later retrieval.

Entities:
 - **Messages**: a mime message belonging to a user, along with their
mailbox context.
 - **Mailboxes**: a group of messages

Sample operations:
 - Mailboxes can be created, deleted, renamed
 - Messages can be appended, deleted, moved, copied, their flags can be
modified

Key interfaces:
 - **MailboxManager**: Enable managing and accessing mailboxes
 - **MessageManager**: Enables accessing messages within a given
mailbox. Can be obtained via the MailboxManager.
 - **MessageIdManager**: Enables accessing messages by their unique
identifier.. Can be obtained via the MailboxManager.

Implementations:
 - Memory implementation
 - Maildir implementation
 - JPA implementation
 - Cassandra implementation

The mailbox components defines the following sub-components:

### The EventBus

James mailbox uses an event driven architecture.
	
It means every meaningful action on mailboxes or messages triggers an
event for any component to react to that event.
`MailboxListener` allows executing actions upon mailbox events. They
could be used for a wide variety of purposes, like
enriching mailbox managers features or enabling user notifications upon
concurrent mailboxes operations.

Entities:
 - **Events**: describes an action that happenened on a mailbox
 - **MailboxListeners**: standard API for acting upon events.

Key interfaces:
 - **EventBus**: enables MailboxListener registration and enables
dispatching events to the MailboxListeners

Implementations:
 - In VM
 - RabbitMQ

Plugins:
 - A user can register its own mailbox listeners.

### EventDeadLetter

Failed event processing is being saved to the EventDeadLetter for both
diagnostic and reprocessing purposes.

Entities:
 - **Events**: As per the EventBus

Key interfaces:
 - **EventDeadLetter**

Implementations:
 - Cassandra
 - Memory

### Search index

Sub components allowing searching emails.

Entities: **Messages**

Key interfaces:
 - **MessageSearchIndex**

Implementations:
 - Scrolling
 - Lucene
 - ElasticSearch

### Text extraction

Allows extracting text from arbtrary files. It empowers a more relevant
search within the MessageSearchIndex

Key interfaces:
 - **TextExtractor**

Implementations:
 - Default: only extracts plain text
 - JSoup: only extracts plain text or HTML
 - Tika: relias on Apache Tika

### Quotas

Defines and enforces limitations on the mailbox resource usage. Keep
track of current resource usage.

Entities:
 - Quota Root: defines a group of mailboxes for which a given quota
applies. James implementation defines QuotaRoot as the mailboxes
belonging to a user.
 - Resources: What resourc ehte quota tracks. Could be the count of
messages or the total sizes of messages.
 - Quota: A limit along with current usage for a given Quota Root.

Key interfaces:
 - **QuotaManager** allows to define limits and retrieve quota usage.

## data/api

Stores server level metadata.

Entities:
 - **Users**: people enables to access the server, along with their
credentials
 - **Domains**: Logical group of users

Key interfaces:
 - **UsersRepository**
 - **DomainList**

Implementation:
 - Memory
 - Cassandra
 - JPA

## Recipient Rewrite Tables

Enables storing rules for rewriting recipient of an email.

Entities:
 - **Mapping source**: defines which address should be rewritten for a
given rule
 - **Mapping**: defines the rewriting that should be performed

Key interface: **RecipientRewriteTable** stores all the Mappings along
with their sources.

Implementations:
 - Memory
 - Cassandra
 - JPA
 - XML (configuration only)

## SieveRepository

Enables users to store Sieve scripts. Enables managing quota applied for
user Sieve scripts.

Entity:
 - **Sieve script**: enable a user to define actions to be performed
upon mail reception.

Key interfaces:
 - **SieveRepository**: Leverages sieve script storage
 - **SieveQuotaRepository**: Leverages sieve script quota storage

## DNS Service

Provides abstraction for DNS resolutions. Helps DNS resolution for mail
processing purposes.

Key interfaces: **DNSService** enables DNS resolution.

# Protocols

## SMTP

Implementation of RFC-5321 to enable receiving emails. Mails are
processed asynchronously.

A user can register additional **ProtocolHandlers** to extend the
capabilities of the SMTP server.

A SMTP **Hook** is a specific Protocol handler being plugged in a
specific location within the default SMTP server stack.

## LMTP

Implementation of RFC-2033 to enable receiving emails. Similar to SMTP
but mails are processed synchronously.

A user can register additional **ProtocolHandlers** to extend the
capabilities of the LMTP server.

A SMTP **Hook** is a specific Protocol handler being plugged in a
specific location within the default SMTP server stack.

## IMAP

Implementationof RFC-3501 to enable a user to access and manage their
mailbox.

A user as of today can not extend the behaviour of the IMAP stack.

## JMAP

Implementation of RFC-8620 and RFC-8621 to enable a user to interact and
manage their mailboxes, and to send emails, on top of HTTP.

The implementation of these specifications is currently a work in
progress. A previous draft implementation is available.

## POP3

Implementation of RFC-1939 to enables users to retrieve the mails within
their mailboxes.

A user can register additional **ProtocolHandlers** to extend the
capabilities of the POP3 server.

## ManagedSieve

Implementation of RFC-5804 to enable a user to upload and manage their
Sieve scripts.

This protocol implementation is known to be buggy.

# Server Administration

## REST administration via WebAdmin

James specific REST APIs to manage other components, and their specific
implementations.

WebAdmin routes are defined in a modular way, the webadmin API will thus
be product specific.

Specification:

Key interface:
 - **Routes** enables registering additional REST endpoints.

Plugin: a user can define custom webadmin routes.

## Administration via the CLI

Command Line interface to manage James.

This CLI is not product specific, some command will not work for some
products.

The CLI relies on JMX protocol and can represent a potential
vulnerabilities. Guice products enables disabing it. A long term work is
to port the CLI to rely on the WebAdmin REST APIs.

# Utility components

These components offers services core-components implementations can
rely on.

## BlobStore

Stores potentially large binary data.

Entity:
 - **Blob**: Binary data

Key interface: BlobStore

Implementations:
 - Memory
 - ObjectStorage (S3/Swift)

## TaskManager

Allows to control and schedule long running tasks run by other
components. Among other it enables scheduling, progress monitoring,
cancelation of long running tasks

Entity:
 - **Task**: An operation performed by the TaskManager
 - **Task additional information**: Task specific information. Exposes
specific details about this task and how its execution went on.

Key interface: **TaskManager**.

Implementations:
 - Memory
 - Distributed (cassandra + RabbitMQ)

## Metrics

Enables recording execution timing for various operation within James.
Enables fine grained performance monitoring of a running James server,
for example using grafana boards.

Key interface: **MetricFactory**.

Implementation:
 - Default
 - DropWizard

## HealthChecks

Enables knowing the status of each components within a running James server.

This is both periodically logged, and exposed via WebAdmin.

Key interface: **HealthCheck**.

## Event sourcing

Event sourcing implementation for the James server. Enables components
to rely on event sourcing technics for taking decisions.

Here the definition of event is not the one of the mailbox, we are in a
different bounded context.

Entity:

 - **Aggregate** represents an entity on which actions are performed
 - **command** represents actions we wish to perform on this aggregate
 - **events** represent modifications of the aggregate. The state of the
aggregate can be computed from its event history.
 - **CommandHandler** applies commands on aggregates and generates the
resulting events.
 - **Subscriber** enables applying actions upon events.

Key interfaces:

 - **EventStore** stores events generated by event sourcing. Implemented
in Memory and on top of Cassandra.

## JSON serialization

Generic mechanism for modular JSON serialization of entities.

Sample usages:
 - Task
 - Task additional details
 - Event sourcing events

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: James architecture: core components, sub-components and utility components

[Eugen Stan <eu...@netdava.com>]

Hi Benoît,

Thanks for the effort you put into this. It's very useful.

I also saw the list of documents
https://github.com/apache/james-project/tree/master/src/adr .  I need to
go through them.

@Raphaël : IMO there are many points of view here:

- the technical / developer view - we work on source to produce
libraries and apps

- the business / product view - we package the code in binary form to
distribute it to our users.

The same code can be packaged in many ways and in many packages (think
products in a gift basket :) ) .

We get to decide what we deliver out of the box and in what packages.

We don't have to build a single package with everything.

It's more valuable to have them easily composable.

Having good documentation is paramount here. Since with docs, we empower
users to take care of themselves, instead of relying on us to provide
them with everything.

We can maintain a few common James offerings - for common use cases /
configurations. We should encourage users to customize their build and
we should point them at the docs.


Again, awesome job Benoît .


La 11.06.2020 18:54, Raphaël Ouazana-Sustowski a scris:
> Hello,
>
> Even if I globally agree with your definition of extension, I think
> import/export is a very valuable feature and should be promoted (when
> finished). Of course from an architecture point of view it could be
> seen as an extension but for an admin it should be well integrated and
> easy to use.
>
> Regards,
>
> Raphaël.
>
> Le 11/06/2020 à 08:21, Tellier Benoit a écrit :
>>
>> On 10/06/2020 22:49, Raphaël Ouazana-Sustowski wrote:
>>> Hi Benoît,
>>>
>>> This inventory is amazing, thanks for having issued it.
>>>
>>> For my part it's just missing the LDAP read only user repository as an
>>> implementation of the user repository.
>> True.
>>
>>> But I probably missed things :)
>>>
>>> Oh for example the (unfinished?) import / export mechanism.
>> I would count it as a mailbox extension. I did not listed them on
>> purpose as I think these subproject parts address very specific needs
>> and should not be packaged as part of out-of-the-box servers.
>>
>> I rather think that each server should have a list of extensions that we
>> can use with it.
>>
>> So far I can come up with the following list of extensions candidates
>> (not only mailbox ones):
>>   - Data Leak Prevention
>>   - SpamAssassin
>>   - OverQuota mailing
>>   - Quota search
>>   - Of course mailbox import/export
>>   - DeletedMessages vault
>>   - Maybe even Sieve support could live as an extension?
>>   - WKD would IMO also qualify...
>>   - ...
>>
>> As a regular user, I don't really care about them. They should not
>> complexify the configuration, documentation or the server itself that I
>> am running.
>>
>> I started detailing my thoughts about extension mechanism in this email
>> https://www.mail-archive.com/server-dev@james.apache.org/msg66206.html
>>
>> I believe that extracting extensions out of default servers would ease
>> reducing build time.
>>
>> But I also believe it would help us building (guice based) servers that
>> are easier to extend.
>>
>> Regards,
>>
>> Benoit
>>
>>> Regards,
>>>
>>> Raphaël.
>>>
>>> Le 25/05/2020 à 07:00, Tellier Benoit a écrit :
>>>> Hi David,
>>>>
>>>> I gave a shot at defining:
>>>>
>>>>    - Core components, that offers services at heart of James
>>>>    - Utility components, that offers services core-components
>>>> implementations can rely on.
>>>>    - Mailbox sub-components.
>>>>
>>>> Here is the result. I encourage other project members to review it.
>>>>
>>>>    - Did I forgot some components?
>>>>    - Is the way I split them relevant?
>>>>    - Is the way to describe a component (descriptions, entities, key
>>>> interfaces, plugin) relevant? Would you add/remove something?
>>>>    - Of course all the component level details...
>>>>
>>>> I hope it helps your documentation effort.
>>>>
>>>> Best regards,
>>>>
>>>> Benoit
>>>>
>>>> -------------------------------------
>>>>
>>>> # Core components
>>>>
>>>> ## MailetContainer
>>>>
>>>> Enables mail processing.
>>>>
>>>> Entity:
>>>>    - **Mails**: Message transiting between a sender and recipients
>>>>
>>>> Key interfaces:
>>>>    - **Mailet**: Enables to act upon an email: modify it, trigger side
>>>> effects
>>>>    - **Matcher**: Condition to trigger a mailet action
>>>>
>>>> Plugin: A user can register their own mailets / matchers
>>>>
>>>> Here is a list of mailets providing central services, and bridges
>>>> other
>>>> core components defined here after :
>>>>
>>>>    - **RemoteDelivery** enables sending emails to other mail servers,
>>>> using the SMTP protocol.
>>>>    - **LocalDelivery** enables delivering emails to local user
>>>> mailboxes.
>>>>    - **RecipientRewriteTable** queries and applied recipient
>>>> rewritting
>>>> rules.
>>>>    - **ToMailRepository** stores emails in a given mail repository.
>>>>    - **Sieve** executes stored sieve scripts against incoming emails.
>>>>
>>>> ## MailQueue
>>>>
>>>> Enables asynchronous mail processing. Enables review and
>>>> administration
>>>> of mail traffic awaiting processing.
>>>>
>>>> Entity: **Mails**
>>>>
>>>> Key interface: **MailQueue**, **ManageableMailQueue**
>>>>
>>>> Implementations:
>>>>    - Memory
>>>>    - (embedded) ActiveMQ
>>>>    - Distributed (RabbitMQ + Cassandra)
>>>>    - Files
>>>>
>>>> ## MailRepositories
>>>>
>>>> Enables storing emails, along with their processing context. Enables
>>>> traffic review.
>>>>
>>>> Can be used to recover error, stores spam, etc...
>>>>
>>>> Entity: **Mails**
>>>>
>>>> Key interfaces:
>>>>    - **MailRepository**
>>>>    - **MailRepositoryStore**: enables instanciation of
>>>> MailRepositories
>>>>
>>>> Plugins:
>>>>    - A user can register new classes of MailRepository, that he can
>>>> then
>>>> configure within it's Mailet Container
>>>>
>>>> Bundled implementation:
>>>>    - Memory
>>>>    - Cassandra
>>>>    - JDBC
>>>>    - File
>>>>
>>>> ## Mailbox
>>>>
>>>> The mailbox stores user messages for later retrieval.
>>>>
>>>> Entities:
>>>>    - **Messages**: a mime message belonging to a user, along with
>>>> their
>>>> mailbox context.
>>>>    - **Mailboxes**: a group of messages
>>>>
>>>> Sample operations:
>>>>    - Mailboxes can be created, deleted, renamed
>>>>    - Messages can be appended, deleted, moved, copied, their flags
>>>> can be
>>>> modified
>>>>
>>>> Key interfaces:
>>>>    - **MailboxManager**: Enable managing and accessing mailboxes
>>>>    - **MessageManager**: Enables accessing messages within a given
>>>> mailbox. Can be obtained via the MailboxManager.
>>>>    - **MessageIdManager**: Enables accessing messages by their unique
>>>> identifier.. Can be obtained via the MailboxManager.
>>>>
>>>> Implementations:
>>>>    - Memory implementation
>>>>    - Maildir implementation
>>>>    - JPA implementation
>>>>    - Cassandra implementation
>>>>
>>>> The mailbox components defines the following sub-components:
>>>>
>>>> ### The EventBus
>>>>
>>>> James mailbox uses an event driven architecture.
>>>>      It means every meaningful action on mailboxes or messages
>>>> triggers an
>>>> event for any component to react to that event.
>>>> `MailboxListener` allows executing actions upon mailbox events. They
>>>> could be used for a wide variety of purposes, like
>>>> enriching mailbox managers features or enabling user notifications
>>>> upon
>>>> concurrent mailboxes operations.
>>>>
>>>> Entities:
>>>>    - **Events**: describes an action that happenened on a mailbox
>>>>    - **MailboxListeners**: standard API for acting upon events.
>>>>
>>>> Key interfaces:
>>>>    - **EventBus**: enables MailboxListener registration and enables
>>>> dispatching events to the MailboxListeners
>>>>
>>>> Implementations:
>>>>    - In VM
>>>>    - RabbitMQ
>>>>
>>>> Plugins:
>>>>    - A user can register its own mailbox listeners.
>>>>
>>>> ### EventDeadLetter
>>>>
>>>> Failed event processing is being saved to the EventDeadLetter for both
>>>> diagnostic and reprocessing purposes.
>>>>
>>>> Entities:
>>>>    - **Events**: As per the EventBus
>>>>
>>>> Key interfaces:
>>>>    - **EventDeadLetter**
>>>>
>>>> Implementations:
>>>>    - Cassandra
>>>>    - Memory
>>>>
>>>> ### Search index
>>>>
>>>> Sub components allowing searching emails.
>>>>
>>>> Entities: **Messages**
>>>>
>>>> Key interfaces:
>>>>    - **MessageSearchIndex**
>>>>
>>>> Implementations:
>>>>    - Scrolling
>>>>    - Lucene
>>>>    - ElasticSearch
>>>>
>>>> ### Text extraction
>>>>
>>>> Allows extracting text from arbtrary files. It empowers a more
>>>> relevant
>>>> search within the MessageSearchIndex
>>>>
>>>> Key interfaces:
>>>>    - **TextExtractor**
>>>>
>>>> Implementations:
>>>>    - Default: only extracts plain text
>>>>    - JSoup: only extracts plain text or HTML
>>>>    - Tika: relias on Apache Tika
>>>>
>>>> ### Quotas
>>>>
>>>> Defines and enforces limitations on the mailbox resource usage. Keep
>>>> track of current resource usage.
>>>>
>>>> Entities:
>>>>    - Quota Root: defines a group of mailboxes for which a given quota
>>>> applies. James implementation defines QuotaRoot as the mailboxes
>>>> belonging to a user.
>>>>    - Resources: What resourc ehte quota tracks. Could be the count of
>>>> messages or the total sizes of messages.
>>>>    - Quota: A limit along with current usage for a given Quota Root.
>>>>
>>>> Key interfaces:
>>>>    - **QuotaManager** allows to define limits and retrieve quota
>>>> usage.
>>>>
>>>> ## data/api
>>>>
>>>> Stores server level metadata.
>>>>
>>>> Entities:
>>>>    - **Users**: people enables to access the server, along with their
>>>> credentials
>>>>    - **Domains**: Logical group of users
>>>>
>>>> Key interfaces:
>>>>    - **UsersRepository**
>>>>    - **DomainList**
>>>>
>>>> Implementation:
>>>>    - Memory
>>>>    - Cassandra
>>>>    - JPA
>>>>
>>>> ## Recipient Rewrite Tables
>>>>
>>>> Enables storing rules for rewriting recipient of an email.
>>>>
>>>> Entities:
>>>>    - **Mapping source**: defines which address should be rewritten
>>>> for a
>>>> given rule
>>>>    - **Mapping**: defines the rewriting that should be performed
>>>>
>>>> Key interface: **RecipientRewriteTable** stores all the Mappings along
>>>> with their sources.
>>>>
>>>> Implementations:
>>>>    - Memory
>>>>    - Cassandra
>>>>    - JPA
>>>>    - XML (configuration only)
>>>>
>>>> ## SieveRepository
>>>>
>>>> Enables users to store Sieve scripts. Enables managing quota
>>>> applied for
>>>> user Sieve scripts.
>>>>
>>>> Entity:
>>>>    - **Sieve script**: enable a user to define actions to be performed
>>>> upon mail reception.
>>>>
>>>> Key interfaces:
>>>>    - **SieveRepository**: Leverages sieve script storage
>>>>    - **SieveQuotaRepository**: Leverages sieve script quota storage
>>>>
>>>> ## DNS Service
>>>>
>>>> Provides abstraction for DNS resolutions. Helps DNS resolution for
>>>> mail
>>>> processing purposes.
>>>>
>>>> Key interfaces: **DNSService** enables DNS resolution.
>>>>
>>>> # Protocols
>>>>
>>>> ## SMTP
>>>>
>>>> Implementation of RFC-5321 to enable receiving emails. Mails are
>>>> processed asynchronously.
>>>>
>>>> A user can register additional **ProtocolHandlers** to extend the
>>>> capabilities of the SMTP server.
>>>>
>>>> A SMTP **Hook** is a specific Protocol handler being plugged in a
>>>> specific location within the default SMTP server stack.
>>>>
>>>> ## LMTP
>>>>
>>>> Implementation of RFC-2033 to enable receiving emails. Similar to SMTP
>>>> but mails are processed synchronously.
>>>>
>>>> A user can register additional **ProtocolHandlers** to extend the
>>>> capabilities of the LMTP server.
>>>>
>>>> A SMTP **Hook** is a specific Protocol handler being plugged in a
>>>> specific location within the default SMTP server stack.
>>>>
>>>> ## IMAP
>>>>
>>>> Implementationof RFC-3501 to enable a user to access and manage their
>>>> mailbox.
>>>>
>>>> A user as of today can not extend the behaviour of the IMAP stack.
>>>>
>>>> ## JMAP
>>>>
>>>> Implementation of RFC-8620 and RFC-8621 to enable a user to
>>>> interact and
>>>> manage their mailboxes, and to send emails, on top of HTTP.
>>>>
>>>> The implementation of these specifications is currently a work in
>>>> progress. A previous draft implementation is available.
>>>>
>>>> ## POP3
>>>>
>>>> Implementation of RFC-1939 to enables users to retrieve the mails
>>>> within
>>>> their mailboxes.
>>>>
>>>> A user can register additional **ProtocolHandlers** to extend the
>>>> capabilities of the POP3 server.
>>>>
>>>> ## ManagedSieve
>>>>
>>>> Implementation of RFC-5804 to enable a user to upload and manage their
>>>> Sieve scripts.
>>>>
>>>> This protocol implementation is known to be buggy.
>>>>
>>>> # Server Administration
>>>>
>>>> ## REST administration via WebAdmin
>>>>
>>>> James specific REST APIs to manage other components, and their
>>>> specific
>>>> implementations.
>>>>
>>>> WebAdmin routes are defined in a modular way, the webadmin API will
>>>> thus
>>>> be product specific.
>>>>
>>>> Specification:
>>>>
>>>> Key interface:
>>>>    - **Routes** enables registering additional REST endpoints.
>>>>
>>>> Plugin: a user can define custom webadmin routes.
>>>>
>>>> ## Administration via the CLI
>>>>
>>>> Command Line interface to manage James.
>>>>
>>>> This CLI is not product specific, some command will not work for some
>>>> products.
>>>>
>>>> The CLI relies on JMX protocol and can represent a potential
>>>> vulnerabilities. Guice products enables disabing it. A long term
>>>> work is
>>>> to port the CLI to rely on the WebAdmin REST APIs.
>>>>
>>>> # Utility components
>>>>
>>>> These components offers services core-components implementations can
>>>> rely on.
>>>>
>>>> ## BlobStore
>>>>
>>>> Stores potentially large binary data.
>>>>
>>>> Entity:
>>>>    - **Blob**: Binary data
>>>>
>>>> Key interface: BlobStore
>>>>
>>>> Implementations:
>>>>    - Memory
>>>>    - ObjectStorage (S3/Swift)
>>>>
>>>> ## TaskManager
>>>>
>>>> Allows to control and schedule long running tasks run by other
>>>> components. Among other it enables scheduling, progress monitoring,
>>>> cancelation of long running tasks
>>>>
>>>> Entity:
>>>>    - **Task**: An operation performed by the TaskManager
>>>>    - **Task additional information**: Task specific information.
>>>> Exposes
>>>> specific details about this task and how its execution went on.
>>>>
>>>> Key interface: **TaskManager**.
>>>>
>>>> Implementations:
>>>>    - Memory
>>>>    - Distributed (cassandra + RabbitMQ)
>>>>
>>>> ## Metrics
>>>>
>>>> Enables recording execution timing for various operation within James.
>>>> Enables fine grained performance monitoring of a running James server,
>>>> for example using grafana boards.
>>>>
>>>> Key interface: **MetricFactory**.
>>>>
>>>> Implementation:
>>>>    - Default
>>>>    - DropWizard
>>>>
>>>> ## HealthChecks
>>>>
>>>> Enables knowing the status of each components within a running James
>>>> server.
>>>>
>>>> This is both periodically logged, and exposed via WebAdmin.
>>>>
>>>> Key interface: **HealthCheck**.
>>>>
>>>> ## Event sourcing
>>>>
>>>> Event sourcing implementation for the James server. Enables components
>>>> to rely on event sourcing technics for taking decisions.
>>>>
>>>> Here the definition of event is not the one of the mailbox, we are
>>>> in a
>>>> different bounded context.
>>>>
>>>> Entity:
>>>>
>>>>    - **Aggregate** represents an entity on which actions are performed
>>>>    - **command** represents actions we wish to perform on this
>>>> aggregate
>>>>    - **events** represent modifications of the aggregate. The state
>>>> of the
>>>> aggregate can be computed from its event history.
>>>>    - **CommandHandler** applies commands on aggregates and
>>>> generates the
>>>> resulting events.
>>>>    - **Subscriber** enables applying actions upon events.
>>>>
>>>> Key interfaces:
>>>>
>>>>    - **EventStore** stores events generated by event sourcing.
>>>> Implemented
>>>> in Memory and on top of Cassandra.
>>>>
>>>> ## JSON serialization
>>>>
>>>> Generic mechanism for modular JSON serialization of entities.
>>>>
>>>> Sample usages:
>>>>    - Task
>>>>    - Task additional details
>>>>    - Event sourcing events
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>> For additional commands, e-mail: server-dev-help@james.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
-- 
Eugen Stan
+40720 898 747 / netdava.com

Re: James architecture: core components, sub-components and utility components

[Raphaël Ouazana-Sustowski <ro...@apache.org>]

Hello,

Even if I globally agree with your definition of extension, I think 
import/export is a very valuable feature and should be promoted (when 
finished). Of course from an architecture point of view it could be seen 
as an extension but for an admin it should be well integrated and easy 
to use.

Regards,

Raphaël.

Le 11/06/2020 à 08:21, Tellier Benoit a écrit :
>
> On 10/06/2020 22:49, Raphaël Ouazana-Sustowski wrote:
>> Hi Benoît,
>>
>> This inventory is amazing, thanks for having issued it.
>>
>> For my part it's just missing the LDAP read only user repository as an
>> implementation of the user repository.
> True.
>
>> But I probably missed things :)
>>
>> Oh for example the (unfinished?) import / export mechanism.
> I would count it as a mailbox extension. I did not listed them on
> purpose as I think these subproject parts address very specific needs
> and should not be packaged as part of out-of-the-box servers.
>
> I rather think that each server should have a list of extensions that we
> can use with it.
>
> So far I can come up with the following list of extensions candidates
> (not only mailbox ones):
>   - Data Leak Prevention
>   - SpamAssassin
>   - OverQuota mailing
>   - Quota search
>   - Of course mailbox import/export
>   - DeletedMessages vault
>   - Maybe even Sieve support could live as an extension?
>   - WKD would IMO also qualify...
>   - ...
>
> As a regular user, I don't really care about them. They should not
> complexify the configuration, documentation or the server itself that I
> am running.
>
> I started detailing my thoughts about extension mechanism in this email
> https://www.mail-archive.com/server-dev@james.apache.org/msg66206.html
>
> I believe that extracting extensions out of default servers would ease
> reducing build time.
>
> But I also believe it would help us building (guice based) servers that
> are easier to extend.
>
> Regards,
>
> Benoit
>
>> Regards,
>>
>> Raphaël.
>>
>> Le 25/05/2020 à 07:00, Tellier Benoit a écrit :
>>> Hi David,
>>>
>>> I gave a shot at defining:
>>>
>>>    - Core components, that offers services at heart of James
>>>    - Utility components, that offers services core-components
>>> implementations can rely on.
>>>    - Mailbox sub-components.
>>>
>>> Here is the result. I encourage other project members to review it.
>>>
>>>    - Did I forgot some components?
>>>    - Is the way I split them relevant?
>>>    - Is the way to describe a component (descriptions, entities, key
>>> interfaces, plugin) relevant? Would you add/remove something?
>>>    - Of course all the component level details...
>>>
>>> I hope it helps your documentation effort.
>>>
>>> Best regards,
>>>
>>> Benoit
>>>
>>> -------------------------------------
>>>
>>> # Core components
>>>
>>> ## MailetContainer
>>>
>>> Enables mail processing.
>>>
>>> Entity:
>>>    - **Mails**: Message transiting between a sender and recipients
>>>
>>> Key interfaces:
>>>    - **Mailet**: Enables to act upon an email: modify it, trigger side
>>> effects
>>>    - **Matcher**: Condition to trigger a mailet action
>>>
>>> Plugin: A user can register their own mailets / matchers
>>>
>>> Here is a list of mailets providing central services, and bridges other
>>> core components defined here after :
>>>
>>>    - **RemoteDelivery** enables sending emails to other mail servers,
>>> using the SMTP protocol.
>>>    - **LocalDelivery** enables delivering emails to local user mailboxes.
>>>    - **RecipientRewriteTable** queries and applied recipient rewritting
>>> rules.
>>>    - **ToMailRepository** stores emails in a given mail repository.
>>>    - **Sieve** executes stored sieve scripts against incoming emails.
>>>
>>> ## MailQueue
>>>
>>> Enables asynchronous mail processing. Enables review and administration
>>> of mail traffic awaiting processing.
>>>
>>> Entity: **Mails**
>>>
>>> Key interface: **MailQueue**, **ManageableMailQueue**
>>>
>>> Implementations:
>>>    - Memory
>>>    - (embedded) ActiveMQ
>>>    - Distributed (RabbitMQ + Cassandra)
>>>    - Files
>>>
>>> ## MailRepositories
>>>
>>> Enables storing emails, along with their processing context. Enables
>>> traffic review.
>>>
>>> Can be used to recover error, stores spam, etc...
>>>
>>> Entity: **Mails**
>>>
>>> Key interfaces:
>>>    - **MailRepository**
>>>    - **MailRepositoryStore**: enables instanciation of MailRepositories
>>>
>>> Plugins:
>>>    - A user can register new classes of MailRepository, that he can then
>>> configure within it's Mailet Container
>>>
>>> Bundled implementation:
>>>    - Memory
>>>    - Cassandra
>>>    - JDBC
>>>    - File
>>>
>>> ## Mailbox
>>>
>>> The mailbox stores user messages for later retrieval.
>>>
>>> Entities:
>>>    - **Messages**: a mime message belonging to a user, along with their
>>> mailbox context.
>>>    - **Mailboxes**: a group of messages
>>>
>>> Sample operations:
>>>    - Mailboxes can be created, deleted, renamed
>>>    - Messages can be appended, deleted, moved, copied, their flags can be
>>> modified
>>>
>>> Key interfaces:
>>>    - **MailboxManager**: Enable managing and accessing mailboxes
>>>    - **MessageManager**: Enables accessing messages within a given
>>> mailbox. Can be obtained via the MailboxManager.
>>>    - **MessageIdManager**: Enables accessing messages by their unique
>>> identifier.. Can be obtained via the MailboxManager.
>>>
>>> Implementations:
>>>    - Memory implementation
>>>    - Maildir implementation
>>>    - JPA implementation
>>>    - Cassandra implementation
>>>
>>> The mailbox components defines the following sub-components:
>>>
>>> ### The EventBus
>>>
>>> James mailbox uses an event driven architecture.
>>>      
>>> It means every meaningful action on mailboxes or messages triggers an
>>> event for any component to react to that event.
>>> `MailboxListener` allows executing actions upon mailbox events. They
>>> could be used for a wide variety of purposes, like
>>> enriching mailbox managers features or enabling user notifications upon
>>> concurrent mailboxes operations.
>>>
>>> Entities:
>>>    - **Events**: describes an action that happenened on a mailbox
>>>    - **MailboxListeners**: standard API for acting upon events.
>>>
>>> Key interfaces:
>>>    - **EventBus**: enables MailboxListener registration and enables
>>> dispatching events to the MailboxListeners
>>>
>>> Implementations:
>>>    - In VM
>>>    - RabbitMQ
>>>
>>> Plugins:
>>>    - A user can register its own mailbox listeners.
>>>
>>> ### EventDeadLetter
>>>
>>> Failed event processing is being saved to the EventDeadLetter for both
>>> diagnostic and reprocessing purposes.
>>>
>>> Entities:
>>>    - **Events**: As per the EventBus
>>>
>>> Key interfaces:
>>>    - **EventDeadLetter**
>>>
>>> Implementations:
>>>    - Cassandra
>>>    - Memory
>>>
>>> ### Search index
>>>
>>> Sub components allowing searching emails.
>>>
>>> Entities: **Messages**
>>>
>>> Key interfaces:
>>>    - **MessageSearchIndex**
>>>
>>> Implementations:
>>>    - Scrolling
>>>    - Lucene
>>>    - ElasticSearch
>>>
>>> ### Text extraction
>>>
>>> Allows extracting text from arbtrary files. It empowers a more relevant
>>> search within the MessageSearchIndex
>>>
>>> Key interfaces:
>>>    - **TextExtractor**
>>>
>>> Implementations:
>>>    - Default: only extracts plain text
>>>    - JSoup: only extracts plain text or HTML
>>>    - Tika: relias on Apache Tika
>>>
>>> ### Quotas
>>>
>>> Defines and enforces limitations on the mailbox resource usage. Keep
>>> track of current resource usage.
>>>
>>> Entities:
>>>    - Quota Root: defines a group of mailboxes for which a given quota
>>> applies. James implementation defines QuotaRoot as the mailboxes
>>> belonging to a user.
>>>    - Resources: What resourc ehte quota tracks. Could be the count of
>>> messages or the total sizes of messages.
>>>    - Quota: A limit along with current usage for a given Quota Root.
>>>
>>> Key interfaces:
>>>    - **QuotaManager** allows to define limits and retrieve quota usage.
>>>
>>> ## data/api
>>>
>>> Stores server level metadata.
>>>
>>> Entities:
>>>    - **Users**: people enables to access the server, along with their
>>> credentials
>>>    - **Domains**: Logical group of users
>>>
>>> Key interfaces:
>>>    - **UsersRepository**
>>>    - **DomainList**
>>>
>>> Implementation:
>>>    - Memory
>>>    - Cassandra
>>>    - JPA
>>>
>>> ## Recipient Rewrite Tables
>>>
>>> Enables storing rules for rewriting recipient of an email.
>>>
>>> Entities:
>>>    - **Mapping source**: defines which address should be rewritten for a
>>> given rule
>>>    - **Mapping**: defines the rewriting that should be performed
>>>
>>> Key interface: **RecipientRewriteTable** stores all the Mappings along
>>> with their sources.
>>>
>>> Implementations:
>>>    - Memory
>>>    - Cassandra
>>>    - JPA
>>>    - XML (configuration only)
>>>
>>> ## SieveRepository
>>>
>>> Enables users to store Sieve scripts. Enables managing quota applied for
>>> user Sieve scripts.
>>>
>>> Entity:
>>>    - **Sieve script**: enable a user to define actions to be performed
>>> upon mail reception.
>>>
>>> Key interfaces:
>>>    - **SieveRepository**: Leverages sieve script storage
>>>    - **SieveQuotaRepository**: Leverages sieve script quota storage
>>>
>>> ## DNS Service
>>>
>>> Provides abstraction for DNS resolutions. Helps DNS resolution for mail
>>> processing purposes.
>>>
>>> Key interfaces: **DNSService** enables DNS resolution.
>>>
>>> # Protocols
>>>
>>> ## SMTP
>>>
>>> Implementation of RFC-5321 to enable receiving emails. Mails are
>>> processed asynchronously.
>>>
>>> A user can register additional **ProtocolHandlers** to extend the
>>> capabilities of the SMTP server.
>>>
>>> A SMTP **Hook** is a specific Protocol handler being plugged in a
>>> specific location within the default SMTP server stack.
>>>
>>> ## LMTP
>>>
>>> Implementation of RFC-2033 to enable receiving emails. Similar to SMTP
>>> but mails are processed synchronously.
>>>
>>> A user can register additional **ProtocolHandlers** to extend the
>>> capabilities of the LMTP server.
>>>
>>> A SMTP **Hook** is a specific Protocol handler being plugged in a
>>> specific location within the default SMTP server stack.
>>>
>>> ## IMAP
>>>
>>> Implementationof RFC-3501 to enable a user to access and manage their
>>> mailbox.
>>>
>>> A user as of today can not extend the behaviour of the IMAP stack.
>>>
>>> ## JMAP
>>>
>>> Implementation of RFC-8620 and RFC-8621 to enable a user to interact and
>>> manage their mailboxes, and to send emails, on top of HTTP.
>>>
>>> The implementation of these specifications is currently a work in
>>> progress. A previous draft implementation is available.
>>>
>>> ## POP3
>>>
>>> Implementation of RFC-1939 to enables users to retrieve the mails within
>>> their mailboxes.
>>>
>>> A user can register additional **ProtocolHandlers** to extend the
>>> capabilities of the POP3 server.
>>>
>>> ## ManagedSieve
>>>
>>> Implementation of RFC-5804 to enable a user to upload and manage their
>>> Sieve scripts.
>>>
>>> This protocol implementation is known to be buggy.
>>>
>>> # Server Administration
>>>
>>> ## REST administration via WebAdmin
>>>
>>> James specific REST APIs to manage other components, and their specific
>>> implementations.
>>>
>>> WebAdmin routes are defined in a modular way, the webadmin API will thus
>>> be product specific.
>>>
>>> Specification:
>>>
>>> Key interface:
>>>    - **Routes** enables registering additional REST endpoints.
>>>
>>> Plugin: a user can define custom webadmin routes.
>>>
>>> ## Administration via the CLI
>>>
>>> Command Line interface to manage James.
>>>
>>> This CLI is not product specific, some command will not work for some
>>> products.
>>>
>>> The CLI relies on JMX protocol and can represent a potential
>>> vulnerabilities. Guice products enables disabing it. A long term work is
>>> to port the CLI to rely on the WebAdmin REST APIs.
>>>
>>> # Utility components
>>>
>>> These components offers services core-components implementations can
>>> rely on.
>>>
>>> ## BlobStore
>>>
>>> Stores potentially large binary data.
>>>
>>> Entity:
>>>    - **Blob**: Binary data
>>>
>>> Key interface: BlobStore
>>>
>>> Implementations:
>>>    - Memory
>>>    - ObjectStorage (S3/Swift)
>>>
>>> ## TaskManager
>>>
>>> Allows to control and schedule long running tasks run by other
>>> components. Among other it enables scheduling, progress monitoring,
>>> cancelation of long running tasks
>>>
>>> Entity:
>>>    - **Task**: An operation performed by the TaskManager
>>>    - **Task additional information**: Task specific information. Exposes
>>> specific details about this task and how its execution went on.
>>>
>>> Key interface: **TaskManager**.
>>>
>>> Implementations:
>>>    - Memory
>>>    - Distributed (cassandra + RabbitMQ)
>>>
>>> ## Metrics
>>>
>>> Enables recording execution timing for various operation within James.
>>> Enables fine grained performance monitoring of a running James server,
>>> for example using grafana boards.
>>>
>>> Key interface: **MetricFactory**.
>>>
>>> Implementation:
>>>    - Default
>>>    - DropWizard
>>>
>>> ## HealthChecks
>>>
>>> Enables knowing the status of each components within a running James
>>> server.
>>>
>>> This is both periodically logged, and exposed via WebAdmin.
>>>
>>> Key interface: **HealthCheck**.
>>>
>>> ## Event sourcing
>>>
>>> Event sourcing implementation for the James server. Enables components
>>> to rely on event sourcing technics for taking decisions.
>>>
>>> Here the definition of event is not the one of the mailbox, we are in a
>>> different bounded context.
>>>
>>> Entity:
>>>
>>>    - **Aggregate** represents an entity on which actions are performed
>>>    - **command** represents actions we wish to perform on this aggregate
>>>    - **events** represent modifications of the aggregate. The state of the
>>> aggregate can be computed from its event history.
>>>    - **CommandHandler** applies commands on aggregates and generates the
>>> resulting events.
>>>    - **Subscriber** enables applying actions upon events.
>>>
>>> Key interfaces:
>>>
>>>    - **EventStore** stores events generated by event sourcing. Implemented
>>> in Memory and on top of Cassandra.
>>>
>>> ## JSON serialization
>>>
>>> Generic mechanism for modular JSON serialization of entities.
>>>
>>> Sample usages:
>>>    - Task
>>>    - Task additional details
>>>    - Event sourcing events
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>>> For additional commands, e-mail: server-dev-help@james.apache.org
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>> For additional commands, e-mail: server-dev-help@james.apache.org
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: James architecture: core components, sub-components and utility components

[Tellier Benoit <bt...@apache.org>]

On 10/06/2020 22:49, Raphaël Ouazana-Sustowski wrote:
> Hi Benoît,
> 
> This inventory is amazing, thanks for having issued it.
> 
> For my part it's just missing the LDAP read only user repository as an
> implementation of the user repository.

True.

> But I probably missed things :)
> 
> Oh for example the (unfinished?) import / export mechanism.

I would count it as a mailbox extension. I did not listed them on
purpose as I think these subproject parts address very specific needs
and should not be packaged as part of out-of-the-box servers.

I rather think that each server should have a list of extensions that we
can use with it.

So far I can come up with the following list of extensions candidates
(not only mailbox ones):
 - Data Leak Prevention
 - SpamAssassin
 - OverQuota mailing
 - Quota search
 - Of course mailbox import/export
 - DeletedMessages vault
 - Maybe even Sieve support could live as an extension?
 - WKD would IMO also qualify...
 - ...

As a regular user, I don't really care about them. They should not
complexify the configuration, documentation or the server itself that I
am running.

I started detailing my thoughts about extension mechanism in this email
https://www.mail-archive.com/server-dev@james.apache.org/msg66206.html

I believe that extracting extensions out of default servers would ease
reducing build time.

But I also believe it would help us building (guice based) servers that
are easier to extend.

Regards,

Benoit

> 
> Regards,
> 
> Raphaël.
> 
> Le 25/05/2020 à 07:00, Tellier Benoit a écrit :
>> Hi David,
>>
>> I gave a shot at defining:
>>
>>   - Core components, that offers services at heart of James
>>   - Utility components, that offers services core-components
>> implementations can rely on.
>>   - Mailbox sub-components.
>>
>> Here is the result. I encourage other project members to review it.
>>
>>   - Did I forgot some components?
>>   - Is the way I split them relevant?
>>   - Is the way to describe a component (descriptions, entities, key
>> interfaces, plugin) relevant? Would you add/remove something?
>>   - Of course all the component level details...
>>
>> I hope it helps your documentation effort.
>>
>> Best regards,
>>
>> Benoit
>>
>> -------------------------------------
>>
>> # Core components
>>
>> ## MailetContainer
>>
>> Enables mail processing.
>>
>> Entity:
>>   - **Mails**: Message transiting between a sender and recipients
>>
>> Key interfaces:
>>   - **Mailet**: Enables to act upon an email: modify it, trigger side
>> effects
>>   - **Matcher**: Condition to trigger a mailet action
>>
>> Plugin: A user can register their own mailets / matchers
>>
>> Here is a list of mailets providing central services, and bridges other
>> core components defined here after :
>>
>>   - **RemoteDelivery** enables sending emails to other mail servers,
>> using the SMTP protocol.
>>   - **LocalDelivery** enables delivering emails to local user mailboxes.
>>   - **RecipientRewriteTable** queries and applied recipient rewritting
>> rules.
>>   - **ToMailRepository** stores emails in a given mail repository.
>>   - **Sieve** executes stored sieve scripts against incoming emails.
>>
>> ## MailQueue
>>
>> Enables asynchronous mail processing. Enables review and administration
>> of mail traffic awaiting processing.
>>
>> Entity: **Mails**
>>
>> Key interface: **MailQueue**, **ManageableMailQueue**
>>
>> Implementations:
>>   - Memory
>>   - (embedded) ActiveMQ
>>   - Distributed (RabbitMQ + Cassandra)
>>   - Files
>>
>> ## MailRepositories
>>
>> Enables storing emails, along with their processing context. Enables
>> traffic review.
>>
>> Can be used to recover error, stores spam, etc...
>>
>> Entity: **Mails**
>>
>> Key interfaces:
>>   - **MailRepository**
>>   - **MailRepositoryStore**: enables instanciation of MailRepositories
>>
>> Plugins:
>>   - A user can register new classes of MailRepository, that he can then
>> configure within it's Mailet Container
>>
>> Bundled implementation:
>>   - Memory
>>   - Cassandra
>>   - JDBC
>>   - File
>>
>> ## Mailbox
>>
>> The mailbox stores user messages for later retrieval.
>>
>> Entities:
>>   - **Messages**: a mime message belonging to a user, along with their
>> mailbox context.
>>   - **Mailboxes**: a group of messages
>>
>> Sample operations:
>>   - Mailboxes can be created, deleted, renamed
>>   - Messages can be appended, deleted, moved, copied, their flags can be
>> modified
>>
>> Key interfaces:
>>   - **MailboxManager**: Enable managing and accessing mailboxes
>>   - **MessageManager**: Enables accessing messages within a given
>> mailbox. Can be obtained via the MailboxManager.
>>   - **MessageIdManager**: Enables accessing messages by their unique
>> identifier.. Can be obtained via the MailboxManager.
>>
>> Implementations:
>>   - Memory implementation
>>   - Maildir implementation
>>   - JPA implementation
>>   - Cassandra implementation
>>
>> The mailbox components defines the following sub-components:
>>
>> ### The EventBus
>>
>> James mailbox uses an event driven architecture.
>>     
>> It means every meaningful action on mailboxes or messages triggers an
>> event for any component to react to that event.
>> `MailboxListener` allows executing actions upon mailbox events. They
>> could be used for a wide variety of purposes, like
>> enriching mailbox managers features or enabling user notifications upon
>> concurrent mailboxes operations.
>>
>> Entities:
>>   - **Events**: describes an action that happenened on a mailbox
>>   - **MailboxListeners**: standard API for acting upon events.
>>
>> Key interfaces:
>>   - **EventBus**: enables MailboxListener registration and enables
>> dispatching events to the MailboxListeners
>>
>> Implementations:
>>   - In VM
>>   - RabbitMQ
>>
>> Plugins:
>>   - A user can register its own mailbox listeners.
>>
>> ### EventDeadLetter
>>
>> Failed event processing is being saved to the EventDeadLetter for both
>> diagnostic and reprocessing purposes.
>>
>> Entities:
>>   - **Events**: As per the EventBus
>>
>> Key interfaces:
>>   - **EventDeadLetter**
>>
>> Implementations:
>>   - Cassandra
>>   - Memory
>>
>> ### Search index
>>
>> Sub components allowing searching emails.
>>
>> Entities: **Messages**
>>
>> Key interfaces:
>>   - **MessageSearchIndex**
>>
>> Implementations:
>>   - Scrolling
>>   - Lucene
>>   - ElasticSearch
>>
>> ### Text extraction
>>
>> Allows extracting text from arbtrary files. It empowers a more relevant
>> search within the MessageSearchIndex
>>
>> Key interfaces:
>>   - **TextExtractor**
>>
>> Implementations:
>>   - Default: only extracts plain text
>>   - JSoup: only extracts plain text or HTML
>>   - Tika: relias on Apache Tika
>>
>> ### Quotas
>>
>> Defines and enforces limitations on the mailbox resource usage. Keep
>> track of current resource usage.
>>
>> Entities:
>>   - Quota Root: defines a group of mailboxes for which a given quota
>> applies. James implementation defines QuotaRoot as the mailboxes
>> belonging to a user.
>>   - Resources: What resourc ehte quota tracks. Could be the count of
>> messages or the total sizes of messages.
>>   - Quota: A limit along with current usage for a given Quota Root.
>>
>> Key interfaces:
>>   - **QuotaManager** allows to define limits and retrieve quota usage.
>>
>> ## data/api
>>
>> Stores server level metadata.
>>
>> Entities:
>>   - **Users**: people enables to access the server, along with their
>> credentials
>>   - **Domains**: Logical group of users
>>
>> Key interfaces:
>>   - **UsersRepository**
>>   - **DomainList**
>>
>> Implementation:
>>   - Memory
>>   - Cassandra
>>   - JPA
>>
>> ## Recipient Rewrite Tables
>>
>> Enables storing rules for rewriting recipient of an email.
>>
>> Entities:
>>   - **Mapping source**: defines which address should be rewritten for a
>> given rule
>>   - **Mapping**: defines the rewriting that should be performed
>>
>> Key interface: **RecipientRewriteTable** stores all the Mappings along
>> with their sources.
>>
>> Implementations:
>>   - Memory
>>   - Cassandra
>>   - JPA
>>   - XML (configuration only)
>>
>> ## SieveRepository
>>
>> Enables users to store Sieve scripts. Enables managing quota applied for
>> user Sieve scripts.
>>
>> Entity:
>>   - **Sieve script**: enable a user to define actions to be performed
>> upon mail reception.
>>
>> Key interfaces:
>>   - **SieveRepository**: Leverages sieve script storage
>>   - **SieveQuotaRepository**: Leverages sieve script quota storage
>>
>> ## DNS Service
>>
>> Provides abstraction for DNS resolutions. Helps DNS resolution for mail
>> processing purposes.
>>
>> Key interfaces: **DNSService** enables DNS resolution.
>>
>> # Protocols
>>
>> ## SMTP
>>
>> Implementation of RFC-5321 to enable receiving emails. Mails are
>> processed asynchronously.
>>
>> A user can register additional **ProtocolHandlers** to extend the
>> capabilities of the SMTP server.
>>
>> A SMTP **Hook** is a specific Protocol handler being plugged in a
>> specific location within the default SMTP server stack.
>>
>> ## LMTP
>>
>> Implementation of RFC-2033 to enable receiving emails. Similar to SMTP
>> but mails are processed synchronously.
>>
>> A user can register additional **ProtocolHandlers** to extend the
>> capabilities of the LMTP server.
>>
>> A SMTP **Hook** is a specific Protocol handler being plugged in a
>> specific location within the default SMTP server stack.
>>
>> ## IMAP
>>
>> Implementationof RFC-3501 to enable a user to access and manage their
>> mailbox.
>>
>> A user as of today can not extend the behaviour of the IMAP stack.
>>
>> ## JMAP
>>
>> Implementation of RFC-8620 and RFC-8621 to enable a user to interact and
>> manage their mailboxes, and to send emails, on top of HTTP.
>>
>> The implementation of these specifications is currently a work in
>> progress. A previous draft implementation is available.
>>
>> ## POP3
>>
>> Implementation of RFC-1939 to enables users to retrieve the mails within
>> their mailboxes.
>>
>> A user can register additional **ProtocolHandlers** to extend the
>> capabilities of the POP3 server.
>>
>> ## ManagedSieve
>>
>> Implementation of RFC-5804 to enable a user to upload and manage their
>> Sieve scripts.
>>
>> This protocol implementation is known to be buggy.
>>
>> # Server Administration
>>
>> ## REST administration via WebAdmin
>>
>> James specific REST APIs to manage other components, and their specific
>> implementations.
>>
>> WebAdmin routes are defined in a modular way, the webadmin API will thus
>> be product specific.
>>
>> Specification:
>>
>> Key interface:
>>   - **Routes** enables registering additional REST endpoints.
>>
>> Plugin: a user can define custom webadmin routes.
>>
>> ## Administration via the CLI
>>
>> Command Line interface to manage James.
>>
>> This CLI is not product specific, some command will not work for some
>> products.
>>
>> The CLI relies on JMX protocol and can represent a potential
>> vulnerabilities. Guice products enables disabing it. A long term work is
>> to port the CLI to rely on the WebAdmin REST APIs.
>>
>> # Utility components
>>
>> These components offers services core-components implementations can
>> rely on.
>>
>> ## BlobStore
>>
>> Stores potentially large binary data.
>>
>> Entity:
>>   - **Blob**: Binary data
>>
>> Key interface: BlobStore
>>
>> Implementations:
>>   - Memory
>>   - ObjectStorage (S3/Swift)
>>
>> ## TaskManager
>>
>> Allows to control and schedule long running tasks run by other
>> components. Among other it enables scheduling, progress monitoring,
>> cancelation of long running tasks
>>
>> Entity:
>>   - **Task**: An operation performed by the TaskManager
>>   - **Task additional information**: Task specific information. Exposes
>> specific details about this task and how its execution went on.
>>
>> Key interface: **TaskManager**.
>>
>> Implementations:
>>   - Memory
>>   - Distributed (cassandra + RabbitMQ)
>>
>> ## Metrics
>>
>> Enables recording execution timing for various operation within James.
>> Enables fine grained performance monitoring of a running James server,
>> for example using grafana boards.
>>
>> Key interface: **MetricFactory**.
>>
>> Implementation:
>>   - Default
>>   - DropWizard
>>
>> ## HealthChecks
>>
>> Enables knowing the status of each components within a running James
>> server.
>>
>> This is both periodically logged, and exposed via WebAdmin.
>>
>> Key interface: **HealthCheck**.
>>
>> ## Event sourcing
>>
>> Event sourcing implementation for the James server. Enables components
>> to rely on event sourcing technics for taking decisions.
>>
>> Here the definition of event is not the one of the mailbox, we are in a
>> different bounded context.
>>
>> Entity:
>>
>>   - **Aggregate** represents an entity on which actions are performed
>>   - **command** represents actions we wish to perform on this aggregate
>>   - **events** represent modifications of the aggregate. The state of the
>> aggregate can be computed from its event history.
>>   - **CommandHandler** applies commands on aggregates and generates the
>> resulting events.
>>   - **Subscriber** enables applying actions upon events.
>>
>> Key interfaces:
>>
>>   - **EventStore** stores events generated by event sourcing. Implemented
>> in Memory and on top of Cassandra.
>>
>> ## JSON serialization
>>
>> Generic mechanism for modular JSON serialization of entities.
>>
>> Sample usages:
>>   - Task
>>   - Task additional details
>>   - Event sourcing events
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>> For additional commands, e-mail: server-dev-help@james.apache.org
>>
>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: James architecture: core components, sub-components and utility components

[Raphaël Ouazana-Sustowski <ro...@apache.org>]

Hi Benoît,

This inventory is amazing, thanks for having issued it.

For my part it's just missing the LDAP read only user repository as an 
implementation of the user repository.

But I probably missed things :)

Oh for example the (unfinished?) import / export mechanism.

Regards,

Raphaël.

Le 25/05/2020 à 07:00, Tellier Benoit a écrit :
> Hi David,
>
> I gave a shot at defining:
>
>   - Core components, that offers services at heart of James
>   - Utility components, that offers services core-components
> implementations can rely on.
>   - Mailbox sub-components.
>
> Here is the result. I encourage other project members to review it.
>
>   - Did I forgot some components?
>   - Is the way I split them relevant?
>   - Is the way to describe a component (descriptions, entities, key
> interfaces, plugin) relevant? Would you add/remove something?
>   - Of course all the component level details...
>
> I hope it helps your documentation effort.
>
> Best regards,
>
> Benoit
>
> -------------------------------------
>
> # Core components
>
> ## MailetContainer
>
> Enables mail processing.
>
> Entity:
>   - **Mails**: Message transiting between a sender and recipients
>
> Key interfaces:
>   - **Mailet**: Enables to act upon an email: modify it, trigger side effects
>   - **Matcher**: Condition to trigger a mailet action
>
> Plugin: A user can register their own mailets / matchers
>
> Here is a list of mailets providing central services, and bridges other
> core components defined here after :
>
>   - **RemoteDelivery** enables sending emails to other mail servers,
> using the SMTP protocol.
>   - **LocalDelivery** enables delivering emails to local user mailboxes.
>   - **RecipientRewriteTable** queries and applied recipient rewritting rules.
>   - **ToMailRepository** stores emails in a given mail repository.
>   - **Sieve** executes stored sieve scripts against incoming emails.
>
> ## MailQueue
>
> Enables asynchronous mail processing. Enables review and administration
> of mail traffic awaiting processing.
>
> Entity: **Mails**
>
> Key interface: **MailQueue**, **ManageableMailQueue**
>
> Implementations:
>   - Memory
>   - (embedded) ActiveMQ
>   - Distributed (RabbitMQ + Cassandra)
>   - Files
>
> ## MailRepositories
>
> Enables storing emails, along with their processing context. Enables
> traffic review.
>
> Can be used to recover error, stores spam, etc...
>
> Entity: **Mails**
>
> Key interfaces:
>   - **MailRepository**
>   - **MailRepositoryStore**: enables instanciation of MailRepositories
>
> Plugins:
>   - A user can register new classes of MailRepository, that he can then
> configure within it's Mailet Container
>
> Bundled implementation:
>   - Memory
>   - Cassandra
>   - JDBC
>   - File
>
> ## Mailbox
>
> The mailbox stores user messages for later retrieval.
>
> Entities:
>   - **Messages**: a mime message belonging to a user, along with their
> mailbox context.
>   - **Mailboxes**: a group of messages
>
> Sample operations:
>   - Mailboxes can be created, deleted, renamed
>   - Messages can be appended, deleted, moved, copied, their flags can be
> modified
>
> Key interfaces:
>   - **MailboxManager**: Enable managing and accessing mailboxes
>   - **MessageManager**: Enables accessing messages within a given
> mailbox. Can be obtained via the MailboxManager.
>   - **MessageIdManager**: Enables accessing messages by their unique
> identifier.. Can be obtained via the MailboxManager.
>
> Implementations:
>   - Memory implementation
>   - Maildir implementation
>   - JPA implementation
>   - Cassandra implementation
>
> The mailbox components defines the following sub-components:
>
> ### The EventBus
>
> James mailbox uses an event driven architecture.
> 	
> It means every meaningful action on mailboxes or messages triggers an
> event for any component to react to that event.
> `MailboxListener` allows executing actions upon mailbox events. They
> could be used for a wide variety of purposes, like
> enriching mailbox managers features or enabling user notifications upon
> concurrent mailboxes operations.
>
> Entities:
>   - **Events**: describes an action that happenened on a mailbox
>   - **MailboxListeners**: standard API for acting upon events.
>
> Key interfaces:
>   - **EventBus**: enables MailboxListener registration and enables
> dispatching events to the MailboxListeners
>
> Implementations:
>   - In VM
>   - RabbitMQ
>
> Plugins:
>   - A user can register its own mailbox listeners.
>
> ### EventDeadLetter
>
> Failed event processing is being saved to the EventDeadLetter for both
> diagnostic and reprocessing purposes.
>
> Entities:
>   - **Events**: As per the EventBus
>
> Key interfaces:
>   - **EventDeadLetter**
>
> Implementations:
>   - Cassandra
>   - Memory
>
> ### Search index
>
> Sub components allowing searching emails.
>
> Entities: **Messages**
>
> Key interfaces:
>   - **MessageSearchIndex**
>
> Implementations:
>   - Scrolling
>   - Lucene
>   - ElasticSearch
>
> ### Text extraction
>
> Allows extracting text from arbtrary files. It empowers a more relevant
> search within the MessageSearchIndex
>
> Key interfaces:
>   - **TextExtractor**
>
> Implementations:
>   - Default: only extracts plain text
>   - JSoup: only extracts plain text or HTML
>   - Tika: relias on Apache Tika
>
> ### Quotas
>
> Defines and enforces limitations on the mailbox resource usage. Keep
> track of current resource usage.
>
> Entities:
>   - Quota Root: defines a group of mailboxes for which a given quota
> applies. James implementation defines QuotaRoot as the mailboxes
> belonging to a user.
>   - Resources: What resourc ehte quota tracks. Could be the count of
> messages or the total sizes of messages.
>   - Quota: A limit along with current usage for a given Quota Root.
>
> Key interfaces:
>   - **QuotaManager** allows to define limits and retrieve quota usage.
>
> ## data/api
>
> Stores server level metadata.
>
> Entities:
>   - **Users**: people enables to access the server, along with their
> credentials
>   - **Domains**: Logical group of users
>
> Key interfaces:
>   - **UsersRepository**
>   - **DomainList**
>
> Implementation:
>   - Memory
>   - Cassandra
>   - JPA
>
> ## Recipient Rewrite Tables
>
> Enables storing rules for rewriting recipient of an email.
>
> Entities:
>   - **Mapping source**: defines which address should be rewritten for a
> given rule
>   - **Mapping**: defines the rewriting that should be performed
>
> Key interface: **RecipientRewriteTable** stores all the Mappings along
> with their sources.
>
> Implementations:
>   - Memory
>   - Cassandra
>   - JPA
>   - XML (configuration only)
>
> ## SieveRepository
>
> Enables users to store Sieve scripts. Enables managing quota applied for
> user Sieve scripts.
>
> Entity:
>   - **Sieve script**: enable a user to define actions to be performed
> upon mail reception.
>
> Key interfaces:
>   - **SieveRepository**: Leverages sieve script storage
>   - **SieveQuotaRepository**: Leverages sieve script quota storage
>
> ## DNS Service
>
> Provides abstraction for DNS resolutions. Helps DNS resolution for mail
> processing purposes.
>
> Key interfaces: **DNSService** enables DNS resolution.
>
> # Protocols
>
> ## SMTP
>
> Implementation of RFC-5321 to enable receiving emails. Mails are
> processed asynchronously.
>
> A user can register additional **ProtocolHandlers** to extend the
> capabilities of the SMTP server.
>
> A SMTP **Hook** is a specific Protocol handler being plugged in a
> specific location within the default SMTP server stack.
>
> ## LMTP
>
> Implementation of RFC-2033 to enable receiving emails. Similar to SMTP
> but mails are processed synchronously.
>
> A user can register additional **ProtocolHandlers** to extend the
> capabilities of the LMTP server.
>
> A SMTP **Hook** is a specific Protocol handler being plugged in a
> specific location within the default SMTP server stack.
>
> ## IMAP
>
> Implementationof RFC-3501 to enable a user to access and manage their
> mailbox.
>
> A user as of today can not extend the behaviour of the IMAP stack.
>
> ## JMAP
>
> Implementation of RFC-8620 and RFC-8621 to enable a user to interact and
> manage their mailboxes, and to send emails, on top of HTTP.
>
> The implementation of these specifications is currently a work in
> progress. A previous draft implementation is available.
>
> ## POP3
>
> Implementation of RFC-1939 to enables users to retrieve the mails within
> their mailboxes.
>
> A user can register additional **ProtocolHandlers** to extend the
> capabilities of the POP3 server.
>
> ## ManagedSieve
>
> Implementation of RFC-5804 to enable a user to upload and manage their
> Sieve scripts.
>
> This protocol implementation is known to be buggy.
>
> # Server Administration
>
> ## REST administration via WebAdmin
>
> James specific REST APIs to manage other components, and their specific
> implementations.
>
> WebAdmin routes are defined in a modular way, the webadmin API will thus
> be product specific.
>
> Specification:
>
> Key interface:
>   - **Routes** enables registering additional REST endpoints.
>
> Plugin: a user can define custom webadmin routes.
>
> ## Administration via the CLI
>
> Command Line interface to manage James.
>
> This CLI is not product specific, some command will not work for some
> products.
>
> The CLI relies on JMX protocol and can represent a potential
> vulnerabilities. Guice products enables disabing it. A long term work is
> to port the CLI to rely on the WebAdmin REST APIs.
>
> # Utility components
>
> These components offers services core-components implementations can
> rely on.
>
> ## BlobStore
>
> Stores potentially large binary data.
>
> Entity:
>   - **Blob**: Binary data
>
> Key interface: BlobStore
>
> Implementations:
>   - Memory
>   - ObjectStorage (S3/Swift)
>
> ## TaskManager
>
> Allows to control and schedule long running tasks run by other
> components. Among other it enables scheduling, progress monitoring,
> cancelation of long running tasks
>
> Entity:
>   - **Task**: An operation performed by the TaskManager
>   - **Task additional information**: Task specific information. Exposes
> specific details about this task and how its execution went on.
>
> Key interface: **TaskManager**.
>
> Implementations:
>   - Memory
>   - Distributed (cassandra + RabbitMQ)
>
> ## Metrics
>
> Enables recording execution timing for various operation within James.
> Enables fine grained performance monitoring of a running James server,
> for example using grafana boards.
>
> Key interface: **MetricFactory**.
>
> Implementation:
>   - Default
>   - DropWizard
>
> ## HealthChecks
>
> Enables knowing the status of each components within a running James server.
>
> This is both periodically logged, and exposed via WebAdmin.
>
> Key interface: **HealthCheck**.
>
> ## Event sourcing
>
> Event sourcing implementation for the James server. Enables components
> to rely on event sourcing technics for taking decisions.
>
> Here the definition of event is not the one of the mailbox, we are in a
> different bounded context.
>
> Entity:
>
>   - **Aggregate** represents an entity on which actions are performed
>   - **command** represents actions we wish to perform on this aggregate
>   - **events** represent modifications of the aggregate. The state of the
> aggregate can be computed from its event history.
>   - **CommandHandler** applies commands on aggregates and generates the
> resulting events.
>   - **Subscriber** enables applying actions upon events.
>
> Key interfaces:
>
>   - **EventStore** stores events generated by event sourcing. Implemented
> in Memory and on top of Cassandra.
>
> ## JSON serialization
>
> Generic mechanism for modular JSON serialization of entities.
>
> Sample usages:
>   - Task
>   - Task additional details
>   - Event sourcing events
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: James architecture: core components, sub-components and utility components

[David Leangen <ap...@leangen.net>]

Hi Benoit:

> I gave a shot at defining:
> 
> - Core components, that offers services at heart of James
> - Utility components, that offers services core-components
> implementations can rely on.
> - Mailbox sub-components.


Wow, there is a lot here! It will take me some time to digest, but this is very helpful.

I will use this as one of my main references. My immediate problem is grasping the whole picture at once, but I will continue that discussion here:

  https://www.mail-archive.com/server-dev@james.apache.org/msg65876.html


I first need to get the entire picture a little clearer in my head, then I will come back here for sure. In the meantime, as you were requesting, if others could also chime in and fill in any missing blanks, that would be great.

Thanks again!
=David



---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: James architecture: core components, sub-components and utility components

[Matthieu Baechler <ma...@apache.org>]

I finally read it and it's very good (I have some comments but will
keep them for when this become a documentation as they are really
minor).

Thank you Benoit, I think it will help building a overall picture of
what James is and how it works.

-- Matthieu Baechler

On Mon, 2020-05-25 at 12:00 +0700, Tellier Benoit wrote:
> Hi David,
> 
> I gave a shot at defining:
> 
>  - Core components, that offers services at heart of James
>  - Utility components, that offers services core-components
> implementations can rely on.
>  - Mailbox sub-components.
> 
> Here is the result. I encourage other project members to review it.
> 
>  - Did I forgot some components?
>  - Is the way I split them relevant?
>  - Is the way to describe a component (descriptions, entities, key
> interfaces, plugin) relevant? Would you add/remove something?
>  - Of course all the component level details...
> 
> I hope it helps your documentation effort.
> 
> Best regards,
> 
> Benoit
> 
> -------------------------------------
> 
> # Core components
> 
> ## MailetContainer
> 
> Enables mail processing.
> 
> Entity:
>  - **Mails**: Message transiting between a sender and recipients
> 
> Key interfaces:
>  - **Mailet**: Enables to act upon an email: modify it, trigger side
> effects
>  - **Matcher**: Condition to trigger a mailet action
> 
> Plugin: A user can register their own mailets / matchers
> 
> Here is a list of mailets providing central services, and bridges
> other
> core components defined here after :
> 
>  - **RemoteDelivery** enables sending emails to other mail servers,
> using the SMTP protocol.
>  - **LocalDelivery** enables delivering emails to local user
> mailboxes.
>  - **RecipientRewriteTable** queries and applied recipient rewritting
> rules.
>  - **ToMailRepository** stores emails in a given mail repository.
>  - **Sieve** executes stored sieve scripts against incoming emails.
> 
> ## MailQueue
> 
> Enables asynchronous mail processing. Enables review and
> administration
> of mail traffic awaiting processing.
> 
> Entity: **Mails**
> 
> Key interface: **MailQueue**, **ManageableMailQueue**
> 
> Implementations:
>  - Memory
>  - (embedded) ActiveMQ
>  - Distributed (RabbitMQ + Cassandra)
>  - Files
> 
> ## MailRepositories
> 
> Enables storing emails, along with their processing context. Enables
> traffic review.
> 
> Can be used to recover error, stores spam, etc...
> 
> Entity: **Mails**
> 
> Key interfaces:
>  - **MailRepository**
>  - **MailRepositoryStore**: enables instanciation of MailRepositories
> 
> Plugins:
>  - A user can register new classes of MailRepository, that he can
> then
> configure within it's Mailet Container
> 
> Bundled implementation:
>  - Memory
>  - Cassandra
>  - JDBC
>  - File
> 
> ## Mailbox
> 
> The mailbox stores user messages for later retrieval.
> 
> Entities:
>  - **Messages**: a mime message belonging to a user, along with their
> mailbox context.
>  - **Mailboxes**: a group of messages
> 
> Sample operations:
>  - Mailboxes can be created, deleted, renamed
>  - Messages can be appended, deleted, moved, copied, their flags can
> be
> modified
> 
> Key interfaces:
>  - **MailboxManager**: Enable managing and accessing mailboxes
>  - **MessageManager**: Enables accessing messages within a given
> mailbox. Can be obtained via the MailboxManager.
>  - **MessageIdManager**: Enables accessing messages by their unique
> identifier.. Can be obtained via the MailboxManager.
> 
> Implementations:
>  - Memory implementation
>  - Maildir implementation
>  - JPA implementation
>  - Cassandra implementation
> 
> The mailbox components defines the following sub-components:
> 
> ### The EventBus
> 
> James mailbox uses an event driven architecture.
> 	
> It means every meaningful action on mailboxes or messages triggers an
> event for any component to react to that event.
> `MailboxListener` allows executing actions upon mailbox events. They
> could be used for a wide variety of purposes, like
> enriching mailbox managers features or enabling user notifications
> upon
> concurrent mailboxes operations.
> 
> Entities:
>  - **Events**: describes an action that happenened on a mailbox
>  - **MailboxListeners**: standard API for acting upon events.
> 
> Key interfaces:
>  - **EventBus**: enables MailboxListener registration and enables
> dispatching events to the MailboxListeners
> 
> Implementations:
>  - In VM
>  - RabbitMQ
> 
> Plugins:
>  - A user can register its own mailbox listeners.
> 
> ### EventDeadLetter
> 
> Failed event processing is being saved to the EventDeadLetter for
> both
> diagnostic and reprocessing purposes.
> 
> Entities:
>  - **Events**: As per the EventBus
> 
> Key interfaces:
>  - **EventDeadLetter**
> 
> Implementations:
>  - Cassandra
>  - Memory
> 
> ### Search index
> 
> Sub components allowing searching emails.
> 
> Entities: **Messages**
> 
> Key interfaces:
>  - **MessageSearchIndex**
> 
> Implementations:
>  - Scrolling
>  - Lucene
>  - ElasticSearch
> 
> ### Text extraction
> 
> Allows extracting text from arbtrary files. It empowers a more
> relevant
> search within the MessageSearchIndex
> 
> Key interfaces:
>  - **TextExtractor**
> 
> Implementations:
>  - Default: only extracts plain text
>  - JSoup: only extracts plain text or HTML
>  - Tika: relias on Apache Tika
> 
> ### Quotas
> 
> Defines and enforces limitations on the mailbox resource usage. Keep
> track of current resource usage.
> 
> Entities:
>  - Quota Root: defines a group of mailboxes for which a given quota
> applies. James implementation defines QuotaRoot as the mailboxes
> belonging to a user.
>  - Resources: What resourc ehte quota tracks. Could be the count of
> messages or the total sizes of messages.
>  - Quota: A limit along with current usage for a given Quota Root.
> 
> Key interfaces:
>  - **QuotaManager** allows to define limits and retrieve quota usage.
> 
> ## data/api
> 
> Stores server level metadata.
> 
> Entities:
>  - **Users**: people enables to access the server, along with their
> credentials
>  - **Domains**: Logical group of users
> 
> Key interfaces:
>  - **UsersRepository**
>  - **DomainList**
> 
> Implementation:
>  - Memory
>  - Cassandra
>  - JPA
> 
> ## Recipient Rewrite Tables
> 
> Enables storing rules for rewriting recipient of an email.
> 
> Entities:
>  - **Mapping source**: defines which address should be rewritten for
> a
> given rule
>  - **Mapping**: defines the rewriting that should be performed
> 
> Key interface: **RecipientRewriteTable** stores all the Mappings
> along
> with their sources.
> 
> Implementations:
>  - Memory
>  - Cassandra
>  - JPA
>  - XML (configuration only)
> 
> ## SieveRepository
> 
> Enables users to store Sieve scripts. Enables managing quota applied
> for
> user Sieve scripts.
> 
> Entity:
>  - **Sieve script**: enable a user to define actions to be performed
> upon mail reception.
> 
> Key interfaces:
>  - **SieveRepository**: Leverages sieve script storage
>  - **SieveQuotaRepository**: Leverages sieve script quota storage
> 
> ## DNS Service
> 
> Provides abstraction for DNS resolutions. Helps DNS resolution for
> mail
> processing purposes.
> 
> Key interfaces: **DNSService** enables DNS resolution.
> 
> # Protocols
> 
> ## SMTP
> 
> Implementation of RFC-5321 to enable receiving emails. Mails are
> processed asynchronously.
> 
> A user can register additional **ProtocolHandlers** to extend the
> capabilities of the SMTP server.
> 
> A SMTP **Hook** is a specific Protocol handler being plugged in a
> specific location within the default SMTP server stack.
> 
> ## LMTP
> 
> Implementation of RFC-2033 to enable receiving emails. Similar to
> SMTP
> but mails are processed synchronously.
> 
> A user can register additional **ProtocolHandlers** to extend the
> capabilities of the LMTP server.
> 
> A SMTP **Hook** is a specific Protocol handler being plugged in a
> specific location within the default SMTP server stack.
> 
> ## IMAP
> 
> Implementationof RFC-3501 to enable a user to access and manage their
> mailbox.
> 
> A user as of today can not extend the behaviour of the IMAP stack.
> 
> ## JMAP
> 
> Implementation of RFC-8620 and RFC-8621 to enable a user to interact
> and
> manage their mailboxes, and to send emails, on top of HTTP.
> 
> The implementation of these specifications is currently a work in
> progress. A previous draft implementation is available.
> 
> ## POP3
> 
> Implementation of RFC-1939 to enables users to retrieve the mails
> within
> their mailboxes.
> 
> A user can register additional **ProtocolHandlers** to extend the
> capabilities of the POP3 server.
> 
> ## ManagedSieve
> 
> Implementation of RFC-5804 to enable a user to upload and manage
> their
> Sieve scripts.
> 
> This protocol implementation is known to be buggy.
> 
> # Server Administration
> 
> ## REST administration via WebAdmin
> 
> James specific REST APIs to manage other components, and their
> specific
> implementations.
> 
> WebAdmin routes are defined in a modular way, the webadmin API will
> thus
> be product specific.
> 
> Specification:
> 
> Key interface:
>  - **Routes** enables registering additional REST endpoints.
> 
> Plugin: a user can define custom webadmin routes.
> 
> ## Administration via the CLI
> 
> Command Line interface to manage James.
> 
> This CLI is not product specific, some command will not work for some
> products.
> 
> The CLI relies on JMX protocol and can represent a potential
> vulnerabilities. Guice products enables disabing it. A long term work
> is
> to port the CLI to rely on the WebAdmin REST APIs.
> 
> # Utility components
> 
> These components offers services core-components implementations can
> rely on.
> 
> ## BlobStore
> 
> Stores potentially large binary data.
> 
> Entity:
>  - **Blob**: Binary data
> 
> Key interface: BlobStore
> 
> Implementations:
>  - Memory
>  - ObjectStorage (S3/Swift)
> 
> ## TaskManager
> 
> Allows to control and schedule long running tasks run by other
> components. Among other it enables scheduling, progress monitoring,
> cancelation of long running tasks
> 
> Entity:
>  - **Task**: An operation performed by the TaskManager
>  - **Task additional information**: Task specific information.
> Exposes
> specific details about this task and how its execution went on.
> 
> Key interface: **TaskManager**.
> 
> Implementations:
>  - Memory
>  - Distributed (cassandra + RabbitMQ)
> 
> ## Metrics
> 
> Enables recording execution timing for various operation within
> James.
> Enables fine grained performance monitoring of a running James
> server,
> for example using grafana boards.
> 
> Key interface: **MetricFactory**.
> 
> Implementation:
>  - Default
>  - DropWizard
> 
> ## HealthChecks
> 
> Enables knowing the status of each components within a running James
> server.
> 
> This is both periodically logged, and exposed via WebAdmin.
> 
> Key interface: **HealthCheck**.
> 
> ## Event sourcing
> 
> Event sourcing implementation for the James server. Enables
> components
> to rely on event sourcing technics for taking decisions.
> 
> Here the definition of event is not the one of the mailbox, we are in
> a
> different bounded context.
> 
> Entity:
> 
>  - **Aggregate** represents an entity on which actions are performed
>  - **command** represents actions we wish to perform on this
> aggregate
>  - **events** represent modifications of the aggregate. The state of
> the
> aggregate can be computed from its event history.
>  - **CommandHandler** applies commands on aggregates and generates
> the
> resulting events.
>  - **Subscriber** enables applying actions upon events.
> 
> Key interfaces:
> 
>  - **EventStore** stores events generated by event sourcing.
> Implemented
> in Memory and on top of Cassandra.
> 
> ## JSON serialization
> 
> Generic mechanism for modular JSON serialization of entities.
> 
> Sample usages:
>  - Task
>  - Task additional details
>  - Event sourcing events
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org