You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by "Otto, Karsten Andreas" <Ka...@akquinet.de.INVALID> on 2021/11/02 16:41:56 UTC
Akquinet contributions to James
Dear James Community,
Over the last two years, we at Akquinet have developed an email solution
for the medical and healthcare sector. We chose Apache James for this
project because it provides
- a robust clustering solution out of the box,
- a comprehensive WebAdmin REST interface for integration with other
product components,
- a flexible Mailet architecture easily adapting to our specific
requirements,
- an open source solution with an active community and commercial
support where needed.
Our deployment uses the distributed-pop3 app variant, with multiple
James server instances running on top of a Cassandra cluster,
S3-compatible blob storage and RabbitMQ. The choice of POP3 may seem
strange, but our customers typically employ third party systems for
semi-automatic mail processing. For this use case, POP3 enables much
simpler systems integration than the more complex protocols.
Despite Apache James being a very flexible solution, we encountered a
few situations during development where we had to change the original
codebase in order to meet our requirements. We believe these changes
might well be of interest to the James community at large, and in the
spirit of open source we would like to share them with you!
Over the next few weeks we plan to create Jira tickes and pull requests
for the features below. Let us know what you think, and which you would
like to see first!
(The list includes a complexity estimate in T-Shirt size, where S is
just a few / localized changes and L is a lot of code / all over the place.)
SECURITY ENHANCEMENTS
# TLS authentication via client certificate (M)
Add options to network server configurations to set certificate-based
authentication mode (none, want, need), and the associated trust store
to validate these client certificates. Useful to limit server access to
trusted partners/users.
# Separate trust store for S3 (M)
Extend blob store configuration to specify a trust store, which is used
to validate the S3 server certificate. Useful if also using TLS client
cert authentication (see above) to keep the security realms separate.
# Delay on authentication failure (S)
Add an option verifyFailureDelay to usersrepository.xml to delay the
response if someone tries to authenticate with a non-existing user or
wrong password. Basic protection against people using James as a
password oracle for brute-force/dictionary attacks.
# Support password salting (M) *
Add extra hashingMode choices in usersrepository.xml ("salted",
"legacy_salted") to include the user name in the password hash. Basic
protection against rainbow table cracking if someone ever manages to
steal the password database.
# Gradual migration of password hash settings (L) *
Add a hashingMode column to the user table, use it together with the
algorithm column to verify password hashes. But use the configured
algorithm and hashingMode from usersrepository.xml when updating the
password. This way, the user database can gradually migrate to a
different (hopefully stronger) security setting. Useful to get rid of
the legacy hashing mode, and to introduce password salting (see above).
*NOTE: Currently only works with Cassandra/Memory, but should be
possible for JPA as well if someone more experienced with this server
variant would like to tackle it.
POP3 ENHANCEMENTS
# Fix WebAdmin routes for POP3 (S) - already submitted
Include WebAdmin routes and example configuration for managing the
DeletedMessagesVault in the distributed-pop3 server variant.
# Add Glowroot instrumentation for POP3 (S)
Minor refactoring of POP3 command handlers, and matching glowroot
configurations. Better diagnostics on what is going on in POP3 handling.
# Configure restore location for deleted messages (S)
Add an option to specify which mailbox receives any restored messages
from the DeletedMessagesVault, e.g. "INBOX" instead of the default
"Restored-Messages". Required since POP3 cannot access other mailboxes
than INBOX.
GENERAL FEATURES
# Extra system properties (S)
Read a properties file immediately at server startup and set them as
system properties. Useful to keep command line clutter down and manageable.
# Check user credentials via WebAdmin (M)
Provide a WebAdmin route to check a username/password combination.
Reports 204 on success and 401 on failure. Useful for integrating James
with 3rd party services, e.g. a web admin GUI for users.
# Generalized vacation handling, including via WebAdmin (L)
Refactor the vacation handling code out of JMAP space and make it a
general James feature. Add a WebAdmin route to query and change a user's
vacation notice. Useful for integrating James with 3rd party services,
e.g. a web admin GUI.
Regards,
Karsten Otto
--
akquinet tech@spree GmbH
Bülowstraße 66 • D-10783 Berlin
Tel: +49 30 235520-0
Fax: +49 30 217520-12
E-Mail: karsten.otto@akquinet.de
Web: www.akquinet.de
Geschäftsführung: Martin Weber, Dr. Torsten Fink, Heinz Wilming
Amtsgericht Berlin HRB 86780 • USt.-Id. Nr.: DE 225 964 680
Re: Akquinet contributions to James
Posted by "Otto, Karsten Andreas" <Ka...@akquinet.de.INVALID>.
On 03.11.21 04:45, btellier@apache.org wrote:
> Hello Otto,
>
It is actually Karsten :-)
> This sounds very promising! Thanks for sharing your feedback as well as
> these enhancements.
>
> I will give my thoughts on each one of these propositions, inlined in
> the email.
>
> Best regards,
>
> Benoit TELLIER
>
> On 02/11/2021 23:41, Otto, Karsten Andreas wrote:
>> Dear James Community,
>>
>> Over the last two years, we at Akquinet have developed an email solution
>> for the medical and healthcare sector. We chose Apache James for this
>> project because it provides
>> - a robust clustering solution out of the box,
>> - a comprehensive WebAdmin REST interface for integration with other
>> product components,
>> - a flexible Mailet architecture easily adapting to our specific
>> requirements,
>> - an open source solution with an active community and commercial
>> support where needed.
>>
>> Our deployment uses the distributed-pop3 app variant, with multiple
>> James server instances running on top of a Cassandra cluster,
>> S3-compatible blob storage and RabbitMQ. The choice of POP3 may seem
>> strange, but our customers typically employ third party systems for
>> semi-automatic mail processing. For this use case, POP3 enables much
>> simpler systems integration than the more complex protocols.
>>
>> Despite Apache James being a very flexible solution, we encountered a
>> few situations during development where we had to change the original
>> codebase in order to meet our requirements. We believe these changes
>> might well be of interest to the James community at large, and in the
>> spirit of open source we would like to share them with you!
>>
>> Over the next few weeks we plan to create Jira tickes and pull requests
>> for the features below. Let us know what you think, and which you would
>> like to see first!
>>
>> (The list includes a complexity estimate in T-Shirt size, where S is
>> just a few / localized changes and L is a lot of code / all over the place.)
>>
>>
>> SECURITY ENHANCEMENTS
>>
>> # TLS authentication via client certificate (M)
>>
>> Add options to network server configurations to set certificate-based
>> authentication mode (none, want, need), and the associated trust store
>> to validate these client certificates. Useful to limit server access to
>> trusted partners/users.
> +1
>
> Here I guess we are speaking of additional options to
> AbstractConfigurableAsyncServer class family within
> server/protocols/protocols-library maven module. Correct?
>
Yes, excactly.
> The hardest part is likely (unit) testing of the developed code. Not
> sure what is the best strategy regarding this...
Same thought here, and there was no test for this class before. I went
with manual testing for now, but if anybody has suggestions for proper
unit testing, I'd appreciate them.
>> # Separate trust store for S3 (M)
>>
>> Extend blob store configuration to specify a trust store, which is used
>> to validate the S3 server certificate. Useful if also using TLS client
>> cert authentication (see above) to keep the security realms separate.
> +1
>
> Just like one can pass certs / truststore for Cassandra and
> ElasticSearch one should be able to tune TLS for S3.
>>
>> # Delay on authentication failure (S)
>>
>> Add an option verifyFailureDelay to usersrepository.xml to delay the
>> response if someone tries to authenticate with a non-existing user or
>> wrong password. Basic protection against people using James as a
>> password oracle for brute-force/dictionary attacks.
> +1
>
> On HTTP I would have been saying "just use an API gateway ;-)" but no
> such things exists for IMAP/SMTP / TCP protocols. Thus I think your
> proposal is beneficial.
Indeed, we needed a central point for enforcing this, regardless of
which protocol triggered the authentication.
>>
>> # Support password salting (M) *
> +1
>
> Historically I have only be seriously using the LDAP repository and the
> default (eg Cassandra) implementation is trivially exposed to hasing
> table dictionnary attacks.
>
> Salting would prevent this. This SHOULD become the default ;-)
>
> Also we need to take care of this for the JPA implementation that
> follows a slightly different code path.
I saw that during development, but since we do not use JPA there was no
time to implement this yet, unfortunately. If necessary I can take
another crack at it.
>>
>> Add extra hashingMode choices in usersrepository.xml ("salted",
>> "legacy_salted") to include the user name in the password hash. Basic
>> protection against rainbow table cracking if someone ever manages to
>> steal the password database.
> As a side note:
> - We are writing something new, thus I do not think adding a
> legacy_salted new hashing mode makes any sense as we don't want to
> maintain something that nobody is using...
> - I would be happy to specify the hash algorithm used in combination
> with the salt. Eg salted_sha512, salted_sha256...
>
Salting is implemented as an extra accessor in Algorithm, isSalted(),
next to isLegacy(). So legacy_salted mode is kind of an artifact of that.
I believe Akquinet is the only current user of legacy_salted, as we
introduced the salting feature *before* we discovered we were using
legacy mode. That said, it is certainly discouraged for new users, and
maybe should just skip it in the documentation.
> Maybe we do not need to implement a generic salted_{hashalg}, but I
> would prefer to include the hashing algorithm in the name of this new
> option (so salted => salted_sha512)
>
In the current Cassandra implementation there is an extra column
"hashingmode" next to "algorithm". It is a very simple migration this
way, just need to add the column once (documented in
upgrade-instructions.md), and configure the default if it is not
set/null. I am not opposed to combining the two columns, but that would
require a more complex schema migration implementation.
>>
>> # Gradual migration of password hash settings (L) *
> +1
>
> The simplest way to achieve this is likely to specify the hash algorithm
> along side the new password in User::setPassword. This should then make
> it easy to implement.
We extended the DefaultUser constructor so that it takes two Algorithm
parameters: The current algorithm from the user database, used in
verifyPassword(), and the preferred algorithm from the configuration,
used in setPassword(). This way we avoided changing the parameters of
these methods, which would have had a much wider impact on the code base.
>>
>> Add a hashingMode column to the user table, use it together with the
>> algorithm column to verify password hashes. But use the configured
>> algorithm and hashingMode from usersrepository.xml when updating the
>> password. This way, the user database can gradually migrate to a
>> different (hopefully stronger) security setting. Useful to get rid of
>> the legacy hashing mode, and to introduce password salting (see above).
>>
>> *NOTE: Currently only works with Cassandra/Memory, but should be
>> possible for JPA as well if someone more experienced with this server
>> variant would like to tackle it.
>>
>>
>> POP3 ENHANCEMENTS
>>
>> # Fix WebAdmin routes for POP3 (S) - already submitted
>>
>> Include WebAdmin routes and example configuration for managing the
>> DeletedMessagesVault in the distributed-pop3 server variant.
>>
>> # Add Glowroot instrumentation for POP3 (S)
>
> +1
>
> Glowroot is useful for debugging, investigating slow calls,
> understanding Cassandra query flow.
>
>
> Bare in mind that Glowroot have a ~25% CPU costs, especially visible on
> Cassandra intensive mail requests.
>>
>> Minor refactoring of POP3 command handlers, and matching glowroot
>> configurations. Better diagnostics on what is going on in POP3 handling.
>>
>> # Configure restore location for deleted messages (S)
> +0.5 I guess it do not hurt
>
> (and I'm happy to see the deleted message vault is used :-) )
>>
>> Add an option to specify which mailbox receives any restored messages
>> from the DeletedMessagesVault, e.g. "INBOX" instead of the default
>> "Restored-Messages". Required since POP3 cannot access other mailboxes
>> than INBOX.
It does not hurt in the general case, but is required to make POP3 work
with DMV. And obviously, "Restored-Messages" is the default if there is
no configuration given.
>>
>>
>> GENERAL FEATURES
>>
>> # Extra system properties (S)
>>
>> Read a properties file immediately at server startup and set them as
>> system properties. Useful to keep command line clutter down and manageable.
> +1
>
> We had a recurring debate with @mbaechler on the use of system
> properties within James.
>
> On the one side there is the ease of use (no need to read a
> configuration file, transform it in a POJO, and rely on abstract
> factories etc to customise stuff).
>
> On the other hand there are concerns around the discoverability of such
> properties.
>
> CF https://issues.apache.org/jira/browse/JAMES-3487
>
> Maybe having this file would be a good place to document all those
> properties (and make the use of System properties an implementation detail).
>
> Would this suite you @mbaechler?
>
> @Otto maybe we could have a list of the system properties you did find
> useful? I think the community have limited knowledge regarding system
> property usage with James... At least me...
>
We mostly use this for TLS tweaks, in particular javax.net.ssl.keyStore
and friends to be used by Java Mail API. This is the client side for
mutually authenticated TLS connections, and worked thios way out of the
box. It might make sense to move this to a more suitable James
configuration file, but I am not sure where this would have to go.
>>
>> # Check user credentials via WebAdmin (M)
>>
>> Provide a WebAdmin route to check a username/password combination.
>> Reports 204 on success and 401 on failure. Useful for integrating James
>> with 3rd party services, e.g. a web admin GUI for users.
> +1
>>
>> # Generalized vacation handling, including via WebAdmin (L)
>>
>> Refactor the vacation handling code out of JMAP space and make it a
>> general James feature. Add a WebAdmin route to query and change a user's
>> vacation notice. Useful for integrating James with 3rd party services,
>> e.g. a web admin GUI.
> +0.1
>
> I do not have oppositions regarding this though we need to keep in mind
> Vacations are tightly coupled to the JMAP spec
> (https://jmap.io/spec-mail.html#vacation-response)
>
> I understand working with a nice JSON format is easier than aweful Sieve
> scripts, I understand using vacations pulls in the all pack of JMAP
> dependencies, thus I agree with such a refactoring.
>
>>
>>
>> Regards,
>> Karsten Otto
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
Re: Akquinet contributions to James
Posted by "btellier@apache.org" <bt...@apache.org>.
Hello Otto,
This sounds very promising! Thanks for sharing your feedback as well as
these enhancements.
I will give my thoughts on each one of these propositions, inlined in
the email.
Best regards,
Benoit TELLIER
On 02/11/2021 23:41, Otto, Karsten Andreas wrote:
> Dear James Community,
>
> Over the last two years, we at Akquinet have developed an email solution
> for the medical and healthcare sector. We chose Apache James for this
> project because it provides
> - a robust clustering solution out of the box,
> - a comprehensive WebAdmin REST interface for integration with other
> product components,
> - a flexible Mailet architecture easily adapting to our specific
> requirements,
> - an open source solution with an active community and commercial
> support where needed.
>
> Our deployment uses the distributed-pop3 app variant, with multiple
> James server instances running on top of a Cassandra cluster,
> S3-compatible blob storage and RabbitMQ. The choice of POP3 may seem
> strange, but our customers typically employ third party systems for
> semi-automatic mail processing. For this use case, POP3 enables much
> simpler systems integration than the more complex protocols.
>
> Despite Apache James being a very flexible solution, we encountered a
> few situations during development where we had to change the original
> codebase in order to meet our requirements. We believe these changes
> might well be of interest to the James community at large, and in the
> spirit of open source we would like to share them with you!
>
> Over the next few weeks we plan to create Jira tickes and pull requests
> for the features below. Let us know what you think, and which you would
> like to see first!
>
> (The list includes a complexity estimate in T-Shirt size, where S is
> just a few / localized changes and L is a lot of code / all over the place.)
>
>
> SECURITY ENHANCEMENTS
>
> # TLS authentication via client certificate (M)
>
> Add options to network server configurations to set certificate-based
> authentication mode (none, want, need), and the associated trust store
> to validate these client certificates. Useful to limit server access to
> trusted partners/users.
+1
Here I guess we are speaking of additional options to
AbstractConfigurableAsyncServer class family within
server/protocols/protocols-library maven module. Correct?
The hardest part is likely (unit) testing of the developed code. Not
sure what is the best strategy regarding this...
> # Separate trust store for S3 (M)
>
> Extend blob store configuration to specify a trust store, which is used
> to validate the S3 server certificate. Useful if also using TLS client
> cert authentication (see above) to keep the security realms separate.
+1
Just like one can pass certs / truststore for Cassandra and
ElasticSearch one should be able to tune TLS for S3.
>
> # Delay on authentication failure (S)
>
> Add an option verifyFailureDelay to usersrepository.xml to delay the
> response if someone tries to authenticate with a non-existing user or
> wrong password. Basic protection against people using James as a
> password oracle for brute-force/dictionary attacks.
+1
On HTTP I would have been saying "just use an API gateway ;-)" but no
such things exists for IMAP/SMTP / TCP protocols. Thus I think your
proposal is beneficial.
>
> # Support password salting (M) *
+1
Historically I have only be seriously using the LDAP repository and the
default (eg Cassandra) implementation is trivially exposed to hasing
table dictionnary attacks.
Salting would prevent this. This SHOULD become the default ;-)
Also we need to take care of this for the JPA implementation that
follows a slightly different code path.
>
> Add extra hashingMode choices in usersrepository.xml ("salted",
> "legacy_salted") to include the user name in the password hash. Basic
> protection against rainbow table cracking if someone ever manages to
> steal the password database.
As a side note:
- We are writing something new, thus I do not think adding a
legacy_salted new hashing mode makes any sense as we don't want to
maintain something that nobody is using...
- I would be happy to specify the hash algorithm used in combination
with the salt. Eg salted_sha512, salted_sha256...
Maybe we do not need to implement a generic salted_{hashalg}, but I
would prefer to include the hashing algorithm in the name of this new
option (so salted => salted_sha512)
>
> # Gradual migration of password hash settings (L) *
+1
The simplest way to achieve this is likely to specify the hash algorithm
along side the new password in User::setPassword. This should then make
it easy to implement.
>
> Add a hashingMode column to the user table, use it together with the
> algorithm column to verify password hashes. But use the configured
> algorithm and hashingMode from usersrepository.xml when updating the
> password. This way, the user database can gradually migrate to a
> different (hopefully stronger) security setting. Useful to get rid of
> the legacy hashing mode, and to introduce password salting (see above).
>
> *NOTE: Currently only works with Cassandra/Memory, but should be
> possible for JPA as well if someone more experienced with this server
> variant would like to tackle it.
>
>
> POP3 ENHANCEMENTS
>
> # Fix WebAdmin routes for POP3 (S) - already submitted
>
> Include WebAdmin routes and example configuration for managing the
> DeletedMessagesVault in the distributed-pop3 server variant.
>
> # Add Glowroot instrumentation for POP3 (S)
+1
Glowroot is useful for debugging, investigating slow calls,
understanding Cassandra query flow.
Bare in mind that Glowroot have a ~25% CPU costs, especially visible on
Cassandra intensive mail requests.
>
> Minor refactoring of POP3 command handlers, and matching glowroot
> configurations. Better diagnostics on what is going on in POP3 handling.
>
> # Configure restore location for deleted messages (S)
+0.5 I guess it do not hurt
(and I'm happy to see the deleted message vault is used :-) )
>
> Add an option to specify which mailbox receives any restored messages
> from the DeletedMessagesVault, e.g. "INBOX" instead of the default
> "Restored-Messages". Required since POP3 cannot access other mailboxes
> than INBOX.
>
>
> GENERAL FEATURES
>
> # Extra system properties (S)
>
> Read a properties file immediately at server startup and set them as
> system properties. Useful to keep command line clutter down and manageable.
+1
We had a recurring debate with @mbaechler on the use of system
properties within James.
On the one side there is the ease of use (no need to read a
configuration file, transform it in a POJO, and rely on abstract
factories etc to customise stuff).
On the other hand there are concerns around the discoverability of such
properties.
CF https://issues.apache.org/jira/browse/JAMES-3487
Maybe having this file would be a good place to document all those
properties (and make the use of System properties an implementation detail).
Would this suite you @mbaechler?
@Otto maybe we could have a list of the system properties you did find
useful? I think the community have limited knowledge regarding system
property usage with James... At least me...
>
> # Check user credentials via WebAdmin (M)
>
> Provide a WebAdmin route to check a username/password combination.
> Reports 204 on success and 401 on failure. Useful for integrating James
> with 3rd party services, e.g. a web admin GUI for users.
+1
>
> # Generalized vacation handling, including via WebAdmin (L)
>
> Refactor the vacation handling code out of JMAP space and make it a
> general James feature. Add a WebAdmin route to query and change a user's
> vacation notice. Useful for integrating James with 3rd party services,
> e.g. a web admin GUI.
+0.1
I do not have oppositions regarding this though we need to keep in mind
Vacations are tightly coupled to the JMAP spec
(https://jmap.io/spec-mail.html#vacation-response)
I understand working with a nice JSON format is easier than aweful Sieve
scripts, I understand using vacations pulls in the all pack of JMAP
dependencies, thus I agree with such a refactoring.
>
>
> Regards,
> Karsten Otto
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Akquinet contributions to James
Posted by "Otto, Karsten Andreas" <Ka...@akquinet.de.INVALID>.
On 03.11.21 05:59, btellier@linagora.com wrote:
> Hello Tung,
>
> Thanks for the feedback.
>
> On 03/11/2021 11:24, Tung Tran Van wrote:
>> Hello Otto,
>> Very interesting,
>>
>>> Delay on authentication failure (S)
>> I will follow how it will be implemented. Right now, I think about Redis
>> with expire time key-value, with key is the fingerprint of the client.
>>
> I think what Otto proposes is "if the authentication fails then just
> Thead.sleep(1000)" which is a simple business rule mitigating
> disctionary attacks using James as an Oracle.
>
This is exactly what we implemented, with a configurable delay. The
current default is 0 delay, to be compatible to the previous behavior.
>
> To implement it you do not need any kind of synchronisations. So it is
> easy to implement.
>
> Of course this is not as efficient as (say) fail2ban [1] (tool scrapping
> the logs and shadowing ips at the firewall level in case of too many bad
> logins...) but of course such a set up would need to be distributed (and
> things would start being complicated with some state synchronization eg
> through REDIS). Fail2ban stuff is inefficient against a bot farm where
> the pool of ips used allows to bypass the checks....
>
> [1] https://www.fail2ban.org/wiki/index.php/Main_Page
>
> My take is that we could implement the easy thing first :-)
>
Exactly our reasoning. This is not intended as a replacement for more
advanced intrusion/abuse detection systems. But anything is better than
nothing :-)
>>> Check user credentials via WebAdmin (M)
>> What is the key difference between webAdmin endpoint and /jmap/session
>> endpoint?
>
> 1. Not every admin deploys JMAP
>
> 2. Discoverability: given that I'm an admin I am likely familiar with
> WebAdmin /users endpoints familly. To think of using JMAP session (basic
> auth) to check a password is complex, non straightforward. Smells like a
> roundabout way to do things ;-)
>
We use this WebAdmin endpoint to authenticate users that log in to our
Web GUI, so we can re-use the James user database. I believe this might
make sense for other deployments too if they do not wish to use LDAP.
>>
>> Regards,
>> Tung, Tran Van
>>
>> On Tue, Nov 2, 2021 at 11:42 PM Otto, Karsten Andreas
>> <Ka...@akquinet.de.invalid> wrote:
>>
>>> Dear James Community,
>>>
>>> Over the last two years, we at Akquinet have developed an email solution
>>> for the medical and healthcare sector. We chose Apache James for this
>>> project because it provides
>>> - a robust clustering solution out of the box,
>>> - a comprehensive WebAdmin REST interface for integration with other
>>> product components,
>>> - a flexible Mailet architecture easily adapting to our specific
>>> requirements,
>>> - an open source solution with an active community and commercial
>>> support where needed.
>>>
>>> Our deployment uses the distributed-pop3 app variant, with multiple
>>> James server instances running on top of a Cassandra cluster,
>>> S3-compatible blob storage and RabbitMQ. The choice of POP3 may seem
>>> strange, but our customers typically employ third party systems for
>>> semi-automatic mail processing. For this use case, POP3 enables much
>>> simpler systems integration than the more complex protocols.
>>>
>>> Despite Apache James being a very flexible solution, we encountered a
>>> few situations during development where we had to change the original
>>> codebase in order to meet our requirements. We believe these changes
>>> might well be of interest to the James community at large, and in the
>>> spirit of open source we would like to share them with you!
>>>
>>> Over the next few weeks we plan to create Jira tickes and pull requests
>>> for the features below. Let us know what you think, and which you would
>>> like to see first!
>>>
>>> (The list includes a complexity estimate in T-Shirt size, where S is
>>> just a few / localized changes and L is a lot of code / all over the
>>> place.)
>>>
>>>
>>> SECURITY ENHANCEMENTS
>>>
>>> # TLS authentication via client certificate (M)
>>>
>>> Add options to network server configurations to set certificate-based
>>> authentication mode (none, want, need), and the associated trust store
>>> to validate these client certificates. Useful to limit server access to
>>> trusted partners/users.
>>>
>>> # Separate trust store for S3 (M)
>>>
>>> Extend blob store configuration to specify a trust store, which is used
>>> to validate the S3 server certificate. Useful if also using TLS client
>>> cert authentication (see above) to keep the security realms separate.
>>>
>>> # Delay on authentication failure (S)
>>>
>>> Add an option verifyFailureDelay to usersrepository.xml to delay the
>>> response if someone tries to authenticate with a non-existing user or
>>> wrong password. Basic protection against people using James as a
>>> password oracle for brute-force/dictionary attacks.
>>>
>>> # Support password salting (M) *
>>>
>>> Add extra hashingMode choices in usersrepository.xml ("salted",
>>> "legacy_salted") to include the user name in the password hash. Basic
>>> protection against rainbow table cracking if someone ever manages to
>>> steal the password database.
>>>
>>> # Gradual migration of password hash settings (L) *
>>>
>>> Add a hashingMode column to the user table, use it together with the
>>> algorithm column to verify password hashes. But use the configured
>>> algorithm and hashingMode from usersrepository.xml when updating the
>>> password. This way, the user database can gradually migrate to a
>>> different (hopefully stronger) security setting. Useful to get rid of
>>> the legacy hashing mode, and to introduce password salting (see above).
>>>
>>> *NOTE: Currently only works with Cassandra/Memory, but should be
>>> possible for JPA as well if someone more experienced with this server
>>> variant would like to tackle it.
>>>
>>>
>>> POP3 ENHANCEMENTS
>>>
>>> # Fix WebAdmin routes for POP3 (S) - already submitted
>>>
>>> Include WebAdmin routes and example configuration for managing the
>>> DeletedMessagesVault in the distributed-pop3 server variant.
>>>
>>> # Add Glowroot instrumentation for POP3 (S)
>>>
>>> Minor refactoring of POP3 command handlers, and matching glowroot
>>> configurations. Better diagnostics on what is going on in POP3 handling.
>>>
>>> # Configure restore location for deleted messages (S)
>>>
>>> Add an option to specify which mailbox receives any restored messages
>>> from the DeletedMessagesVault, e.g. "INBOX" instead of the default
>>> "Restored-Messages". Required since POP3 cannot access other mailboxes
>>> than INBOX.
>>>
>>>
>>> GENERAL FEATURES
>>>
>>> # Extra system properties (S)
>>>
>>> Read a properties file immediately at server startup and set them as
>>> system properties. Useful to keep command line clutter down and manageable.
>>>
>>> # Check user credentials via WebAdmin (M)
>>>
>>> Provide a WebAdmin route to check a username/password combination.
>>> Reports 204 on success and 401 on failure. Useful for integrating James
>>> with 3rd party services, e.g. a web admin GUI for users.
>>>
>>> # Generalized vacation handling, including via WebAdmin (L)
>>>
>>> Refactor the vacation handling code out of JMAP space and make it a
>>> general James feature. Add a WebAdmin route to query and change a user's
>>> vacation notice. Useful for integrating James with 3rd party services,
>>> e.g. a web admin GUI.
>>>
>>>
>>> Regards,
>>> Karsten Otto
>>>
>>> --
>>>
>>> akquinet tech@spree GmbH
>>> Bülowstraße 66 • D-10783 Berlin
>>> Tel: +49 30 235520-0
>>> Fax: +49 30 217520-12
>>>
>>> E-Mail: karsten.otto@akquinet.de
>>> Web: www.akquinet.de
>>>
>>> Geschäftsführung: Martin Weber, Dr. Torsten Fink, Heinz Wilming
>>> Amtsgericht Berlin HRB 86780 • USt.-Id. Nr.: DE 225 964 680
>>>
>>>
>>>
>
Re: Akquinet contributions to James
Posted by "btellier@linagora.com" <bt...@linagora.com>.
Hello Tung,
Thanks for the feedback.
On 03/11/2021 11:24, Tung Tran Van wrote:
> Hello Otto,
> Very interesting,
>
>> Delay on authentication failure (S)
> I will follow how it will be implemented. Right now, I think about Redis
> with expire time key-value, with key is the fingerprint of the client.
>
I think what Otto proposes is "if the authentication fails then just
Thead.sleep(1000)" which is a simple business rule mitigating
disctionary attacks using James as an Oracle.
To implement it you do not need any kind of synchronisations. So it is
easy to implement.
Of course this is not as efficient as (say) fail2ban [1] (tool scrapping
the logs and shadowing ips at the firewall level in case of too many bad
logins...) but of course such a set up would need to be distributed (and
things would start being complicated with some state synchronization eg
through REDIS). Fail2ban stuff is inefficient against a bot farm where
the pool of ips used allows to bypass the checks....
[1] https://www.fail2ban.org/wiki/index.php/Main_Page
My take is that we could implement the easy thing first :-)
>> Check user credentials via WebAdmin (M)
> What is the key difference between webAdmin endpoint and /jmap/session
> endpoint?
1. Not every admin deploys JMAP
2. Discoverability: given that I'm an admin I am likely familiar with
WebAdmin /users endpoints familly. To think of using JMAP session (basic
auth) to check a password is complex, non straightforward. Smells like a
roundabout way to do things ;-)
>
> Regards,
> Tung, Tran Van
>
> On Tue, Nov 2, 2021 at 11:42 PM Otto, Karsten Andreas
> <Ka...@akquinet.de.invalid> wrote:
>
>> Dear James Community,
>>
>> Over the last two years, we at Akquinet have developed an email solution
>> for the medical and healthcare sector. We chose Apache James for this
>> project because it provides
>> - a robust clustering solution out of the box,
>> - a comprehensive WebAdmin REST interface for integration with other
>> product components,
>> - a flexible Mailet architecture easily adapting to our specific
>> requirements,
>> - an open source solution with an active community and commercial
>> support where needed.
>>
>> Our deployment uses the distributed-pop3 app variant, with multiple
>> James server instances running on top of a Cassandra cluster,
>> S3-compatible blob storage and RabbitMQ. The choice of POP3 may seem
>> strange, but our customers typically employ third party systems for
>> semi-automatic mail processing. For this use case, POP3 enables much
>> simpler systems integration than the more complex protocols.
>>
>> Despite Apache James being a very flexible solution, we encountered a
>> few situations during development where we had to change the original
>> codebase in order to meet our requirements. We believe these changes
>> might well be of interest to the James community at large, and in the
>> spirit of open source we would like to share them with you!
>>
>> Over the next few weeks we plan to create Jira tickes and pull requests
>> for the features below. Let us know what you think, and which you would
>> like to see first!
>>
>> (The list includes a complexity estimate in T-Shirt size, where S is
>> just a few / localized changes and L is a lot of code / all over the
>> place.)
>>
>>
>> SECURITY ENHANCEMENTS
>>
>> # TLS authentication via client certificate (M)
>>
>> Add options to network server configurations to set certificate-based
>> authentication mode (none, want, need), and the associated trust store
>> to validate these client certificates. Useful to limit server access to
>> trusted partners/users.
>>
>> # Separate trust store for S3 (M)
>>
>> Extend blob store configuration to specify a trust store, which is used
>> to validate the S3 server certificate. Useful if also using TLS client
>> cert authentication (see above) to keep the security realms separate.
>>
>> # Delay on authentication failure (S)
>>
>> Add an option verifyFailureDelay to usersrepository.xml to delay the
>> response if someone tries to authenticate with a non-existing user or
>> wrong password. Basic protection against people using James as a
>> password oracle for brute-force/dictionary attacks.
>>
>> # Support password salting (M) *
>>
>> Add extra hashingMode choices in usersrepository.xml ("salted",
>> "legacy_salted") to include the user name in the password hash. Basic
>> protection against rainbow table cracking if someone ever manages to
>> steal the password database.
>>
>> # Gradual migration of password hash settings (L) *
>>
>> Add a hashingMode column to the user table, use it together with the
>> algorithm column to verify password hashes. But use the configured
>> algorithm and hashingMode from usersrepository.xml when updating the
>> password. This way, the user database can gradually migrate to a
>> different (hopefully stronger) security setting. Useful to get rid of
>> the legacy hashing mode, and to introduce password salting (see above).
>>
>> *NOTE: Currently only works with Cassandra/Memory, but should be
>> possible for JPA as well if someone more experienced with this server
>> variant would like to tackle it.
>>
>>
>> POP3 ENHANCEMENTS
>>
>> # Fix WebAdmin routes for POP3 (S) - already submitted
>>
>> Include WebAdmin routes and example configuration for managing the
>> DeletedMessagesVault in the distributed-pop3 server variant.
>>
>> # Add Glowroot instrumentation for POP3 (S)
>>
>> Minor refactoring of POP3 command handlers, and matching glowroot
>> configurations. Better diagnostics on what is going on in POP3 handling.
>>
>> # Configure restore location for deleted messages (S)
>>
>> Add an option to specify which mailbox receives any restored messages
>> from the DeletedMessagesVault, e.g. "INBOX" instead of the default
>> "Restored-Messages". Required since POP3 cannot access other mailboxes
>> than INBOX.
>>
>>
>> GENERAL FEATURES
>>
>> # Extra system properties (S)
>>
>> Read a properties file immediately at server startup and set them as
>> system properties. Useful to keep command line clutter down and manageable.
>>
>> # Check user credentials via WebAdmin (M)
>>
>> Provide a WebAdmin route to check a username/password combination.
>> Reports 204 on success and 401 on failure. Useful for integrating James
>> with 3rd party services, e.g. a web admin GUI for users.
>>
>> # Generalized vacation handling, including via WebAdmin (L)
>>
>> Refactor the vacation handling code out of JMAP space and make it a
>> general James feature. Add a WebAdmin route to query and change a user's
>> vacation notice. Useful for integrating James with 3rd party services,
>> e.g. a web admin GUI.
>>
>>
>> Regards,
>> Karsten Otto
>>
>> --
>>
>> akquinet tech@spree GmbH
>> Bülowstraße 66 • D-10783 Berlin
>> Tel: +49 30 235520-0
>> Fax: +49 30 217520-12
>>
>> E-Mail: karsten.otto@akquinet.de
>> Web: www.akquinet.de
>>
>> Geschäftsführung: Martin Weber, Dr. Torsten Fink, Heinz Wilming
>> Amtsgericht Berlin HRB 86780 • USt.-Id. Nr.: DE 225 964 680
>>
>>
>>
Re: Akquinet contributions to James
Posted by Tung Tran Van <tu...@gmail.com>.
Hello Otto,
Very interesting,
> Delay on authentication failure (S)
I will follow how it will be implemented. Right now, I think about Redis
with expire time key-value, with key is the fingerprint of the client.
> Check user credentials via WebAdmin (M)
What is the key difference between webAdmin endpoint and /jmap/session
endpoint?
Regards,
Tung, Tran Van
On Tue, Nov 2, 2021 at 11:42 PM Otto, Karsten Andreas
<Ka...@akquinet.de.invalid> wrote:
> Dear James Community,
>
> Over the last two years, we at Akquinet have developed an email solution
> for the medical and healthcare sector. We chose Apache James for this
> project because it provides
> - a robust clustering solution out of the box,
> - a comprehensive WebAdmin REST interface for integration with other
> product components,
> - a flexible Mailet architecture easily adapting to our specific
> requirements,
> - an open source solution with an active community and commercial
> support where needed.
>
> Our deployment uses the distributed-pop3 app variant, with multiple
> James server instances running on top of a Cassandra cluster,
> S3-compatible blob storage and RabbitMQ. The choice of POP3 may seem
> strange, but our customers typically employ third party systems for
> semi-automatic mail processing. For this use case, POP3 enables much
> simpler systems integration than the more complex protocols.
>
> Despite Apache James being a very flexible solution, we encountered a
> few situations during development where we had to change the original
> codebase in order to meet our requirements. We believe these changes
> might well be of interest to the James community at large, and in the
> spirit of open source we would like to share them with you!
>
> Over the next few weeks we plan to create Jira tickes and pull requests
> for the features below. Let us know what you think, and which you would
> like to see first!
>
> (The list includes a complexity estimate in T-Shirt size, where S is
> just a few / localized changes and L is a lot of code / all over the
> place.)
>
>
> SECURITY ENHANCEMENTS
>
> # TLS authentication via client certificate (M)
>
> Add options to network server configurations to set certificate-based
> authentication mode (none, want, need), and the associated trust store
> to validate these client certificates. Useful to limit server access to
> trusted partners/users.
>
> # Separate trust store for S3 (M)
>
> Extend blob store configuration to specify a trust store, which is used
> to validate the S3 server certificate. Useful if also using TLS client
> cert authentication (see above) to keep the security realms separate.
>
> # Delay on authentication failure (S)
>
> Add an option verifyFailureDelay to usersrepository.xml to delay the
> response if someone tries to authenticate with a non-existing user or
> wrong password. Basic protection against people using James as a
> password oracle for brute-force/dictionary attacks.
>
> # Support password salting (M) *
>
> Add extra hashingMode choices in usersrepository.xml ("salted",
> "legacy_salted") to include the user name in the password hash. Basic
> protection against rainbow table cracking if someone ever manages to
> steal the password database.
>
> # Gradual migration of password hash settings (L) *
>
> Add a hashingMode column to the user table, use it together with the
> algorithm column to verify password hashes. But use the configured
> algorithm and hashingMode from usersrepository.xml when updating the
> password. This way, the user database can gradually migrate to a
> different (hopefully stronger) security setting. Useful to get rid of
> the legacy hashing mode, and to introduce password salting (see above).
>
> *NOTE: Currently only works with Cassandra/Memory, but should be
> possible for JPA as well if someone more experienced with this server
> variant would like to tackle it.
>
>
> POP3 ENHANCEMENTS
>
> # Fix WebAdmin routes for POP3 (S) - already submitted
>
> Include WebAdmin routes and example configuration for managing the
> DeletedMessagesVault in the distributed-pop3 server variant.
>
> # Add Glowroot instrumentation for POP3 (S)
>
> Minor refactoring of POP3 command handlers, and matching glowroot
> configurations. Better diagnostics on what is going on in POP3 handling.
>
> # Configure restore location for deleted messages (S)
>
> Add an option to specify which mailbox receives any restored messages
> from the DeletedMessagesVault, e.g. "INBOX" instead of the default
> "Restored-Messages". Required since POP3 cannot access other mailboxes
> than INBOX.
>
>
> GENERAL FEATURES
>
> # Extra system properties (S)
>
> Read a properties file immediately at server startup and set them as
> system properties. Useful to keep command line clutter down and manageable.
>
> # Check user credentials via WebAdmin (M)
>
> Provide a WebAdmin route to check a username/password combination.
> Reports 204 on success and 401 on failure. Useful for integrating James
> with 3rd party services, e.g. a web admin GUI for users.
>
> # Generalized vacation handling, including via WebAdmin (L)
>
> Refactor the vacation handling code out of JMAP space and make it a
> general James feature. Add a WebAdmin route to query and change a user's
> vacation notice. Useful for integrating James with 3rd party services,
> e.g. a web admin GUI.
>
>
> Regards,
> Karsten Otto
>
> --
>
> akquinet tech@spree GmbH
> Bülowstraße 66 • D-10783 Berlin
> Tel: +49 30 235520-0
> Fax: +49 30 217520-12
>
> E-Mail: karsten.otto@akquinet.de
> Web: www.akquinet.de
>
> Geschäftsführung: Martin Weber, Dr. Torsten Fink, Heinz Wilming
> Amtsgericht Berlin HRB 86780 • USt.-Id. Nr.: DE 225 964 680
>
>
>
--
Tung, Tran Van
*Phone:* (+84) 35 757 6258
*Skype:* tung.tv202