You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by David Leangen <ap...@leangen.net> on 2020/05/17 06:01:51 UTC
Questions about Guice design
Hi!
As I just mentioned in a different email:
> As part of my documentation project, I am starting to turn my attention towards the code itself. I am trying to understand how the system works. […] To understand the system better, I need to take a step back and understand Guice. I am learning about Guice now. So far, guice looks quite easy to understand and use (at least for me because as a long-time OSGi user I understand very well the concepts of DI, api/impl separation, services, etc.). I can understand very well the motivation for using a framework like Guice, and I think (hope!) it should help me to understand what is going on in the system.
You’ll have to excuse my inexperience with Guice. However, to understand James, it is imperative (ok, actually “declarative” hahahaha I’m so funny) to understand its organization into Guice Modules. I have not read through all of the Guice documentation, but I think I get the idea.
In this context, I have started looking at the James code base. There are many very nice things about it. Despite its size and complexity, I was able to understand quite a lot thanks to the good organization and naming conventions. That said, I think we could still do a lot better.
Organizing into Guice Modules is very nice. One of the important benefits is that it helps a lot to break the system down into reasonable chunks that are more understandable. It is simply impossible for a normal human to understand the system all at once. It is necessary to have several levels of abstraction. The use of Guice Modules to provide a “just right" level of abstraction (not too high so as to be useless when trying to run it, but not so low as to be much too detailed) is essential. I would wager that in most situations, this is the level of abstraction that is most useful for a developer or system operator (and definitely for an application assembler). The only time a package or class level is necessary is when actually making changes to the code.
I would even double down on my statement and say that the organization of the Guice Modules is perhaps THE most important abstraction available to allow people to understand the system.
Ideally, to help provide a better understanding of the system and its compile-time (and even to some extent its runtime) organization, I think it is important to:
* Match the Guice Modules with the Maven modules, matching them exactly if possible
—> At first glance, this seems pretty good
* Ensure that each Module is well-contained (i.e. no “leaks” or coupling to other implementations)
—> I found this part to be quite problematic (topic for a different email)
* Understand the API surface area of each Module
—> I am having a lot of trouble with this (again, a topic for a different email)
Below is a list of all the Guice Modules I was able to find in the system. I simply did a search and manually extracted these. There are 144!!
Although I could guess a little bit as to what they do, I was otherwise unable to understand many things:
* What is the purpose?
* How does it relate to other Modules?
* Which Modules are implementing the same API, so are “swappable”?
* What is the hierarchy? (Assuming that there are Modules of Modules)?
It is a great start! However, to be able to put more order into all of this, these are my initial thoughts:
1. Document each module individually (even if only a line or two of text)
2. Understand if there is any hierarchy (i.e. Modules of Modules)
—> Refactor / Rename the Modules to make the hierarchy immediately clear
3. Validate that each Guice Module is perfectly aligned with its Maven module
4. With this clearer picture, evaluate whether or not the current project structure is still appropriate
5. Update the documentation of each Maven module to show all of its Guice Modules
With this clearer understanding, it should be much easier to wire together a “product” without having to rely on a “supported product”. I think the system appears to be well-designed. It’s just very difficult to understand right now. If we can make it easier to grok, it will be much easier to use.
Thoughts?
Cheers,
=David
BlobStoreAPIModule
BlobExportMechanismModule
LinshareBlobExportMechanismModule
LocalFileBlobExportMechanismModule
BlobMemoryModule
ObjectStorageBlobStoreModule
ObjectStorageDependenciesModule
MyExtensionModule
CassandraDLPConfigurationStoreModule
CassandraDomainListModule
CassandraJmapModule
CassandraMailRepositoryModule
CassandraRecipientRewriteTableModule
CassandraSieveRepositoryModule
CassandraUsersRepositoryModule
CassandraEventStoreModule
CassandraBlobStoreModule
CassandraCacheSessionModule
CassandraDeadLetterModule
CassandraDeletedMessageVaultModule
CassandraMailboxModule
CassandraQuotaMailingModule
CassandraQuotaModule
CassandraSessionModule
ElasticSearchClientModule
ElasticSearchMailboxModule
ElasticSearchQuotaSearcherModule
TikaMailboxModule
CassandraMetricsModule
CassandraRoutesModule
InconsistencySolvingRoutesModule
SolveMailboxInconsistenciesModules
SolveMessageInconsistenciesModules
TestDockerElasticSearchModule
TestDockerESMetricReporterModule
TestTikaModule
LdapUsersRepositoryModule
BlobStoreChoosingModule
RabbitMQEventBusModule
DistributedTaskManagerModule
TaskSerializationModule
TestAwsS3BlobStoreModule
TestRabbitMQModule
TestSwiftBlobStoreModule
ActiveMQQueueModule
ProtocolHandlerModule
DefaultProcessorsConfigurationProviderModule
DNSServiceModule
DropWizardMetricsModule
HostnameModule
LoggingMetricsModule
MailStoreRepositoryModule
RawPostDequeueDecoratorModule
TaskManagerModule
CleanupTaskModule
ClockModule
CommonServicesModule
IsStartedProbeModule
MailetProcessingModule
MimeMessageModule
PeriodicalHealthChecksModule
StartablesModule
StartUpChecksModule
ElasticSearchMetricReporterModule
IMAPServerModule
JMAPCommonModule
JMAPModule
MethodsModule
JMAPDraftServerModule
TestJMAPServerModule
SearchModule
JMXServerModule
LMTPServerModule
DefaultEventModule
FastRetryBackoffModule
MemoryDeadLetterModule
PreDeletionHookModule
MailboxModule
SpamAssassinListenerModule
CamelMailetContainerModule
DKIMMailetModule
ManageSieveServerModule
SieveModule
NettyServerModule
POP3ServerModule
RabbitMQModule
SieveFileRepositoryModule
SieveJPARepositoryModules
JSPFModule
SMTPServerModule
MyExtensionModule
ExtensionModule
HealthCheckRoutesModule
NoJwtModule
TaskRoutesModule
WebAdminServerModule
DataRoutesModules
DLPRoutesModule
SieveRoutesModule
JmapTasksModule
InconsistencyQuotasSolvingRoutesModule
MailboxesBackupModule
MailboxesExportRoutesModule
MailboxesRoutesModule
MailboxRoutesModule
MessagesRoutesModule
ReIndexingModule
MailQueueRoutesModule
MailRepositoriesRoutesModule
SwaggerRoutesModule
SpamAssassinModule
JPADataModule
JPADomainListModule
JPAEntityManagerModule
JPAMailRepositoryModule
JPARecipientRewriteTableModule
JPAUsersRepositoryModule
TestJPAConfigurationModule
TestJPAConfigurationModuleWithSqlValidation
NoDatabaseAuthentication
WithDatabaseAuthentication
JPAMailboxModule
JpaQuotaModule
JPAQuotaSearchModule
LuceneSearchMailboxModule
TestJPAConfigurationModule
DeletedMessageVaultModule
DeletedMessageVaultRetentionModule
DeletedMessageVaultRoutesModule
TestDeleteMessageVaultPreDeletionHookModule
MemoryDataJmapModule
MemoryDataModule
MemoryEventStoreModule
MemoryMailboxModule
MemoryQuotaModule
MemoryQuotaSearchModule
MemoryMailQueueModule
FakeSearchMailboxModule
LifeCycleModule
MultiLifeCycleTestCase
UnauthorizedModule
WebadminIntegrationTestModule
SpamAssassinModule
TestingSessionModule
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Questions about Guice design
Posted by David Leangen <ap...@leangen.net>.
> A hexagon defines some ports (the component it needs to work), some adapters (the implementations it provides to other hexagons) and a core domain: the logic of this hexagon.
Thank you for elaborating your thoughts. I see much better now what you are trying to do. I like many of the ideas of the hexagonal architecture that you are trying to achieve.
In that case, let’s just focus on the core. The rest is really not that important at this time if the immediate goal is to document the project.
I am a long-time enthusiast of DDD. If the “core” of the hexagonal architecture matches the “core domain” in the DDD sense, it would make the system very easy to understand. The “core” domain ought to be very explicit and should exactly match the domain concepts. When reading the code, it ought to be really obvious what is happening.
My impression is that we agree on pretty much everything except maybe the details about how the core is developed. I don’t think it’s important at this time to go into that, though.
From where I am coming from, in general the core domain should be able to pop out to somebody like me with very little effort. The ports and adapters merely provide access to the core (or support for the core, depending on your perspective), so if the core is clear and well understood, then the rest ought to be quite secondary. When using the system, the core does not change, only the “outside” components that interact with the core change. For this reason, the wiring that happens (for example with Guice Modules) is basically the connection between the outside components to the core via the ports/adapters, but the ports/adapters ought to be obvious if the core communicates its intentions well.
If that describes James, then the path forward is now clear.
> `Core domains` are a bit hidden in the current code
With your help, I will try to figure out what the core is, and the rest should follow quite nicely.
I will start a separate thread relating to the “core”, because this topic has drifted from the original discussion about Guice. Unless you think I am way off track with your thinking, I would suggest that you not reply to this email, but wait for me to start the new thread. :-)
I think it’s good that we got here. Thanks for your patience.
Cheers,
=David
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Questions about Guice design
Posted by ma...@gmail.com.
I realize while looking for resources about hexagonal architecture that
I probably used port/adaptor words differently from what they are
supposed to mean.
I'll ask some friends to correct me on that matter.
On Tue, 2020-05-19 at 20:41 +0900, David Leangen wrote:
> Hi Matthieu,
>
>
[...]
>
> > * `ports` that define some APIs required for `domain cores`
> > (mailbox-
> > api is a port)
>
> Ok, this is interesting because it is exactly the opposite approach
> from what I am used to. :-)
>
> I am used to designing the API first. Since the API is exposed to the
> outside, it usually requires a lot of thought. Then an implementation
> gets built that satisfies the API. Since the implementation is
> “private”, it can be changed or updated whenever necessary (or there
> can even be multiple implementations at once), so it doesn’t require
> quite as much thought as the API. Definitely the API should remain as
> stable as possible, as changing it affects all of its users.
>
> It sounds from your description that the implementation (core) is
> done first, then the API is more of an afterthought.
>
> Or did I misunderstand?
We have the same definition of what an API and an implementation is.
It's the reason why APIs have contract testsuites: they are important
and require stability.
What you miss in the picture, and I guess you should read more about
hexagonal architecture, is that a core domain is a component that is
written in pure Java (no technical dependency) that encapsulate the
logic of a given domain.
The `port` concept is here to say "I will need to store that email
somewhere, so I define an API because I don't care how it works as long
as it works with plain Java objects. Please write an implementation of
that API and give it to me".
> > * `adapters` that provides the implementation of a `port` (mailbox-
> > cassandra)
>
> But if mailbox is a “core” component implementation, that has an
> adapter, and also has multiple other implementations… Sorry, I’m a
> bit confused. Maybe an image would help?
An hexagon defines some ports (the component it needs to work), some
adapters (the implementations it provides to other hexagons) and a core
domain: the logic of this hexagon.
Note that an hexagonal `core domain` can very well be just
`infrastructure` (adaptor) for another hexagon.
It's not layers but a recursive architecture.
And sorry, I can't find illustration about that.
>
> > When we define a `port` API we provide a contract testsuite to
> > check
> > that the implementations will work well plugged-in.
>
> Yeah… like I said, this is backwards from what I am used to. :-)
>
> My fear is: if the API is just an afterthought that is placed over
> the implementation, how do you know if it was well designed?
Because it fits the `core domain`. The `core domains` are what matters
in a software, everything else is just technical details.
>
> > So to come back to the analysis: the right level of abstraction for
> > James architecture is these three entities.
>
> Those sounds like very abstract architectural entities to me,
> something like a “database table” or a “queue” or some other general
> construct. Or am I misunderstanding?
>
> In my mind, if you were to say that they are just “categories” of
> actual James entities, perhaps that would make more sense to me. For
> example: “A Mailbox is a core entity in the James domain, and happens
> to be one of the core entities” makes sense to me, but it is the
> Mailbox that is the entity, not “a core” that is an entity. Does that
> make sense? Or again, am I completely misunderstanding what you are
> trying to say?
>
>
> > Unfortunately, there's not way to visualize the hexagons from the
> > code
> > right now.
>
> :-)
>
> Ok, that should be our objective I guess, then. The code should match
> the domain model, so it should be easy to find our way around in the
> code.
`Core domains` are a bit hidden in the current code (it can be an
abstract class for example) so the easiest way to draw our hexagons is
to review *-api projects: they describe the boundaries between
hexagons.
It could be fun to draw the hexagons for various profiles actually.
>
> > Finally, I see Guice module documentation as a problem: we usually
> > document things that are supposed to last. Documentation reduces
> > the
> > ability to refactor things.
>
> Yes, that makes sense. Really it should be the API that is
> documented, as normally (again from my experience) the API is the
> representation of the domain model in code.
APIs are just boundaries. That means it can be the boundary for the
user facing features but also the boundary between your imap protocol
and your mailbox `core domain`.
>
> > As Guice modules are technical classes, I would not like to be
> > refrained from refactorings because of this documentation.
>
> Completely agree!!
>
>
> > It's why I think we have to find a way to document `ports` and
> > `adapters` because they are definitely more stable.
>
> I agree with this statement, notwithstanding the things above that I
> said I was having trouble understanding.
>
> If you could give me a list of the cores, ports, and adapters, I’ll
> see what I can do with that, and perhaps make progress in my
> understanding of the code base.
>
Let's look at quota-mailing for example.
The `core domain` is located in `quota-mailing`.
It provides an implementation of `GroupMailboxListener`.
It requires an implementation of `EventStore`.
Then you have `quota-mailing-memory` which provides a memory
implementation of `EventStore` for `quota-mailing`.
And `quota-mailing-cassandra` which provides a cassandra implementation
of `EventStore` for `quota-mailing`.
I hope it helps.
Cheers,
--
Matthieu Baechler
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Questions about Guice design
Posted by David Leangen <ap...@leangen.net>.
Hi Matthieu,
>> I would wager that in most situations, this is the level of abstraction that is most useful for a developer or system operator (and definitely for an application assembler). The only time a package or class level is necessary is when actually making changes to the code.
> Let me disagree.
Yes, please. :-)
> Guice is a technology we use but it is not mandatory, we could change
> this part of James tomorrow if we want, it's an implementation concern.
I completely agree with this statement. Just to show to what extent I agree, allow me to even add to your statement.
It could be Spring, OSGi, or whatever. It just happens to be Guice. Guice isn’t even the implementation, it is just the framework code that wraps the implementation, so I would argue that “Guice” is even lower-level and less important than what you are stating. I have no argument at all with this idea.
The only point of Guice, as far as I can tell, is to inject implementations into APIs. If it were in OSGi, it would be the service object. I am sure that the principle is much the same in most frameworks.
The reason I was dwelling on the Guice Components is because I am unable to figure out what the API of the domain model is, and the API interfaces should have a very clear 1:1 correspondence with the James model. (Actually, I usually tend to think that the API **is** the model, expressed in code.)
Since I am having a hard time with understanding the code base, I thought I could use the Guice Components as a proxy for the domain model. My thinking is that in general terms, given a domain concept, there will be one or more interfaces, and for each of these, there will be an implementation. For each of those, it will be wrapped as a Guice Module. So by understanding the Guice Modules, I should be able to work my way back to the domain model and eventually understand what is going on in the code.
So I thought it was a safe assumption to give me at least a starting point as to where to eat this elephant.
But reading below, I see that our ideas of what is API vs. implementation seem to be a little different...
> Basically, you have:
>
> * `domain cores` (say managing emails and mailboxes, managing users)
> that implement the important domain logic
Ok, sounds good. If I could understand better what these “cores” are and where they are represented in the code, that would already give me a start.
> * `ports` that define some APIs required for `domain cores` (mailbox-
> api is a port)
Ok, this is interesting because it is exactly the opposite approach from what I am used to. :-)
I am used to designing the API first. Since the API is exposed to the outside, it usually requires a lot of thought. Then an implementation gets built that satisfies the API. Since the implementation is “private”, it can be changed or updated whenever necessary (or there can even be multiple implementations at once), so it doesn’t require quite as much thought as the API. Definitely the API should remain as stable as possible, as changing it affects all of its users.
It sounds from your description that the implementation (core) is done first, then the API is more of an afterthought.
Or did I misunderstand?
> * `adapters` that provides the implementation of a `port` (mailbox-
> cassandra)
But if mailbox is a “core” component implementation, that has an adapter, and also has multiple other implementations… Sorry, I’m a bit confused. Maybe an image would help?
> When we define a `port` API we provide a contract testsuite to check
> that the implementations will work well plugged-in.
Yeah… like I said, this is backwards from what I am used to. :-)
My fear is: if the API is just an afterthought that is placed over the implementation, how do you know if it was well designed?
> So to come back to the analysis: the right level of abstraction for
> James architecture is these three entities.
Those sounds like very abstract architectural entities to me, something like a “database table” or a “queue” or some other general construct. Or am I misunderstanding?
In my mind, if you were to say that they are just “categories” of actual James entities, perhaps that would make more sense to me. For example: “A Mailbox is a core entity in the James domain, and happens to be one of the core entities” makes sense to me, but it is the Mailbox that is the entity, not “a core” that is an entity. Does that make sense? Or again, am I completely misunderstanding what you are trying to say?
> Unfortunately, there's not way to visualize the hexagons from the code
> right now.
:-)
Ok, that should be our objective I guess, then. The code should match the domain model, so it should be easy to find our way around in the code.
> Finally, I see Guice module documentation as a problem: we usually
> document things that are supposed to last. Documentation reduces the
> ability to refactor things.
Yes, that makes sense. Really it should be the API that is documented, as normally (again from my experience) the API is the representation of the domain model in code.
> As Guice modules are technical classes, I would not like to be
> refrained from refactorings because of this documentation.
Completely agree!!
> It's why I think we have to find a way to document `ports` and
> `adapters` because they are definitely more stable.
I agree with this statement, notwithstanding the things above that I said I was having trouble understanding.
If you could give me a list of the cores, ports, and adapters, I’ll see what I can do with that, and perhaps make progress in my understanding of the code base.
Cheers,
=David
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Questions about Guice design
Posted by Matthieu Baechler <ma...@apache.org>.
Hi David,
On Sun, 2020-05-17 at 15:01 +0900, David Leangen wrote:
> Hi!
>
>
[...]
> Organizing into Guice Modules is very nice. One of the important
> benefits is that it helps a lot to break the system down into
> reasonable chunks that are more understandable. It is simply
> impossible for a normal human to understand the system all at once.
> It is necessary to have several levels of abstraction. The use of
> Guice Modules to provide a “just right" level of abstraction (not too
> high so as to be useless when trying to run it, but not so low as to
> be much too detailed) is essential. I would wager that in most
> situations, this is the level of abstraction that is most useful for
> a developer or system operator (and definitely for an application
> assembler). The only time a package or class level is necessary is
> when actually making changes to the code.
Let me disagree.
Guice is a technology we use but it is not mandatory, we could change
this part of James tomorrow if we want, it's an implementation concern.
What makes James special is the hexagonal(-ish) architecture it uses.
I won't go into details here about why it's not exactly how it should
be but to understand James you have to understand this design.
Basically, you have:
* `domain cores` (say managing emails and mailboxes, managing users)
that implement the important domain logic
* `ports` that define some APIs required for `domain cores` (mailbox-
api is a port)
* `adapters` that provides the implementation of a `port` (mailbox-
cassandra)
When we define a `port` API we provide a contract testsuite to check
that the implementations will work well plugged-in.
So to come back to the analysis: the right level of abstraction for
James architecture is these three entities.
Unfortunately, there's not way to visualize the hexagons from the code
right now.
Finally, I see Guice module documentation as a problem: we usually
document things that are supposed to last. Documentation reduces the
ability to refactor things.
As Guice modules are technical classes, I would not like to be
refrained from refactorings because of this documentation.
It's why I think we have to find a way to document `ports` and
`adapters` because they are definitely more stable.
Cheers,
-- Matthieu Baechler
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Questions about Guice design
Posted by Tellier Benoit <bt...@apache.org>.
On 18/05/2020 16:16, David Leangen wrote:
>
> Thank you for the explanations. I now understand much better the migration from Spring to Guice, and the intent behind Guice.
>
> Instead of directly answering the questions you posed, I will make a proposal that I think goes in the same direction as your comments.
>
>> This documentation effort would be a great move toward James as a
>> toolkit to write your own email server. We of course had it in mind
>> during Guice adoption, but not yet had feedback on the topic.
>
> Although quite tedious, I propose as a first step that I simply ask questions about each of the 144 Guice Modules in the system. I will take notes in the form of Javadoc in order to keep the information together with the code. Just knowing what they are intended for will be a good start. Once each of them is documented, I think it should be easier to analyze what to do with it.
>
> I am sure that in some cases, maybe even many, I should be able to guess about the purpose of the Module. So to save time, I will try, as much as possible, to propose an explanation and ask for feedback.
>
> When I don’t understand, I’ll have to ask for help.
>
> In any case, this tedious project should be a big step forward, but it depends on whether or not the developers here are willing / able to help. :-)
>
>
> Please let me know if I should or should not continue in this direction.
Please continue. That is an awesome proposition!
I am available to answer Guice related question and promote James usage
as a toolkit.
Cheers,
Benoit
>
>
> Cheers,
> =David
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Questions about Guice design
Posted by David Leangen <ap...@leangen.net>.
Thank you for the explanations. I now understand much better the migration from Spring to Guice, and the intent behind Guice.
Instead of directly answering the questions you posed, I will make a proposal that I think goes in the same direction as your comments.
> This documentation effort would be a great move toward James as a
> toolkit to write your own email server. We of course had it in mind
> during Guice adoption, but not yet had feedback on the topic.
Although quite tedious, I propose as a first step that I simply ask questions about each of the 144 Guice Modules in the system. I will take notes in the form of Javadoc in order to keep the information together with the code. Just knowing what they are intended for will be a good start. Once each of them is documented, I think it should be easier to analyze what to do with it.
I am sure that in some cases, maybe even many, I should be able to guess about the purpose of the Module. So to save time, I will try, as much as possible, to propose an explanation and ask for feedback.
When I don’t understand, I’ll have to ask for help.
In any case, this tedious project should be a big step forward, but it depends on whether or not the developers here are willing / able to help. :-)
Please let me know if I should or should not continue in this direction.
Cheers,
=David
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Questions about Guice design
Posted by Tellier Benoit <bt...@apache.org>.
Hi David!
A pretty long and interesting email!
On 17/05/2020 13:01, David Leangen wrote:
>
> Hi!
>
> As I just mentioned in a different email:
>
>> As part of my documentation project, I am starting to turn my attention towards the code itself. I am trying to understand how the system works. […] To understand the system better, I need to take a step back and understand Guice. I am learning about Guice now. So far, guice looks quite easy to understand and use (at least for me because as a long-time OSGi user I understand very well the concepts of DI, api/impl separation, services, etc.). I can understand very well the motivation for using a framework like Guice, and I think (hope!) it should help me to understand what is going on in the system.
>
> You’ll have to excuse my inexperience with Guice. However, to understand James, it is imperative (ok, actually “declarative” hahahaha I’m so funny) to understand its organization into Guice Modules. I have not read through all of the Guice documentation, but I think I get the idea.
>
> In this context, I have started looking at the James code base. There are many very nice things about it. Despite its size and complexity, I was able to understand quite a lot thanks to the good organization and naming conventions. That said, I think we could still do a lot better.
>
> Organizing into Guice Modules is very nice. One of the important benefits is that it helps a lot to break the system down into reasonable chunks that are more understandable. It is simply impossible for a normal human to understand the system all at once. It is necessary to have several levels of abstraction. The use of Guice Modules to provide a “just right" level of abstraction (not too high so as to be useless when trying to run it, but not so low as to be much too detailed) is essential. I would wager that in most situations, this is the level of abstraction that is most useful for a developer or system operator (and definitely for an application assembler). The only time a package or class level is necessary is when actually making changes to the code.
>
> I would even double down on my statement and say that the organization of the Guice Modules is perhaps THE most important abstraction available to allow people to understand the system.
I had a write at "why we chose Guice" and your statements perfectly
complete it, I will add it in
https://github.com/apache/james-project/blob/master/src/adr/0036-against-use-of-conditional-statements-in-guice-modules.md
>
> Ideally, to help provide a better understanding of the system and its compile-time (and even to some extent its runtime) organization, I think it is important to:
>
> * Match the Guice Modules with the Maven modules, matching them exactly if possible
> —> At first glance, this seems pretty good
That is what we tried to do, however we also wanted to limit the
cardinality of maven modules, and split guice-maven-modules when it was
used by different profiles.
>
> * Ensure that each Module is well-contained (i.e. no “leaks” or coupling to other implementations)
> —> I found this part to be quite problematic (topic for a different email)
>
> * Understand the API surface area of each Module
> —> I am having a lot of trouble with this (again, a topic for a different email)
>
>
> Below is a list of all the Guice Modules I was able to find in the system. I simply did a search and manually extracted these. There are 144!!
>
> Although I could guess a little bit as to what they do, I was otherwise unable to understand many things:
>
> * What is the purpose?
I don't understand this question.
If you take into account the cardinality (profile x components) and the
fact that profiles don't enable the same administration features (that
thus needs to be split), a high cardinality is in my opinion understandable.
> * How does it relate to other Modules?
> * Which Modules are implementing the same API, so are “swappable”?
> * What is the hierarchy? (Assuming that there are Modules of Modules)?
Modules can "install" other modules, we use that to better split the
logic, for instance for the JMAP protocol.
>
>
> It is a great start! However, to be able to put more order into all of this, these are my initial thoughts:
>
> 1. Document each module individually (even if only a line or two of text)
> 2. Understand if there is any hierarchy (i.e. Modules of Modules)
> —> Refactor / Rename the Modules to make the hierarchy immediately clear
> 3. Validate that each Guice Module is perfectly aligned with its Maven module
> 4. With this clearer picture, evaluate whether or not the current project structure is still appropriate
> 5. Update the documentation of each Maven module to show all of its Guice Modules
This documentation effort would be a great move toward James as a
toolkit to write your own email server. We of course had it in mind
during Guice adoption, but not yet had feedback on the topic.
I am not aware of community usage of James Guice modules to build their
own mail server.
Such a documentation effort could of course help!
* Maybe marker interfaces to 'tag' modules as 'ComponentModule' or
'ProtocolModule' could help the module discovery? That would allow to
quickly identify top-level modules.
Also, some JavaDoc in the modules themselves would of course help. We
have for the mailet some documentation page generated from the javadoc,
writing a documentation page for guice modules based on this model can
be doable.
>
> With this clearer understanding, it should be much easier to wire together a “product” without having to rely on a “supported product”. I think the system appears to be well-designed. It’s just very difficult to understand right now. If we can make it easier to grok, it will be much easier to use.
For sure!
Until now we had only main developers contributing to the part of the
code and cruelly lacked feedback.
Benoit
>
>
> Thoughts?
>
> Cheers,
> =David
>
>
> BlobStoreAPIModule
> BlobExportMechanismModule
> LinshareBlobExportMechanismModule
> LocalFileBlobExportMechanismModule
> BlobMemoryModule
> ObjectStorageBlobStoreModule
> ObjectStorageDependenciesModule
> MyExtensionModule
> CassandraDLPConfigurationStoreModule
> CassandraDomainListModule
> CassandraJmapModule
> CassandraMailRepositoryModule
> CassandraRecipientRewriteTableModule
> CassandraSieveRepositoryModule
> CassandraUsersRepositoryModule
> CassandraEventStoreModule
> CassandraBlobStoreModule
> CassandraCacheSessionModule
> CassandraDeadLetterModule
> CassandraDeletedMessageVaultModule
> CassandraMailboxModule
> CassandraQuotaMailingModule
> CassandraQuotaModule
> CassandraSessionModule
> ElasticSearchClientModule
> ElasticSearchMailboxModule
> ElasticSearchQuotaSearcherModule
> TikaMailboxModule
> CassandraMetricsModule
> CassandraRoutesModule
> InconsistencySolvingRoutesModule
> SolveMailboxInconsistenciesModules
> SolveMessageInconsistenciesModules
> TestDockerElasticSearchModule
> TestDockerESMetricReporterModule
> TestTikaModule
> LdapUsersRepositoryModule
> BlobStoreChoosingModule
> RabbitMQEventBusModule
> DistributedTaskManagerModule
> TaskSerializationModule
> TestAwsS3BlobStoreModule
> TestRabbitMQModule
> TestSwiftBlobStoreModule
> ActiveMQQueueModule
> ProtocolHandlerModule
> DefaultProcessorsConfigurationProviderModule
> DNSServiceModule
> DropWizardMetricsModule
> HostnameModule
> LoggingMetricsModule
> MailStoreRepositoryModule
> RawPostDequeueDecoratorModule
> TaskManagerModule
> CleanupTaskModule
> ClockModule
> CommonServicesModule
> IsStartedProbeModule
> MailetProcessingModule
> MimeMessageModule
> PeriodicalHealthChecksModule
> StartablesModule
> StartUpChecksModule
> ElasticSearchMetricReporterModule
> IMAPServerModule
> JMAPCommonModule
> JMAPModule
> MethodsModule
> JMAPDraftServerModule
> TestJMAPServerModule
> SearchModule
> JMXServerModule
> LMTPServerModule
> DefaultEventModule
> FastRetryBackoffModule
> MemoryDeadLetterModule
> PreDeletionHookModule
> MailboxModule
> SpamAssassinListenerModule
> CamelMailetContainerModule
> DKIMMailetModule
> ManageSieveServerModule
> SieveModule
> NettyServerModule
> POP3ServerModule
> RabbitMQModule
> SieveFileRepositoryModule
> SieveJPARepositoryModules
> JSPFModule
> SMTPServerModule
> MyExtensionModule
> ExtensionModule
> HealthCheckRoutesModule
> NoJwtModule
> TaskRoutesModule
> WebAdminServerModule
> DataRoutesModules
> DLPRoutesModule
> SieveRoutesModule
> JmapTasksModule
> InconsistencyQuotasSolvingRoutesModule
> MailboxesBackupModule
> MailboxesExportRoutesModule
> MailboxesRoutesModule
> MailboxRoutesModule
> MessagesRoutesModule
> ReIndexingModule
> MailQueueRoutesModule
> MailRepositoriesRoutesModule
> SwaggerRoutesModule
> SpamAssassinModule
> JPADataModule
> JPADomainListModule
> JPAEntityManagerModule
> JPAMailRepositoryModule
> JPARecipientRewriteTableModule
> JPAUsersRepositoryModule
> TestJPAConfigurationModule
> TestJPAConfigurationModuleWithSqlValidation
> NoDatabaseAuthentication
> WithDatabaseAuthentication
> JPAMailboxModule
> JpaQuotaModule
> JPAQuotaSearchModule
> LuceneSearchMailboxModule
> TestJPAConfigurationModule
> DeletedMessageVaultModule
> DeletedMessageVaultRetentionModule
> DeletedMessageVaultRoutesModule
> TestDeleteMessageVaultPreDeletionHookModule
> MemoryDataJmapModule
> MemoryDataModule
> MemoryEventStoreModule
> MemoryMailboxModule
> MemoryQuotaModule
> MemoryQuotaSearchModule
> MemoryMailQueueModule
> FakeSearchMailboxModule
> LifeCycleModule
> MultiLifeCycleTestCase
> UnauthorizedModule
> WebadminIntegrationTestModule
> SpamAssassinModule
> TestingSessionModule
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org