[Discussion] Road to 4.0

[David Leangen <ap...@leangen.net>]

Hello!

I wanted to tie together a few ongoing threads and make a proposal for the road to a 4.0 release. My assumption is that there currently is not roadmap to 4.0. If I am mistaken, please do let me know and I will adapt.

I think by now it is clear that, although a great project with TONS of great work put into it by some very dedicated people over several years, over time James has drifted away from having a user focus. (I’ll say “user” here because it is more natural, but I really mean “Operator” as we defined recently). What this means is:

 * It is difficult for new users to understand how James works
 * It is difficult to get James working
 * It is not easy to understand how to configure James
     (or even to understand what all the different configurations are)

I don’t think this was ever intended, but it just kinda happened over time. I think it is understandable, as James is a very complex project. Maintaining a working system of this scale by a small team of volunteers is not at all an easy task. I am not at all trying to suggest that this is a “bad” thing in any way, so please don’t get me wrong.

I get the feeling that there has been a lot of support in the community to shift towards making James more user-friendly, and that it shouldn’t require a huge effort because there are so many good things already baked into the project. The current direction seems to be:

 * Redo the documentation (already underway, though taking more time than expected)
 * Have a “Basic Server” entry-level offering that is ideally very easy to install and operate
 * Have an “Extensible Server” offering that shows where James really shines (i.e. its extensibility)
     —> Include some kind of “build project” that helps operators build the James they want
  * Have a “Distributed Server” offering for more heavy-duty environments

(There are a few other servers as well, but maybe they are more development-oriented??)

My intention in this thread is to set a very clear objective for a 4.0 release. I would like to propose that for the 4.0 release, we start to make the objectives more user-oriented. I would like to propose that as part of setting our objectives for 4.0, we adhere to the following:

 * Set clear metrics to determine if the objectives are met or not
 * Make the metrics user-oriented

As part of this release, I would like to resolve how the community sees its role. In the discussion we had last time, it seemed to be very inward focused (what do I get out of this; how do I ensure that I don’t feel stuck doing something I don’t want to do) instead of being outward focused (what does the user get, how should the user expect to benefit). I would argue that any user-oriented documentation should be for the user’s benefit, and ought to be user-focused. I think we need a clear resolution, and I think it is related to the 4.0 objective.

By the way, to coincide with the release, if the objectives are clear, perhaps there is a commercial organization (or individual) who would be willing to provide paid-for professional support starting from 4.0? I think it would help complete the offering, and hopefully provide a commercial opportunity as well in a way that is beneficial to all, including the James community. We just need to ensure that the “contract” is very clear, and that we avoid any potential conflicts of interest. I think we should include this item in the scope of the discussion as well. (Just a thought, but maybe the “Distributed James” could be a commercial offering, rather than a community offering??)


To resolve this thread, I will be satisfied once we have a clear statement of objectives regarding the 4.0 release.

Cheers,
=David



---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[Tellier Benoit <bt...@apache.org>]

Le 29/09/2020 à 17:47, Matthieu Baechler a écrit :
> On Tue, 2020-09-29 at 08:59 +0700, Tellier Benoit wrote:
> 
> [...]
> 
>> Spring deprecation could be seen as this big event for most users ?
> 
> You are not very good at public relation, do you? (:
> I don't see feature deprecation as a good opportunity to increment
> major version number.

With Spring deprecation comes Guice applications.

Today they are not even available as a download option, so nobody uses them.

(this should be fixed by the way for the next minor release)

I believe, with all the changes and features that comes with Guice
application, a major release could be justified.

Of course we should not communicate in a "stop spring" way as I did it.

Also I found this link helpfull (and I should try applying it more to my
writings here): https://infra.apache.org/contrib-email-tips.html

" Try not to use the word "you". People get defensive when they think
that comments are directed at them personally. Consider using "one" or
"we" or even "me". "

> 
>>> 4. About defining a vision
>>>
>>> [...]
>>>
>>> What I would really like is to break things! Let's remove all
>>> these anachronic modules, or even better: let's build James 4 by
>>> adopting only well selected modules, ones that are here for a purpose.
>> I do think that such an effort will end up with similar effects for the
>> community than the 3.0 release effort.
> 
> I agree to some extent.
> 
>> What I am looking for is a scalable, modern mail server, we mostly have
>> that already (after 5 years of development which is already a huge
>> commitment).
> 
> I know that it's Linagora intent. However, you are still slow to reach
> that goal partly because of James codebase size. Also, you will keep
> having a clutered product that most people won't see as a "scalable,
> modern mail server".

We agree on the symptoms.

The problem here is that "Linagora" problem about code base size becomes
a community problem.

>> [...]
> 
> How it is different from what we did for years? Did it solve velocity
> issues? Is the project fun now?
> 
> I may start this 3.99 thing on my fork to see how it goes and will keep
> you posted about that.

We deprecated unmaintained outdated components with no active contributors.

Maybe what we should do is discuss what we want to reach as a project.

As a project (as I understood it) we want to answer our end user needs
(self hosting, the distributed server, easy to extend behavior). We
spoke a lot about it lately.

That being said maybe our goal as a project is not to offer a fully
featured email server that we can plug directly into in a collaborative
platform. (our goal at Linagora is to do just that, but our goal as an
Apache project is to *enable* people doing that).

But our goal is more to enable people to do that.

I believe that, going down that road, and removing things, even
maintained, well-written, that ended up in the Apache James code base by
accident (with the classic deprecation / removal procedure)

Third parties, including Linagora, will of course be free to backport them.

I believe it will:
 - enhance the governance of the project
 - simplify the code base
 - force us to have well working extension systems

However (relative) API stability will be needed.

But this could be done "reasonably quickly" with the 3.x code base.

This is what I meant, and as far as I know we never done that.

Sorry for explaining myself elusively.

> 
> Cheers,
> 
> -- Matthieu Baechler

As a conclusion, I'm sure there is a way to arrange the way Linagora
contributes to the Apache James project so that I feel for confident
endorsing both hats at the same time.

It's a long lasting problem, but trying to solve it will in my opinion
solve some of the problems we encounter as a community.

Cheers,

Benoit

> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[Matthieu Baechler <ma...@apache.org>]

On Tue, 2020-09-29 at 08:59 +0700, Tellier Benoit wrote:

[...]

> Spring deprecation could be seen as this big event for most users ?

You are not very good at public relation, do you? (:
I don't see feature deprecation as a good opportunity to increment
major version number.

> > 4. About defining a vision
> > 
> > [...]
> > 
> > What I would really like is to break things! Let's remove all
> > these anachronic modules, or even better: let's build James 4 by
> > adopting only well selected modules, ones that are here for a purpose.
> I do think that such an effort will end up with similar effects for the
> community than the 3.0 release effort.

I agree to some extent.

> What I am looking for is a scalable, modern mail server, we mostly have
> that already (after 5 years of development which is already a huge
> commitment).

I know that it's Linagora intent. However, you are still slow to reach
that goal partly because of James codebase size. Also, you will keep
having a clutered product that most people won't see as a "scalable,
modern mail server".

> Note that I am also convinced that we need better documentation, and
> that we need to simplify the project structure.
> 
> With my Linagora hat on now, I see no way to convince my management to
> dedicate any effort toward a completely reworked 4.0 with a several
> years ETA.

I think you disagreement is mainly about what you assume 4.0 would be.
Let's dig this topic. A plan could be:

1. create a 3.99 branch which would be a new James-from-scratch project
2. define progressive goals to implement
3. progressively import modules to 3.99 following defined goals,
breaking as many things as we want to simplify the project
4. release 3.99.x versions along the way with no guarantee about API
and operation stability
5. as 3.99 reach a product we like and want to support, switch from
3.99 to 4.0

3.0 was stuck in alpha/beta stage for years. People started using it by
forking it or using a fixed snapshot. Given the work left to release
the 3.0, nobody care enough because they had a working project locally.

By the way, 3.0 wanted to keep API stability on some topics while at
the same time never provided any decent upgrade path.

Now our components are basically working: we don't have to write much
things to build this 3.99, we have to combine things and get rid of the
weight preventing us to have a leaner product.

If it takes years, it's because there's no workforce on it, not because
it's a huge work.

The good thing about this strategy is: we can go ahead with 3.x branch
at the same time and can drop this 3.99 strategy at any point if we
conclude it's a dead end.

What Linagora wants or not is not relevant in this case.

> > People could either jump to this fresh version of James or keep
> > maintaining the 3.x branch. If they lack some modules that were not
> > selected for James 4, they could just port these modules to the new
> > APIs.
> > 
> > By doing such a move, we could focus to finally solve our longstanding
> > problems like: developer experience, newcomers welcoming, having a
> > decent and up-to-date documentation, very easy first deployment, etc.
> > 
> > What would you think of such a move ?
> I would prefer a more pragmatic alternative.
> 
> We as a community could be identifying features / modules that should
> not have made it in the 3.x release line. We could decide to deprecate
> then remove these modules, hopefully letting time for third party to
> backport alternatives.

How it is different from what we did for years? Did it solve velocity
issues? Is the project fun now?

I may start this 3.99 thing on my fork to see how it goes and will keep
you posted about that.

Cheers,

-- Matthieu Baechler


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[Tellier Benoit <bt...@apache.org>]

A big thread, let's jump in.

Le 28/09/2020 à 02:34, Matthieu Baechler a écrit :
> Hi,
>
> I'm not sure which message to answer in the thread so I start a new
> thread to summarize my thoughts on the various topics discussed.
>
> 1. About Roadmap
>
> [...]
>
> 2. About documentation
>
>    [...]
>
>    TL;DR we have to explain how it already works
+1
>
> 3. About versioning (3.x vs 4.0)
>
>    Like roadmaps, major version numbers give people expectations: there
>    must be something very new and/or very different because the
>    community decided to increment major version number.
>
>    Last time the community tried that (3.0) the projects almost died
>    because too many things had to ship at the same time and then was
>    never ready. It took years to finally release it after Linagora
>    started to invest a lot it and by the way, we never finished what
>    was supposed to, we just decided that, no software being perfect, we
>    had to release this much-better-than-2.x version.
>
>    I would like to take the Linux Kernel path: 
>
>    * only increment minor version for the time being
>    * don't build a backlog or any list of things we want to achieve
>    before incrementing
>    * release 2 to 4 times a years with what is ready
>    * increment major version when what will be ready deserves it or
>    when minor number get to big
Spring deprecation could be seen as this big event for most users ?
> 4. About defining a vision
>
> [...]
>
> What I would really like is to break things! Let's remove all
> these anachronic modules, or even better: let's build James 4 by
> adopting only well selected modules, ones that are here for a purpose.
I do think that such an effort will end up with similar effects for the
community than the 3.0 release effort.

What I am looking for is a scalable, modern mail server, we mostly have
that already (after 5 years of development which is already a huge
commitment).

Note that I am also convinced that we need better documentation, and
that we need to simplify the project structure.

With my Linagora hat on now, I see no way to convince my management to
dedicate any effort toward a completely reworked 4.0 with a several
years ETA.

>
> People could either jump to this fresh version of James or keep
> maintaining the 3.x branch. If they lack some modules that were not
> selected for James 4, they could just port these modules to the new
> APIs.
>
> By doing such a move, we could focus to finally solve our longstanding
> problems like: developer experience, newcomers welcoming, having a
> decent and up-to-date documentation, very easy first deployment, etc.
>
> What would you think of such a move ?
I would prefer a more pragmatic alternative.

We as a community could be identifying features / modules that should
not have made it in the 3.x release line. We could decide to deprecate
then remove these modules, hopefully letting time for third party to
backport alternatives.

As such we would end up with a much simpler code base.

The downside is that core API stability would be (to some extent) needed.
>
> Cheers,
>
> -- Matthieu Baechler
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[Matthieu Baechler <ma...@apache.org>]

Hi,

I will only answer some parts of your email, see below.

On Mon, 2020-09-28 at 11:33 +0900, David Leangen wrote:

[...]

> 

> 
> > This legacy is putting us in a very difficult situation: the
> > codebase
> > is huge, the test suite takes ages to execute, a lot of things are
> > here
> > for historical reasons but as we are careful about not breaking too
> > much people deployments we don't remove enough things.
> 
> Yes, it is a difficult situation, but I think we can overcome this.
> The first step is recognizing the problems, which I think you have
> quite well.
> 

We know that for years. We tried the "work hard" strategy for now and
it allowed us to deliver things. But It didn't allowed to make the
project appealing enough to new users.

> 
> > In short: we are not able to move fast, to simplify the codebase,
> > to
> > implement new things easily and finally it's hard to have fun for
> > newcomers or even James veterans.
> 
> I agree. I was initially excited about using James, but I realized
> along the way that it would be a long-term commitment. I was not able
> to accomplish what I wanted because I was not able to use it, and I
> was also unable to understand it enough to make changes to be able to
> use it. So I was not able to complete my project with any success. I
> have changed tactic to try to accomplish this over the long term
> instead. I hope to continue to invest maybe an hour or so each day,
> but that is easier said than done.

You personal experience with James is what most people experience. In
my opinion there are two scenarios with James:

* people who achieve to make it work for their purpose and move to the
next thing they want to do, meaning little involvment on the project (I
don't intend to be negative, it's ok to do that in my opinion)

* people who want to make the project better and never reach their
expectations

James is thus a heap of unfinished initiatives, that's why we need to
cut some parts.

[...]

> > 3. About versioning (3.x vs 4.0)
> > 
> >   Like roadmaps, major version numbers give people expectations:
> > there
> >   must be something very new and/or very different because the
> >   community decided to increment major version number.
> 
> Yes, you are absolutely right that a version number sets
> expectations. And so it should. That is actually kind of the purpose.
> 
> If possible, we should consider moving away from soft requirements
> and use semantic versioning. I think that the concept of semantic
> versioning is specified well enough to be considered as being quite
> objective. Hopefully that should avoid this type of debate.

I like semantic versioning, for libraries. I don't find that very
relevant for end-user software.

> 
> >   Last time the community tried that (3.0) the projects almost died
> >   because too many things had to ship at the same time and then was
> >   never ready. It took years to finally release it after Linagora
> >   started to invest a lot it and by the way, we never finished what
> >   was supposed to, we just decided that, no software being perfect,
> > we
> >   had to release this much-better-than-2.x version.
> 
> Oh, that is interesting. It is always good to learn from experience.
> 
> 
> >   I would like to take the Linux Kernel path: 
> > 
> >   * only increment minor version for the time being
> >   * don't build a backlog or any list of things we want to achieve
> >   before incrementing
> 
> Can you explain this point more?

What I mean is: ship what's ready, not what is planned. That way, we
don't delay releases.

> 
> >   * release 2 to 4 times a years with what is ready
> >   * increment major version when what will be ready deserves it or
> >   when minor number get to big
> 
> This seems a bit arbitrary. Maybe we could instead figure out what
> semantic versioning means to James, so we can have a more objective
> criterion?

As said previously, semantic versioning is very good for libraries but
not much for end-user software: every single minor release to date
contained at least one breaking change so far. Is a database schema
migration a breaking change for example?

I prefer arbitrary community decision rather than loosing energy
figuring out a rule for that.

> > 4. About defining a vision
> > 
> >   You defined different kinds of software project styles and it's
> >   interesting to try to define where James is.
> > 
> >   Let me define how I see it.
> > 
> >   The community and the project doesn't have a vision yet. The
> >   community is small, mostly composed of people using James in
> >   enterprise context or paid to work on it.
> > 
> >   Apache Foundation projects should not be led by a single company
> > so
> >   Linagora is not taking the lead on this side.
> 
> Why do you state this? Is there some rule? Or you are worried about
> appearance?

It's a rule as far as I know: projects have to be neutral enough (not
tied to a single vendor).

> There is indeed a really heavy Linagora presence here. I think it’s
> great, and I really appreciate your caution. 

Just to be clear, I'm not working on Linagora's projects anymore. I'm
here as a individual contributor.

> Without yet agreeing to this point, I don’t mind acting like a
> moderator to capture the vision. As a moderator I can stamp my name
> on it to assure the community that it is not too biased in favor of a
> single company.
> 
> 
> >   It means to me that the best we can do as contributors is to
> >   contribute to the project in a way that is useful to us and try
> > to
> >   welcome newcomers as well as we can.
> > 
> >   If the project is valuable to us, it will eventually be valuable
> > to
> >   others. We have to stay commited to it for the time being,
> > continue
> >   as we did for the past years.
> > 
> >   I can't see any vision that the community would be able to commit
> >   to, so let's say it's just a project that wait for its hour of
> >   glory.
> 
> 😀
> 
> Well, I think we’ll have to be a bit more proactive, but let’s
> continue this discussion to see where it takes us. I think from what
> we experienced over the past few months that for a slowly evolving
> project like this, the Documentation actually seems to be a great way
> to capture all these ideas. We can debate the contents of the docs
> and as a nice side-effect resolve issues like: should there be a
> roadmap, and if so, what should it contain?
> 
> Thanks a lot for always putting in so much effort and thought into
> this. I hope we can have another live chat soon.
> 

Thanks for your feedback too.

-- Matthieu Baechler


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[David Leangen <ap...@leangen.net>]

Hi Matthieu,

My apologies for be absence. I expect to continue to be absent for the next couple of weeks, but I wanted to respond to your email.

> I'm not sure which message to answer in the thread so I start a new
> thread to summarize my thoughts on the various topics discussed.

No problem. However, you touch on so many points that I am not sure of the best way to organize this type of complex discussion. In absence of any good ideas, I will just respond to everything in this one thread.

I will start with one important point, then move on to the conclusion before responding to the rest.

>   James already works for a lot of use cases, there's nothing really
>   lacking in term of code, we should stop thinking the software should
>   be better before being happy with it. A lot of successful softwares
>   are less mature than James.

Yes, I agree. My problem was that I was never able to actually use it. So, my natural reaction was to try to figure out why, but I was lacking the documentation to do that. That got me digging into the documentation much more deeply than I expected to do, and took me further and further away from just trying to use it. 😆


> My conclusion is that this project is a very valuable one, written by
> some very talented developers for something like 20 years or so.

Absolutely!!


> This legacy is putting us in a very difficult situation: the codebase
> is huge, the test suite takes ages to execute, a lot of things are here
> for historical reasons but as we are careful about not breaking too
> much people deployments we don't remove enough things.

Yes, it is a difficult situation, but I think we can overcome this. The first step is recognizing the problems, which I think you have quite well.


> In short: we are not able to move fast, to simplify the codebase, to
> implement new things easily and finally it's hard to have fun for
> newcomers or even James veterans.

I agree. I was initially excited about using James, but I realized along the way that it would be a long-term commitment. I was not able to accomplish what I wanted because I was not able to use it, and I was also unable to understand it enough to make changes to be able to use it. So I was not able to complete my project with any success. I have changed tactic to try to accomplish this over the long term instead. I hope to continue to invest maybe an hour or so each day, but that is easier said than done.


> What I would really like is to break things! Let's remove all
> these anachronic modules, or even better: let's build James 4 by
> adopting only well selected modules, ones that are here for a purpose.

Yes, great idea.


> People could either jump to this fresh version of James or keep
> maintaining the 3.x branch. If they lack some modules that were not
> selected for James 4, they could just port these modules to the new
> APIs.

 Speaking for myself, my preference would be to jump into this version I think.


> By doing such a move, we could focus to finally solve our longstanding
> problems like: developer experience, newcomers welcoming, having a
> decent and up-to-date documentation, very easy first deployment, etc.

Ok, nice.


> What would you think of such a move ?

100% agree.


> 1. About Roadmap
> 
>   Having a roadmap is a very good way to deceive users. It's ok for a
>   company because you can somehow foresee what you'll be able to
>   achieve in the future if you care about but when it comes to people
>   and their spare time, well, life is unprecdictable.
> 
>   The less we say about the future, the better. People interested into
>   the future can always get in touch with the community and ask.

I understand your argument that we don’t want to make false promises. Hopefully it is clear by now that I totally agree with that idea, and that is why I have been pushing (hopefully not too hard) to be more “honest” about the state of James.

I think it is still possible to provide a “roadmap” while not making false promises, though. I don’t think it is an either/or. It is a “fact” that “at this time, this is our intent”. It does not mean that we are making any “promises”. So long as we make this clear, stating the current thinking of the community as a fact is not at all deceiving. In fact, it is helpful (1) for the community to have an agreement, and in writing, and (2) useful for new members to better understand the current thinking.

The roadmap can always be changed.


> 2. About documentation
> 
>   You definitely deserve much praises for what you did already. We
>   know that it's the missing piece for James to shine.

Ahahaha. Thanks, but I feel badly about not completing it yet. I have not given up, just changed gears.

>   I would just like to list what I think we lack the most:
> 
>   * examples of working setups
>   * easy-to-search documentation for details like configuration or    
>   mailets
>   * guides for the most usual things: configure some mailets, write my
>   own, how to debug a mailet pipeling, how to plug my app in James
>   efficiently

Yes, all good points.

>   James already works for a lot of use cases, there's nothing really
>   lacking in term of code, we should stop thinking the software should
>   be better before being happy with it. A lot of successful softwares
>   are less mature than James.
> 
>   TL;DR we have to explain how it already works

👍


> 3. About versioning (3.x vs 4.0)
> 
>   Like roadmaps, major version numbers give people expectations: there
>   must be something very new and/or very different because the
>   community decided to increment major version number.

Yes, you are absolutely right that a version number sets expectations. And so it should. That is actually kind of the purpose.

If possible, we should consider moving away from soft requirements and use semantic versioning. I think that the concept of semantic versioning is specified well enough to be considered as being quite objective. Hopefully that should avoid this type of debate.


>   Last time the community tried that (3.0) the projects almost died
>   because too many things had to ship at the same time and then was
>   never ready. It took years to finally release it after Linagora
>   started to invest a lot it and by the way, we never finished what
>   was supposed to, we just decided that, no software being perfect, we
>   had to release this much-better-than-2.x version.

Oh, that is interesting. It is always good to learn from experience.


>   I would like to take the Linux Kernel path: 
> 
>   * only increment minor version for the time being
>   * don't build a backlog or any list of things we want to achieve
>   before incrementing

Can you explain this point more?

>   * release 2 to 4 times a years with what is ready
>   * increment major version when what will be ready deserves it or
>   when minor number get to big

This seems a bit arbitrary. Maybe we could instead figure out what semantic versioning means to James, so we can have a more objective criterion?

> 
> 4. About defining a vision
> 
>   You defined different kinds of software project styles and it's
>   interesting to try to define where James is.
> 
>   Let me define how I see it.
> 
>   The community and the project doesn't have a vision yet. The
>   community is small, mostly composed of people using James in
>   enterprise context or paid to work on it.
> 
>   Apache Foundation projects should not be led by a single company so
>   Linagora is not taking the lead on this side.

Why do you state this? Is there some rule? Or you are worried about appearance?

There is indeed a really heavy Linagora presence here. I think it’s great, and I really appreciate your caution. Without yet agreeing to this point, I don’t mind acting like a moderator to capture the vision. As a moderator I can stamp my name on it to assure the community that it is not too biased in favor of a single company.


>   It means to me that the best we can do as contributors is to
>   contribute to the project in a way that is useful to us and try to
>   welcome newcomers as well as we can.
> 
>   If the project is valuable to us, it will eventually be valuable to
>   others. We have to stay commited to it for the time being, continue
>   as we did for the past years.
> 
>   I can't see any vision that the community would be able to commit
>   to, so let's say it's just a project that wait for its hour of
>   glory.

😀

Well, I think we’ll have to be a bit more proactive, but let’s continue this discussion to see where it takes us. I think from what we experienced over the past few months that for a slowly evolving project like this, the Documentation actually seems to be a great way to capture all these ideas. We can debate the contents of the docs and as a nice side-effect resolve issues like: should there be a roadmap, and if so, what should it contain?

Thanks a lot for always putting in so much effort and thought into this. I hope we can have another live chat soon.


Cheers,
=David



---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[Matthieu Baechler <ma...@apache.org>]

Hi,

I'm not sure which message to answer in the thread so I start a new
thread to summarize my thoughts on the various topics discussed.

1. About Roadmap

   Having a roadmap is a very good way to deceive users. It's ok for a
   company because you can somehow foresee what you'll be able to
   achieve in the future if you care about but when it comes to people
   and their spare time, well, life is unprecdictable.

   The less we say about the future, the better. People interested into
   the future can always get in touch with the community and ask.

2. About documentation

   You definitely deserve much praises for what you did already. We
   know that it's the missing piece for James to shine.

   I would just like to list what I think we lack the most:

   * examples of working setups
   * easy-to-search documentation for details like configuration or    
   mailets
   * guides for the most usual things: configure some mailets, write my
   own, how to debug a mailet pipeling, how to plug my app in James
   efficiently

   James already works for a lot of use cases, there's nothing really
   lacking in term of code, we should stop thinking the software should
   be better before being happy with it. A lot of successful softwares
   are less mature than James.

   TL;DR we have to explain how it already works

3. About versioning (3.x vs 4.0)

   Like roadmaps, major version numbers give people expectations: there
   must be something very new and/or very different because the
   community decided to increment major version number.

   Last time the community tried that (3.0) the projects almost died
   because too many things had to ship at the same time and then was
   never ready. It took years to finally release it after Linagora
   started to invest a lot it and by the way, we never finished what
   was supposed to, we just decided that, no software being perfect, we
   had to release this much-better-than-2.x version.

   I would like to take the Linux Kernel path: 

   * only increment minor version for the time being
   * don't build a backlog or any list of things we want to achieve
   before incrementing
   * release 2 to 4 times a years with what is ready
   * increment major version when what will be ready deserves it or
   when minor number get to big

4. About defining a vision

   You defined different kinds of software project styles and it's
   interesting to try to define where James is.

   Let me define how I see it.

   The community and the project doesn't have a vision yet. The
   community is small, mostly composed of people using James in
   enterprise context or paid to work on it.

   Apache Foundation projects should not be led by a single company so
   Linagora is not taking the lead on this side.

   It means to me that the best we can do as contributors is to
   contribute to the project in a way that is useful to us and try to
   welcome newcomers as well as we can.

   If the project is valuable to us, it will eventually be valuable to
   others. We have to stay commited to it for the time being, continue
   as we did for the past years.

   I can't see any vision that the community would be able to commit
   to, so let's say it's just a project that wait for its hour of
   glory.


My conclusion is that this project is a very valuable one, written by
some very talented developers for something like 20 years or so.

This legacy is putting us in a very difficult situation: the codebase
is huge, the test suite takes ages to execute, a lot of things are here
for historical reasons but as we are careful about not breaking too
much people deployments we don't remove enough things.

In short: we are not able to move fast, to simplify the codebase, to
implement new things easily and finally it's hard to have fun for
newcomers or even James veterans.

What I would really like is to break things! Let's remove all
these anachronic modules, or even better: let's build James 4 by
adopting only well selected modules, ones that are here for a purpose.

People could either jump to this fresh version of James or keep
maintaining the 3.x branch. If they lack some modules that were not
selected for James 4, they could just port these modules to the new
APIs.

By doing such a move, we could focus to finally solve our longstanding
problems like: developer experience, newcomers welcoming, having a
decent and up-to-date documentation, very easy first deployment, etc.

What would you think of such a move ?

Cheers,

-- Matthieu Baechler


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[David Leangen <ap...@leangen.net>]

Hi Raphaël,

> First thank you very much for your involvement into the community, I hope you'll be able to continue to do so.

Well, thanks to you for your support.


> From my point of view the most important point is to describe what James does now. It is what is more helpful to users (operators), and also to brand the product as it allows to have a view of several capabilities of the product.

Yes, I agree with you.

For me, it is helpful to understand what the aspirations are as well, but I agree that “what James is now” is more important.

On that note, “what James does” to me is a part of “what James is”. Understanding how the community operates is also an important part of what James is.


Cheers,
=David


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[Tellier Benoit <bt...@apache.org>]

Le 31/08/2020 à 18:28, David Leangen a écrit :
> Hi Benoit,
>
> Ok, cool. I think we are making progress. 👍🏻
>
>> Would it answer some of your concerns?
> Let me answer this first: yes. Thank you for patiently tolerating my persistence. 😀
>
>
>> Well from what I recall, we had a nice community meeting answering that
>> very question.
>>
>>  - We will rework product definition, boundaries and branding. Using
>> guice servers we will provide a basic/advanced/distributed server
>>  - We will improve the build experience through Apache CI builds and
>> migrating to graddle.
>>  - Also, some contributors are implementing the final RFC-8621 JMAP release.
>>
>> Maybe we should have a roadmap page somewhere so that people don't have
>> to read the DEV mailing list to find these pieces of information?
> That is indeed my intention, and why I am asking these questions. I am ok with referencing other artifacts, but I think it is important to publish the roadmap on the website. Of course, it can always change, that is fine. It is not a set contract that MUST be undertaken, but it ought to represent the current thoughts of the community.
It turns out there is a roadmap published on our website (
http://james.apache.org/#roadmap ) - I just discovered it, shame on me...

I did open a pull request to add the items of the community call to it.
See https://github.com/apache/james-project/pull/245

Cheers

Benoit


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[David Leangen <ap...@leangen.net>]

Hi Benoit,

Ok, cool. I think we are making progress. 👍🏻

> Would it answer some of your concerns?

Let me answer this first: yes. Thank you for patiently tolerating my persistence. 😀


> Well from what I recall, we had a nice community meeting answering that
> very question.
> 
>  - We will rework product definition, boundaries and branding. Using
> guice servers we will provide a basic/advanced/distributed server
>  - We will improve the build experience through Apache CI builds and
> migrating to graddle.
>  - Also, some contributors are implementing the final RFC-8621 JMAP release.
> 
> Maybe we should have a roadmap page somewhere so that people don't have
> to read the DEV mailing list to find these pieces of information?

That is indeed my intention, and why I am asking these questions. I am ok with referencing other artifacts, but I think it is important to publish the roadmap on the website. Of course, it can always change, that is fine. It is not a set contract that MUST be undertaken, but it ought to represent the current thoughts of the community.


> 
>> On that note, I would propose that for the 4.0 release, we clearly indicate that Spring will **NOT** be available.
> 
> +1, and a 4.0 would make the switch very clear. No ambiguity.

Ok, cool! So I think we have just now opened up a path forward.


> I believe the only work remaining toward this is
> https://github.com/apache/james-project/pull/221 (and for other guice
> servers)

I will take a look. Thanks for pointing this out.


>> It should be deprecated in an upcoming 3.x release, then “completely” eliminated in 4.0. (Perhaps some code could remain if some people really want it, but we need to make it clear in the “user contract" that it is not supported.) That would be precisely what a major release is for, IMO.
> We discussed (can't find the thread though) some years ago about the
> adoption of time based release for James server.

Hmmmm. Intuitively, I am a bit skeptical, but if you ever find that discussion, I will read through it to hear the arguments.



> Here we could maybe:
>  - Better communicate about the release calendar (why not having it on
> the roadmap page?)
>  - Prior the release date, discuss the reach of this release (major vs
> minor) and see how we are regarding roadmap items.
>  - Also, the release process needs to be run faster...

Thank you for sharing these ideas.

I will put my thoughts together and get back to this discussion in a few days. I have a few other things I need to attend to now.


Cheers,
=David


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[Tellier Benoit <bt...@apache.org>]

Le 31/08/2020 à 09:43, David Leangen a écrit :
>>> From my point of view the most important point is to describe what
>>> James does now. 
>> +1
> Yes, I agree.
>
> Again, I do not disagree that this is “most important”. Let me use a more concrete example to illustrate why it is not the **only** concern.
>
> When I first looked into James, there was a complex grid to choose from between Spring and Guice. Based on the grid, it seemed to me that Spring was better.
>
> When I started asking questions, though, I understood that Spring is not really being maintained anymore, or at least, more energy is being put into Guice.
>
> Well, for me as a user trying to understand what to do, that is pretty darn important information! I **WANT** to know where the project is heading, because I need to be able to make decisions. So not only do I need to know what the project’s vision is (which IMO needs a lot of clarification currently), I need to know more concretely what the plans are for at least the next major release. Else, how can I decide if it’s worth investing or not?

Well from what I recall, we had a nice community meeting answering that
very question.

 - We will rework product definition, boundaries and branding. Using
guice servers we will provide a basic/advanced/distributed server
 - We will improve the build experience through Apache CI builds and
migrating to graddle.
 - Also, some contributors are implementing the final RFC-8621 JMAP release.

Maybe we should have a roadmap page somewhere so that people don't have
to read the DEV mailing list to find these pieces of information?

> On that note, I would propose that for the 4.0 release, we clearly indicate that Spring will **NOT** be available.

+1, and a 4.0 would make the switch very clear. No ambiguity.

I believe the only work remaining toward this is
https://github.com/apache/james-project/pull/221 (and for other guice
servers)

>  It should be deprecated in an upcoming 3.x release, then “completely” eliminated in 4.0. (Perhaps some code could remain if some people really want it, but we need to make it clear in the “user contract" that it is not supported.) That would be precisely what a major release is for, IMO.
We discussed (can't find the thread though) some years ago about the
adoption of time based release for James server.

This ensures we keep delivering improvement and fixing bugs even if we
struggle delivering some major changes.

Here we could maybe:
 - Better communicate about the release calendar (why not having it on
the roadmap page?)
 - Prior the release date, discuss the reach of this release (major vs
minor) and see how we are regarding roadmap items.
 - Also, the release process needs to be run faster...

Would it answer some of your concerns?

Benoit


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[David Leangen <ap...@leangen.net>]

>> From my point of view the most important point is to describe what
>> James does now. 
> 
> +1

Yes, I agree.

Again, I do not disagree that this is “most important”. Let me use a more concrete example to illustrate why it is not the **only** concern.

When I first looked into James, there was a complex grid to choose from between Spring and Guice. Based on the grid, it seemed to me that Spring was better.

When I started asking questions, though, I understood that Spring is not really being maintained anymore, or at least, more energy is being put into Guice.

Well, for me as a user trying to understand what to do, that is pretty darn important information! I **WANT** to know where the project is heading, because I need to be able to make decisions. So not only do I need to know what the project’s vision is (which IMO needs a lot of clarification currently), I need to know more concretely what the plans are for at least the next major release. Else, how can I decide if it’s worth investing or not?


On that note, I would propose that for the 4.0 release, we clearly indicate that Spring will **NOT** be available. It should be deprecated in an upcoming 3.x release, then “completely” eliminated in 4.0. (Perhaps some code could remain if some people really want it, but we need to make it clear in the “user contract" that it is not supported.) That would be precisely what a major release is for, IMO.


Thoughts?

=David



---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[Tellier Benoit <bt...@apache.org>]

Le 28/08/2020 à 14:48, Raphaël Ouazana-Sustowski a écrit :
> Hi David,
>
> First thank you very much for your involvement into the community, I
> hope you'll be able to continue to do so.
>
> Le 28/08/2020 à 05:11, David Leangen a écrit :
>> Hi Rene,
>>
>>>> I have observed a few different types of approaches to OSS:
>>>>   * Haphazard “me” approach.
>>> I think you are correct, I would say that now it is more later
>>> approach as well. Well at least since I started working on the
>>> project (might have been different before).
>> Thank you very much for your comments. Yes, that is very helpful!
>>
>> I fall into this trap often, and I think I am not the only one, but
>> sometimes I explain what I **aspire** a project to be, not really
>> what it is now. I think it is crucial to explain both. We need to be
>> able to give proper information to newcomers so they can make a
>> correct assessment. I think what you mention makes sense and I can
>> work with that.
>>
>> Perhaps we should just wait in case anybody else would also like to
>> add something?
>>
>> So I would describe James as:
>>
>>   * Currently in a state of transition (this is what James is now)
>>   * What we aspire it to be for version xxx (I am proposing v4.0)
>>
>> Just a crazy thought, but technically, I could even create the v4.0
>> of the documentation, and we could even use it almost like a
>> specification to guide the development. Of course, the real work gets
>> done based on JIRA issues, but the issues that get created in the
>> first place could be matched against the v4.0 docs.
>
>
> From my point of view the most important point is to describe what
> James does now. 

+1

> [...]
>
> Raphaël.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[Raphaël Ouazana-Sustowski <ro...@apache.org>]

Hi David,

First thank you very much for your involvement into the community, I 
hope you'll be able to continue to do so.

Le 28/08/2020 à 05:11, David Leangen a écrit :
> Hi Rene,
>
>>> I have observed a few different types of approaches to OSS:
>>>   * Haphazard “me” approach.
>> I think you are correct, I would say that now it is more later approach as well. Well at least since I started working on the project (might have been different before).
> Thank you very much for your comments. Yes, that is very helpful!
>
> I fall into this trap often, and I think I am not the only one, but sometimes I explain what I **aspire** a project to be, not really what it is now. I think it is crucial to explain both. We need to be able to give proper information to newcomers so they can make a correct assessment. I think what you mention makes sense and I can work with that.
>
> Perhaps we should just wait in case anybody else would also like to add something?
>
> So I would describe James as:
>
>   * Currently in a state of transition (this is what James is now)
>   * What we aspire it to be for version xxx (I am proposing v4.0)
>
> Just a crazy thought, but technically, I could even create the v4.0 of the documentation, and we could even use it almost like a specification to guide the development. Of course, the real work gets done based on JIRA issues, but the issues that get created in the first place could be matched against the v4.0 docs.


 From my point of view the most important point is to describe what 
James does now. It is what is more helpful to users (operators), and 
also to brand the product as it allows to have a view of several 
capabilities of the product.

What we aspire to is also interesting, but less important, and above all 
subject to change, so it brings less value to users. On the contrary it 
can bring value to contributors as it helps them to get where we want to go.

Anyway we are always encouraging contributions, so if you prefer working 
on our aspiration (as a community) than on the current state, this is 
also very welcome.

Cheers,

Raphaël.


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[Rene Cordier <rc...@apache.org>]

Hi again David,

On 28/08/2020 10:11, David Leangen wrote:

> Perhaps we should just wait in case anybody else would also like to add something?
> 
> So I would describe James as:
> 
>   * Currently in a state of transition (this is what James is now)
>   * What we aspire it to be for version xxx (I am proposing v4.0)
> 
> Just a crazy thought, but technically, I could even create the v4.0 of the documentation, and we could even use it almost like a specification to guide the development. Of course, the real work gets done based on JIRA issues, but the issues that get created in the first place could be matched against the v4.0 docs.
> 
> 
> As we make progress, the two different versions will resemble each other more and more (i.e. what James **is** gets closer to what we **aspire** it to be, at least for v4.0).
> 
> I don’t think we need to (or ought to) set a timeline, though. There are not enough resources allocated to do that.
> 
> 
> What do you think?

I think for such a move there is probably a need of consensus through 
the PMC members via a vote.

Yes we need a good change of approach in our documentation and branding, 
and it looks like it's taking the right direction, but the code itself 
is not really having a big breaking change with this...

So is it enough to move towards a 4.0 or should we keep going with 3.x 
version? I'm not sure myself to be honest.

Cheers,
Rene.

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[David Leangen <ap...@leangen.net>]

Hi Rene,

>> I have observed a few different types of approaches to OSS:
>>  * Haphazard “me” approach.
> I think you are correct, I would say that now it is more later approach as well. Well at least since I started working on the project (might have been different before).

Thank you very much for your comments. Yes, that is very helpful!

I fall into this trap often, and I think I am not the only one, but sometimes I explain what I **aspire** a project to be, not really what it is now. I think it is crucial to explain both. We need to be able to give proper information to newcomers so they can make a correct assessment. I think what you mention makes sense and I can work with that.

Perhaps we should just wait in case anybody else would also like to add something?

So I would describe James as:

 * Currently in a state of transition (this is what James is now)
 * What we aspire it to be for version xxx (I am proposing v4.0)

Just a crazy thought, but technically, I could even create the v4.0 of the documentation, and we could even use it almost like a specification to guide the development. Of course, the real work gets done based on JIRA issues, but the issues that get created in the first place could be matched against the v4.0 docs.


As we make progress, the two different versions will resemble each other more and more (i.e. what James **is** gets closer to what we **aspire** it to be, at least for v4.0).

I don’t think we need to (or ought to) set a timeline, though. There are not enough resources allocated to do that.


What do you think?

Cheers,
=David


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[Rene Cordier <rc...@apache.org>]

Hi David,

On 27/08/2020 10:21, David Leangen wrote:
> I have observed a few different types of approaches to OSS:
> 
>   * Centered on a specification. The spec development has a formal process, and the spec becomes the driving force. There is a “reference implementation” to show that the spec works as intended. In this case, the clarity relates to the spec. There is engagement from the community to make implement the spec in a way that “works”. The clarity for the user in this case is that they know what they can expect. (Perhaps OSGi is a great example of this approach.)
> 
>   * Centered on a clear corporate vision. As a typical case, a company has developed a product and has decided to open source it. What the product does is clearly documented, as is the type of support that a user can receive (community version or commercial version). Again, for the user, there is clarity in terms of what they can expect to receive. (Maybe something like Docker would fit this model? Or Elasticsearch?)
> 
>   * Driven by a clear vision and objective. The scope of the product is very clear, and there is a commitment (for whatever reason) to make it “work”. By extension, this means that the meaning of “it works” is very precisely understood. (Maybe Ant, Maven, and Gradle are good examples, because what is meant by a “build tool” is pretty well understood.)
> 
>   * Haphazard “me” approach. In this case, there is no central driver (spec, company, clear vision). The product evolves more based on the individual contributions of the members, which themselves are based almost entirely on what the contributors need for themselves. Unless a contributor has their own specific need and evolves the product to suite that need, then there is little chance that the product will move. Because it is unpredictable what each individual contributor will need, from the user’s perspective the development is haphazard. There is little clarity in terms of what they can expect to get, or if the product will continue to be maintained. If they want it to work, they will have to become a contributor themselves, as there there are no guarantees.
> 
> 
> There is no “right” or “wrong”, or even “better” by the way.
> 
> 
>  From my understanding so far, I think that James seems to fit the latter approach. Yes, it relates to a few specifications, but James is not (as I understood) willing to “commit” to being a reference implementation for SMTP, IMAP, JMAP etc. Yes, there is some vision and some objectives, but nobody is willing to commit to ensuring that the product works according to the objectives, and as far as I can tell, the objectives are not entirely precisely defined. So, based on what I know so far, it seems to fit best the last model.

I think you are correct, I would say that now it is more later approach 
as well. Well at least since I started working on the project (might 
have been different before).

I think the devs at Linagora are working more towards filling the needs 
for the company, to be honest, which is mainly centered around the Guice 
distributed server. But all the other Guice implementations are growing 
as well with it along the way I would say, as they reuse pretty much the 
same components (distributed server being the most complete and complex 
one, others are like lighter versions of it).

I believe that we are of course mainly working on distributed server 
version, but maintaining the rest as well... Trying to fix some IMAP or 
POP bugs, etc.

But the biggest issue with the community would be I think because the 
Spring version is still the default promoted one on our current 
documentation. So people (as we say doc is quite confusing) start using 
this one usually, and ask questions on it. The choice to keep this as 
default seems to be because James JPA on Guice is still not as complete 
as the Spring one. For example, the Guice version is only running on a 
Derby embedded db, among other things missing (if I understood 
correctly, but if I'm wrong I'm sure someone else will correct me)

We would like to get rid of the Spring JPA version (also in term of 
maintenance, it's always a win to have less different technologies 
used), but we can't really as long as the guice JPA is missing things 
compared to Spring. But it is not a priority for us and we don't have 
enough resources.

But I believe the documentation you started would help clarify things to 
users and maybe push newcomers to contribute more on things we try to 
maintain but don't have the resources to spend enough time on it.

I would like to add I think we are willing to commit being a reference 
implementation to some things though, like lately we are spending energy 
on JMAP. First we develop what we need but we will try to complete it 
over time to be as close as possible from the specs. Would like to 
mention as well some devs here are participating in the writing of the 
JMAP specs:

* Benoit Tellier wrote the part regarding VacationResponse in the JMAP 
mail specs
* Raphael Ouazana wrote an extension for JMAP MDN that is in the 
finalization phase
* myself is trying to contribute an extension on JMAP quotas as well

I hope this helps you having a better vision of the project current 
situation!

Cheers,
Rene.

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[David Leangen <ap...@leangen.net>]

Hi Benoit,

> I think I did answer in that thread already.

Yes, you did. And thank you.

>> To continue with the documentation (at least on the path I have taken so far) I need to better understand the vision. That will allow me to resolve and clarify the topics previously raised regarding the community’s approach to dealing with newbies.
> This sentence sounds like the "support/warranty" discussion turned into
> a blocker for you.

It is not a blocker in the sense that regardless of the approach the community wants to take, I will continue one way or the other.

However, in order to document, I need to understand. In that respect it is a blocker because I still do not fully understand how this community is working. I will write more below about what I mean.

> I think instead of using blockers we should propose consensual solutions.

All for it!

> The approach that emerged from ongoing proposal would be "Through better
> branding and documentation, end user would need less support" eventually
> making most people happy, without requiring SLAs from community members.

It is interesting that you interpreted it this way. Allow me to try to clarify my thinking.



What I am trying to understand is (1) this community’s vision, and (2) a slightly deeper understanding of its approach to OSS.

I don’t really have proper vocabulary, so I’ll try to explain in my own words. Perhaps somebody has already done research on this topic so can express these ideas better. If you know of any such document, please do let me know. In the meantime, here is my layman’s explanation.



I have observed a few different types of approaches to OSS:

 * Centered on a specification. The spec development has a formal process, and the spec becomes the driving force. There is a “reference implementation” to show that the spec works as intended. In this case, the clarity relates to the spec. There is engagement from the community to make implement the spec in a way that “works”. The clarity for the user in this case is that they know what they can expect. (Perhaps OSGi is a great example of this approach.)

 * Centered on a clear corporate vision. As a typical case, a company has developed a product and has decided to open source it. What the product does is clearly documented, as is the type of support that a user can receive (community version or commercial version). Again, for the user, there is clarity in terms of what they can expect to receive. (Maybe something like Docker would fit this model? Or Elasticsearch?)

 * Driven by a clear vision and objective. The scope of the product is very clear, and there is a commitment (for whatever reason) to make it “work”. By extension, this means that the meaning of “it works” is very precisely understood. (Maybe Ant, Maven, and Gradle are good examples, because what is meant by a “build tool” is pretty well understood.)

 * Haphazard “me” approach. In this case, there is no central driver (spec, company, clear vision). The product evolves more based on the individual contributions of the members, which themselves are based almost entirely on what the contributors need for themselves. Unless a contributor has their own specific need and evolves the product to suite that need, then there is little chance that the product will move. Because it is unpredictable what each individual contributor will need, from the user’s perspective the development is haphazard. There is little clarity in terms of what they can expect to get, or if the product will continue to be maintained. If they want it to work, they will have to become a contributor themselves, as there there are no guarantees.


There is no “right” or “wrong”, or even “better” by the way.


From my understanding so far, I think that James seems to fit the latter approach. Yes, it relates to a few specifications, but James is not (as I understood) willing to “commit” to being a reference implementation for SMTP, IMAP, JMAP etc. Yes, there is some vision and some objectives, but nobody is willing to commit to ensuring that the product works according to the objectives, and as far as I can tell, the objectives are not entirely precisely defined. So, based on what I know so far, it seems to fit best the last model.

All I want to know is: is this intended?


Again, all I am trying to do is get clarity. I need the clarity if I am to describe James to outsiders. The first, fundamental, basic questions anyone will want to know is: What is James? Can I use this? What can I expect?

If we do not provide a clear and honest answer, as far as I am concerned it is a misrepresentation, and is not ethical. I am not trying suggest which approach to take, I am simply trying to understand this community’s intentions with more clarity in order to properly represent it.

Your help would be greatly appreciated. 😀


Cheers,
=David


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[Tellier Benoit <bt...@apache.org>]

Hello David,

I did actually came back from vacation.

I think I did answer in that thread already. I would love to have other
people feedback as well on the opportunity to market our current
"community roadmap" (documentation, product re-branding, CI....) into a
major release.

Le 26/08/2020 à 18:10, David Leangen a écrit :
> Hello,
>
> I am still trying to get a sense of the community’s vision, and it’s thoughts relating to how to treat users. Unfortunately I have not heard too many comments so far on this topic. Trying to read between the lines, I could think of 4 potential conclusions:
>
>  1. People are still on summer holidays and don’t have the time to spend
>  2. I have taken a wrong or confusing approach to the topic, so people don’t know what to answer
>  3. This is already documented somewhere so the assumption is that I already know it
>  4. There is not much interest in discussing the the topic of “users”
>
> To continue with the documentation (at least on the path I have taken so far) I need to better understand the vision. That will allow me to resolve and clarify the topics previously raised regarding the community’s approach to dealing with newbies.
This sentence sounds like the "support/warranty" discussion turned into
a blocker for you.

I think instead of using blockers we should propose consensual solutions.

The approach that emerged from ongoing proposal would be "Through better
branding and documentation, end user would need less support" eventually
making most people happy, without requiring SLAs from community members.

> By the way, this project was much more difficult that I had anticipated. I no longer have much time to dedicate, but I pledge to continue, albeit at a slower pace and in the background, so long as I continue to get engagement from the community members. My main goal is to help document the state of James and the community, and the secondary goal that I discovered in the process is to point out gaps and potential areas of improvement. Thanks again for all the support!!
Thank you again for your involvement!

Cheers,

Benoit
>
> Cheers,
> =David
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[David Leangen <ap...@leangen.net>]

Hi Rene,

Thanks for your reply. Sorry I just sent out my last email without having seen yours. My comments inline.

>> To continue with the documentation (at least on the path I have taken so far) I need to better understand the vision. That will allow me to resolve and clarify the topics previously raised regarding the community’s approach to dealing with newbies.
> 
> Well I believe a bit like Benoit on this. I think most of the time, users are asking stuff because, as I raised in point 3, our documentation is outdated, mixed between different products all together, not versioned, and very confusing.
> 
> With the efforts made on rebranding our products towards a more understandable user approach and reworking the documentation with Antora, with a versioning, and having a clear separation between James flavors, it would already help a lot more the users on how to use James regarding their needs, and might decrease the overall need for help with the community.
> 
> Also the release process was a bit long and painful as I understood, thus we were only updating the doc when doing a release, so lots of fixes stay hanging for a while. I believe with Antora and Eugen work to automate the build and website release he did (I hope I'm not wrong?), it might make it much easier as well to keep our documentation up-to-date.

Great! So it sounds like there **is** some interest in taking a more user-centric approach. That is exactly what I am trying to gauge and understand. 😀


> I believe as well trying to add a bunch of how-tos regarding the topics that an admin might likely be interested to setup with James or what seems to be recurrent struggles of users might help too.

That would be great!!


> Honestly thank you greatly for all the time and thinking you have been putting into this new James documentation with a more user-focused mindset. I believe it helped the project greatly being more community friendly !

Thank you for letting me know. I am relieved that it is useful. 😅 In that case, I am happy to continue, albeit at an even slower pace.

Cheers,
=David


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[Rene Cordier <rc...@apache.org>]

Hello David,

On 26/08/2020 18:10, David Leangen wrote:
> Hello,
> 
> I am still trying to get a sense of the community’s vision, and it’s thoughts relating to how to treat users. Unfortunately I have not heard too many comments so far on this topic. Trying to read between the lines, I could think of 4 potential conclusions:
> 
>   1. People are still on summer holidays and don’t have the time to spend

Everything always slows down on each summer, with holidays and such yes.

>   2. I have taken a wrong or confusing approach to the topic, so people don’t know what to answer

Well when people are just saying : "This is not working" without 
details, and which James version they use, etc, it is hard to understand 
the issue and answer yes. But that is not specific to James I believe, 
it is a more global communication problem that you can see for any project.

>   3. This is already documented somewhere so the assumption is that I already know it

Not really... We know we have a problem with our current documentation. 
Some parts are outdated, there is no versioning so if someone use an 
older version of James, it's nearly impossible to find back the 
documentation corresponding to it, all information is mixed together.

It is often I can see people asking questions why when they try to use 
something it doesn't work, while actually the answer is because they use 
the wrong product. But from our current documentation, it's not obvious 
and confusing (like for example people trying to use webadmin with the 
Spring version of James... where it's not supported).

>   4. There is not much interest in discussing the the topic of “users”

Why there would be no interest? ^^

> 
> To continue with the documentation (at least on the path I have taken so far) I need to better understand the vision. That will allow me to resolve and clarify the topics previously raised regarding the community’s approach to dealing with newbies.

Well I believe a bit like Benoit on this. I think most of the time, 
users are asking stuff because, as I raised in point 3, our 
documentation is outdated, mixed between different products all 
together, not versioned, and very confusing.

With the efforts made on rebranding our products towards a more 
understandable user approach and reworking the documentation with 
Antora, with a versioning, and having a clear separation between James 
flavors, it would already help a lot more the users on how to use James 
regarding their needs, and might decrease the overall need for help with 
the community.

Also the release process was a bit long and painful as I understood, 
thus we were only updating the doc when doing a release, so lots of 
fixes stay hanging for a while. I believe with Antora and Eugen work to 
automate the build and website release he did (I hope I'm not wrong?), 
it might make it much easier as well to keep our documentation up-to-date.

I believe as well trying to add a bunch of how-tos regarding the topics 
that an admin might likely be interested to setup with James or what 
seems to be recurrent struggles of users might help too.

> By the way, this project was much more difficult that I had anticipated. I no longer have much time to dedicate, but I pledge to continue, albeit at a slower pace and in the background, so long as I continue to get engagement from the community members. My main goal is to help document the state of James and the community, and the secondary goal that I discovered in the process is to point out gaps and potential areas of improvement. Thanks again for all the support!!

Honestly thank you greatly for all the time and thinking you have been 
putting into this new James documentation with a more user-focused 
mindset. I believe it helped the project greatly being more community 
friendly !

Cheers,
Rene.

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[David Leangen <ap...@leangen.net>]

Hello,

I am still trying to get a sense of the community’s vision, and it’s thoughts relating to how to treat users. Unfortunately I have not heard too many comments so far on this topic. Trying to read between the lines, I could think of 4 potential conclusions:

 1. People are still on summer holidays and don’t have the time to spend
 2. I have taken a wrong or confusing approach to the topic, so people don’t know what to answer
 3. This is already documented somewhere so the assumption is that I already know it
 4. There is not much interest in discussing the the topic of “users”

To continue with the documentation (at least on the path I have taken so far) I need to better understand the vision. That will allow me to resolve and clarify the topics previously raised regarding the community’s approach to dealing with newbies.

By the way, this project was much more difficult that I had anticipated. I no longer have much time to dedicate, but I pledge to continue, albeit at a slower pace and in the background, so long as I continue to get engagement from the community members. My main goal is to help document the state of James and the community, and the secondary goal that I discovered in the process is to point out gaps and potential areas of improvement. Thanks again for all the support!!

Cheers,
=David



---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

Re: [Discussion] Road to 4.0

[David Leangen <ap...@leangen.net>]

Hi Benoit,

Thank you for responding.

>> I wanted to tie together a few ongoing threads and make a proposal for the road to a 4.0 release. 
> 
> I'm uncomfortable with the 4.0 release switch. It gives the impression
> that "much" needs to change whilewhat we need is to re-brand, and better
> document what exists.

I see.

I agree that there isn’t really that much that needs to be changed from a technical perspective. My thought was that from the user perspective, there really are some very large changes being made. It seems to me that even if technically the changes are small, from an objective / orientation perspective this is a huge change.

Do you think it will be as easy to explain these changes to the public even with a minor release?


>> By the way, to coincide with the release, if the objectives are clear, perhaps there is a commercial organization (or individual) who would be willing to provide paid-for professional support starting from 4.0?
> Linagora (company for which I am working) already proposes these
> services. We already carried out several support / development contracts
> for Apache James.

Thanks! Is there a website page or something with details of this service offering?

I did a quick search for “Linagora James development” and got this hit:

  —> https://linagora.com/open-source-technologies/apache-james <https://linagora.com/open-source-technologies/apache-james>


>> I think it would help complete the offering, and hopefully provide a commercial opportunity as well in a way that is beneficial to all, including the James community. We just need to ensure that the “contract” is very clear, and that we avoid any potential conflicts of interest. I think we should include this item in the scope of the discussion as well. 
> 
> I'm not sure an Apache project is entitled to broadcast details of
> comercial offers. I believe "listing" service provider is enough. It
> should be up to the service providers to detail their offer on their own
> website (referenced via an hyper-link)

Good point. Need to follow Apache rules. My concern is about clarity, especially because there could be a potential appearance of a conflict of interest between Linagora and the James project.

My thought was that if the objectives and scope of James is made very clear, then any potential conflict of interest would be resolved, and also it would provide clarity to current and potential users of James.



>> (Just a thought, but maybe the “Distributed James” could be a commercial offering, rather than a community offering??)
> I strongly believe the "distributed server" is a differentiator for Apache James and helps satisfy some of the community needs.

Ok, that’s fair. Then I guess we’ll have to figure out a different boundary.



>> To resolve this thread, I will be satisfied once we have a clear statement of objectives regarding the 4.0 release.
> I do agree with the already discussed items (documentation & re-branding)

For my benefit, would you mind pasting a very short summary here so that I can understand more precisely what it is you are referring to?

By the way, I also want to resolve the “community” discussion because I think it is related, but perhaps that is too ambitious for this thread.


Cheers,
=David

Re: [Discussion] Road to 4.0

[Tellier Benoit <bt...@apache.org>]

Hi David!

Le 15/08/2020 à 08:20, David Leangen a écrit :
> Hello!
>
> I wanted to tie together a few ongoing threads and make a proposal for the road to a 4.0 release. 

I'm uncomfortable with the 4.0 release switch. It gives the impression
that "much" needs to change whilewhat we need is to re-brand, and better
document what exists.

> My assumption is that there currently is not roadmap to 4.0. If I am mistaken, please do let me know and I will adapt.
>
> [...]
>
> By the way, to coincide with the release, if the objectives are clear, perhaps there is a commercial organization (or individual) who would be willing to provide paid-for professional support starting from 4.0?
Linagora (company for which I am working) already proposes these
services. We already carried out several support / development contracts
for Apache James.
>  I think it would help complete the offering, and hopefully provide a commercial opportunity as well in a way that is beneficial to all, including the James community. We just need to ensure that the “contract” is very clear, and that we avoid any potential conflicts of interest. I think we should include this item in the scope of the discussion as well. 

I'm not sure an Apache project is entitled to broadcast details of
comercial offers. I believe "listing" service provider is enough. It
should be up to the service providers to detail their offer on their own
website (referenced via an hyper-link)

> (Just a thought, but maybe the “Distributed James” could be a commercial offering, rather than a community offering??)
I strongly believe the "distributed server" is a differentiator for
Apache James and helps satisfy some of the community needs.
> To resolve this thread, I will be satisfied once we have a clear statement of objectives regarding the 4.0 release.
I do agree with the already discussed items (documentation & re-branding)
>
> Cheers,
> =David
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org