Re: [incubator-ponymail-foal] branch master updated: Add a disclaimer for Foal, and a small roadmap

[sebb <se...@gmail.com>]

On Mon, 24 Aug 2020 at 19:41, <hu...@apache.org> wrote:
>
> This is an automated email from the ASF dual-hosted git repository.
>
> humbedooh pushed a commit to branch master
> in repository https://gitbox.apache.org/repos/asf/incubator-ponymail-foal.git
>
>
> The following commit(s) were added to refs/heads/master by this push:
>      new 92eb3f9  Add a disclaimer for Foal, and a small roadmap
> 92eb3f9 is described below
>
> commit 92eb3f9698b65f5c94bb90e361c6adf52c495901
> Author: Daniel Gruno <hu...@apache.org>
> AuthorDate: Mon Aug 24 20:41:46 2020 +0200
>
>     Add a disclaimer for Foal, and a small roadmap
>
>     This disclaimer generally speaking applies to all PM setups pre-foal as well, but might as well put it out there.
> ---
>  README.md | 20 ++++++++++++++++++++
>  1 file changed, 20 insertions(+)
>
> diff --git a/README.md b/README.md
> index c7f2843..6260fe0 100644
> --- a/README.md
> +++ b/README.md
> @@ -5,3 +5,23 @@ _Next generation suite of services and tools for Apache Pony Mail_
>
>  This repository aims to contain the next generation of Apache Pony Mail,
>  a pure python version of Apache Pony Mail with support for ElasticSearch 6.x and above.
> +
> +
> +### Roadmap
> +Work is underway on the following items:
> +
> +- Improved archiver and import tools   **[DONE]**
> +- New UI for the end user              **[DONE]**
> +- 100% python backend, no mod_lua required. **[COMING SOON]**
> +
> +
> +### Migration disclaimer:
> +_While compatible with the original Pony Mail, this will not be a drop-in replacement.
> +Migration of the old database will be needed, and there may be edge cases where

They are not just edge cases, see below.

> +re-imaging a database from scratch (re-importing from mbox files as opposed to migrating
> +your ElasticSearch database) will not produce the exact same permalinks as before, if
> +the input options don't match exactly. For lists with a List-ID header present in all
> +emails (and no override used), this should not be the case for any permalinks, and
> +they should be the same.

It is not true that re-importing mails from mbox files will produce
the same Permalinks (depending on the generator that was used).
For example, if a generator used the archived-at date as part of the
hash, then this will have been added at time of import, and will only
be present in the ES database.

There are various other issues with all the existing generators that
mean it is impossible to guarantee recreation of the original
Permalinks even if there are no list name changes or lid overrides.

For example, early versions of the code used local TZ rather than UTC.
If a site originally did not use html2text, introducing it changes the
text that is hashed for some emails.
If a site changed generators at any point (or updated the code in a
way that changed the output) then it won't be possible to guarantee
the same links.

There are currently no test cases to show that re-importing works at all.
Even if it does work now, the existing generators depend on the parsed
message which may change with a new release.

>  If you have an existing Pony Mail database, migrating it
> +with the provided migration tool is highly recommended if you wish to preserve old
> +permalinks._

It should be possible to make that work.