You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ponymail.apache.org by hu...@apache.org on 2022/03/16 20:50:29 UTC
[incubator-ponymail-foal] branch master updated: clear up some limitations
This is an automated email from the ASF dual-hosted git repository.
humbedooh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-ponymail-foal.git
The following commit(s) were added to refs/heads/master by this push:
new 3a1aff5 clear up some limitations
3a1aff5 is described below
commit 3a1aff5777c2abe7970642ad1c7c7cb6240b9a6d
Author: Daniel Gruno <hu...@apache.org>
AuthorDate: Wed Mar 16 21:49:37 2022 +0100
clear up some limitations
---
README.md | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/README.md b/README.md
index 7668d5e..cfd140f 100644
--- a/README.md
+++ b/README.md
@@ -47,16 +47,17 @@ Migration of the old database is required, and most of the older ID generators h
been dropped in favor of collision-secure generators._
### Known Limitations:
-* Not currently suitable as a list archive, as not all emails received for a list are stored.
* If an email is re-imported or archived, entries are currently replaced.
This can result in loss of attributes such as alternate Permalinks and deleted status.
-* The migration tool drops Permalinks if two existing entries point to a sufficiently similar email
-* The migration tool does not fix up badly-parsed message-ids etc
-* There is no longer a 1-to-1 relationship between mbox and source entries.
- This can result in orphan source entries, with implications for privacy redaction.
* emails are filed according to the Date: header, rather than arrival time.
This can cause emails to appear in the wrong month or year, or even be future-dated.
* Whilst the underlying database can handle any number of emails in a month,
- the UI and much of the API cannot, and are not scalable.
+ the UI and much of the API does not scale well beyond around 10,000 emails per month per list.
+
+#### Known limitations when migrating from older Pony Mail instances:
+* The migration tool can drop Permalinks if two existing entries point to a sufficiently similar email
+* The migration tool does not fix up badly-parsed message-ids etc
+* There is no longer a 1-to-1 relationship between mbox and source entries.
+ This can result in orphan source entries, with implications for privacy redaction.
* Header parsing is stricter than before; in particular some unusual Message-IDs are not handled correctly.
This affects use of Foal as a replacement for Apache mod_mbox mail archives.
Re: [incubator-ponymail-foal] branch master updated: clear up some limitations
Posted by sebb <se...@gmail.com>.
On Fri, 18 Mar 2022 at 13:25, Daniel Gruno <hu...@apache.org> wrote:
>
> On 18/03/2022 14.20, sebb wrote:
> >
> > Go to https://lists.apache.org/list.html?announce@apache.org
> > Click through to last page.
> > Open an email and click on Thread or Source.
> > Click back
> >
>
> How about we make it open the source/thread links in a new tab if it's
> opened within the main UI?
>
That would do, I suppose, but it would be better if it remembered the
current page.
Re: [incubator-ponymail-foal] branch master updated: clear up some limitations
Posted by Daniel Gruno <hu...@apache.org>.
On 18/03/2022 14.20, sebb wrote:
>
> Go to https://lists.apache.org/list.html?announce@apache.org
> Click through to last page.
> Open an email and click on Thread or Source.
> Click back
>
How about we make it open the source/thread links in a new tab if it's
opened within the main UI?
Re: [incubator-ponymail-foal] branch master updated: clear up some limitations
Posted by sebb <se...@gmail.com>.
On Fri, 18 Mar 2022 at 14:25, Rich Bowen <rb...@rcbowen.com> wrote:
>
> On Fri, Mar 18, 2022, 14:20 sebb <se...@gmail.com> wrote:
>
> >
> > > Could we add a 1....N page drop-down or something? Probably easy to do
> > yeah.
> >
> > But at present that is a limitation.
> >
>
> What a strange perspective. The software also doesn't display flashing
> rainbows. Is that a limitation?
>
> The lack of a feature that it *could* have isn't a limitation. You could
> presumably list thousands of such "limitations"
Browsing is an example of "should", not "could"
I regard being able to browse emails as a basic feature of Ponymail.
Indeed the blurb says:
"Apache Pony Mail (Incubating) is a web-based mail archive browser
built to scale to millions of archived messages with hundreds of
requests per second. It allows you to browse, search, and interact
with mailing lists including creating replies to mailing list
threads."
Pony Mail does not promise rainbows, but it does promise browsing.
> >
Re: [incubator-ponymail-foal] branch master updated: clear up some limitations
Posted by Rich Bowen <rb...@rcbowen.com>.
On Fri, Mar 18, 2022, 14:20 sebb <se...@gmail.com> wrote:
>
> > Could we add a 1....N page drop-down or something? Probably easy to do
> yeah.
>
> But at present that is a limitation.
>
What a strange perspective. The software also doesn't display flashing
rainbows. Is that a limitation?
The lack of a feature that it *could* have isn't a limitation. You could
presumably list thousands of such "limitations"
>
Re: [incubator-ponymail-foal] branch master updated: clear up some limitations
Posted by sebb <se...@gmail.com>.
On Fri, 18 Mar 2022 at 13:16, Daniel Gruno <hu...@apache.org> wrote:
>
> On 18/03/2022 14.08, sebb wrote:
>
> >
> > The GUI shows about 25 emails per page on a desktop; far fewer than
> > that on a mobile device.
> > The only option for browsing all emails is to do so one page at a time.
> > It will take ages to get to the last page of a list than has more than
> > a few hundreds of emails.
> > Furthermore, if you do manage to get to the last page of such a list,
> > if you display a thread or source of an email, on return you will be
> > taken back to the first page.
> >
> > Compare this with mod_mbox, which not only allows direct navigation to
> > individual pages, it also allows them to be sorted by date or sender.
> > It's reasonably easy to get to any part of a month by estimating the
> > page number and adjusting as necessary.
> >
> > This could potentially be fixed, but at present it is a limitation
> > compared with other solutions.
> >
>
> First off, it's not a competition here. It's great that mod_mbox allows
> to sort by sender, but that doesn't mean our UI is not useable.
>
> Could we add a 1....N page drop-down or something? Probably easy to do yeah.
But at present that is a limitation.
> I don't understand the "taken back to the first page" thing, emails open
> in the interface and are closed again with the escape key or by clicking
> on the subject. There is no navigating away unless you open the email in
> a new window...
Go to https://lists.apache.org/list.html?announce@apache.org
Click through to last page.
Open an email and click on Thread or Source.
Click back
>
> As for sorting by sender, if someone wants that as a feature, they can
> create a ticket and we can look at how to potentially implement that,
> keeping mobile friendliness in mind.
Re: [incubator-ponymail-foal] branch master updated: clear up some limitations
Posted by Daniel Gruno <hu...@apache.org>.
On 18/03/2022 14.08, sebb wrote:
>
> The GUI shows about 25 emails per page on a desktop; far fewer than
> that on a mobile device.
> The only option for browsing all emails is to do so one page at a time.
> It will take ages to get to the last page of a list than has more than
> a few hundreds of emails.
> Furthermore, if you do manage to get to the last page of such a list,
> if you display a thread or source of an email, on return you will be
> taken back to the first page.
>
> Compare this with mod_mbox, which not only allows direct navigation to
> individual pages, it also allows them to be sorted by date or sender.
> It's reasonably easy to get to any part of a month by estimating the
> page number and adjusting as necessary.
>
> This could potentially be fixed, but at present it is a limitation
> compared with other solutions.
>
First off, it's not a competition here. It's great that mod_mbox allows
to sort by sender, but that doesn't mean our UI is not useable.
Could we add a 1....N page drop-down or something? Probably easy to do yeah.
I don't understand the "taken back to the first page" thing, emails open
in the interface and are closed again with the escape key or by clicking
on the subject. There is no navigating away unless you open the email in
a new window...
As for sorting by sender, if someone wants that as a feature, they can
create a ticket and we can look at how to potentially implement that,
keeping mobile friendliness in mind.
Re: [incubator-ponymail-foal] branch master updated: clear up some limitations
Posted by sebb <se...@gmail.com>.
On Fri, 18 Mar 2022 at 12:36, Daniel Gruno <hu...@apache.org> wrote:
>
> On 17/03/2022 00.17, sebb wrote:
> > On Wed, 16 Mar 2022 at 20:50, <hu...@apache.org> wrote:
> >>
> >> This is an automated email from the ASF dual-hosted git repository.
> >>
> >> humbedooh pushed a commit to branch master
> >> in repository https://gitbox.apache.org/repos/asf/incubator-ponymail-foal.git
> >>
> >>
> >> The following commit(s) were added to refs/heads/master by this push:
> >> new 3a1aff5 clear up some limitations
> >> 3a1aff5 is described below
> >>
> >> commit 3a1aff5777c2abe7970642ad1c7c7cb6240b9a6d
> >> Author: Daniel Gruno <hu...@apache.org>
> >> AuthorDate: Wed Mar 16 21:49:37 2022 +0100
> >>
> >> clear up some limitations
> >
> > -1
> >
> > Some of the limitations have not been addressed.
> >
> >> ---
> >> README.md | 13 +++++++------
> >> 1 file changed, 7 insertions(+), 6 deletions(-)
> >>
> >> diff --git a/README.md b/README.md
> >> index 7668d5e..cfd140f 100644
> >> --- a/README.md
> >> +++ b/README.md
> >> @@ -47,16 +47,17 @@ Migration of the old database is required, and most of the older ID generators h
> >> been dropped in favor of collision-secure generators._
> >>
> >> ### Known Limitations:
> >> -* Not currently suitable as a list archive, as not all emails received for a list are stored.
> >
> > That limitation is still true, AFAICT.
>
> If you have this assumption, please provide some evidence. I find it
> very discouraging to both devs and users if we write "don't use this
> software".
We need to be clear on the limitations of the software.
If they don't apply to the user, then fine, they can use it.
> >
> >> * If an email is re-imported or archived, entries are currently replaced.
> >> This can result in loss of attributes such as alternate Permalinks and deleted status.
> >> -* The migration tool drops Permalinks if two existing entries point to a sufficiently similar email
> >> -* The migration tool does not fix up badly-parsed message-ids etc
> >> -* There is no longer a 1-to-1 relationship between mbox and source entries.
> >> - This can result in orphan source entries, with implications for privacy redaction.
> >> * emails are filed according to the Date: header, rather than arrival time.
> >> This can cause emails to appear in the wrong month or year, or even be future-dated.
> >> * Whilst the underlying database can handle any number of emails in a month,
> >> - the UI and much of the API cannot, and are not scalable.
> >> + the UI and much of the API does not scale well beyond around 10,000 emails per month per list.
> >
> > The UI is not really usable for browsing anything other than a few
> > hundred emails per month.
> > Above a certain limit it breaks down completely, as not all mails are returned.
>
> Above 10,000 emails perhaps, or whatever is configured.
If the limit is set too high, it is likely to break clients or at
least cause serious memory issues, as all emails have to be returned.
I fixed that limitation for downloads by using stream output, but the
UI still has to grab everything.
This is fixable, but at present it is a limitation, which is why it
needs to be documented.
> I personally see no issues with loading up a month with 6,000 emails in
> it. What do you mean when you say it's not useable above a few hundred
> emails?
The GUI shows about 25 emails per page on a desktop; far fewer than
that on a mobile device.
The only option for browsing all emails is to do so one page at a time.
It will take ages to get to the last page of a list than has more than
a few hundreds of emails.
Furthermore, if you do manage to get to the last page of such a list,
if you display a thread or source of an email, on return you will be
taken back to the first page.
Compare this with mod_mbox, which not only allows direct navigation to
individual pages, it also allows them to be sorted by date or sender.
It's reasonably easy to get to any part of a month by estimating the
page number and adjusting as necessary.
This could potentially be fixed, but at present it is a limitation
compared with other solutions.
> >
> >> +
> >> +#### Known limitations when migrating from older Pony Mail instances:
> >> +* The migration tool can drop Permalinks if two existing entries point to a sufficiently similar email
> >> +* The migration tool does not fix up badly-parsed message-ids etc
> >> +* There is no longer a 1-to-1 relationship between mbox and source entries.
> >> + This can result in orphan source entries, with implications for privacy redaction.
> >> * Header parsing is stricter than before; in particular some unusual Message-IDs are not handled correctly.
> >> This affects use of Foal as a replacement for Apache mod_mbox mail archives.
>
Re: [incubator-ponymail-foal] branch master updated: clear up some limitations
Posted by Daniel Gruno <hu...@apache.org>.
On 18/03/2022 14.16, sebb wrote:
>
> If an email is duplicated in transit (this does happen sometimes),
> subscribers to the list will generally see both copies, depending on
> their email client.
> However the duplicates do not appear in Ponymail.
That's a feature, not a bug. Many email clients do the same thing, there
is no need to show duplicates if they are an exact match.
Why would you need to see the duplicate?
Re: [incubator-ponymail-foal] branch master updated: clear up some limitations
Posted by sebb <se...@gmail.com>.
On Fri, 18 Mar 2022 at 13:07, Daniel Gruno <hu...@apache.org> wrote:
>
> On 18/03/2022 14.01, sebb wrote:
> > On Fri, 18 Mar 2022 at 12:58, Rich Bowen <rb...@rcbowen.com> wrote:
> >>
> >>> . I find it
> >>> very discouraging to both devs and users if we write "don't use this
> >>> software".
> >>>
> >>
> >> +1
> >>
> >> Point to a list of open/known issues for sure. But putting "don't use this"
> >> in the readme has long term effects on user trust. So many times over the
> >> years I have heard people cite long-fixed issues as reasons for avoiding a
> >> particular package because it was made part of primary messaging.
> >>
> >> This kind of blanket "don't use this" messaging is weird and damaging.
> >
> > The point is that all software has limitations, and we should be open
> > about the known issues.
> > Users can then decide whether or not the limitations affect them.
> >
>
> It's not whether there are limitations or not, it's a matter of wording.
> What was written was a very wide opinionated statement that this
> software doesn't work well and should not be used in production, which
> is patently false and VERY discouraging to read.
It does not say that.
It just says that it is not suitable as an archival solution.
> If there are *specific* cases where archiving doesn't work (I don't know
> of any specifics here), put that in a document, maybe make a
> LIMITATIONS.md file and link to that. But putting in the main readme
> that people shouldn't use our software is just rude.
If an email is duplicated in transit (this does happen sometimes),
subscribers to the list will generally see both copies, depending on
their email client.
However the duplicates do not appear in Ponymail.
Re: [incubator-ponymail-foal] branch master updated: clear up some limitations
Posted by Daniel Gruno <hu...@apache.org>.
On 18/03/2022 14.01, sebb wrote:
> On Fri, 18 Mar 2022 at 12:58, Rich Bowen <rb...@rcbowen.com> wrote:
>>
>>> . I find it
>>> very discouraging to both devs and users if we write "don't use this
>>> software".
>>>
>>
>> +1
>>
>> Point to a list of open/known issues for sure. But putting "don't use this"
>> in the readme has long term effects on user trust. So many times over the
>> years I have heard people cite long-fixed issues as reasons for avoiding a
>> particular package because it was made part of primary messaging.
>>
>> This kind of blanket "don't use this" messaging is weird and damaging.
>
> The point is that all software has limitations, and we should be open
> about the known issues.
> Users can then decide whether or not the limitations affect them.
>
It's not whether there are limitations or not, it's a matter of wording.
What was written was a very wide opinionated statement that this
software doesn't work well and should not be used in production, which
is patently false and VERY discouraging to read.
If there are *specific* cases where archiving doesn't work (I don't know
of any specifics here), put that in a document, maybe make a
LIMITATIONS.md file and link to that. But putting in the main readme
that people shouldn't use our software is just rude.
We strive for close-to-perfection, sure, but it's not a dichotomy where
software is either good or bad and nowhere in between, and we shouldn't
act as if that is the case.
>>>
Re: [incubator-ponymail-foal] branch master updated: clear up some limitations
Posted by sebb <se...@gmail.com>.
On Fri, 18 Mar 2022 at 12:58, Rich Bowen <rb...@rcbowen.com> wrote:
>
> > . I find it
> > very discouraging to both devs and users if we write "don't use this
> > software".
> >
>
> +1
>
> Point to a list of open/known issues for sure. But putting "don't use this"
> in the readme has long term effects on user trust. So many times over the
> years I have heard people cite long-fixed issues as reasons for avoiding a
> particular package because it was made part of primary messaging.
>
> This kind of blanket "don't use this" messaging is weird and damaging.
The point is that all software has limitations, and we should be open
about the known issues.
Users can then decide whether or not the limitations affect them.
> >
Re: [incubator-ponymail-foal] branch master updated: clear up some limitations
Posted by Rich Bowen <rb...@rcbowen.com>.
> . I find it
> very discouraging to both devs and users if we write "don't use this
> software".
>
+1
Point to a list of open/known issues for sure. But putting "don't use this"
in the readme has long term effects on user trust. So many times over the
years I have heard people cite long-fixed issues as reasons for avoiding a
particular package because it was made part of primary messaging.
This kind of blanket "don't use this" messaging is weird and damaging.
>
Re: [incubator-ponymail-foal] branch master updated: clear up some limitations
Posted by Daniel Gruno <hu...@apache.org>.
On 17/03/2022 00.17, sebb wrote:
> On Wed, 16 Mar 2022 at 20:50, <hu...@apache.org> wrote:
>>
>> This is an automated email from the ASF dual-hosted git repository.
>>
>> humbedooh pushed a commit to branch master
>> in repository https://gitbox.apache.org/repos/asf/incubator-ponymail-foal.git
>>
>>
>> The following commit(s) were added to refs/heads/master by this push:
>> new 3a1aff5 clear up some limitations
>> 3a1aff5 is described below
>>
>> commit 3a1aff5777c2abe7970642ad1c7c7cb6240b9a6d
>> Author: Daniel Gruno <hu...@apache.org>
>> AuthorDate: Wed Mar 16 21:49:37 2022 +0100
>>
>> clear up some limitations
>
> -1
>
> Some of the limitations have not been addressed.
>
>> ---
>> README.md | 13 +++++++------
>> 1 file changed, 7 insertions(+), 6 deletions(-)
>>
>> diff --git a/README.md b/README.md
>> index 7668d5e..cfd140f 100644
>> --- a/README.md
>> +++ b/README.md
>> @@ -47,16 +47,17 @@ Migration of the old database is required, and most of the older ID generators h
>> been dropped in favor of collision-secure generators._
>>
>> ### Known Limitations:
>> -* Not currently suitable as a list archive, as not all emails received for a list are stored.
>
> That limitation is still true, AFAICT.
If you have this assumption, please provide some evidence. I find it
very discouraging to both devs and users if we write "don't use this
software".
>
>> * If an email is re-imported or archived, entries are currently replaced.
>> This can result in loss of attributes such as alternate Permalinks and deleted status.
>> -* The migration tool drops Permalinks if two existing entries point to a sufficiently similar email
>> -* The migration tool does not fix up badly-parsed message-ids etc
>> -* There is no longer a 1-to-1 relationship between mbox and source entries.
>> - This can result in orphan source entries, with implications for privacy redaction.
>> * emails are filed according to the Date: header, rather than arrival time.
>> This can cause emails to appear in the wrong month or year, or even be future-dated.
>> * Whilst the underlying database can handle any number of emails in a month,
>> - the UI and much of the API cannot, and are not scalable.
>> + the UI and much of the API does not scale well beyond around 10,000 emails per month per list.
>
> The UI is not really usable for browsing anything other than a few
> hundred emails per month.
> Above a certain limit it breaks down completely, as not all mails are returned.
Above 10,000 emails perhaps, or whatever is configured.
I personally see no issues with loading up a month with 6,000 emails in
it. What do you mean when you say it's not useable above a few hundred
emails?
>
>> +
>> +#### Known limitations when migrating from older Pony Mail instances:
>> +* The migration tool can drop Permalinks if two existing entries point to a sufficiently similar email
>> +* The migration tool does not fix up badly-parsed message-ids etc
>> +* There is no longer a 1-to-1 relationship between mbox and source entries.
>> + This can result in orphan source entries, with implications for privacy redaction.
>> * Header parsing is stricter than before; in particular some unusual Message-IDs are not handled correctly.
>> This affects use of Foal as a replacement for Apache mod_mbox mail archives.
Re: [incubator-ponymail-foal] branch master updated: clear up some limitations
Posted by sebb <se...@gmail.com>.
On Wed, 16 Mar 2022 at 20:50, <hu...@apache.org> wrote:
>
> This is an automated email from the ASF dual-hosted git repository.
>
> humbedooh pushed a commit to branch master
> in repository https://gitbox.apache.org/repos/asf/incubator-ponymail-foal.git
>
>
> The following commit(s) were added to refs/heads/master by this push:
> new 3a1aff5 clear up some limitations
> 3a1aff5 is described below
>
> commit 3a1aff5777c2abe7970642ad1c7c7cb6240b9a6d
> Author: Daniel Gruno <hu...@apache.org>
> AuthorDate: Wed Mar 16 21:49:37 2022 +0100
>
> clear up some limitations
-1
Some of the limitations have not been addressed.
> ---
> README.md | 13 +++++++------
> 1 file changed, 7 insertions(+), 6 deletions(-)
>
> diff --git a/README.md b/README.md
> index 7668d5e..cfd140f 100644
> --- a/README.md
> +++ b/README.md
> @@ -47,16 +47,17 @@ Migration of the old database is required, and most of the older ID generators h
> been dropped in favor of collision-secure generators._
>
> ### Known Limitations:
> -* Not currently suitable as a list archive, as not all emails received for a list are stored.
That limitation is still true, AFAICT.
> * If an email is re-imported or archived, entries are currently replaced.
> This can result in loss of attributes such as alternate Permalinks and deleted status.
> -* The migration tool drops Permalinks if two existing entries point to a sufficiently similar email
> -* The migration tool does not fix up badly-parsed message-ids etc
> -* There is no longer a 1-to-1 relationship between mbox and source entries.
> - This can result in orphan source entries, with implications for privacy redaction.
> * emails are filed according to the Date: header, rather than arrival time.
> This can cause emails to appear in the wrong month or year, or even be future-dated.
> * Whilst the underlying database can handle any number of emails in a month,
> - the UI and much of the API cannot, and are not scalable.
> + the UI and much of the API does not scale well beyond around 10,000 emails per month per list.
The UI is not really usable for browsing anything other than a few
hundred emails per month.
Above a certain limit it breaks down completely, as not all mails are returned.
> +
> +#### Known limitations when migrating from older Pony Mail instances:
> +* The migration tool can drop Permalinks if two existing entries point to a sufficiently similar email
> +* The migration tool does not fix up badly-parsed message-ids etc
> +* There is no longer a 1-to-1 relationship between mbox and source entries.
> + This can result in orphan source entries, with implications for privacy redaction.
> * Header parsing is stricter than before; in particular some unusual Message-IDs are not handled correctly.
> This affects use of Foal as a replacement for Apache mod_mbox mail archives.