You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by Jerry Malcolm <te...@malcolms.com> on 2022/04/26 04:41:11 UTC
iPhone/Android mail apps with JAMES
This is a critical problem. I really need some direction on this.
Please!!!....
I started this thread about two and half years ago when I moved my James
installation to AWS EC2. All of my clients lost all of their mail on
their mobile devices, even though all of the mail still shows up fine on
desktop Thunderbird. I tried all of the suggestions, but never could
get the older emails restored. Since then I have added another very
large customer account on a completely different james installation, and
I've upgraded to pretty much the latest GitHub version of James on my
original installation. For the new client, there was no migration. It
was completely start from scratch on a new domain. Yet across the
board, NONE of my clients on either system can reliably get their mail
on iPhone or Android. On my own iPhone, I have replaced the native
iPhone mail app with Edison mail app and later with Outlook mail app. I
get one or two emails downloaded periodically on each app on my various
email accounts, and some email accounts just start saying there's no
mail in the last month or two (and there's actually typically 10-20
emails each day on those accounts). Today, a client configured their
mail account on an iPhone. It immediately said 21 unread emails, and
the inbox then downloaded 3 emails. Open TBird on the same account.
There's there's the 21 emails.
I've tried to re-index the Lucene cache. But I'm still getting the same
error that I got in Oct 2019 about wrong parameters or something when I
try to do that. So I completely erased the Lucene cache folder. No
change. I'm running Spring. I tried to move to Guice a month or so ago,
but gave up when I couldn't fix the errors I was getting , so I moved
back to Spring. Is this whole iPhone problem due to me using the Spring
build? I have no problem trying again to get Guice up and running. But
I don't want to waste a week trying to get Guice up and find out the
same problem exists in the Guice build.
It's obvious that for some reason all of the mobile email apps (native,
Edison, Outlook) ask for email differently than Thunderbird. But I'm at
a loss to explain why JAMES refuses to send the mail that is there when
these same email client apps have no problem getting mail from other
mail servers.
It's hard for me to understand how every other JAMES user in the world
is working totally successfully with mobile phones when I have two
completely independent JAMES environments with a huge number of clients
on each and NOT ONE of them can get more than 5% of their real mail on
their phone.
If somebody can just educate me just a little on the differences between
how JAMES responds to IMAP queries on mobile devices vs. IMAP queries
from Thunderbird, and point me to the handling code, I'll start seeing
what I can do to resolve this. Or better yet, is there someway to
disable all of the Lucene or whatever caching completely and just make
JAMES think it's talking to Thunderbird instead of an iPhone?
I'm to the point that my major client is refusing to use the JAMES
environment for their company mail accounts since their phone email apps
are not receiving most of their critical corporate emails. To say the
least, they are NOT happy. Somebody PLEASE respond before my customer
fires me.
Please HELP!!!! Give me SOMETHING I can work with!!!! I just want to
get a conversation going.
Thanks
Jerry
On 11/8/2019 8:03 AM, Matthieu Baechler wrote:
> Hi Jerry,
>
> On Tue, 2019-10-29 at 15:12 -0500, Jerry Malcolm wrote:
>> Ok, I need an IMAP expert.... Below is a very brief trace of the
>> communications between iPhone mail and JAMES (3.4). I completely
>> deleted an account on my iPhone, then recreated it while in airplane
>> mode to make sure I didn't miss any communications in my trace. I
>> started the trace, exited airplane mode and let the iPhone do an
>> initial
>> sync with the account. The inbox folder in this account has over
>> 1000
>> emails going back to early 2019.
>>
>> I'm not an expert in IMAP. But it appears that the iPhone mail app
>> requests all of the emails 1:* (see line 812), but JAMES returns a
>> single id plus two ranges (line 813). But the total count JAMES
>> reports
>> is nowhere near the full 1000. Subsequently (line 822), iPhone
>> requests
>> the emails JAMES told it about in line 813. From what I can tell,
>> the
>> problem is in line 813. JAMES did not report all of the actual mail
>> that is in the INBOX, which is exactly what I'm seeing on the
>> phone.
>> Shouldn't JAMES return all 1000+ email ids that exist in INBOX? Or
>> am I
>> reading the IMAP trace incorrectly?
> You are right, if there's 1000+ messages, it should be returned by
> these calls.
>
>> Note that the only emails JAMES reports came in AFTER I migrated to
>> the
>> new server and to the James 3.4 from v3b5. It appears that there is
>> something different about pre-existing emails in the mailbox folder
>> that
>> is causing JAMES not to recognize them. But this problem ONLY exists
>> on
>> the iPhone mail app. Thunderbird gets all of the mail for the
>> inbox.
>> So is iPhone must be sending a slightly different request command
>> syntax
>> than other clients do (??) Just speculating....
>>
> The best thing to do is to get a trace from Thunderbird for the same
> scenario.
>
> My best guess is that one client trigger a search whereas the other is
> just triggering a full-scan.
>
> Are you using JPA? With with indexing backend?
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: iPhone/Android mail apps with JAMES
Posted by Jerry Malcolm <te...@malcolms.com>.
Benoit,
Thank you so much for the info. I'm going to start pulling together
some wireshark captures and logs and hopefully we can figure out what it
going wrong.
I really appreciate your help.
Jerry
On 4/26/2022 9:40 PM, btellier@linagora.com wrote:
> Hello Jerry,
>
> I confirm that different email applications uses the IMAP protocol in
> different ways.
>
> I confirm that some clients uses the IMAP SEARCH results upon
> resynchronisation, and thus that inacurate results could result in bad
> synchronisation.
>
> That being said, having traffic capture for bith apps would be
> valuable to diagnose what is going on. Things like wireshark of James
> debug logs (that include IMAP command).
>
> FYI I succeeded to reinde email on the JPA Guice based distribution on
> the 3.7.0 demo image.
>
> Regards,
>
> Benoit
>
> On 4/26/22 11:41, Jerry Malcolm wrote:
>> This is a critical problem. I really need some direction on this.
>> Please!!!....
>>
>> I started this thread about two and half years ago when I moved my
>> James installation to AWS EC2. All of my clients lost all of their
>> mail on their mobile devices, even though all of the mail still shows
>> up fine on desktop Thunderbird. I tried all of the suggestions, but
>> never could get the older emails restored. Since then I have added
>> another very large customer account on a completely different james
>> installation, and I've upgraded to pretty much the latest GitHub
>> version of James on my original installation. For the new client,
>> there was no migration. It was completely start from scratch on a
>> new domain. Yet across the board, NONE of my clients on either
>> system can reliably get their mail on iPhone or Android. On my own
>> iPhone, I have replaced the native iPhone mail app with Edison mail
>> app and later with Outlook mail app. I get one or two emails
>> downloaded periodically on each app on my various email accounts, and
>> some email accounts just start saying there's no mail in the last
>> month or two (and there's actually typically 10-20 emails each day on
>> those accounts). Today, a client configured their mail account on an
>> iPhone. It immediately said 21 unread emails, and the inbox then
>> downloaded 3 emails. Open TBird on the same account. There's
>> there's the 21 emails.
>>
>> I've tried to re-index the Lucene cache. But I'm still getting the
>> same error that I got in Oct 2019 about wrong parameters or something
>> when I try to do that. So I completely erased the Lucene cache
>> folder. No change. I'm running Spring. I tried to move to Guice a
>> month or so ago, but gave up when I couldn't fix the errors I was
>> getting , so I moved back to Spring. Is this whole iPhone problem
>> due to me using the Spring build? I have no problem trying again to
>> get Guice up and running. But I don't want to waste a week trying to
>> get Guice up and find out the same problem exists in the Guice build.
>>
>> It's obvious that for some reason all of the mobile email apps
>> (native, Edison, Outlook) ask for email differently than
>> Thunderbird. But I'm at a loss to explain why JAMES refuses to send
>> the mail that is there when these same email client apps have no
>> problem getting mail from other mail servers.
>>
>> It's hard for me to understand how every other JAMES user in the
>> world is working totally successfully with mobile phones when I have
>> two completely independent JAMES environments with a huge number of
>> clients on each and NOT ONE of them can get more than 5% of their
>> real mail on their phone.
>>
>> If somebody can just educate me just a little on the differences
>> between how JAMES responds to IMAP queries on mobile devices vs. IMAP
>> queries from Thunderbird, and point me to the handling code, I'll
>> start seeing what I can do to resolve this. Or better yet, is there
>> someway to disable all of the Lucene or whatever caching completely
>> and just make JAMES think it's talking to Thunderbird instead of an
>> iPhone?
>>
>> I'm to the point that my major client is refusing to use the JAMES
>> environment for their company mail accounts since their phone email
>> apps are not receiving most of their critical corporate emails. To
>> say the least, they are NOT happy. Somebody PLEASE respond before my
>> customer fires me.
>>
>> Please HELP!!!! Give me SOMETHING I can work with!!!! I just want to
>> get a conversation going.
>>
>> Thanks
>>
>> Jerry
>>
>>
>> On 11/8/2019 8:03 AM, Matthieu Baechler wrote:
>>> Hi Jerry,
>>>
>>> On Tue, 2019-10-29 at 15:12 -0500, Jerry Malcolm wrote:
>>>> Ok, I need an IMAP expert.... Below is a very brief trace of the
>>>> communications between iPhone mail and JAMES (3.4). I completely
>>>> deleted an account on my iPhone, then recreated it while in airplane
>>>> mode to make sure I didn't miss any communications in my trace. I
>>>> started the trace, exited airplane mode and let the iPhone do an
>>>> initial
>>>> sync with the account. The inbox folder in this account has over
>>>> 1000
>>>> emails going back to early 2019.
>>>>
>>>> I'm not an expert in IMAP. But it appears that the iPhone mail app
>>>> requests all of the emails 1:* (see line 812), but JAMES returns a
>>>> single id plus two ranges (line 813). But the total count JAMES
>>>> reports
>>>> is nowhere near the full 1000. Subsequently (line 822), iPhone
>>>> requests
>>>> the emails JAMES told it about in line 813. From what I can tell,
>>>> the
>>>> problem is in line 813. JAMES did not report all of the actual mail
>>>> that is in the INBOX, which is exactly what I'm seeing on the
>>>> phone.
>>>> Shouldn't JAMES return all 1000+ email ids that exist in INBOX? Or
>>>> am I
>>>> reading the IMAP trace incorrectly?
>>> You are right, if there's 1000+ messages, it should be returned by
>>> these calls.
>>>
>>>> Note that the only emails JAMES reports came in AFTER I migrated to
>>>> the
>>>> new server and to the James 3.4 from v3b5. It appears that there is
>>>> something different about pre-existing emails in the mailbox folder
>>>> that
>>>> is causing JAMES not to recognize them. But this problem ONLY exists
>>>> on
>>>> the iPhone mail app. Thunderbird gets all of the mail for the
>>>> inbox.
>>>> So is iPhone must be sending a slightly different request command
>>>> syntax
>>>> than other clients do (??) Just speculating....
>>>>
>>> The best thing to do is to get a trace from Thunderbird for the same
>>> scenario.
>>>
>>> My best guess is that one client trigger a search whereas the other is
>>> just triggering a full-scan.
>>>
>>> Are you using JPA? With with indexing backend?
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>> For additional commands, e-mail: server-dev-help@james.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: iPhone/Android mail apps with JAMES
Posted by "btellier@linagora.com" <bt...@linagora.com>.
Hello Jerry,
I confirm that different email applications uses the IMAP protocol in
different ways.
I confirm that some clients uses the IMAP SEARCH results upon
resynchronisation, and thus that inacurate results could result in bad
synchronisation.
That being said, having traffic capture for bith apps would be valuable
to diagnose what is going on. Things like wireshark of James debug logs
(that include IMAP command).
FYI I succeeded to reinde email on the JPA Guice based distribution on
the 3.7.0 demo image.
Regards,
Benoit
On 4/26/22 11:41, Jerry Malcolm wrote:
> This is a critical problem. I really need some direction on this.
> Please!!!....
>
> I started this thread about two and half years ago when I moved my
> James installation to AWS EC2. All of my clients lost all of their
> mail on their mobile devices, even though all of the mail still shows
> up fine on desktop Thunderbird. I tried all of the suggestions, but
> never could get the older emails restored. Since then I have added
> another very large customer account on a completely different james
> installation, and I've upgraded to pretty much the latest GitHub
> version of James on my original installation. For the new client,
> there was no migration. It was completely start from scratch on a new
> domain. Yet across the board, NONE of my clients on either system can
> reliably get their mail on iPhone or Android. On my own iPhone, I
> have replaced the native iPhone mail app with Edison mail app and
> later with Outlook mail app. I get one or two emails downloaded
> periodically on each app on my various email accounts, and some email
> accounts just start saying there's no mail in the last month or two
> (and there's actually typically 10-20 emails each day on those
> accounts). Today, a client configured their mail account on an
> iPhone. It immediately said 21 unread emails, and the inbox then
> downloaded 3 emails. Open TBird on the same account. There's there's
> the 21 emails.
>
> I've tried to re-index the Lucene cache. But I'm still getting the
> same error that I got in Oct 2019 about wrong parameters or something
> when I try to do that. So I completely erased the Lucene cache
> folder. No change. I'm running Spring. I tried to move to Guice a
> month or so ago, but gave up when I couldn't fix the errors I was
> getting , so I moved back to Spring. Is this whole iPhone problem due
> to me using the Spring build? I have no problem trying again to get
> Guice up and running. But I don't want to waste a week trying to get
> Guice up and find out the same problem exists in the Guice build.
>
> It's obvious that for some reason all of the mobile email apps
> (native, Edison, Outlook) ask for email differently than Thunderbird.
> But I'm at a loss to explain why JAMES refuses to send the mail that
> is there when these same email client apps have no problem getting
> mail from other mail servers.
>
> It's hard for me to understand how every other JAMES user in the world
> is working totally successfully with mobile phones when I have two
> completely independent JAMES environments with a huge number of
> clients on each and NOT ONE of them can get more than 5% of their real
> mail on their phone.
>
> If somebody can just educate me just a little on the differences
> between how JAMES responds to IMAP queries on mobile devices vs. IMAP
> queries from Thunderbird, and point me to the handling code, I'll
> start seeing what I can do to resolve this. Or better yet, is there
> someway to disable all of the Lucene or whatever caching completely
> and just make JAMES think it's talking to Thunderbird instead of an
> iPhone?
>
> I'm to the point that my major client is refusing to use the JAMES
> environment for their company mail accounts since their phone email
> apps are not receiving most of their critical corporate emails. To say
> the least, they are NOT happy. Somebody PLEASE respond before my
> customer fires me.
>
> Please HELP!!!! Give me SOMETHING I can work with!!!! I just want to
> get a conversation going.
>
> Thanks
>
> Jerry
>
>
> On 11/8/2019 8:03 AM, Matthieu Baechler wrote:
>> Hi Jerry,
>>
>> On Tue, 2019-10-29 at 15:12 -0500, Jerry Malcolm wrote:
>>> Ok, I need an IMAP expert.... Below is a very brief trace of the
>>> communications between iPhone mail and JAMES (3.4). I completely
>>> deleted an account on my iPhone, then recreated it while in airplane
>>> mode to make sure I didn't miss any communications in my trace. I
>>> started the trace, exited airplane mode and let the iPhone do an
>>> initial
>>> sync with the account. The inbox folder in this account has over
>>> 1000
>>> emails going back to early 2019.
>>>
>>> I'm not an expert in IMAP. But it appears that the iPhone mail app
>>> requests all of the emails 1:* (see line 812), but JAMES returns a
>>> single id plus two ranges (line 813). But the total count JAMES
>>> reports
>>> is nowhere near the full 1000. Subsequently (line 822), iPhone
>>> requests
>>> the emails JAMES told it about in line 813. From what I can tell,
>>> the
>>> problem is in line 813. JAMES did not report all of the actual mail
>>> that is in the INBOX, which is exactly what I'm seeing on the
>>> phone.
>>> Shouldn't JAMES return all 1000+ email ids that exist in INBOX? Or
>>> am I
>>> reading the IMAP trace incorrectly?
>> You are right, if there's 1000+ messages, it should be returned by
>> these calls.
>>
>>> Note that the only emails JAMES reports came in AFTER I migrated to
>>> the
>>> new server and to the James 3.4 from v3b5. It appears that there is
>>> something different about pre-existing emails in the mailbox folder
>>> that
>>> is causing JAMES not to recognize them. But this problem ONLY exists
>>> on
>>> the iPhone mail app. Thunderbird gets all of the mail for the
>>> inbox.
>>> So is iPhone must be sending a slightly different request command
>>> syntax
>>> than other clients do (??) Just speculating....
>>>
>> The best thing to do is to get a trace from Thunderbird for the same
>> scenario.
>>
>> My best guess is that one client trigger a search whereas the other is
>> just triggering a full-scan.
>>
>> Are you using JPA? With with indexing backend?
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org