Fetchmail (was Re: Is this a valid From: address?)

[tobe <to...@swipnet.se>]

Steve Brewin wrote:

>
>I do not think this is a viable approach as...
>1) If fetchmail bypasses the SMTP spool we lose all the benefits of James's
>delivery chain.
>2) Without going through mailet processing, we cannot determine "the correct
>message stores".
>
I agree with this, because the mailet capabilities are probably one of 
the reasons for ever wanting to use fetchmail anyway.

My main reason for it was to get some spam "weeding". However that 
reason has mostly disappeared now because of Mozillas excellent Junk 
mail handling (IMO, the client is the only place for content-based 
spam-handling, and that is the only type that seems to be successful now)

>
>It seems to me that to progress we need to agree that...
>
>1) fetchmail will only deliver locally
>2) Further consideration is required regarding James' policy for dealing
>with invalid SMTP envelopes however they are received. This should be at a
>different time in a different thread.
>3) As an >interim< measure, in my fetchmail enhancements, I will modify the
>code so that a From: header with a missing domain part is made valid by
>appending the local host name (didn't Danny suggest that a while back)?
>
>  
>

I am a bit concerned about "will only deliver locally". How do you 
ensure that?

The first problem is that the account you fetch from is usually never a 
local account, and should not be set up as a handled domain else you 
will get all sorts of other problems.

I found that mail with multiple recipients caused problems when used 
with fetchpop, either generating duplicates if I forwarded all to my 
local account or risked being resent outwards if I tried to be smarter.

So there seems to be a lot of reasons for not injecting fetched mail 
into James's delivery chain. And yet the main reason for using fetchmail 
is to be able to use mailets on it.

Let me try to invent some "user stories" for fetchmail:

1 - As an email user I want to collect the mail from all my separate 
accounts and have them put in my local account.
Note: Personally, it works better for me to keep my accounts separate, 
like putting on different caps for "public internet mail", "personal 
friends", "business". My mail client handles that well for me.

2 - As an email user I want James to help clean out the spam.
Note: My mail client handles this better.

3 - As a user of a joint e-mail account I would like to use a mailet 
that that delivers mail to the separate owners, e.g. mail for "John 
<fa...@isp.com>" to john@localnetwork, mail for "Mary <fa...@isp.com>" 
to mary@localnetwork and other mail for "family@isp.com" to both.

4 - variations of 3 for business, e.g. with content-based routing

5 - As an owner of a B2B e-business setup I fetch mail from the incoming 
account and use mailets to distribute the content of e-mails to 
appropriate applications, via message queues, EJBs, files, SMTP, 
whatever. I would also handle logging of messages and protocols like 
ebXML Reliable Messaging in custom mailets. For ebXML there might be a 
corresponding mailet for outgoing messages if SMTP is used internally, 
but most probably I have another process that gets its input from the 
message queue, etc. Therefore I would usually in this scenario set up a 
James process to only handle fetched mail.
Note: In this scenario one could relay the incoming messages to my James 
server and handle them directly in the SMTP spool, but there might be 
reasons for avoiding it.

I am sure there are other scenarios I have not listed here, please add 
to the list.

My conclusion so far is that fetched mail should go into a separate 
spool. The handling is completely different from what I want to do with 
the SMTP-spool and there are inherent dangers from injecting fetched 
mail into the SMTP spool.

There are probably scenarios where I actually do want to forward or 
reply to a fetched mail (via SMTP), but this is, IMO, the exception 
rather than the rule and could probably be solved by a special mailet.

Regards,
tobe


---------------------------------------------------------------------
To unsubscribe, e-mail: james-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-user-help@jakarta.apache.org

RE: Fetchmail (was Re: Is this a valid From: address?)

[Steve Brewin <sb...@synsys.com>]

> -----Original Message-----
> From: tobe [mailto:torbjorn.gannholm@swipnet.se]
> Sent: 17 June 2003 00:15
> To: James Users List
> Subject: Re: Fetchmail (was Re: Is this a valid From: address?)
>
>
> Steve,
>
> you seem to have thought a lot about this.

Light relief from the day job.

> As long as I can handle the fetched mail fairly easily
> without risking
> something being sent out again by mistake, I don't get lots of
> duplicates,

The version of fetchmail in CVS HEAD will do this. I didn't write this
version, I think Danny did(?), so thank him.

> and I can still send mail to other people in the fetched
> domain, great!

That is a simple mod. to config.xml. After the check for a local account,
check if the recipient host is in the fetched domain...

         <!-- Is the recipient is for a local account, deliver it
locally -->
         <mailet match="RecipientIsLocal" class="LocalDelivery"/>

	   <!-- Process 'local' servernames which are not entirely local. -->
	   <!-- eg: joe@myisp.com IS local, paula@myisp.com IS NOT local. -->
	   <!-- If the host is matched, use remote delivery. -->
	   <mailet match="HostIs=myisp.com" class="ToProcessor">
	      <processor> remoteTransport </processor>
         </mailet>

         <!-- If the host is handled by this server and it did not get -->
         <!-- locally delivered, this is an invalid recipient -->
         <mailet match="HostIsLocal" class="ToProcessor">
            <processor>error</processor>
            <notice> Host is local but recipient not found </notice>
         </mailet>

Hope this helps.

-- Steve




---------------------------------------------------------------------
To unsubscribe, e-mail: james-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-user-help@jakarta.apache.org

Re: Fetchmail (was Re: Is this a valid From: address?)

[tobe <to...@swipnet.se>]

Steve,

you seem to have thought a lot about this.

As long as I can handle the fetched mail fairly easily without risking 
something being sent out again by mistake, I don't get lots of 
duplicates, and I can still send mail to other people in the fetched 
domain, great!

Thanks for your work on this.

/tobe



---------------------------------------------------------------------
To unsubscribe, e-mail: james-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-user-help@jakarta.apache.org

RE: Fetchmail (was Re: Is this a valid From: address?)

[Steve Brewin <sb...@synsys.com>]

> -----Original Message-----
> From: tobe [mailto:torbjorn.gannholm@swipnet.se]
> Sent: 15 June 2003 12:27
> To: James Users List
> Subject: Fetchmail (was Re: Is this a valid From: address?)
>
> Steve Brewin wrote:
>
>
Tobe,

> I am a bit concerned about "will only deliver locally". How do you
> ensure that?

My musings have moved on to a new version of fetchmail that has been
submitted, but not as yet accepted.

In the submitted code, for each of the mail filter conditions, such as
rejecting remote mail, rejection is now optional and the status of the
rejected mail on the POP3 server can be deleted, marked as seen or left
asis.

The main reasons for rejecting mail at the fetchmail stage are to avoid the
overhead of fetching mail that is simply going to be junked and the extra
work of defining matcher/mailets to weed out such fetched mail.

If it is decided to accept mail for which a filter condition applies, this
is indicated by an X-Header added to the mail which describes the filter
match. This allows the injected message to be detected by a matcher and
processed in a mailet as required.

Tere are many possible fetch 'stories'. The intent is to provide the
flexibility to accomodate as many as possible.

> The first problem is that the account you fetch from is
> usually never a
> local account, and should not be set up as a handled domain else you
> will get all sorts of other problems.

Depends on the story. There are a several where *not* defining the domain
locally causes all sorts of other problems. As described above, the
rejection of remote mail is now optional and can be done at the fetch stage
or during matcher/mailet processing.

> I found that mail with multiple recipients caused problems when used
> with fetchpop, either generating duplicates if I forwarded all to my
> local account or risked being resent outwards if I tried to
> be smarter.

A key difference between fetchpop and fetchmail is that the latter injects
mail for a sole intended recipient. This avoids the problems you had with
fetchpop. There are circumstances when it is not possible to determine the
sole intended recipient, in this case the released version of fetchmail uses
a default recipient. The submitted version adds the option of leaving it on
the POP3 server. The latter accomodates scenarios in which the default
recipient can be determined by refetching with a different POP3 account.
This is explained in the comments within the submitted config.xml.

> So there seems to be a lot of reasons for not injecting fetched mail
> into James's delivery chain. And yet the main reason for
> using fetchmail
> is to be able to use mailets on it.
>
> Let me try to invent some "user stories" for fetchmail:

The beauty of James is that it is extremely flexible and can accomodate
these and a huge number of 'stories'.

I can't really see the motivation for delivering fetched mail to another
spool in any of your 'stories'. If you want to do something special with all
fetched mail, use a matcher that checks for the X-fetched-from header, such
as "fetchedfrom". All mail injected into the spool by fetchpop and fetchmail
has this header appended for just this purpose.

-- Steve



---------------------------------------------------------------------
To unsubscribe, e-mail: james-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-user-help@jakarta.apache.org