[GitHub] [incubator-ponymail-foal] sbp opened a new pull request #20: Make Archiver.archive_message compatible with Mailman3

[GitBox <gi...@apache.org>]

sbp opened a new pull request #20:
URL: https://github.com/apache/incubator-ponymail-foal/pull/20


   The signature of `Archiver.archive_message` is incompatible with Mailman3. This PR fixes that.
   
   Mailman3 archives messages by sending two arguments, `(mlist, msg)`, to the archiver. But the `Archiver.archive_message` signature is `(args, mlist, msg, raw_message)`, where `args` is used to provide flags for custom functionality, and `raw_message` provides accurate input for the DKIM-ID generator.
   
   It is not possible to use the alternative stdin mode with `archiver.py` in Mailman3 because Mailman3 also injects messages internally in addition to receiving them via LMTP. The injected messages would not be sent to `archiver.py`.
   
   In this PR, the `args` flags become keyword arguments and the `raw_message` argument is removed. The functionality originally provided by `raw_message` is instead provided by the non-standard Mailman3 attribute `msg.original_message` if present, and `msg.to_bytes()` if not.
   
   The following two trivial patches to Mailman3 add support for `msg.original_message`:
   
   ```patch
   --- a/app/inject.py
   +++ b/app/inject.py
   @@ -58,6 +58,7 @@
        if 'date' not in msg:
            msg['Date'] = formatdate(localtime=True)
        msg.original_size = len(msg.as_string())
   +    msg.original_content = bytes(msg.as_string(), "utf-8")
        msgdata = dict(
            listid=mlist.list_id,
            original_size=msg.original_size,
   ```
   
   ```patch
   --- a/runners/lmtp.py
   +++ b/runners/lmtp.py
   @@ -154,6 +154,7 @@
            if msg.defects:
                return ERR_501
            msg.original_size = len(envelope.content)
   +        msg.envelope_content = envelope.content
            add_message_hash(msg)
            msg['X-MailFrom'] = envelope.mail_from
            # RFC 2033 requires us to return a status code for every recipient.
   ```
   
   This PR also fixes a potential issue with detecting the existence of the auditlog index. If access to the auditlog existence check endpoint is unauthorised, the code now correctly skips creating an auditlog entry instead of throwing an exception.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-ponymail-foal] sbp closed pull request #20: Make Archiver.archive_message compatible with Mailman3

[GitBox <gi...@apache.org>]

sbp closed pull request #20:
URL: https://github.com/apache/incubator-ponymail-foal/pull/20


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-ponymail-foal] sbp commented on pull request #20: Make Archiver.archive_message compatible with Mailman3

[GitBox <gi...@apache.org>]

sbp commented on pull request #20:
URL: https://github.com/apache/incubator-ponymail-foal/pull/20#issuecomment-835372590


   The `original_content` attribute preserves the message delivered from the MTA to Mailman3, before Mailman3 potentially modifies the message. This is necessary because the potential modifications made by Mailman3 could affect the DKIM-ID. But it may be possible to implement deduplication in a better way than DKIM-ID, by manipulating the message database *post hoc*, either in `archiver.py` or using a background task or both. This would mean that `original_content` would not be necessary, at least for Mailman3: `msg.to_bytes()` or `bytes(msg.as_string(), "utf-8")` could be used instead.
   
   I'm going to close the present PR to make way for a new one using this alternative approach.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org