You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@james.apache.org by GitBox <gi...@apache.org> on 2020/12/10 06:01:59 UTC
[GitHub] [james-project] chibenwa opened a new pull request #278: JAMES-3431 Optional DSN support for SMTP server
chibenwa opened a new pull request #278:
URL: https://github.com/apache/james-project/pull/278
I decided to write DSN support as a completely optional extension to the James SMTP server.
Why? Because RFC-3461 compliance needs the mail processing to be correctly configured (and handle DSN bounces correctly). Which we cannot assume. Hence, for the sake of compliance, this should be optional.
For the sake of the exercise, I decided to challenge leveraging SMTP hooks to implement new ESMTP extensions. Needless to say, I had bad surprises.
I needed to modify "core" SMTP server code in order to:
- Pass new supported ESMTP extensions to EHLO command
- Cary over RCPT parameters to the RcptHooks.
- Rcpt hooks needs to announce which RCPT parameters they can handle. Rcpt command should not reject those.
Default methods where used in order NOT to introduce braking changes.
To enable DSN in JAMES, add the following handler to smtpserver.xml :
```
<handler class="org.apache.james.smtpserver.dsn.DSNHeloHook"/>
<handler class="org.apache.james.smtpserver.dsn.DSNMailParameterHook"/>
<handler class="org.apache.james.smtpserver.dsn.DSNRcptParameterHook"/>
<handler class="org.apache.james.smtpserver.dsn.DSNMessageHook"/>
```
Not only does this contribution enables opt-in DSN support,
But also it makes it easier for contributors to write their own ESMTP extensions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on a change in pull request #278: JAMES-3431 Optional DSN support for SMTP server
Posted by GitBox <gi...@apache.org>.
chibenwa commented on a change in pull request #278:
URL: https://github.com/apache/james-project/pull/278#discussion_r542161780
##########
File path: server/protocols/protocols-smtp/src/main/java/org/apache/james/smtpserver/dsn/DSNHeloHook.java
##########
@@ -0,0 +1,40 @@
+/****************************************************************
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information *
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the *
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the *
+ * specific language governing permissions and limitations *
+ * under the License. *
+ ****************************************************************/
+
+package org.apache.james.smtpserver.dsn;
+
+import java.util.Set;
+
+import org.apache.james.protocols.smtp.SMTPSession;
+import org.apache.james.protocols.smtp.hook.HeloHook;
+import org.apache.james.protocols.smtp.hook.HookResult;
+
+import com.google.common.collect.ImmutableSet;
+
+public class DSNHeloHook implements HeloHook {
Review comment:
:+1: to change this one.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] Arsnael commented on a change in pull request #278: JAMES-3431 Optional DSN support for SMTP server
Posted by GitBox <gi...@apache.org>.
Arsnael commented on a change in pull request #278:
URL: https://github.com/apache/james-project/pull/278#discussion_r542106707
##########
File path: protocols/smtp/src/main/java/org/apache/james/protocols/smtp/hook/HeloHook.java
##########
@@ -19,12 +19,19 @@
package org.apache.james.protocols.smtp.hook;
+import java.util.Set;
+
import org.apache.james.protocols.smtp.SMTPSession;
+import com.google.common.collect.ImmutableSet;
+
/**
* Implement this interfaces to hook in the HELO Command
*/
public interface HeloHook extends Hook {
Review comment:
Not your fault but I think there is a typo in this hook's name... should be `EhloHook`?
##########
File path: server/protocols/protocols-smtp/src/main/java/org/apache/james/smtpserver/dsn/DSNMailParameterHook.java
##########
@@ -0,0 +1,63 @@
+/****************************************************************
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information *
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the *
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the *
+ * specific language governing permissions and limitations *
+ * under the License. *
+ ****************************************************************/
+
+package org.apache.james.smtpserver.dsn;
+
+import static org.apache.james.protocols.api.ProtocolSession.State.Transaction;
+import static org.apache.mailet.DsnParameters.ENVID_PARAMETER;
+import static org.apache.mailet.DsnParameters.RET_PARAMETER;
+
+import java.util.Optional;
+
+import org.apache.james.protocols.api.ProtocolSession;
+import org.apache.james.protocols.smtp.SMTPSession;
+import org.apache.james.protocols.smtp.hook.HookResult;
+import org.apache.james.protocols.smtp.hook.MailParametersHook;
+import org.apache.mailet.DsnParameters;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class DSNMailParameterHook implements MailParametersHook {
+ private static final Logger LOGGER = LoggerFactory.getLogger(DSNMailParameterHook.class);
+
+ public static final ProtocolSession.AttachmentKey<DsnParameters.Ret> DSN_RET = ProtocolSession.AttachmentKey.of("DSN_RET", DsnParameters.Ret.class);
+ public static final ProtocolSession.AttachmentKey<DsnParameters.EnvId> DSN_ENVID = ProtocolSession.AttachmentKey.of("DSN_ENVID", DsnParameters.EnvId.class);
+
+ @Override
+ public HookResult doMailParameter(SMTPSession session, String paramName, String paramValue) {
+ if (paramName.equals(RET_PARAMETER)) {
+ DsnParameters.Ret.parse(paramValue)
+ .or(() -> {
+ LOGGER.debug("Invalid DSN RET value: {}", paramValue);
+ return Optional.empty();
+ })
+ .ifPresent(ret -> session.setAttachment(DSN_RET, ret, Transaction));
+ }
+ if (paramName.equals(ENVID_PARAMETER)) {
+ DsnParameters.EnvId envId = DsnParameters.EnvId.of(paramValue);
+ session.setAttachment(DSN_ENVID, envId, Transaction);
+ }
+ return HookResult.DECLINED;
Review comment:
why always returning DECLINED status?
##########
File path: server/protocols/protocols-smtp/src/main/java/org/apache/james/smtpserver/dsn/DSNHeloHook.java
##########
@@ -0,0 +1,40 @@
+/****************************************************************
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information *
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the *
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the *
+ * specific language governing permissions and limitations *
+ * under the License. *
+ ****************************************************************/
+
+package org.apache.james.smtpserver.dsn;
+
+import java.util.Set;
+
+import org.apache.james.protocols.smtp.SMTPSession;
+import org.apache.james.protocols.smtp.hook.HeloHook;
+import org.apache.james.protocols.smtp.hook.HookResult;
+
+import com.google.common.collect.ImmutableSet;
+
+public class DSNHeloHook implements HeloHook {
Review comment:
here too... Helo -> Ehlo ?
##########
File path: server/protocols/protocols-smtp/src/test/java/org/apache/james/smtpserver/DSNTest.java
##########
@@ -0,0 +1,354 @@
+/****************************************************************
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information *
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the *
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the *
+ * specific language governing permissions and limitations *
+ * under the License. *
+ ****************************************************************/
+package org.apache.james.smtpserver;
+
+import static org.apache.mailet.DsnParameters.Notify.DELAY;
+import static org.apache.mailet.DsnParameters.Notify.FAILURE;
+import static org.apache.mailet.DsnParameters.Notify.SUCCESS;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+import java.net.InetSocketAddress;
+import java.util.EnumSet;
+
+import org.apache.commons.configuration2.BaseHierarchicalConfiguration;
+import org.apache.commons.net.smtp.SMTPClient;
+import org.apache.james.core.Domain;
+import org.apache.james.core.MailAddress;
+import org.apache.james.dnsservice.api.DNSService;
+import org.apache.james.dnsservice.api.InMemoryDNSService;
+import org.apache.james.domainlist.api.DomainList;
+import org.apache.james.domainlist.memory.MemoryDomainList;
+import org.apache.james.filesystem.api.FileSystem;
+import org.apache.james.mailrepository.api.MailRepositoryStore;
+import org.apache.james.mailrepository.api.Protocol;
+import org.apache.james.mailrepository.memory.MailRepositoryStoreConfiguration;
+import org.apache.james.mailrepository.memory.MemoryMailRepository;
+import org.apache.james.mailrepository.memory.MemoryMailRepositoryStore;
+import org.apache.james.mailrepository.memory.MemoryMailRepositoryUrlStore;
+import org.apache.james.mailrepository.memory.SimpleMailRepositoryLoader;
+import org.apache.james.metrics.api.Metric;
+import org.apache.james.metrics.api.MetricFactory;
+import org.apache.james.metrics.tests.RecordingMetricFactory;
+import org.apache.james.protocols.api.utils.ProtocolServerUtils;
+import org.apache.james.protocols.lib.mock.MockProtocolHandlerLoader;
+import org.apache.james.queue.api.MailQueueFactory;
+import org.apache.james.queue.api.RawMailQueueItemDecoratorFactory;
+import org.apache.james.queue.memory.MemoryMailQueueFactory;
+import org.apache.james.rrt.api.AliasReverseResolver;
+import org.apache.james.rrt.api.CanSendFrom;
+import org.apache.james.rrt.api.RecipientRewriteTable;
+import org.apache.james.rrt.api.RecipientRewriteTableConfiguration;
+import org.apache.james.rrt.lib.AliasReverseResolverImpl;
+import org.apache.james.rrt.lib.CanSendFromImpl;
+import org.apache.james.rrt.memory.MemoryRecipientRewriteTable;
+import org.apache.james.server.core.configuration.Configuration;
+import org.apache.james.server.core.configuration.FileConfigurationProvider;
+import org.apache.james.server.core.filesystem.FileSystemImpl;
+import org.apache.james.smtpserver.netty.SMTPServer;
+import org.apache.james.smtpserver.netty.SmtpMetricsImpl;
+import org.apache.james.user.api.UsersRepository;
+import org.apache.james.user.memory.MemoryUsersRepository;
+import org.apache.mailet.DsnParameters;
+import org.apache.mailet.Mail;
+import org.assertj.core.api.SoftAssertions;
+import org.jboss.netty.util.HashedWheelTimer;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+
+import com.google.common.collect.ImmutableList;
+import com.google.inject.TypeLiteral;
+
+public class DSNTest {
+ public static final String LOCAL_DOMAIN = "example.local";
+
+ protected HashedWheelTimer hashedWheelTimer;
+ protected MemoryDomainList domainList;
+ protected MemoryUsersRepository usersRepository;
+ protected SMTPServerTest.AlterableDNSServer dnsServer;
+ protected MemoryMailRepositoryStore mailRepositoryStore;
+ protected FileSystemImpl fileSystem;
+ protected Configuration configuration;
+ protected MockProtocolHandlerLoader chain;
+ protected MemoryMailQueueFactory queueFactory;
+ protected MemoryMailQueueFactory.MemoryCacheableMailQueue queue;
+
+ private SMTPServer smtpServer;
+
+ @Before
+ public void setUp() throws Exception {
+ domainList = new MemoryDomainList(new InMemoryDNSService()
+ .registerMxRecord(Domain.LOCALHOST.asString(), "127.0.0.1")
+ .registerMxRecord(LOCAL_DOMAIN, "127.0.0.1")
+ .registerMxRecord("examplebis.local", "127.0.0.1")
+ .registerMxRecord("127.0.0.1", "127.0.0.1"));
+ domainList.setAutoDetect(false);
+ domainList.setAutoDetectIP(false);
Review comment:
https://github.com/linagora/james-project/pull/4115 just got merged, rebase using this syntax for DomainList
##########
File path: server/protocols/protocols-smtp/src/test/java/org/apache/james/smtpserver/DSNTest.java
##########
@@ -79,6 +83,8 @@
public class DSNTest {
public static final String LOCAL_DOMAIN = "example.local";
+ public static final Username BOB = Username.of("bob@localhost");
Review comment:
this commit title misses the ticket number
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on a change in pull request #278: JAMES-3431 Optional DSN support for SMTP server
Posted by GitBox <gi...@apache.org>.
chibenwa commented on a change in pull request #278:
URL: https://github.com/apache/james-project/pull/278#discussion_r542161675
##########
File path: server/protocols/protocols-smtp/src/main/java/org/apache/james/smtpserver/dsn/DSNMailParameterHook.java
##########
@@ -0,0 +1,63 @@
+/****************************************************************
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information *
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the *
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the *
+ * specific language governing permissions and limitations *
+ * under the License. *
+ ****************************************************************/
+
+package org.apache.james.smtpserver.dsn;
+
+import static org.apache.james.protocols.api.ProtocolSession.State.Transaction;
+import static org.apache.mailet.DsnParameters.ENVID_PARAMETER;
+import static org.apache.mailet.DsnParameters.RET_PARAMETER;
+
+import java.util.Optional;
+
+import org.apache.james.protocols.api.ProtocolSession;
+import org.apache.james.protocols.smtp.SMTPSession;
+import org.apache.james.protocols.smtp.hook.HookResult;
+import org.apache.james.protocols.smtp.hook.MailParametersHook;
+import org.apache.mailet.DsnParameters;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class DSNMailParameterHook implements MailParametersHook {
+ private static final Logger LOGGER = LoggerFactory.getLogger(DSNMailParameterHook.class);
+
+ public static final ProtocolSession.AttachmentKey<DsnParameters.Ret> DSN_RET = ProtocolSession.AttachmentKey.of("DSN_RET", DsnParameters.Ret.class);
+ public static final ProtocolSession.AttachmentKey<DsnParameters.EnvId> DSN_ENVID = ProtocolSession.AttachmentKey.of("DSN_ENVID", DsnParameters.EnvId.class);
+
+ @Override
+ public HookResult doMailParameter(SMTPSession session, String paramName, String paramValue) {
+ if (paramName.equals(RET_PARAMETER)) {
+ DsnParameters.Ret.parse(paramValue)
+ .or(() -> {
+ LOGGER.debug("Invalid DSN RET value: {}", paramValue);
+ return Optional.empty();
+ })
+ .ifPresent(ret -> session.setAttachment(DSN_RET, ret, Transaction));
+ }
+ if (paramName.equals(ENVID_PARAMETER)) {
+ DsnParameters.EnvId envId = DsnParameters.EnvId.of(paramValue);
+ session.setAttachment(DSN_ENVID, envId, Transaction);
+ }
+ return HookResult.DECLINED;
Review comment:
Declined = I'm not taking decisions on whether the hooks should accept or reject the command.
I agree the naming is ambiguous here (but same thing I want to limit breaking changes in this part of the code!)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on a change in pull request #278: JAMES-3431 Optional DSN support for SMTP server
Posted by GitBox <gi...@apache.org>.
chibenwa commented on a change in pull request #278:
URL: https://github.com/apache/james-project/pull/278#discussion_r540720313
##########
File path: protocols/smtp/src/main/java/org/apache/james/protocols/smtp/hook/HeloHook.java
##########
@@ -19,12 +19,19 @@
package org.apache.james.protocols.smtp.hook;
+import java.util.Set;
+
import org.apache.james.protocols.smtp.SMTPSession;
+import com.google.common.collect.ImmutableSet;
+
/**
* Implement this interfaces to hook in the HELO Command
*/
public interface HeloHook extends Hook {
+ default Set<String> implementedEsmtpFeatures() {
Review comment:
We need Java Doc here!
##########
File path: protocols/smtp/src/main/java/org/apache/james/protocols/smtp/hook/RcptHook.java
##########
@@ -51,4 +62,8 @@ default HookResult doRcpt(SMTPSession session, MaybeSender sender, MailAddress r
return doRcpt(session, sender.asOptional().orElse(null), rcpt);
}
+ default HookResult doRcpt(SMTPSession session, MaybeSender sender, MailAddress rcpt, Map<String, String> parameters) {
Review comment:
We need Java doc here!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on a change in pull request #278: JAMES-3431 Optional DSN support for SMTP server
Posted by GitBox <gi...@apache.org>.
chibenwa commented on a change in pull request #278:
URL: https://github.com/apache/james-project/pull/278#discussion_r542161132
##########
File path: protocols/smtp/src/main/java/org/apache/james/protocols/smtp/hook/HeloHook.java
##########
@@ -19,12 +19,19 @@
package org.apache.james.protocols.smtp.hook;
+import java.util.Set;
+
import org.apache.james.protocols.smtp.SMTPSession;
+import com.google.common.collect.ImmutableSet;
+
/**
* Implement this interfaces to hook in the HELO Command
*/
public interface HeloHook extends Hook {
Review comment:
I do not want to change that as it will cause breaking changes for SMTP hooks implementors.
But you are right, there is a typo.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #278: JAMES-3431 Optional DSN support for SMTP server
Posted by GitBox <gi...@apache.org>.
chibenwa commented on pull request #278:
URL: https://github.com/apache/james-project/pull/278#issuecomment-745810897
Merged
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa closed pull request #278: JAMES-3431 Optional DSN support for SMTP server
Posted by GitBox <gi...@apache.org>.
chibenwa closed pull request #278:
URL: https://github.com/apache/james-project/pull/278
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org