You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@james.apache.org by GitBox <gi...@apache.org> on 2021/11/01 03:59:46 UTC
[GitHub] [james-project] Arsnael commented on a change in pull request #718: JAMES-3665 - Add Kubernetes support document
Arsnael commented on a change in pull request #718:
URL: https://github.com/apache/james-project/pull/718#discussion_r739934232
##########
File path: server/apps/distributed-app/docs/modules/ROOT/pages/helm/checklist.adoc
##########
@@ -0,0 +1,43 @@
+# Checklist
+
+There is some things that you must have before being able to run James on your Kubernetes Cluster.
+
+## MUST have
+
+Those are absolutely necessary if you want at least James to start without crashing.
+
+- [ ] Setup RabbitMQ and create James user in it.
+- [ ] Create a Kubernetes namespace for James deployment.
+- [ ] A `distributed` version of James image.
+- [ ] Have an appropriate email domain (Contact your Domain Service Provider)
+- [ ] Cassandra nodes deployed.
+- [ ] An ObjectStorage compatible with S3 APIs.
+- [ ] Configure JWT keys, keystores, SPF and DKIM. See document [here](https://james.apache.org/howTo/)
+- [ ] Adding the proper `mailetcontainer.xml` file with other conf files in James Helm package
+
+With those you should be able to startup James for first tests without having it crashing. However it's still not enough to have it fully functional.
+
+## Should have
+
+- [ ] Cassandra nodes deployed.
Review comment:
That's said in the previous list already... (line 13)
##########
File path: server/apps/distributed-app/docs/modules/ROOT/pages/helm/logsMetrics.adoc
##########
@@ -0,0 +1,27 @@
+== Logs and Metrics
+
+=== Logs
+
+James exposes its logs on the console. Then a fluentbit instance scraps
+and collect them, before being indexed and used by Kibana, with nice
Review comment:
```suggestion
and collects them, before being indexed and used by Kibana, with nice
```
##########
File path: server/apps/distributed-app/docs/modules/ROOT/pages/helm/values.adoc
##########
@@ -0,0 +1,178 @@
+== Values for Helm James template
+
+Helm James has many visible values to configure before being able to use
+it correctly. Values are injected into the deployment template as
+environment variables. An example with default values can be found in
+the `+values.yaml+` file.
+
+This document presents a quick description of each value field.
+
+=== DNS values
+[cols="<,<,<",options="header",]
+|===
+|Field |Default value |Description
+|emailDomain |mail.example.com |The domain name for your tenant’s email
+addresses
+
+|jmapUrl |jmap.example.com |The url for the ingress jmap endpoint
+
+|adminUrl |admin.example.com |The url for the ingress admin endpoint
+
+|smtpHostname |smtp.example.com |The SMTP Ehlo hostname
+|===
+
+=== James values
+[cols="<,<,<",options="header",]
+|===
+|Field |Default value |Description
+|replicaJmapInstanceCount |1 |The number of pods we deploy for James
+JMAP instance in the cluster
+
+|replicaImapSmtpInstanceCount |1 |The number of pods we deploy for James
+IMAP/SMTP instance in the cluster
+
+|image |apache/james:distributed-3.7.0 |The James image used for
+deployment (Cassandra - RabbitMQ - LDAP distributed James flavor)
+|===
+
+=== James environment values
+[cols="<,<,<",options="header",]
+|===
+|Field |Default value |Description
+|jamesAdminID |admin@mail.example.com |The James admin account (mainly
+used for imapsync)
+
+|jamesCassandraKeyspace |sandbox_james |The keyspace used for Cassandra
+James data
+
+|jamesCassandraCacheKeyspace |sandbox_james_cache |The keyspace used for
+the Cassandra cache
+
+|jamesEsMailboxIndex |mailbox_v1 |The index used for mailbox in
+Elasticsearch. The default value should be enough, but in case you had
+issues and needed to create a new index, this will allow you to point
+James to it
+
+|jamesEsClusterName |change-me |Name of the cluster
+
+|jamesEsHostScheme |http |Only http or https are accepted
+
+|jamesEsSslValidationStrategy |ignore |Choosing the SSL check strategy
+when using https scheme: `+default+` (use the default SSL TrustStore of
+the system), `+ignore+` (Ignore SSL Validation check (not recommended)),
+`+override+` (override the SSL Context to use a custom TrustStore
+containing ES server’s certificate)
+
+|jamesEsHostNameVerifier |default |Configure Elasticsearch rest client
+to use host name verifier during SSL handshake: `+default+` (using the
+default hostname verifier provided by apache http client),
+`+accept_any_hostname+` (accept any host (not recommended))
+
+|jamesRabbitHost |change-me |The host url for RabbitMQ used by James for
+mail processing
+
+|jamesMessageSize |25M |The default maximum size for each email is 25MB,
+Must be a positive integer, optionally with a unit: B, K, M, G.
+
+|jamesDkimSignSmtp |dkimselector |SMTP signature for DKIM, also called
+DKIM Selector
+
+|jamesDkimSignDomain |mail.example.com |Domain signature for DKIM
+
+|cassandraReplicationFactor |3 |The replication factor for Cassandra
+configuration
+
+|remoteDeliveryGateway |1.2.3.4 |The tenant’s gateway for outgoing SMTP
+mails from James
+
+|remoteDeliveryGatewayPort |587 |The port of the gateway
+
+|remoteDeliveryHelo |outgoing.mail.example.com |The SMTP Ehlo hostname
+for remote delivery
+
+|jamesHELOMessage |change-me |The SMTP Ehlo message
+
+|jamesRelayAllowedIps |127.0.0.1 |The allowed IPs for message relay
+inside the lcuster (usually locally from James pods and from Postfix)
Review comment:
```suggestion
inside the cluster (usually locally from James pods and from Postfix)
```
##########
File path: server/apps/distributed-app/docs/modules/ROOT/pages/run-kubernetes.adoc
##########
@@ -0,0 +1,183 @@
+= Distributed James Server — Run with Kubernetes
+:navtitle: Run with Kubernetes
+
+== Deploy Apache James on Kubernetes with Helm chart.
+
+This chart bootstraps a James mail server on a link:http://kubernetes.io[Kubernetes]
+cluster using the link:https://helm.sh[Helm] package manager.
+
+== Before Starting
+
+
+Before runnning the Helm chart, James should have particularly access to the following external products:
+
+Apache Cassandra::
+- You can deploy Cassandra on Kubernetes with a link:https://github.com/bitnami/charts/tree/master/bitnami/cassandra[Cassandra chart] for testing purpose. However we would recommend running Cassandra on VirtualMachine for better stablilty.
+ElastiSearch::
+- Starting from `james:distributed-3.6.0` we have been using `elasticsearch-oss-7.10.2` as default version. Please check ElasticSearch link:https://www.elastic.co/downloads/past-releases#elasticsearch-oss[Release Page] for download.
+RabbitMQ::
+- link:https://github.com/rabbitmq/cluster-operator[RabbitMQ Cluster Operator] is a custom Kubernetes resource designed for the lifecycle (creation, upgrade, graceful shutdown) of a RabbitMQ cluster. Using Quickstart setup is enough for James deployment.
+
+
+== Configuration
+
+=== Helm Configuration
+
+The James helm package take parameters that will configure and determine the behavior of James.
Review comment:
```suggestion
The James helm package takes parameters that will configure and determine the behavior of James.
```
##########
File path: server/apps/distributed-app/docs/modules/ROOT/pages/helm/logsMetrics.adoc
##########
@@ -0,0 +1,27 @@
+== Logs and Metrics
+
+=== Logs
+
+James exposes its logs on the console. Then a fluentbit instance scraps
+and collect them, before being indexed and used by Kibana, with nice
+search possibilities.
+
+The way James exposes its logs on the console is defined in the
+`logback.xml` configuration file. James currently exposes logs in JSON
+format using Jackson Json formatter.
+
+=== Metrics
+
+James exposes its metrics over an HTTP Webadmin endpoint, something like
+`https://james-web-admin/metrics`. Those metrics can be scraped by
+Prometheus before being visualized on Grafana.
+
+This Helm chart support exposes James metrics through Prometheus
Review comment:
```suggestion
This Helm chart exposes James metrics through Prometheus
```
##########
File path: server/apps/distributed-app/helm-chart/james/configs/smtpserver.xml
##########
@@ -0,0 +1,68 @@
+<?xml version="1.0"?>
+
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+ -->
+
+<smtpservers>
+ <smtpserver enabled="${env:JAMES_IMAP_SMTP_ENABLED}">
Review comment:
Should we add the starttls port here?
##########
File path: server/apps/distributed-app/docs/modules/ROOT/pages/helm/secrets.adoc
##########
@@ -0,0 +1,67 @@
+== Secrets for James Helm template
+
+Helm James has many secret values to configure before being able to use
+it correctly. Those secrets are linked to the template through their
+declaration in `james-secrets.yaml`. A sample with dumb secrets can be
+found in the `secrets.sample.yaml` file.
+
+This document presents a quick description of each secret field.
+
+For more information about encrypt/descrypt secret with Helm, see:
+https://github.com/jkroepke/helm-secrets
+
+=== James secret values
+
+[cols="<,<,<",options="header",]
+|===
+|Field |Mandatory or optional |Description
+|jwtPublicKey |Mandatory |The JWT public key used for JWT tokens.
+
+|jwtPrivateKey |Mandatory |The JWT private key used for JWT tokens.
+
+|cassandraUser |Mandatory |The user for Cassandra access
+
+|cassandraPassword |Mandatory |The password fort Cassandra access
+
+|esUser |Optional |The user for Elasticsearch access
+
+|esPassword |Optional |The password for Elasticsearch access
+
+|keystorePassword |Optional |The password of the keystore used for
+generating JWT keystore
+
+|dkimPrivateKey |Optional |The private key for the generated DKIM
+
+|keystoreJWTinBase64 |Optional |The JWT keystore (encoded in base64)
+
+|s3.endpoint |Mandatory |The S3 endpoint. ie:
+http://s3-us-east-1.amazonaws.com
+
+|s3.region |Mandatory |The region of the S3 object storage.
+
+|s3.accessKeyId |Mandatory |The access key to the S3 object storage
+
+|s3.secretKey |Mandatory |The secret key to the S3 object storage
+
+|s3.bucketNameSuffix |Mandatory |All buckets in S3 related to james
+storage start by ``james-''. This allows to have a suffix for each
+tenant
+|===
+
+=== jwtPublicKey
+
+Use `openssl` to generate self-signed certificate and convert to Java
+Key Store JKS. You can find an example
+https://loneidealist.medium.com/generating-signing-certificates-with-openssl-and-converting-to-java-key-store-jks-3c8185dbf8fe[here]
+
+=== DKIM
+
+To generate DKIM, you can follow our guide:
+https://james.apache.org/howTo/dkim.html[howTo Dkim]
+
+=== External secrets
+
+Some declared secrets in `james-deployment.yaml` are external to the
+James configuration. They are often being created by other component
Review comment:
```suggestion
James configuration. They are often being created by other components
```
##########
File path: server/apps/distributed-app/docs/modules/ROOT/pages/run-kubernetes.adoc
##########
@@ -0,0 +1,183 @@
+= Distributed James Server — Run with Kubernetes
+:navtitle: Run with Kubernetes
+
+== Deploy Apache James on Kubernetes with Helm chart.
+
+This chart bootstraps a James mail server on a link:http://kubernetes.io[Kubernetes]
+cluster using the link:https://helm.sh[Helm] package manager.
+
+== Before Starting
+
+
+Before runnning the Helm chart, James should have particularly access to the following external products:
+
+Apache Cassandra::
+- You can deploy Cassandra on Kubernetes with a link:https://github.com/bitnami/charts/tree/master/bitnami/cassandra[Cassandra chart] for testing purpose. However we would recommend running Cassandra on VirtualMachine for better stablilty.
+ElastiSearch::
+- Starting from `james:distributed-3.6.0` we have been using `elasticsearch-oss-7.10.2` as default version. Please check ElasticSearch link:https://www.elastic.co/downloads/past-releases#elasticsearch-oss[Release Page] for download.
+RabbitMQ::
+- link:https://github.com/rabbitmq/cluster-operator[RabbitMQ Cluster Operator] is a custom Kubernetes resource designed for the lifecycle (creation, upgrade, graceful shutdown) of a RabbitMQ cluster. Using Quickstart setup is enough for James deployment.
+
+
+== Configuration
+
+=== Helm Configuration
+
+The James helm package take parameters that will configure and determine the behavior of James.
+There are 2 configuration files:
+
+* One contains global, non-confidential, configuration: `values.yaml`
+* The other one contains secrets: `secrets.yaml`. Values are encrypted with a key stored generated by link:https://github.com/mozilla/sops[sops].
+
+You need to pass these 2 files as parameters to the James Helm package.
+To verify the coherence and avoid errors, the packages can validate the input.
+
+This package contains a default `values.yaml` file, but that you should override with proper configuration for your deployment (`conf.yaml`).
+The file `secrets.sample.yaml` will show you the possible values of the secrets file. To generate the `secrets.yaml` file,
+you will need a plugin for Helm called link:https://github.com/jkroepke/helm-secrets)[helm-scecrets].
+
+Usually, you will save those configurations in different repositories, per deployment.
+
+Have a look at `values.md` and `secrets.md` in our Chart link:https://github.com/apache/james-project/tree/master/server/apps/distributed-app/helm-chart/james/doc/[documents] to have more information on the configuration James is using.
+
+You can find our Helm chart for James in link:https://github.com/apache/james-project/tree/master/server/apps/distributed-app/helm-chart[here]
+
+=== James Configuration
+
+The configuration files proper to James are stored in the Chart `james/configs` folder. They are being stored in a `ConfigMap` and then being mounted in the James pod at /root/conf path.
+
+If you need to change a conf in James, this is were you need to look at.
+
+For now all configuration files are the same for every deployment, except for `mailetcontainer.xml` that has been excluded (see section below).
+
+You can get the template `mailetcontainer.xml` in our link:https://github.com/apache/james-project/blob/master/server/apps/distributed-app/sample-configuration/[sample-configuration] folder.
+
+[NOTE]
+===============================
+The `mailetcontainer.xml` is the configuration file in James responsible of mail processing and delivery. It can differ greatly from one deployment to another. For that reason, we have excluded it from `james/configs` folder for the moment. You need to draft your own version of `mailetcontainer.xml` and add it to `james/configs` before start deploying Helm chart, as we can't actually mount a separate file into the same mounting point as the one sued by the `ConfigMap` on `/root/conf` path.
Review comment:
Yeah as said in one of my comments above, I'm not sure we should need that for community doc. Maybe should put a default file in the configmap with maybe a note here that it should be modified regarding needs of the deployment?
##########
File path: server/apps/distributed-app/helm-chart/james/configs/imapserver.xml
##########
@@ -0,0 +1,44 @@
+<?xml version="1.0"?>
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+<!--
+http://james.apache.org/server/config-imap4.html
+-->
+<imapservers>
+ <imapserver enabled="${env:JAMES_IMAP_SMTP_ENABLED}">
Review comment:
Shouldn't we put the starttls port here? (even if I know we don't want it, it's for the community here, that option should be available?)
##########
File path: server/apps/distributed-app/helm-chart/james/configs/blob.properties
##########
@@ -0,0 +1,81 @@
+# ============================================= BlobStore Implementation ==================================
Review comment:
And it's because of the reason that people download the whole thing and start testing that I think we should add the mailetcontainer.xml file too here :)
##########
File path: server/apps/distributed-app/docs/modules/ROOT/pages/helm/checklist.adoc
##########
@@ -0,0 +1,43 @@
+# Checklist
+
+There is some things that you must have before being able to run James on your Kubernetes Cluster.
+
+## MUST have
+
+Those are absolutely necessary if you want at least James to start without crashing.
+
+- [ ] Setup RabbitMQ and create James user in it.
+- [ ] Create a Kubernetes namespace for James deployment.
+- [ ] A `distributed` version of James image.
+- [ ] Have an appropriate email domain (Contact your Domain Service Provider)
+- [ ] Cassandra nodes deployed.
+- [ ] An ObjectStorage compatible with S3 APIs.
+- [ ] Configure JWT keys, keystores, SPF and DKIM. See document [here](https://james.apache.org/howTo/)
+- [ ] Adding the proper `mailetcontainer.xml` file with other conf files in James Helm package
Review comment:
I'm not sure I agree on this one. We did like this because we were trying to have our conf files centralized and for `mailetcontainer.xml` it was just not possible with different tenants.
I think we should provide a default mail processing pipeline here for the community doc, as it's easy to forget. It's up to the people using it after to reorganize it as they want (you could think as well other conf files than mailetcontainer could be hard to be centralized depending on multiple setups for other people, but our doc should not solve that)
##########
File path: server/apps/distributed-app/helm-chart/james/templates/deployment.yaml
##########
@@ -0,0 +1,425 @@
+{{- $elasticsearchUrl := include "elasticsearch.url.list" . }}
+{{- $cassandraUrl := include "cassandra.url.list" . }}
+{{- $jamesOpts := include "james.jvmOpts" . }}
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: james-jmap
Review comment:
Open question: Shouldn't we go back to something simpler here for the community? Only one James server deployment? No jmap / imap-smtp separate instances?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org