You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@solr.apache.org by ho...@apache.org on 2021/11/16 23:18:54 UTC
[solr-operator] branch gh-pages updated: Upgrade documentation for v0.5.0 release
This is an automated email from the ASF dual-hosted git repository.
houston pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/solr-operator.git
The following commit(s) were added to refs/heads/gh-pages by this push:
new 5b112ce Upgrade documentation for v0.5.0 release
5b112ce is described below
commit 5b112ce7db83e2179bd7f974f02908bfb3b7bd89
Author: Houston Putman <ho...@apache.org>
AuthorDate: Tue Nov 16 18:18:52 2021 -0500
Upgrade documentation for v0.5.0 release
---
.gitignore | 1 +
NOTICE | 2 +
README.md | 17 +
docs/README.md | 17 +
docs/development.md | 17 +
docs/local_tutorial.md | 25 +-
docs/release-instructions.md | 17 +
docs/running-the-operator.md | 47 ++-
docs/solr-backup/README.md | 374 ++++++++++++++++++++-
docs/solr-cloud/README.md | 32 +-
docs/solr-cloud/dependencies.md | 17 +
docs/solr-cloud/managed-updates.md | 34 ++
docs/solr-cloud/solr-cloud-crd.md | 102 +++++-
docs/solr-prometheus-exporter/README.md | 17 +
docs/upgrade-notes.md | 93 ++++-
docs/upgrading-to-apache.md | 17 +
example/README.md | 4 +-
.../{test_solrbackup.yaml => test_backup_gcs.yaml} | 12 +-
...est_solrbackup.yaml => test_backup_volume.yaml} | 11 +-
example/test_solrcloud.yaml | 7 +-
example/test_solrcloud_backuprepos.yaml | 74 ++++
21 files changed, 864 insertions(+), 73 deletions(-)
diff --git a/.gitignore b/.gitignore
index efe19ad..bf3ff90 100644
--- a/.gitignore
+++ b/.gitignore
@@ -20,6 +20,7 @@
*.so
*.dylib
bin
+testbin
release-artifacts
.DS_Store
diff --git a/NOTICE b/NOTICE
index e3db3cb..9ccae0b 100644
--- a/NOTICE
+++ b/NOTICE
@@ -38,4 +38,6 @@ Copyright (c) 2020 Dell Inc., or its subsidiaries. All Rights Reserved.
The Zookeeper Operator is used as an optional dependency, however the ZookeeperCluster CRD is used within the repository.
config/dependencies/zookeeper_cluster_crd.yaml
+The API code was also copied to the following directory
+ controllers/zk_api
This was taken from https://github.com/pravega/zookeeper-operator (version v0.2.12)
diff --git a/README.md b/README.md
index cfb76e1..07a9765 100644
--- a/README.md
+++ b/README.md
@@ -1,3 +1,20 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+
# Solr Operator
[](https://github.com/apache/solr-operator/releases)
[](http://www.apache.org/licenses/LICENSE-2.0.html)
diff --git a/docs/README.md b/docs/README.md
index e8ebcba..d2a97cb 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -1,3 +1,20 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+
# Documentation
Please visit the following pages for documentation on using and developing the Solr Operator:
diff --git a/docs/development.md b/docs/development.md
index 6fe577a..7a0d16f 100644
--- a/docs/development.md
+++ b/docs/development.md
@@ -1,3 +1,20 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+
# Developing the Solr Operator
This page details the steps for developing the Solr Operator, and all necessary steps to follow before creating a PR to the repo.
diff --git a/docs/local_tutorial.md b/docs/local_tutorial.md
index 1620dc6..7ff80cb 100644
--- a/docs/local_tutorial.md
+++ b/docs/local_tutorial.md
@@ -1,3 +1,20 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+
# Solr on Kubernetes on local Mac
This tutorial shows how to setup Solr under Kubernetes on your local mac. The plan is as follows:
@@ -73,9 +90,9 @@ This will install the [Zookeeper Operator](https://github.com/pravega/zookeeper-
```bash
# Install the Solr & Zookeeper CRDs
-$ kubectl create -f https://solr.apache.org/operator/downloads/crds/v0.4.0/all-with-dependencies.yaml
+$ kubectl create -f https://solr.apache.org/operator/downloads/crds/v0.5.0/all-with-dependencies.yaml
# Install the Solr operator and Zookeeper Operator
-$ helm install solr-operator apache-solr/solr-operator --version 0.4.0
+$ helm install solr-operator apache-solr/solr-operator --version 0.5.0
```
_Note that the Helm chart version does not contain a `v` prefix, which the downloads version does. The Helm chart version is the only part of the Solr Operator release that does not use the `v` prefix._
@@ -106,7 +123,7 @@ To start a Solr Cloud cluster, we will create a yaml that will tell the Solr Ope
```bash
# Create a 3-node cluster v8.3 with 300m Heap each:
-helm install example-solr apache-solr/solr --version 0.4.0 \
+helm install example-solr apache-solr/solr --version 0.5.0 \
--set image.tag=8.3 \
--set solrOptions.javaMemory="-Xms300m -Xmx300m" \
--set addressability.external.method=Ingress \
@@ -193,7 +210,7 @@ So we wish to upgrade to a newer Solr version:
curl -s http://default-example-solrcloud.ing.local.domain/solr/admin/info/system | grep solr-i
# Update the solrCloud configuration with the new version, keeping all previous settings and the number of nodes set by the autoscaler.
-helm upgrade example-solr apache-solr/solr --version 0.4.0 \
+helm upgrade example-solr apache-solr/solr --version 0.5.0 \
--reuse-values \
--set image.tag=8.7
diff --git a/docs/release-instructions.md b/docs/release-instructions.md
index 88b9996..59c7161 100644
--- a/docs/release-instructions.md
+++ b/docs/release-instructions.md
@@ -1,3 +1,20 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+
# Releasing a New Verson of the Solr Operator
This page details the steps for releasing new versions of the Solr Operator.
diff --git a/docs/running-the-operator.md b/docs/running-the-operator.md
index 855981c..cd9e489 100644
--- a/docs/running-the-operator.md
+++ b/docs/running-the-operator.md
@@ -1,3 +1,20 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+
# Running the Solr Operator
## Using the Solr Operator Helm Chart
@@ -21,8 +38,8 @@ Next, install the Solr Operator chart. Note this is using Helm v3, use the offic
This will install the [Zookeeper Operator](https://github.com/pravega/zookeeper-operator) by default.
```bash
-$ kubectl create -f https://solr.apache.org/operator/downloads/crds/v0.4.0/all-with-dependencies.yaml
-$ helm install solr-operator apache-solr/solr-operator --version 0.4.0
+$ kubectl create -f https://solr.apache.org/operator/downloads/crds/v0.5.0/all-with-dependencies.yaml
+$ helm install solr-operator apache-solr/solr-operator --version 0.5.0
```
_Note that the Helm chart version does not contain a `v` prefix, which the downloads version does. The Helm chart version is the only part of the Solr Operator release that does not use the `v` prefix._
@@ -109,10 +126,28 @@ The final image will only contain the solr-operator binary and necessary License
## Solr Operator Input Args
-* **-zookeeper-operator** Whether or not to use the Zookeeper Operator to create dependency Zookeeepers.
- Required to use the `spec.zookeeperRef.provided` option.
- If _true_, then a Zookeeper Operator must be running for the cluster.
- (_true_ | _false_ , defaults to _false_)
+* **--zookeeper-operator** Whether or not to use the Zookeeper Operator to create dependency Zookeeepers.
+ Required to use the `spec.zookeeperRef.provided` option.
+ If _true_, then a Zookeeper Operator must be running for the cluster.
+ (_true_ | _false_ , defaults to _false_)
+
+* **--watch-namespaces** Watch certain namespaces in the Kubernetes cluster.
+ If flag is omitted, or given an empty string, then the whole cluster will be watched.
+ If the operator should watch multiple namespaces, provide them all separated by commas.
+ (_string_ , defaults to _empty_)
+
+* **--leader-elect** Whether or not to use leader election for the Solr Operator.
+ If set to true, then only one operator pod will be functional for the namespaces given through `--watch-namespaces`.
+ If multiple namespaces are provided, leader election will use the first namespace sorted alphabetically.
+ (_true_ | _false_ , defaults to _true_)
+
+* **--metrics-bind-address** The address to bind the metrics servlet on.
+ If only a port is provided (e.g. `:8080`), then the metrics server will respond to requests with any Host header.
+ (defaults to _:8080_)
+
+* **--health-probe-bind-address=** The address to bind the health probe servlet on.
+ If only a port is provided (e.g. `:8081`), then the metrics server will respond to requests with any Host header.
+ (defaults to _:8081_)
## Client Auth for mTLS-enabled Solr clusters
diff --git a/docs/solr-backup/README.md b/docs/solr-backup/README.md
index 3cd6a29..d2dcd93 100644
--- a/docs/solr-backup/README.md
+++ b/docs/solr-backup/README.md
@@ -1,15 +1,363 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+
# Solr Backups
-Solr backups require 3 things:
-- A solr cloud running in kubernetes to backup
-- The list of collections to backup
-- A shared volume reference that can be written to from many clouds
- - This could be a NFS volume, a persistent volume claim (that has `ReadWriteMany` access), etc.
- - The same volume can be used for many solr clouds in the same namespace, as the data stored within the volume is namespaced.
-- A way to persist the data. The currently supported persistence methods are:
- - A volume reference (this does not have to be `ReadWriteMany`)
- - An S3 endpoint.
-
-Backups will be tarred before they are persisted.
-
-There is no current way to restore these backups, but that is in the roadmap to implement.
+The Solr Operator supports triggering the backup of arbitrary Solr collections.
+
+Triggering these backups involves setting configuration options on both the SolrCloud and SolrBackup CRDs.
+The SolrCloud instance is responsible for defining one or more backup "repositories" (metadata describing where and how the backup data should be stored).
+SolrBackup instances then trigger backups by referencing these repositories by name, listing the Solr collections to back up, and optionally scheduling recurring backups.
+
+For detailed information on how to best configure backups for your use case, please refer to the detailed schema information provided by `kubectl explain solrcloud.spec.backupRepositories` and its child elements, as well as `kubectl explain solrbackup`.
+
+This page outlines how to create and delete a Kubernetes SolrBackup.
+
+- [Creation](#creating-an-example-solrbackup)
+- [Recurring/Scheduled Backups](#recurring-backups)
+- [Deletion](#deleting-an-example-solrbackup)
+- [Repository Types](#supported-repository-types)
+ - [GCS](#gcs-backup-repositories)
+ - [S3](#s3-backup-repositories)
+ - [Volume](#volume-backup-repositories)
+
+## Creating an example SolrBackup
+
+A prerequisite for taking a backup is having something to take a backup _of_.
+SolrCloud creation generally is covered in more detail [here](../solr-cloud/README.md), so if you don't have one already, create a SolrCloud instance as per those instructions.
+
+Now that you have a Solr cluster to backup data from, you need a place to store the backup data.
+In this example, we'll create a Kubernetes persistent volume to mount on each Solr node.
+
+A volume for this purpose can be created as below:
+
+```yaml
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+ name: collection-backup-pvc
+spec:
+ accessModes:
+ - ReadWriteMany
+ resources:
+ requests:
+ storage: 1Gi
+ storageClassName: hostpath
+ volumeMode: Filesystem
+```
+
+Note that this PVC specifies `ReadWriteMany` access, which is required for Solr clusters with more than node.
+Note also that it uses a `storageClassName` of `hostpath`.
+Not all Kubernetes clusters support this `storageClassName` value - you may need to choose a different `ReadWriteMany`-compatible storage class based on your Kubernetes version and cluster setup.
+
+Next, modify your existing SolrCloud instance by adding a backup repository definition that uses the recently created volume.
+To do this, run `kubectl edit solrcloud example`, adding the following YAML nested under the `spec` property:
+
+```yaml
+spec:
+ backupRepositories:
+ - name: "local-collection-backups-1"
+ volume:
+ source:
+ persistentVolumeClaim:
+ claimName: "collection-backup-pvc"
+```
+
+This defines a backup repository called "local-collection-backups-1" which is setup to store backup data on the volume we've just created.
+The operator will notice this change and create new Solr pods that have the 'collection-backup-pvc' volume mounted.
+
+Now that there's a backup repository available to use, a backup can be triggered anytime by creating a new SolrBackup instance.
+
+```yaml
+apiVersion: solr.apache.org/v1beta1
+kind: SolrBackup
+metadata:
+ name: local-backup
+ namespace: default
+spec:
+ repositoryName: "local-collection-backups-1"
+ solrCloud: example
+ collections:
+ - techproducts
+ - books
+```
+
+This will create a backup of both the 'techproducts' and 'books' collections, storing the data on the 'collection-backup-pvc' volume.
+The status of our triggered backup can be checked with the command below.
+
+```bash
+$ kubectl get solrbackups
+NAME CLOUD STARTED FINISHED SUCCESSFUL NEXTBACKUP AGE
+test example 123m true false 161m
+```
+
+## Recurring Backups
+_Since v0.5.0_
+
+The Solr Operator enables taking recurring updates, at a set interval.
+Note that this feature requires a SolrCloud running Solr `8.9.0` or older, because it relies on `Incremental` backups.
+
+By default the Solr Operator will save a maximum of **5** backups at a time, however users can override this using `SolrBackup.spec.recurrence.maxSaved`.
+When using `recurrence`, users must provide a Cron-style `schedule` for the interval at which backups should be taken.
+Please refer to the [GoLang cron-spec](https://pkg.go.dev/github.com/robfig/cron/v3?utm_source=godoc#hdr-CRON_Expression_Format) for more information on allowed syntax.
+
+```yaml
+apiVersion: solr.apache.org/v1beta1
+kind: SolrBackup
+metadata:
+ name: local-backup
+ namespace: default
+spec:
+ repositoryName: "local-collection-backups-1"
+ solrCloud: example
+ collections:
+ - techproducts
+ - books
+ recurrence: # Store one backup daily, and keep a week at a time.
+ schedule: "@daily"
+ maxSaved: 7
+```
+
+If using `kubectl`, the standard `get` command will return the time the backup was last started and when the next backup will occur.
+
+```bash
+$ kubectl get solrbackups
+NAME CLOUD STARTED FINISHED SUCCESSFUL NEXTBACKUP AGE
+test example 123m true true 2021-11-09T00:00:00Z 161m
+```
+
+Much like when not taking a recurring backup, `SolrBackup.status` will contain the information from the latest, or currently running, backup.
+The results of previous backup attempts are stored under `SolrBackup.status.history` (sorted from most recent to oldest).
+
+You are able to **add or remove** `recurrence` to/from an existing `SolrBackup` object, no matter what stage that `SolrBackup` object is in.
+If you add recurrence, then a new backup will be scheduled based on the `startTimestamp` of the last backup.
+If you remove recurrence, then the `nextBackupTime` will be removed.
+However, if the recurrent backup is already underway, it will not be stopped.
+
+### Backup Scheduling
+
+Backups are scheduled based on the `startTimestamp` of the last backup.
+Therefore, if an interval schedule such as `@every 1h` is used, and a backup starts on `2021-11-09T03:10:00Z` and ends on `2021-11-09T05:30:00Z`, then the next backup will be started at `2021-11-09T04:10:00Z`.
+If the interval is shorter than the time it takes to complete a backup, then the next backup will started directly after the previous backup completes (even though it is delayed from its given schedule).
+And the next backup will be scheduled based on the `startTimestamp` of the delayed backup.
+So there is a possibility of skew overtime if backups take longer than the allotted schedule.
+
+If a guaranteed schedule is important, it is recommended to use intervals that are guaranteed to be longer than the time it takes to complete a backup.
+
+### Temporarily Disabling Recurring Backups
+
+It is also easy to temporarily disable backups for a time.
+Merely add `disabled: true` under the `recurrence` section of the `SolrBackup` resource.
+And set `disabled: false`, or just remove the property to re-enable backups.
+
+Since backups are scheduled based on the `startTimestamp` of the last backup, a new backup may start immediately after you re-enable the recurrence.
+
+```yaml
+apiVersion: solr.apache.org/v1beta1
+kind: SolrBackup
+metadata:
+ name: local-backup
+ namespace: default
+spec:
+ repositoryName: "local-collection-backups-1"
+ solrCloud: example
+ collections:
+ - techproducts
+ - books
+ recurrence: # Store one backup daily, and keep a week at a time.
+ schedule: "@daily"
+ maxSaved: 7
+ disabled: true
+```
+
+**Note: this will not stop any backups running at the time that `disabled: true` is set, it will only affect scheduling future backups.**
+
+## Deleting an example SolrBackup
+
+Once the operator completes a backup, the SolrBackup instance can be safely deleted.
+
+```bash
+$ kubectl delete solrbackup local-backup
+```
+
+Note that deleting SolrBackup instances doesn't delete the backed up data, which the operator views as already persisted and outside its control.
+In our example this data can still be found on the volume we created earlier
+
+```bash
+$ kubectl exec example-solrcloud-0 -- ls -lh /var/solr/data/backup-restore/local-collection-backups-1/backups/
+total 8K
+drwxr-xr-x 3 solr solr 4.0K Sep 16 11:48 local-backup-books
+drwxr-xr-x 3 solr solr 4.0K Sep 16 11:48 local-backup-techproducts
+```
+
+Volume backup data, as in our example, can always be deleted using standard shell commands if desired:
+
+```bash
+kubectl exec example-solrcloud-0 -- rm -r /var/solr/data/backup-restore/local-collection-backups-1/backups/local-backup-books
+kubectl exec example-solrcloud-0 -- rm -r /var/solr/data/backup-restore/local-collection-backups-1/backups/local-backup-techproducts
+```
+
+## Supported Repository Types
+_Since v0.5.0_
+
+Note all repositories are defined in the `SolrCloud` specification.
+In order to use a repository in the `SolrBackup` CRD, it must be defined in the `SolrCloud` spec.
+All yaml examples below are `SolrCloud` resources, not `SolrBackup` resources.
+
+The Solr-operator currently supports three different backup repository types: Google Cloud Storage ("GCS"), AWS S3 ("S3"), and Volume ("local").
+The cloud backup solutions (GCS and S3) are strongly suggested as they are cloud-native backup solutions, however they require newer Solr versions.
+
+Multiple repositories can be defined under the `SolrCloud.spec.backupRepositories` field.
+Specify a unique name and single repo type that you want to connect to.
+Repository-type specific options are found under the object named with the repository-type.
+Examples can be found below under each repository-type section below.
+Feel free to mix and match multiple backup repository types to fit your use case (or multiple repositories of the same type):
+
+```yaml
+spec:
+ backupRepositories:
+ - name: "local-collection-backups-1"
+ volume:
+ ...
+ - name: "gcs-collection-backups-1"
+ gcs:
+ ...
+ - name: "s3-collection-backups-1"
+ s3:
+ ...
+ - name: "s3-collection-backups-2"
+ s3:
+ ...
+```
+
+### GCS Backup Repositories
+_Since v0.5.0_
+
+GCS Repositories store backup data remotely in Google Cloud Storage.
+This repository type is only supported in deployments that use a Solr version >= `8.9.0`.
+
+Each repository must specify the GCS bucket to store data in (the `bucket` property), and the name of a Kubernetes secret containing credentials for accessing GCS (the `gcsCredentialSecret` property).
+This secret must have a key `service-account-key.json` whose value is a JSON service account key as described [here](https://cloud.google.com/iam/docs/creating-managing-service-account-keys)
+If you already have your service account key, this secret can be created using a command like the one below.
+
+```bash
+kubectl create secret generic <secretName> --from-file=service-account-key.json=<path-to-service-account-key>
+```
+
+An example of a SolrCloud spec with only one backup repository, with type GCS:
+
+```yaml
+spec:
+ backupRepositories:
+ - name: "gcs-backups-1"
+ gcs:
+ bucket: "backup-bucket" # Required
+ gcsCredentialSecret: # Required
+ name: "secretName"
+ key: "service-account-key.json"
+ baseLocation: "/store/here" # Optional
+```
+
+
+### S3 Backup Repositories
+_Since v0.5.0_
+
+S3 Repositories store backup data remotely in AWS S3 (or a supported S3 compatible interface).
+This repository type is only supported in deployments that use a Solr version >= `8.10.0`.
+
+Each repository must specify an S3 bucket and region to store data in (the `bucket` and `region` properties).
+Users will want to setup credentials so that the SolrCloud can connect to the S3 bucket and region, more information can be found in the [credentials section](#s3-credentials).
+
+```yaml
+spec:
+ backupRepositories:
+ - name: "s3-backups-1"
+ s3:
+ region: "us-west-2" # Required
+ bucket: "backup-bucket" # Required
+ credentials: {} # Optional
+ proxyUrl: "https://proxy-url-for-s3:3242" # Optional
+ endpoint: "https://custom-s3-endpoint:3242" # Optional
+```
+
+Users can also optionally set a `proxyUrl` or `endpoint` for the S3Repository.
+More information on these settings can be found in the [Ref Guide](https://solr.apache.org/guide/8_10/making-and-restoring-backups.html#s3backuprepository).
+
+#### S3 Credentials
+
+The Solr `S3Repository` module uses the [default credential chain for AWS](https://docs.aws.amazon.com/sdk-for-java/v2/developer-guide/credentials.html#credentials-chain).
+All of the options below are designed to be utilized by this credential chain.
+
+There are a few options for giving a SolrCloud the credentials for connecting to S3.
+The two most straightforward ways can be used via the `spec.backupRepositories.s3.credentials` property.
+
+```yaml
+spec:
+ backupRepositories:
+ - name: "s3-backups-1"
+ s3:
+ region: "us-west-2"
+ bucket: "backup-bucket"
+ credentials:
+ accessKeyIdSecret: # Optional
+ name: aws-secrets
+ key: access-key-id
+ secretAccessKeySecret: # Optional
+ name: aws-secrets
+ key: secret-access-key
+ sessionTokenSecret: # Optional
+ name: aws-secrets
+ key: session-token
+ credentialsFileSecret: # Optional
+ name: aws-credentials
+ key: credentials
+```
+
+All options in the `credentials` property are optional, as users can pick and choose which ones to use.
+If you have all of your credentials setup in an [AWS Credentials File](https://docs.aws.amazon.com/sdkref/latest/guide/file-format.html#file-format-creds),
+then `credentialsFileSecret` will be the only property you need to set.
+However, if you don't have a credentials file, you will likely need to set at least the `accessKeyIdSecret` and `secretAccessKeySecret` properties.
+All of these options require the referenced Kuberentes secrets to already exist before creating the SolrCloud resource.
+_(If desired, all options can be combined. e.g. Use `accessKeyIdSecret` and `credentialsFileSecret` together. The ordering of the default credentials chain will determine which options are used.)_
+
+The options in the credentials file above merely set environment variables on the pod, or in the case of `credentialsFileSecret` use an environment variable and a volume mount.
+Users can decide to not use the `credentials` section of the s3 repository config, and instead set these environment variables themselves via `spec.customSolrKubeOptions.podOptions.env`.
+
+Lastly, if running in EKS, it is possible to add [IAM information to Kubernetes serviceAccounts](https://aws.amazon.com/blogs/opensource/introducing-fine-grained-iam-roles-service-accounts/).
+If this is done correctly, you will only need to specify the serviceAccount for the SolrCloud pods via `spec.customSolrKubeOptions.podOptions.serviceAccount`.
+
+_NOTE: Because the Solr S3 Repository is using system-wide settings for AWS credentials, you cannot specify different credentials for different S3 repositories.
+This may be addressed in future Solr versions, but for now use the same credentials for all s3 repos._
+
+### Volume Backup Repositories
+_Since v0.5.0_
+
+Volume repositories store backup data "locally" on a Kubernetes volume mounted to each Solr pod.
+An example of a SolrCloud spec with only one backup repository, with type Volume:
+
+```yaml
+spec:
+ backupRepositories:
+ - name: "local-collection-backups-1"
+ volume:
+ source: # Required
+ persistentVolumeClaim:
+ claimName: "collection-backup-pvc"
+ directory: "store/here" # Optional
+```
+
+**NOTE: All persistent volumes used with Volume Repositories must have `accessMode: ReadWriteMany` set, otherwise the backups will not succeed.**
diff --git a/docs/solr-cloud/README.md b/docs/solr-cloud/README.md
index 6c2cf8b..3b430b1 100644
--- a/docs/solr-cloud/README.md
+++ b/docs/solr-cloud/README.md
@@ -1,3 +1,20 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+
# Solr Clouds
The Solr Operator supports creating and managing Solr Clouds.
@@ -15,7 +32,7 @@ This page outlines how to create, update and delete a SolrCloud in Kubernetes.
## Creating an example SolrCloud
-Make sure that the solr-operator and a zookeeper-operator are running.
+Make sure that the Solr Operator and a Zookeeper Operator are running.
Create an example Solr cloud, with the following configuration.
@@ -81,11 +98,20 @@ $ kubectl delete solrcloud example
### Official Solr Images
-The solr-operator will work with any of the [official Solr images](https://hub.docker.com/_/solr) currently available.
+The Solr Operator is only guaranteed to work with [official Solr images](https://hub.docker.com/_/solr).
+However, as long as your custom image is built to be compatible with the official image, things should go smoothly.
+This is especially true starting with Solr 9, where the docker image creation is bundled within Solr.
+Run `./gradlew docker` in the Solr repository, and your custom Solr additions will be packaged into an officially compliant Solr Docker image.
+
+Please refer to the [Version Compatibility Matrix](../upgrade-notes.md#solr-versions) for more information on what Solr Versions are compatible with the Solr Operator.
+
+Also note that certain features available within the Solr Operator are only supported in newer Solr Versions.
+The version compatibility matrix shows the minimum Solr version supported for **most** options.
+Please refer to the Solr Reference guide to see what features are enabled for the Solr version you are running.
### Build Your Own Private Solr Images
-The solr-operator supports private Docker repo access for Solr images you may want to store in a private Docker repo. It is recommended to source your image from the official Solr images.
+The Solr Operator supports private Docker repo access for Solr images you may want to store in a private Docker repo. It is recommended to source your image from the official Solr images.
Using a private image requires you have a K8s secret preconfigured with appropriate access to the image. (type: kubernetes.io/dockerconfigjson)
diff --git a/docs/solr-cloud/dependencies.md b/docs/solr-cloud/dependencies.md
index 3b21ee4..c162048 100644
--- a/docs/solr-cloud/dependencies.md
+++ b/docs/solr-cloud/dependencies.md
@@ -1,3 +1,20 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+
## Dependent Kubernetes Resources
What actually gets created when the Solr Cloud is spun up?
diff --git a/docs/solr-cloud/managed-updates.md b/docs/solr-cloud/managed-updates.md
index 449abb5..86ec236 100644
--- a/docs/solr-cloud/managed-updates.md
+++ b/docs/solr-cloud/managed-updates.md
@@ -1,3 +1,20 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+
# Managed SolrCloud Rolling Updates
_Since v0.2.7_
@@ -55,3 +72,20 @@ This maximum is calculated by taking the given, or default, [`maxPodsUnavailable
- Some replicas in the shard may already be in a non-active state, or may reside on Solr Nodes that are not "live".
The `maxShardReplicasUnavailable` calculation will take these replicas into account, as a starting point.
- If a pod contains non-active replicas, and the pod is chosen to be updated, then the pods that are already non-active will not be double counted for the `maxShardReplicasUnavailable` calculation.
+
+## Triggering a Manual Rolling Restart
+
+Given these complex requirements, `kubectl rollout restart statefulset` will generally not work on a SolrCloud.
+
+One option to trigger a manual restart is to change one of the podOptions annotations. For example you could set this to the date and time of the manual restart.
+
+
+```yaml
+apiVersion: solr.apache.org/v1beta1
+kind: SolrCloud
+spec:
+ customSolrKubeOptions:
+ podOptions:
+ annotations:
+ manualrestart: "2021-10-20T08:37:00Z"
+```
diff --git a/docs/solr-cloud/solr-cloud-crd.md b/docs/solr-cloud/solr-cloud-crd.md
index e9b42cc..2e28f36 100644
--- a/docs/solr-cloud/solr-cloud-crd.md
+++ b/docs/solr-cloud/solr-cloud-crd.md
@@ -1,8 +1,44 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+
# The SolrCloud CRD
The SolrCloud CRD allows users to spin up a Solr cloud in a very configurable way.
Those configuration options are laid out on this page.
+## Solr Options
+
+The SolrCloud CRD gives users the ability to customize how Solr is run.
+
+Please note that the options described below are shown using the base SolrCloud resource, not the helm chart.
+Most options will have the same name and path, however there are differences such as `customSolrKubeOptions`.
+If using Helm, refer to the [Helm Chart documentation](https://artifacthub.io/packages/helm/apache-solr/solr#chart-values) to see the names for the options you are looking to use.
+This document should still be used to see how the SolrCloud options can be used.
+
+### Solr Modules and Additional Libraries
+_Since v0.5.0_
+
+Solr comes packaged with modules that can be loaded optionally, known as either Solr Modules or Solr Contrib Modules.
+By default they are not included in the classpath of Solr, so they have to be explicitly enabled.
+Use the **`SolrCloud.spec.solrModules`** property to add a list of module names, not paths, and they will automatically be enabled for the solrCloud.
+
+However, users might want to include custom code that is not an official Solr Module.
+In order to facilitate this, the **`SolrCloud.spec.additionalLibs`** property takes a list of paths to folders, containing jars to load in the classpath of the SolrCloud.
+
## Data Storage
The SolrCloud CRD gives the option for users to use either
@@ -92,6 +128,14 @@ Under `SolrCloud.Spec.solrAddressability`:
**Note:** Unless both `external.method=Ingress` and `external.hideNodes=false`, a headless service will be used to make each Solr Node in the statefulSet addressable.
If both of those criteria are met, then an individual ClusterIP Service will be created for each Solr Node/Pod.
+If you are using an `Ingress` for external addressability, you can customize the created `Ingress` through `SolrCloud.spec.customSolrKubeOptions.ingressOptions`.
+Under this property, you can set custom `annotations`, `labels` and an `ingressClassName`.
+
+## Backups
+
+Solr Backups are enabled via the Solr Operator.
+Please refer to the [SolrBackup documentation](../solr-backup) for more information on setting up a SolrCloud with backups enabled.
+
## Zookeeper Reference
Solr Clouds require an Apache Zookeeper to connect to.
@@ -148,6 +192,11 @@ each solrCloud that has this option specified.
The startup parameter `zookeeper-operator` must be provided on startup of the solr-operator for this parameter to be available.
+To find all Provided zookeeper options, run `kubectl explain solrcloud.spec.zookeeperRef.provided`.
+Zookeeper Conf and PodOptions provided in the linked Zookeeper Operator version should be supported in the SolrCloud CRD.
+However, this is a manual task, so not all options might be available.
+If there is an option available in the ZookeeperCluster CRD that is not exposed via the SolrCloud CRD, please create a Github Issue.
+
#### Zookeeper Storage Options
_Since v0.4.0_
@@ -767,13 +816,12 @@ basic authentication with TLS to ensure credentials are never passed in clear te
For background on Solr security, please refer to the [Reference Guide](https://solr.apache.org/guide) for your version of Solr.
-Basic authentication is the only authentication scheme supported by the Solr operator at this time. In general, you have
-two basic options for configuring basic authentication with the Solr operator:
+The Solr operator only supports the `Basic` authentication scheme. In general, you have two primary options for configuring authentication with the Solr operator:
1. Let the Solr operator bootstrap the `security.json` to configure *basic authentication* for Solr.
2. Supply your own `security.json` to Solr, which must define a user account that the operator can use to make API requests to secured Solr pods.
-If you choose option 2, then you need to provide the credentials to the Solr operator using a Kubernetes [Basic Authentication Secret](https://kubernetes.io/docs/concepts/configuration/secret/#basic-authentication-secret).
-With option 1, the operator creates the Basic Authentication Secret for you.
+If you choose option 2, then you need to provide the credentials the Solr operator should use to make requests to Solr via a Kubernetes secret.
+With option 1, the operator creates a Basic Authentication Secret for you, which contains the username and password for the `k8s-oper` user.
### Option 1: Bootstrap Security
@@ -978,6 +1026,12 @@ The exporter also hits the `/admin/ping` endpoint for every collection, which re
"collection": "*",
"path": "/admin/ping"
},
+ {
+ "name": "k8s-zk",
+ "role":"k8s",
+ "collection": null,
+ "path":"/admin/zookeeper/status"
+ },
```
The `"collection":"*"` setting indicates this path applies to all collections, which maps to endpoint `/collections/<COLL>/admin/ping` at runtime.
@@ -985,9 +1039,33 @@ The initial authorization config grants the `read` permission to the `users` rol
For instance, the `solr` user is mapped to the `users` role, so the `solr` user can send query requests only.
In general, please verify the initial authorization rules for each role before sharing user credentials.
-### Option 2: User-provided Basic Auth Secret
+### Option 2: User-provided `security.json` and credentials secret
+
+If users want full control over their cluster's security config, then they can provide the Solr `security.json` via a Secret and the credentials the operator should use
+to make requests to Solr in a Secret.
+
+#### Custom `security.json` Secret
+_Since v0.5.0_
+
+For full control over the Solr security configuration, supply a `security.json` in a Secret. The following example illustrates how to point the operator to a Secret containing a custom `security.json`:
-Alternatively, if users want full control over their cluster's security config, then they can provide a `kubernetes.io/basic-auth` secret containing the credentials for the user they want the operator to make API requests as:
+```yaml
+spec:
+ ...
+ solrSecurity:
+ authenticationType: Basic
+ bootstrapSecurityJson:
+ name: my-custom-security-json
+ key: security.json
+```
+For `Basic` authentication, if you don't supply a `security.json` Secret, then the operator assumes you are bootstrapping the security configuration via some other means.
+
+Refer to the example `security.json` shown in the Authorization section above to help you get started crafting your own custom configuration.
+
+#### Basic Authentication
+
+For `Basic` authentication, the supplied secret must be of type [Basic Authentication Secret](https://kubernetes.io/docs/concepts/configuration/secret/#basic-authentication-secret) and define both a `username` and `password`.
+
```yaml
spec:
...
@@ -995,7 +1073,7 @@ spec:
authenticationType: Basic
basicAuthSecret: user-provided-secret
```
-The supplied secret must be of type [Basic Authentication Secret](https://kubernetes.io/docs/concepts/configuration/secret/#basic-authentication-secret) and define both a `username` and `password`.
+
Here is an example of how to define a basic auth secret using YAML:
```yaml
apiVersion: v1
@@ -1010,25 +1088,23 @@ stringData:
With this config, the operator will make API requests to secured Solr pods as the `k8s-oper` user.
_Note: be sure to use a stronger password for real deployments_
-If users supply their own basic auth secret, then the operator *does not* bootstrap the `security.json`;
-the reasoning is that if you're supplying your own basic auth credentials then you're also assuming the responsibility for configuring the desired access for this user.
-
-Users need to ensure their `security.json` contains the user supplied in the `basicAuthSecret` with read access to:
+Users need to ensure their `security.json` contains the user supplied in the `basicAuthSecret` has read access to the following endpoints:
```
/admin/info/system
/admin/info/health
/admin/collections
/admin/metrics
/admin/ping (for collection="*")
+/admin/zookeeper/status
```
_Tip: see the authorization rules defined by the default `security.json` as a guide for configuring access for the operator user_
-#### Changing the Password
+##### Changing the Password
If you change the password for the user configured in your `basicAuthSecret` using the Solr security API, then you **must** update the secret with the new password or the operator will be locked out.
Also, changing the password for this user in the K8s secret will not update Solr! You're responsible for changing the password in both places.
-### Prometheus Exporter with Basic Auth
+##### Prometheus Exporter with Basic Auth
If you enable basic auth for your SolrCloud cluster, then you need to point the Prometheus exporter at the basic auth secret;
refer to [Prometheus Exporter with Basic Auth](../solr-prometheus-exporter/README.md#prometheus-exporter-with-basic-auth) for more details.
diff --git a/docs/solr-prometheus-exporter/README.md b/docs/solr-prometheus-exporter/README.md
index cc93f59..b00d0bf 100644
--- a/docs/solr-prometheus-exporter/README.md
+++ b/docs/solr-prometheus-exporter/README.md
@@ -1,3 +1,20 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+
# Solr Prometheus Exporter
Solr metrics can be collected from solr clouds/standalone solr both residing within the kubernetes cluster and outside.
diff --git a/docs/upgrade-notes.md b/docs/upgrade-notes.md
index 86e5e43..eadd685 100644
--- a/docs/upgrade-notes.md
+++ b/docs/upgrade-notes.md
@@ -1,18 +1,40 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+
# Solr Operator Upgrade Notes
Please carefully read the entries for all versions between the version you are running and the version you want to upgrade to.
+Ensure to read the [Upgrade Warnings and Notes](#upgrade-warnings-and-notes) for the version you are upgrading to as well as the versions you are skipping.
+
+If you want to skip versions when upgrading, be sure to check out the [upgrading minor versions](#upgrading-minor-versions-v_x_) and [upgrading patch versions](#upgrading-patch-versions-v__x) sections.
+
## Version Compatibility Matrixes
### Kubernetes Versions
-| Solr Operator Version | `1.15` | `1.16` - `1.21` | `1.22`+ |
-| :---: | :---: | :---: | :---: |
-| `v0.2.6` | :heavy_check_mark: | :heavy_check_mark: | :x: |
-| `v0.2.7` | :x: | :heavy_check_mark: | :x: |
-| `v0.2.8` | :x: | :heavy_check_mark: | :x: |
-| `v0.3.0` | :x: | :heavy_check_mark: | :x: |
-| `v0.4.0` | :x: | :heavy_check_mark: | :x: |
+| Solr Operator Version | `1.15` | `1.16` - `1.18` | `1.19` - `1.21` | `1.22`+ |
+| :---: | :---: | :---: | :---: | :---: |
+| `v0.2.6` | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |
+| `v0.2.7` | :x: | :heavy_check_mark: | :heavy_check_mark: | :x: |
+| `v0.2.8` | :x: | :heavy_check_mark: | :heavy_check_mark: | :x: |
+| `v0.3.0` | :x: | :heavy_check_mark: | :heavy_check_mark: | :x: |
+| `v0.4.0` | :x: | :heavy_check_mark: | :heavy_check_mark: | :x: |
+| `v0.5.0` | :x: | :x: | :heavy_check_mark: | :heavy_check_mark: |
### Solr Versions
@@ -23,19 +45,26 @@ Please carefully read the entries for all versions between the version you are r
| `v0.2.8` | :grey_question: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| `v0.3.0` | :grey_question: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| `v0.4.0` | :grey_question: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
+| `v0.5.0` | :grey_question: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
+
+Please note that this represents basic compatibility with the Solr Operator.
+There may be options and features that require newer versions of Solr.
+(e.g. S3/GCS Backup Support)
+Please test to make sure the features you plan to use are compatible with the version of Solr you choose to run.
-## Upgrading from `v0.2.x` to `v0.3.x`
+
+### Upgrading from `v0.2.x` to `v0.3.x`
If you are upgrading from `v0.2.x` to `v0.3.x`, please follow the [Upgrading to Apache guide](upgrading-to-apache.md).
This is a special upgrade that requires different instructions.
-## Upgrading minor versions (`v_.X._`)
+### Upgrading minor versions (`v_.X._`)
In order to upgrade minor versions (e.g. `v0.2.5` -> `v0.3.0`), you must upgrade one minor version at a time (e.g. `v0.2.0` -> `v0.3.0` -> `v0.4.0`).
It is also necessary to upgrade to the latest patch version before upgrading to the next minor version.
Therefore if you are running `v0.2.5` and you want to upgrade to `v0.3.0`, you must first upgrade to `v0.2.8` before upgrading to `v0.3.0`.
-## Upgrading patch versions (`v_._.X`)
+### Upgrading patch versions (`v_._.X`)
You should be able to upgrade from a version to any patch version with the same minor and major versions.
It is always encouraged to upgrade to the latest patch version of the minor and major version you are running.
@@ -68,14 +97,48 @@ If you are using the Solr Helm chart to deploy the Zookeeper operator, then you
```bash
# Just replace the Solr CRDs and all CRDs it might depend on (e.g. ZookeeperCluster)
-kubectl replace -f "http://solr.apache.org/operator/downloads/crds/v0.4.0/all-with-dependencies.yaml"
-helm upgrade solr-operator apache-solr/solr-operator --version 0.4.0
+kubectl replace -f "http://solr.apache.org/operator/downloads/crds/v0.5.0/all-with-dependencies.yaml"
+helm upgrade solr-operator apache-solr/solr-operator --version 0.5.0
```
_Note that the Helm chart version does not contain a `v` prefix, which the downloads version does. The Helm chart version is the only part of the Solr Operator release that does not use the `v` prefix._
## Upgrade Warnings and Notes
+### v0.5.0
+- Due to the deprecation and removal of `networking.k8s.io/v1beta1` in Kubernetes v1.22, `networking.k8s.io/v1` will be used for Ingresses.
+
+ **This means that Kubernetes support is now limited to 1.19+.**
+ If you are unable to use a newer version of Kubernetes, please install the `v0.4.0` version of the Solr Operator for use with Kubernetes 1.18 and below.
+ See the [version compatibility matrix](#kubernetes-versions) for more information.
+
+ This also means that if you specify a custom `ingressClass` via an annotation, you should change to use the `SolrCloud.spec.customSolrKubeOptions.ingressOptions.ingressClassName` instead.
+ The ability to set the class through annotations is now deprecated in Kubernetes and will be removed in future versions.
+
+- The legacy way of specifying a backupRepository has been **DEPRECATED**.
+ Instead of using `SolrCloud.spec.dataStorage.backupRestoreOptions`, use `SolrCloud.spec.backupRepositories`.
+ The `SolrCloud.spec.dataStorage.backupRestoreOptions` option **will be removed in `v0.6.0`**.
+ **Note**: Do not take backups while upgrading from the Solr Operator `v0.4.0` to `v0.5.0`.
+ Wait for the SolrClouds to be updated, after the Solr Operator is upgraded, and complete their rolling restarts before continuing to use the Backup functionality.
+
+- The location of Solr backup data as well as the name of the Solr backups have been changed, when using volume repositories.
+ Previously the name of the backup (in solr) was set to the name of the collection.
+ Now the name given to the backup in Solr will be set to `<backup-resource-name>-<collection-name>`, without the `<` or `>` characters, where the `backup-resource-name` is the name of the SolrBackup resource.
+
+ The directory in the Read-Write-Many Volume, required for volume repositories, that backups are written to is now `/cloud/<solr-cloud-name>/backups` by default, instead of `/cloud/<solr-cloud-name>/backups/<backup-name>`.
+ Because the backup name in Solr uses both the SolrBackup resource name and the collection name, there should be no collisions in this directory.
+ However, this can be overridden using the `SolrBackup.spec.location` option, which is appended to `/cloud/<solr-cloud-name>`.
+
+- The SolrBackup persistence option has been removed as of `v0.5.0`.
+ Users should plan to keep their backup data in the shared volume if using a Volume Backup repository.
+ If `SolrBackup.spec.persistence` is provided, it will be removed and written back to Kubernetes.
+
+ Users using the S3 persistence option should try to use the [S3 backup repository](solr-backup/README.md#s3-backup-repositories) instead. This requires Solr 8.10 or higher.
+
+- Default ports when using TLS are now set to 443 instead of 80.
+ This affects `solrCloud.Spec.SolrAddressability.CommonServicePort` and `solrCloud.Spec.SolrAddressability.CommonServicePort` field defaulting.
+ Users already explicitly setting these values will not be affected.
+
### v0.4.0
- The required version of the [Zookeeper Operator](https://github.com/pravega/zookeeper-operator) to use with this version has been upgraded from `v0.2.9` to `v0.2.12`.
If you use the Solr Operator helm chart, then by default the new version of the Zookeeper Operator will be installed as well.
@@ -85,11 +148,11 @@ _Note that the Helm chart version does not contain a `v` prefix, which the downl
- The deprecated Solr Operator Helm chart option `useZkOperator` has been removed, use `zookeeper-operator.use` instead.
**Note**: The old option takes a _string_ `"true"`/`"false"`, while the new option takes a _boolean_ `true`/`false`.
-
+
- The default Solr version for `SolrCloud` and `SolrPrometheusExporter` resources has been upgraded from `7.7.0` to `8.9`.
This will not effect any existing resources, as default versions are hard-written to the resources immediately.
Only new resources created after the Solr Operator is upgraded to `v0.4.0` will be affected.
-
+
- In previous versions of the Solr Operator, the provided Zookeeper instances could only use Persistent Storage.
Now ephemeral storage is enabled, and used by default if Solr is using ephemeral storage.
The ZK storage type can be explicitly set via `Spec.zookeeperRef.provided.ephemeral` or `Spec.zookeeperRef.provided.persistence`,
@@ -109,7 +172,7 @@ _Note that the Helm chart version does not contain a `v` prefix, which the downl
Please refer to the [Zookeeper Operator release notes](https://github.com/pravega/zookeeper-operator/releases) before upgrading.
### v0.2.7
-- Do to the addition of possible sidecar/initContainers for SolrClouds, the version of CRDs used had to be upgraded to `apiextensions.k8s.io/v1`.
+- Due to the addition of possible sidecar/initContainers for SolrClouds, the version of CRDs used had to be upgraded to `apiextensions.k8s.io/v1`.
**This means that Kubernetes support is now limited to 1.16+.**
If you are unable to use a newer version of Kubernetes, please install the `v0.2.6` version of the Solr Operator for use with Kubernetes 1.15 and below.
diff --git a/docs/upgrading-to-apache.md b/docs/upgrading-to-apache.md
index ca420b3..2e58db8 100644
--- a/docs/upgrading-to-apache.md
+++ b/docs/upgrading-to-apache.md
@@ -1,3 +1,20 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+
# Upgrading the Solr Operator from Bloomberg to Apache
Please read the guide fully before attempting to upgrade your system.
diff --git a/example/README.md b/example/README.md
index c47cbdb..61b7cb7 100644
--- a/example/README.md
+++ b/example/README.md
@@ -7,7 +7,9 @@ The following examples are provided in helping explain various options provided
- [Custom Addressability](test_solrcloud_addressability.yaml)
- [Private Repo Solr Image](test_solrcloud_private_repo.yaml)
- [Pod Tolerations](test_solrcloud_toleration_example.yaml)
+ - [Multiple Backup Repositories](test_solrcloud_backuprepos.yaml)
- Solr Prometheus Exporter
- [Basic](test_solrprometheusexporter.yaml)
- Solr Backup
- - [Basic](test_solrbackup.yaml)
\ No newline at end of file
+ - [Volume Backup](test_backup_volume.yaml)
+ - [GCS Backup](test_backup_gcs.yaml)
diff --git a/example/test_solrbackup.yaml b/example/test_backup_gcs.yaml
similarity index 76%
copy from example/test_solrbackup.yaml
copy to example/test_backup_gcs.yaml
index fc583c0..778fabc 100644
--- a/example/test_solrbackup.yaml
+++ b/example/test_backup_gcs.yaml
@@ -13,17 +13,15 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+# Assumes a 'solrcloud' whose 'backupRestoreOptions' match the "Multiple Backup
+# Repositories" solrcloud example ('test_solrcloud_backuprepos.yaml')
apiVersion: solr.apache.org/v1beta1
kind: SolrBackup
metadata:
- name: solrbackup-test
+ name: main-collection-gcs-backups
namespace: default
spec:
- persistence:
- volume:
- source:
- persistentVolumeClaim:
- claimName: "pvc-test"
- solrCloud: example
+ repositoryName: "main_collection_backup_repository"
+ solrCloud: multiple-backup-repos
collections:
- example
diff --git a/example/test_solrbackup.yaml b/example/test_backup_volume.yaml
similarity index 78%
rename from example/test_solrbackup.yaml
rename to example/test_backup_volume.yaml
index fc583c0..ac31610 100644
--- a/example/test_solrbackup.yaml
+++ b/example/test_backup_volume.yaml
@@ -13,17 +13,14 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+# Assumes a 'solrcloud' whose 'backupRestoreOptions' match the "Multiple Backup Repositories" solrcloud example ('test_solrcloud_backuprepos.yaml')
apiVersion: solr.apache.org/v1beta1
kind: SolrBackup
metadata:
- name: solrbackup-test
+ name: volume-backup
namespace: default
spec:
- persistence:
- volume:
- source:
- persistentVolumeClaim:
- claimName: "pvc-test"
- solrCloud: example
+ repositoryName: "volume_repository_1"
+ solrCloud: multiple-backup-repos
collections:
- example
diff --git a/example/test_solrcloud.yaml b/example/test_solrcloud.yaml
index 9fdf64b..b8ba247 100644
--- a/example/test_solrcloud.yaml
+++ b/example/test_solrcloud.yaml
@@ -26,14 +26,13 @@ spec:
resources:
requests:
storage: "5Gi"
- backupRestoreOptions:
- volume:
- persistentVolumeClaim:
- claimName: "pvc-test"
replicas: 3
solrImage:
tag: 8.7.0
solrJavaMem: "-Xms1g -Xmx3g"
+ solrModules:
+ - jaegertracer-configurator
+ - ltr
customSolrKubeOptions:
podOptions:
resources:
diff --git a/example/test_solrcloud_backuprepos.yaml b/example/test_solrcloud_backuprepos.yaml
new file mode 100644
index 0000000..6c0f734
--- /dev/null
+++ b/example/test_solrcloud_backuprepos.yaml
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: solr.apache.org/v1beta1
+kind: SolrCloud
+metadata:
+ name: multiple-backup-repos
+spec:
+ replicas: 1
+ solrImage:
+ tag: 8.10
+ backupRepositories:
+ # "Volume" repositories store backup data in a Kubernetes volume.
+ #
+ # This syntax replaces the deprecated local-backup syntax.
+ - name: "volume_repository_1"
+ volume:
+ source:
+ persistentVolumeClaim:
+ claimName: "pvc-test-1" # PVC expected to already exist!
+
+ # Creates repositories that backup data to Google Cloud Storage ("GCS").
+ # GCS credentials are taken from the secret 'gcsCredentialSecret', which
+ # should have a key 'service-account-key.json' whose value is a JSON token
+ # as described here: https://cloud.google.com/iam/docs/creating-managing-service-account-keys
+ #
+ # Requires Solr >= 8.9.
+ - name: "main_collection_backup_repository"
+ gcs:
+ bucket: "product_catalog"
+ gcsCredentialSecret:
+ name: "some-secret-name"
+ key: "service-account-key.json"
+
+ # Creates repositories that backup data to AWS S3.
+ #
+ # Requires Solr >= 8.10.
+ - name: "s3-backup-repo"
+ s3:
+ region: "us-west-2"
+ bucket: "product-catalog"
+ credentials:
+ accessKeyIdSecret: # Optional
+ name: aws-secrets
+ key: access-key-id
+ secretAccessKeySecret: # Optional
+ name: aws-secrets
+ key: secret-access-key
+ sessionTokenSecret: # Optional
+ name: aws-secrets
+ key: session-token
+ credentialsFileSecret: # Optional
+ name: aws-credentials
+ key: credentials
+
+ - name: "main_collection_backup_repository_log"
+ gcs:
+ bucket: "log_data"
+ gcsCredentialSecret:
+ name: "some-secret-name-2"
+ key: "service-account-key.json"
+ baseLocation: "/logs"