You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by ho...@apache.org on 2021/10/29 17:11:22 UTC
[lucene-solr] branch branch_8x updated: SOLR-15707: Add the ability
to set the AWS Profile per S3Repository (#357)
This is an automated email from the ASF dual-hosted git repository.
houston pushed a commit to branch branch_8x
in repository https://gitbox.apache.org/repos/asf/lucene-solr.git
The following commit(s) were added to refs/heads/branch_8x by this push:
new 55a0a41 SOLR-15707: Add the ability to set the AWS Profile per S3Repository (#357)
55a0a41 is described below
commit 55a0a4140803e713ff06f9dc5486b00dab81ea85
Author: Houston Putman <ho...@apache.org>
AuthorDate: Fri Oct 29 12:47:55 2021 -0400
SOLR-15707: Add the ability to set the AWS Profile per S3Repository (#357)
Updating the AWS SDK version for better retry options.
---
lucene/ivy-versions.properties | 4 +-
solr/CHANGES.txt | 3 +-
solr/contrib/s3-repository/ivy.xml | 2 +
.../apache/solr/s3/S3BackupRepositoryConfig.java | 15 +-
.../java/org/apache/solr/s3/S3StorageClient.java | 64 ++++++-
.../org/apache/solr/s3/AbstractS3ClientTest.java | 8 +-
solr/licenses/annotations-2.17.63.jar.sha1 | 1 +
solr/licenses/apache-client-2.16.93.jar.sha1 | 1 -
solr/licenses/apache-client-2.17.63.jar.sha1 | 1 +
solr/licenses/arns-2.16.93.jar.sha1 | 1 -
solr/licenses/arns-2.17.63.jar.sha1 | 1 +
solr/licenses/auth-2.16.93.jar.sha1 | 1 -
solr/licenses/auth-2.17.63.jar.sha1 | 1 +
solr/licenses/aws-core-2.16.93.jar.sha1 | 1 -
solr/licenses/aws-core-2.17.63.jar.sha1 | 1 +
solr/licenses/aws-query-protocol-2.16.93.jar.sha1 | 1 -
solr/licenses/aws-query-protocol-2.17.63.jar.sha1 | 1 +
solr/licenses/aws-xml-protocol-2.16.93.jar.sha1 | 1 -
solr/licenses/aws-xml-protocol-2.17.63.jar.sha1 | 1 +
solr/licenses/http-client-spi-2.16.93.jar.sha1 | 1 -
solr/licenses/http-client-spi-2.17.63.jar.sha1 | 1 +
solr/licenses/json-utils-2.17.63.jar.sha1 | 1 +
solr/licenses/json-utils-LICENSE-ASL.txt | 206 +++++++++++++++++++++
solr/licenses/json-utils-NOTICE.txt | 25 +++
solr/licenses/metrics-spi-2.16.93.jar.sha1 | 1 -
solr/licenses/metrics-spi-2.17.63.jar.sha1 | 1 +
solr/licenses/profiles-2.16.93.jar.sha1 | 1 -
solr/licenses/profiles-2.17.63.jar.sha1 | 1 +
solr/licenses/protocol-core-2.16.93.jar.sha1 | 1 -
solr/licenses/protocol-core-2.17.63.jar.sha1 | 1 +
solr/licenses/regions-2.16.93.jar.sha1 | 1 -
solr/licenses/regions-2.17.63.jar.sha1 | 1 +
solr/licenses/s3-2.16.93.jar.sha1 | 1 -
solr/licenses/s3-2.17.63.jar.sha1 | 1 +
solr/licenses/sdk-core-2.16.93.jar.sha1 | 1 -
solr/licenses/sdk-core-2.17.63.jar.sha1 | 1 +
.../third-party-jackson-core-2.17.63.jar.sha1 | 1 +
.../third-party-jackson-core-LICENSE-ASL.txt | 206 +++++++++++++++++++++
solr/licenses/third-party-jackson-core-NOTICE.txt | 25 +++
.../url-connection-client-2.16.93.jar.sha1 | 1 -
.../url-connection-client-2.17.63.jar.sha1 | 1 +
solr/licenses/utils-2.16.93.jar.sha1 | 1 -
solr/licenses/utils-2.17.63.jar.sha1 | 1 +
.../src/making-and-restoring-backups.adoc | 61 ++++--
44 files changed, 615 insertions(+), 37 deletions(-)
diff --git a/lucene/ivy-versions.properties b/lucene/ivy-versions.properties
index 92d7f8a..82c3655 100644
--- a/lucene/ivy-versions.properties
+++ b/lucene/ivy-versions.properties
@@ -412,7 +412,7 @@ org.xerial.snappy.version = 1.1.7.6
/org.yaml/snakeyaml = 1.26
-software.amazon.awssdk.version = 2.16.93
+software.amazon.awssdk.version = 2.17.63
/software.amazon.awssdk/apache-client = ${software.amazon.awssdk.version}
/software.amazon.awssdk/arns = ${software.amazon.awssdk.version}
/software.amazon.awssdk/auth = ${software.amazon.awssdk.version}
@@ -420,12 +420,14 @@ software.amazon.awssdk.version = 2.16.93
/software.amazon.awssdk/aws-query-protocol = ${software.amazon.awssdk.version}
/software.amazon.awssdk/aws-xml-protocol = ${software.amazon.awssdk.version}
/software.amazon.awssdk/http-client-spi = ${software.amazon.awssdk.version}
+/software.amazon.awssdk/json-utils = ${software.amazon.awssdk.version}
/software.amazon.awssdk/metrics-spi = ${software.amazon.awssdk.version}
/software.amazon.awssdk/profiles = ${software.amazon.awssdk.version}
/software.amazon.awssdk/protocol-core = ${software.amazon.awssdk.version}
/software.amazon.awssdk/regions = ${software.amazon.awssdk.version}
/software.amazon.awssdk/s3 = ${software.amazon.awssdk.version}
/software.amazon.awssdk/sdk-core = ${software.amazon.awssdk.version}
+/software.amazon.awssdk/third-party-jackson-core = ${software.amazon.awssdk.version}
/software.amazon.awssdk/url-connection-client = ${software.amazon.awssdk.version}
/software.amazon.awssdk/utils = ${software.amazon.awssdk.version}
diff --git a/solr/CHANGES.txt b/solr/CHANGES.txt
index b7fa6f5..103fa2e 100644
--- a/solr/CHANGES.txt
+++ b/solr/CHANGES.txt
@@ -18,7 +18,8 @@ New Features
Improvements
---------------------
-(No changes)
+
+* SOLR-15707: Add the ability to set the AWS Profile per S3Repository (Houston Putman)
Optimizations
---------------------
diff --git a/solr/contrib/s3-repository/ivy.xml b/solr/contrib/s3-repository/ivy.xml
index 013a93c..8c16ced 100644
--- a/solr/contrib/s3-repository/ivy.xml
+++ b/solr/contrib/s3-repository/ivy.xml
@@ -34,12 +34,14 @@
<dependency org="software.amazon.awssdk" name="aws-query-protocol" rev="${/software.amazon.awssdk/aws-query-protocol}" conf="compile"/>
<dependency org="software.amazon.awssdk" name="aws-xml-protocol" rev="${/software.amazon.awssdk/aws-xml-protocol}" conf="compile"/>
<dependency org="software.amazon.awssdk" name="http-client-spi" rev="${/software.amazon.awssdk/http-client-spi}" conf="compile"/>
+ <dependency org="software.amazon.awssdk" name="json-utils" rev="${/software.amazon.awssdk/json-utils}" conf="compile"/>
<dependency org="software.amazon.awssdk" name="metrics-spi" rev="${/software.amazon.awssdk/metrics-spi}" conf="compile"/>
<dependency org="software.amazon.awssdk" name="profiles" rev="${/software.amazon.awssdk/profiles}" conf="compile"/>
<dependency org="software.amazon.awssdk" name="protocol-core" rev="${/software.amazon.awssdk/protocol-core}" conf="compile"/>
<dependency org="software.amazon.awssdk" name="regions" rev="${/software.amazon.awssdk/regions}" conf="compile"/>
<dependency org="software.amazon.awssdk" name="s3" rev="${/software.amazon.awssdk/s3}" conf="compile"/>
<dependency org="software.amazon.awssdk" name="sdk-core" rev="${/software.amazon.awssdk/sdk-core}" conf="compile"/>
+ <dependency org="software.amazon.awssdk" name="third-party-jackson-core" rev="${/software.amazon.awssdk/third-party-jackson-core}" conf="compile"/>
<dependency org="software.amazon.awssdk" name="utils" rev="${/software.amazon.awssdk/utils}" conf="compile"/>
<dependency org="software.amazon.ion" name="ion-java" rev="${/software.amazon.ion/ion-java}" conf="compile"/>
diff --git a/solr/contrib/s3-repository/src/java/org/apache/solr/s3/S3BackupRepositoryConfig.java b/solr/contrib/s3-repository/src/java/org/apache/solr/s3/S3BackupRepositoryConfig.java
index 229b224..666fd5f 100644
--- a/solr/contrib/s3-repository/src/java/org/apache/solr/s3/S3BackupRepositoryConfig.java
+++ b/solr/contrib/s3-repository/src/java/org/apache/solr/s3/S3BackupRepositoryConfig.java
@@ -26,29 +26,36 @@ import org.apache.solr.common.util.NamedList;
*/
public class S3BackupRepositoryConfig {
+ public static final String PROFILE = "s3.profile";
public static final String BUCKET_NAME = "s3.bucket.name";
public static final String REGION = "s3.region";
public static final String ENDPOINT = "s3.endpoint";
public static final String PROXY_URL = "s3.proxy.url";
public static final String PROXY_USE_SYSTEM_SETTINGS = "s3.proxy.useSystemSettings";
+ public static final String RETRIES_DISABLE = "s3.retries.disable";
+ private final String profile;
private final String bucketName;
private final String region;
private final String proxyURL;
private final boolean proxyUseSystemSettings;
private final String endpoint;
+ private final boolean disableRetries;
public S3BackupRepositoryConfig(NamedList<?> config) {
+ profile = getStringConfig(config, PROFILE);
region = getStringConfig(config, REGION);
bucketName = getStringConfig(config, BUCKET_NAME);
proxyURL = getStringConfig(config, PROXY_URL);
proxyUseSystemSettings = getBooleanConfig(config, PROXY_USE_SYSTEM_SETTINGS, true);
endpoint = getStringConfig(config, ENDPOINT);
+ disableRetries = getBooleanConfig(config, RETRIES_DISABLE, false);
}
/** Construct a {@link S3StorageClient} from the provided config. */
public S3StorageClient buildClient() {
- return new S3StorageClient(bucketName, region, proxyURL, proxyUseSystemSettings, endpoint);
+ return new S3StorageClient(
+ bucketName, profile, region, proxyURL, proxyUseSystemSettings, endpoint, disableRetries);
}
private static String getStringConfig(NamedList<?> config, String property) {
@@ -62,10 +69,14 @@ public class S3BackupRepositoryConfig {
}
private static int getIntConfig(NamedList<?> config, String property) {
+ return getIntConfig(config, property, 0);
+ }
+
+ private static int getIntConfig(NamedList<?> config, String property, int def) {
String envProp = System.getenv().get(toEnvVar(property));
if (envProp == null) {
Object configProp = config.get(property);
- return configProp instanceof Integer ? (int) configProp : 0;
+ return configProp instanceof Integer ? (int) configProp : def;
} else {
return Integer.parseInt(envProp);
}
diff --git a/solr/contrib/s3-repository/src/java/org/apache/solr/s3/S3StorageClient.java b/solr/contrib/s3-repository/src/java/org/apache/solr/s3/S3StorageClient.java
index e080ca0..47b536d 100644
--- a/solr/contrib/s3-repository/src/java/org/apache/solr/s3/S3StorageClient.java
+++ b/solr/contrib/s3-repository/src/java/org/apache/solr/s3/S3StorageClient.java
@@ -35,15 +35,19 @@ import java.util.stream.Stream;
import org.apache.solr.common.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
import software.amazon.awssdk.awscore.exception.AwsServiceException;
import software.amazon.awssdk.core.exception.SdkClientException;
import software.amazon.awssdk.core.exception.SdkException;
+import software.amazon.awssdk.core.retry.RetryMode;
+import software.amazon.awssdk.core.retry.RetryPolicy;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.http.apache.ApacheHttpClient;
import software.amazon.awssdk.http.apache.ProxyConfiguration;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.S3ClientBuilder;
+import software.amazon.awssdk.services.s3.S3Configuration;
import software.amazon.awssdk.services.s3.model.CommonPrefix;
import software.amazon.awssdk.services.s3.model.Delete;
import software.amazon.awssdk.services.s3.model.DeleteObjectsRequest;
@@ -86,11 +90,16 @@ public class S3StorageClient {
S3StorageClient(
String bucketName,
+ String profile,
String region,
String proxyUrl,
boolean proxyUseSystemSettings,
- String endpoint) {
- this(createInternalClient(region, proxyUrl, proxyUseSystemSettings, endpoint), bucketName);
+ String endpoint,
+ boolean disableRetries) {
+ this(
+ createInternalClient(
+ profile, region, proxyUrl, proxyUseSystemSettings, endpoint, disableRetries),
+ bucketName);
}
@VisibleForTesting
@@ -100,7 +109,17 @@ public class S3StorageClient {
}
private static S3Client createInternalClient(
- String region, String proxyUrl, boolean proxyUseSystemSettings, String endpoint) {
+ String profile,
+ String region,
+ String proxyUrl,
+ boolean proxyUseSystemSettings,
+ String endpoint,
+ boolean disableRetries) {
+ S3Configuration.Builder configBuilder = S3Configuration.builder().pathStyleAccessEnabled(true);
+ if (!StringUtils.isEmpty(profile)) {
+ configBuilder.profileName(profile);
+ }
+
ApacheHttpClient.Builder sdkHttpClientBuilder = ApacheHttpClient.builder();
// If configured, add proxy
ProxyConfiguration.Builder proxyConfigurationBuilder = ProxyConfiguration.builder();
@@ -113,17 +132,52 @@ public class S3StorageClient {
sdkHttpClientBuilder.useIdleConnectionReaper(false);
/*
+ * Retry logic
+ */
+ RetryPolicy retryPolicy;
+ if (disableRetries) {
+ retryPolicy = RetryPolicy.none();
+ } else {
+ RetryMode.Resolver retryModeResolver = RetryMode.resolver();
+ if (!StringUtils.isEmpty(profile)) {
+ retryModeResolver.profileName(profile);
+ }
+ RetryMode retryMode = retryModeResolver.resolve();
+ RetryPolicy.Builder retryPolicyBuilder = RetryPolicy.builder(retryMode);
+
+ // Do not fail fast on rate limiting
+ if (retryMode == RetryMode.ADAPTIVE) {
+ retryPolicyBuilder.fastFailRateLimiting(false);
+ }
+
+ retryPolicy = retryPolicyBuilder.build();
+ }
+
+ /*
+ * Set the default credentials provider
+ */
+ DefaultCredentialsProvider.Builder credentialsProviderBuilder =
+ DefaultCredentialsProvider.builder();
+ if (!StringUtils.isEmpty(profile)) {
+ credentialsProviderBuilder.profileName(profile);
+ }
+
+ /*
* Default s3 client builder loads credentials from disk and handles token refreshes
*/
S3ClientBuilder clientBuilder =
S3Client.builder()
- .serviceConfiguration(builder -> builder.pathStyleAccessEnabled(true))
+ .credentialsProvider(credentialsProviderBuilder.build())
+ .overrideConfiguration(builder -> builder.retryPolicy(retryPolicy))
+ .serviceConfiguration(configBuilder.build())
.httpClient(sdkHttpClientBuilder.build());
if (!StringUtils.isEmpty(endpoint)) {
clientBuilder.endpointOverride(URI.create(endpoint));
}
- clientBuilder.region(Region.of(region));
+ if (!StringUtils.isEmpty(region)) {
+ clientBuilder.region(Region.of(region));
+ }
return clientBuilder.build();
}
diff --git a/solr/contrib/s3-repository/src/test/org/apache/solr/s3/AbstractS3ClientTest.java b/solr/contrib/s3-repository/src/test/org/apache/solr/s3/AbstractS3ClientTest.java
index ec0eb03..c0dc20b 100644
--- a/solr/contrib/s3-repository/src/test/org/apache/solr/s3/AbstractS3ClientTest.java
+++ b/solr/contrib/s3-repository/src/test/org/apache/solr/s3/AbstractS3ClientTest.java
@@ -48,7 +48,13 @@ public class AbstractS3ClientTest extends SolrTestCaseJ4 {
client =
new S3StorageClient(
- BUCKET_NAME, "us-east-1", "", false, "http://localhost:" + S3_MOCK_RULE.getHttpPort());
+ BUCKET_NAME,
+ null,
+ "us-east-1",
+ "",
+ false,
+ "http://localhost:" + S3_MOCK_RULE.getHttpPort(),
+ false);
}
@After
diff --git a/solr/licenses/annotations-2.17.63.jar.sha1 b/solr/licenses/annotations-2.17.63.jar.sha1
new file mode 100644
index 0000000..bc853b6
--- /dev/null
+++ b/solr/licenses/annotations-2.17.63.jar.sha1
@@ -0,0 +1 @@
+599f644340477536866076c5e255197111ebac72
diff --git a/solr/licenses/apache-client-2.16.93.jar.sha1 b/solr/licenses/apache-client-2.16.93.jar.sha1
deleted file mode 100644
index 065efaf..0000000
--- a/solr/licenses/apache-client-2.16.93.jar.sha1
+++ /dev/null
@@ -1 +0,0 @@
-2220df06b69e797a75b6acbbbcfeae0c0ce4066a
diff --git a/solr/licenses/apache-client-2.17.63.jar.sha1 b/solr/licenses/apache-client-2.17.63.jar.sha1
new file mode 100644
index 0000000..cb39231
--- /dev/null
+++ b/solr/licenses/apache-client-2.17.63.jar.sha1
@@ -0,0 +1 @@
+252e49b4abce929c30848b33bd6e9620412c6731
diff --git a/solr/licenses/arns-2.16.93.jar.sha1 b/solr/licenses/arns-2.16.93.jar.sha1
deleted file mode 100644
index 33c6c30..0000000
--- a/solr/licenses/arns-2.16.93.jar.sha1
+++ /dev/null
@@ -1 +0,0 @@
-42bd656446bee5dd10ded2559a2238ade359e265
diff --git a/solr/licenses/arns-2.17.63.jar.sha1 b/solr/licenses/arns-2.17.63.jar.sha1
new file mode 100644
index 0000000..9dd868e
--- /dev/null
+++ b/solr/licenses/arns-2.17.63.jar.sha1
@@ -0,0 +1 @@
+8b76ee1514a65b5891bcf957ad118c8f8ab441b8
diff --git a/solr/licenses/auth-2.16.93.jar.sha1 b/solr/licenses/auth-2.16.93.jar.sha1
deleted file mode 100644
index 0d3793c..0000000
--- a/solr/licenses/auth-2.16.93.jar.sha1
+++ /dev/null
@@ -1 +0,0 @@
-0b90309577ca250cd765e34c0235871884aae63f
diff --git a/solr/licenses/auth-2.17.63.jar.sha1 b/solr/licenses/auth-2.17.63.jar.sha1
new file mode 100644
index 0000000..5567c2d
--- /dev/null
+++ b/solr/licenses/auth-2.17.63.jar.sha1
@@ -0,0 +1 @@
+93adf1d64823a5c81a141b69cf8d9a0a0a3f15e1
diff --git a/solr/licenses/aws-core-2.16.93.jar.sha1 b/solr/licenses/aws-core-2.16.93.jar.sha1
deleted file mode 100644
index 7fa96f7..0000000
--- a/solr/licenses/aws-core-2.16.93.jar.sha1
+++ /dev/null
@@ -1 +0,0 @@
-bd338bb4096ce58aa699fef9062bdca48d21f681
diff --git a/solr/licenses/aws-core-2.17.63.jar.sha1 b/solr/licenses/aws-core-2.17.63.jar.sha1
new file mode 100644
index 0000000..2e4d4b8
--- /dev/null
+++ b/solr/licenses/aws-core-2.17.63.jar.sha1
@@ -0,0 +1 @@
+dceb84de5c839f3b9888d18806235155b2769471
diff --git a/solr/licenses/aws-query-protocol-2.16.93.jar.sha1 b/solr/licenses/aws-query-protocol-2.16.93.jar.sha1
deleted file mode 100644
index 3432dda..0000000
--- a/solr/licenses/aws-query-protocol-2.16.93.jar.sha1
+++ /dev/null
@@ -1 +0,0 @@
-13f5747f759f477b6490c7c98d20db329dd4f931
diff --git a/solr/licenses/aws-query-protocol-2.17.63.jar.sha1 b/solr/licenses/aws-query-protocol-2.17.63.jar.sha1
new file mode 100644
index 0000000..f8a83c0
--- /dev/null
+++ b/solr/licenses/aws-query-protocol-2.17.63.jar.sha1
@@ -0,0 +1 @@
+4fb16d65dde6f2279ac0823de038b11d1b1296a7
diff --git a/solr/licenses/aws-xml-protocol-2.16.93.jar.sha1 b/solr/licenses/aws-xml-protocol-2.16.93.jar.sha1
deleted file mode 100644
index 1ec582f..0000000
--- a/solr/licenses/aws-xml-protocol-2.16.93.jar.sha1
+++ /dev/null
@@ -1 +0,0 @@
-eda32ce27be35ff773e9c26f582284eda83b830b
diff --git a/solr/licenses/aws-xml-protocol-2.17.63.jar.sha1 b/solr/licenses/aws-xml-protocol-2.17.63.jar.sha1
new file mode 100644
index 0000000..ff5c09f
--- /dev/null
+++ b/solr/licenses/aws-xml-protocol-2.17.63.jar.sha1
@@ -0,0 +1 @@
+91259bb6f7be19ec8f2dcb780bb1a8f7681ea790
diff --git a/solr/licenses/http-client-spi-2.16.93.jar.sha1 b/solr/licenses/http-client-spi-2.16.93.jar.sha1
deleted file mode 100644
index 2141fc0..0000000
--- a/solr/licenses/http-client-spi-2.16.93.jar.sha1
+++ /dev/null
@@ -1 +0,0 @@
-cb8b9047e05e416ca1eb00255772a27da5118a61
diff --git a/solr/licenses/http-client-spi-2.17.63.jar.sha1 b/solr/licenses/http-client-spi-2.17.63.jar.sha1
new file mode 100644
index 0000000..9a81213
--- /dev/null
+++ b/solr/licenses/http-client-spi-2.17.63.jar.sha1
@@ -0,0 +1 @@
+3d34c848cb6f007360da6dd75e57b4ff32a712ef
diff --git a/solr/licenses/json-utils-2.17.63.jar.sha1 b/solr/licenses/json-utils-2.17.63.jar.sha1
new file mode 100644
index 0000000..6af220f
--- /dev/null
+++ b/solr/licenses/json-utils-2.17.63.jar.sha1
@@ -0,0 +1 @@
+2bfd957071b0c3c6d5a413b4f00501b67669c3dd
diff --git a/solr/licenses/json-utils-LICENSE-ASL.txt b/solr/licenses/json-utils-LICENSE-ASL.txt
new file mode 100644
index 0000000..1eef70a
--- /dev/null
+++ b/solr/licenses/json-utils-LICENSE-ASL.txt
@@ -0,0 +1,206 @@
+
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+ Note: Other license terms may apply to certain, identified software files contained within or distributed
+ with the accompanying software if such terms are included in the directory containing the accompanying software.
+ Such other license terms will then apply in lieu of the terms of the software license above.
diff --git a/solr/licenses/json-utils-NOTICE.txt b/solr/licenses/json-utils-NOTICE.txt
new file mode 100644
index 0000000..7b5a068
--- /dev/null
+++ b/solr/licenses/json-utils-NOTICE.txt
@@ -0,0 +1,25 @@
+AWS SDK for Java 2.0
+Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+This product includes software developed by
+Amazon Technologies, Inc (http://www.amazon.com/).
+
+**********************
+THIRD PARTY COMPONENTS
+**********************
+This software includes third party software subject to the following copyrights:
+- XML parsing and utility functions from JetS3t - Copyright 2006-2009 James Murty.
+- PKCS#1 PEM encoded private key parsing and utility functions from oauth.googlecode.com - Copyright 1998-2010 AOL Inc.
+- Apache Commons Lang - https://github.com/apache/commons-lang
+- Netty Reactive Streams - https://github.com/playframework/netty-reactive-streams
+- Jackson-core - https://github.com/FasterXML/jackson-core
+- Jackson-dataformat-cbor - https://github.com/FasterXML/jackson-dataformats-binary
+
+The licenses for these third party components are included in LICENSE.txt
+
+- For Apache Commons Lang see also this required NOTICE:
+ Apache Commons Lang
+ Copyright 2001-2020 The Apache Software Foundation
+
+ This product includes software developed at
+ The Apache Software Foundation (https://www.apache.org/).
diff --git a/solr/licenses/metrics-spi-2.16.93.jar.sha1 b/solr/licenses/metrics-spi-2.16.93.jar.sha1
deleted file mode 100644
index 84da088..0000000
--- a/solr/licenses/metrics-spi-2.16.93.jar.sha1
+++ /dev/null
@@ -1 +0,0 @@
-02f5edddd16f42f49e38dcc8dae426d64886e23b
diff --git a/solr/licenses/metrics-spi-2.17.63.jar.sha1 b/solr/licenses/metrics-spi-2.17.63.jar.sha1
new file mode 100644
index 0000000..fbf0817
--- /dev/null
+++ b/solr/licenses/metrics-spi-2.17.63.jar.sha1
@@ -0,0 +1 @@
+0e28a7880239d3a3adf33a72591f3b943793445d
diff --git a/solr/licenses/profiles-2.16.93.jar.sha1 b/solr/licenses/profiles-2.16.93.jar.sha1
deleted file mode 100644
index ae2662c..0000000
--- a/solr/licenses/profiles-2.16.93.jar.sha1
+++ /dev/null
@@ -1 +0,0 @@
-92b5d98f8c4752f26625b9c6ab794f4e89fd224b
diff --git a/solr/licenses/profiles-2.17.63.jar.sha1 b/solr/licenses/profiles-2.17.63.jar.sha1
new file mode 100644
index 0000000..f896e38
--- /dev/null
+++ b/solr/licenses/profiles-2.17.63.jar.sha1
@@ -0,0 +1 @@
+58fab0a5d8cc836465c1124de6a723a322389073
diff --git a/solr/licenses/protocol-core-2.16.93.jar.sha1 b/solr/licenses/protocol-core-2.16.93.jar.sha1
deleted file mode 100644
index 653eae4..0000000
--- a/solr/licenses/protocol-core-2.16.93.jar.sha1
+++ /dev/null
@@ -1 +0,0 @@
-59e92e128a6e6f25d103013cf36edcbf39b9b69d
diff --git a/solr/licenses/protocol-core-2.17.63.jar.sha1 b/solr/licenses/protocol-core-2.17.63.jar.sha1
new file mode 100644
index 0000000..a0846bf
--- /dev/null
+++ b/solr/licenses/protocol-core-2.17.63.jar.sha1
@@ -0,0 +1 @@
+dd41fa62ba7ccf1171279b4e6e29e1cb4706250e
diff --git a/solr/licenses/regions-2.16.93.jar.sha1 b/solr/licenses/regions-2.16.93.jar.sha1
deleted file mode 100644
index c354a3b..0000000
--- a/solr/licenses/regions-2.16.93.jar.sha1
+++ /dev/null
@@ -1 +0,0 @@
-64f03ccb2641588870568b6c2063153cb2ed6966
diff --git a/solr/licenses/regions-2.17.63.jar.sha1 b/solr/licenses/regions-2.17.63.jar.sha1
new file mode 100644
index 0000000..33cc3cb
--- /dev/null
+++ b/solr/licenses/regions-2.17.63.jar.sha1
@@ -0,0 +1 @@
+dc592a4db459766bcedd1df94db11a64ba82477c
diff --git a/solr/licenses/s3-2.16.93.jar.sha1 b/solr/licenses/s3-2.16.93.jar.sha1
deleted file mode 100644
index 898c230..0000000
--- a/solr/licenses/s3-2.16.93.jar.sha1
+++ /dev/null
@@ -1 +0,0 @@
-a585c446adf1ef94153bc2e18e4b35c8005e2816
diff --git a/solr/licenses/s3-2.17.63.jar.sha1 b/solr/licenses/s3-2.17.63.jar.sha1
new file mode 100644
index 0000000..5eb2306
--- /dev/null
+++ b/solr/licenses/s3-2.17.63.jar.sha1
@@ -0,0 +1 @@
+bcd787e10b955fd2e222063a8768fdc66422e8da
diff --git a/solr/licenses/sdk-core-2.16.93.jar.sha1 b/solr/licenses/sdk-core-2.16.93.jar.sha1
deleted file mode 100644
index 3a0511a..0000000
--- a/solr/licenses/sdk-core-2.16.93.jar.sha1
+++ /dev/null
@@ -1 +0,0 @@
-b8e60bed704cff3563f09ad7fb0fdbd0a14f1406
diff --git a/solr/licenses/sdk-core-2.17.63.jar.sha1 b/solr/licenses/sdk-core-2.17.63.jar.sha1
new file mode 100644
index 0000000..26f8f5d
--- /dev/null
+++ b/solr/licenses/sdk-core-2.17.63.jar.sha1
@@ -0,0 +1 @@
+ccee98793365d0c99fa7436b815d317cc35253f2
diff --git a/solr/licenses/third-party-jackson-core-2.17.63.jar.sha1 b/solr/licenses/third-party-jackson-core-2.17.63.jar.sha1
new file mode 100644
index 0000000..0d68109
--- /dev/null
+++ b/solr/licenses/third-party-jackson-core-2.17.63.jar.sha1
@@ -0,0 +1 @@
+2bf16296795e79e4e97eb6c747c47e7437d69741
diff --git a/solr/licenses/third-party-jackson-core-LICENSE-ASL.txt b/solr/licenses/third-party-jackson-core-LICENSE-ASL.txt
new file mode 100644
index 0000000..1eef70a
--- /dev/null
+++ b/solr/licenses/third-party-jackson-core-LICENSE-ASL.txt
@@ -0,0 +1,206 @@
+
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+ Note: Other license terms may apply to certain, identified software files contained within or distributed
+ with the accompanying software if such terms are included in the directory containing the accompanying software.
+ Such other license terms will then apply in lieu of the terms of the software license above.
diff --git a/solr/licenses/third-party-jackson-core-NOTICE.txt b/solr/licenses/third-party-jackson-core-NOTICE.txt
new file mode 100644
index 0000000..7b5a068
--- /dev/null
+++ b/solr/licenses/third-party-jackson-core-NOTICE.txt
@@ -0,0 +1,25 @@
+AWS SDK for Java 2.0
+Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+This product includes software developed by
+Amazon Technologies, Inc (http://www.amazon.com/).
+
+**********************
+THIRD PARTY COMPONENTS
+**********************
+This software includes third party software subject to the following copyrights:
+- XML parsing and utility functions from JetS3t - Copyright 2006-2009 James Murty.
+- PKCS#1 PEM encoded private key parsing and utility functions from oauth.googlecode.com - Copyright 1998-2010 AOL Inc.
+- Apache Commons Lang - https://github.com/apache/commons-lang
+- Netty Reactive Streams - https://github.com/playframework/netty-reactive-streams
+- Jackson-core - https://github.com/FasterXML/jackson-core
+- Jackson-dataformat-cbor - https://github.com/FasterXML/jackson-dataformats-binary
+
+The licenses for these third party components are included in LICENSE.txt
+
+- For Apache Commons Lang see also this required NOTICE:
+ Apache Commons Lang
+ Copyright 2001-2020 The Apache Software Foundation
+
+ This product includes software developed at
+ The Apache Software Foundation (https://www.apache.org/).
diff --git a/solr/licenses/url-connection-client-2.16.93.jar.sha1 b/solr/licenses/url-connection-client-2.16.93.jar.sha1
deleted file mode 100644
index f8eef33..0000000
--- a/solr/licenses/url-connection-client-2.16.93.jar.sha1
+++ /dev/null
@@ -1 +0,0 @@
-34e15b62cf3ea01172272e3410d48320f619006d
diff --git a/solr/licenses/url-connection-client-2.17.63.jar.sha1 b/solr/licenses/url-connection-client-2.17.63.jar.sha1
new file mode 100644
index 0000000..4113710
--- /dev/null
+++ b/solr/licenses/url-connection-client-2.17.63.jar.sha1
@@ -0,0 +1 @@
+2bf1844b1ed514967a61643cba95433db4cacbd2
diff --git a/solr/licenses/utils-2.16.93.jar.sha1 b/solr/licenses/utils-2.16.93.jar.sha1
deleted file mode 100644
index 33d38eb..0000000
--- a/solr/licenses/utils-2.16.93.jar.sha1
+++ /dev/null
@@ -1 +0,0 @@
-51452e9260390fea9b17559668cd3530cc12bfce
diff --git a/solr/licenses/utils-2.17.63.jar.sha1 b/solr/licenses/utils-2.17.63.jar.sha1
new file mode 100644
index 0000000..11bc29c
--- /dev/null
+++ b/solr/licenses/utils-2.17.63.jar.sha1
@@ -0,0 +1 @@
+95d31e6c70180cb5b85713c51b1d5b945777db76
diff --git a/solr/solr-ref-guide/src/making-and-restoring-backups.adoc b/solr/solr-ref-guide/src/making-and-restoring-backups.adoc
index 4a6bb41..85fc954 100644
--- a/solr/solr-ref-guide/src/making-and-restoring-backups.adoc
+++ b/solr/solr-ref-guide/src/making-and-restoring-backups.adoc
@@ -433,6 +433,18 @@ Either way, if your **location** (or s3 object prefix) starts with a `/`, it wil
The repository does not allow backup locations that begin with a `/`.
====
+An example configuration to enable S3 backups and restore can be seen below:
+
+[source,xml]
+----
+<backup>
+ <repository name="s3" class="org.apache.solr.s3.S3BackupRepository" default="false">
+ <str name="s3.bucket.name">my-s3-bucket</str>
+ <str name="s3.region">us-west-2</str>
+ </repository>
+</backup>
+----
+
S3BackupRepository accepts the following options (in `solr.xml`) for overall configuration:
`s3.bucket.name`::
@@ -444,6 +456,18 @@ S3BackupRepository accepts the following options (in `solr.xml`) for overall con
+
The S3 bucket to read and write all backup files to. Can be overridden by setting `S3_BUCKET_NAME` environment variable.
+`s3.profile`::
++
+[%autowidth,frame=none]
+|===
+|Optional |Default: none
+|===
++
+A profile to load AWS settings for from config files.
+Profiles allow for independent settings for multiple S3Repositories.
+Can be overridden by setting `AWS_PROFILE` environment variable or `-Daws.profile` system property.
+For more information on setting configuration per-profile, refer to the https://docs.aws.amazon.com/sdkref/latest/guide/file-format.html#file-format-config[AWS Java SDK documentation]
+
`s3.region`::
+
[%autowidth,frame=none]
@@ -453,7 +477,7 @@ The S3 bucket to read and write all backup files to. Can be overridden by settin
+
A valid Amazon S3 region string where your bucket is provisioned. You must have read and write permissions for this bucket.
For a full list of regions, please reference the https://docs.aws.amazon.com/general/latest/gr/s3.html[S3 documentation].
-Can be overridden by setting `S3_REGION` environment variable.
+Can be overridden by setting `S3_REGION` environment variable, or setting the region in the AWS Configuration file.
`s3.endpoint`::
+
@@ -466,7 +490,6 @@ Explicit S3 endpoint. Not needed under normal operations when using AWS S3 (the
This parameter is helpful if using a mock S3 framework and want to explicitly override where S3 requests are routed, such as when using S3Mock.
Can be overridden by setting `S3_ENDPOINT` environment variable.
-
[NOTE]
====
You can use the `s3.endpoint` option to use this BackupRepository with _s3-compatible_ endpoints.
@@ -505,14 +528,28 @@ The supported proxy system properties are:
* `http.proxyUser`
* `http.proxyPassword`
-An example configuration to enable S3 backups and restore can be seen below:
+`s3.retries.disable`::
++
+[%autowidth,frame=none]
+|===
+|Optional |Default: false
+|===
++
+Disable retries for all S3 operations. This is not recommended.
-[source,xml]
-----
-<backup>
- <repository name="s3" class="org.apache.solr.s3.S3BackupRepository" default="false">
- <str name="s3.bucket.name">my-s3-bucket</str>
- <str name="s3.region">us-west-2</str>
- </repository>
-</backup>
-----
+==== S3 Client Configuration
+
+The AWS Java SDKs provide many ways of setting the configuration for an S3 Client.
+The Solr S3Repository allows these configurations to be set via:
+
+* Environment Variables
+* Java System Properties
+* AWS Configuration File (possibly per-profile)
+
+https://docs.aws.amazon.com/sdkref/latest/guide/settings-global.html[These options] include:
+
+* Region
+* Access Keys
+* Retries
+** RetryMode (`LEGACY`, `STANDARD`, `ADAPTIVE`)
+** Max Attempts