You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@asterixdb.apache.org by mb...@apache.org on 2026/09/02 01:39:15 UTC
(asterixdb) 06/06: Merge branch 'gerrit/morpheus' into 'master'
This is an automated email from the ASF dual-hosted git repository.
asf-gitbox-commits pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/asterixdb.git
commit e262f07819e6791a21d35a4cce93de91f9d20291
Merge: f0235c25c8 e4fc7506a6
Author: Michael Blow <mi...@couchbase.com>
AuthorDate: Tue Sep 1 14:00:35 2026 -0400
Merge branch 'gerrit/morpheus' into 'master'
* [NO ISSUE][MISC] Update Dependencies to address CVEs (MB-73268)
* [NO ISSUE][MISC] Advance awssdk to 2.54.7 and hadoop to 3.4.3
(MB-73268)
* [NO ISSUE][MISC] Replace io.findify S3 mock with adobe's (MB-73268)
* [NO ISSUE][MISC] Advance parquet to 1.18.0 (MB-73268)
Three conflicts, all of them this branch being ahead of morpheus rather
than behind, so most of the incoming change does not apply.
The version block keeps this branch's values throughout: hadoop stays at
3.5.0 (morpheus is on 3.4.3 and pins it to its awssdk; this branch is
past both and carries its own note about HADOOP-19906), log4j at 2.26.1,
parquet at 1.17.1 rather than 1.18.0 because advancing it here means
advancing the transitives its own NOTICE calls for, and
algebricks/hyracks at 0.3.10-SNAPSHOT. testcontainers stays 1.21.4,
which is @locked.
awssdk is the exception and does move, 2.50.3 to 2.54.7. This branch
manages it by importing software.amazon.awssdk:bom at
${awsjavasdk.version}, so the bump carries every awssdk artifact and the
per-artifact declarations morpheus adds are redundant here. Its
awsjavasdk.crt.version is @derived-from that bom's awscrt.version, so it
moves with it, 0.48.2 to 0.48.4 -- confirmed against
aws-sdk-java-pom:2.54.7. The apache-client dependency morpheus adds to
asterix-external-data is dropped: this branch already declares it there,
and a second declaration fails the ensure-no-duplicate-deps enforcer
rule.
The S3 mock migration is dropped: this branch already runs adobe's
s3mock-testcontainers at 4.12.4, @locked because 5.x wants
testcontainers 2.x which its kafka, postgresql and azure modules cannot
take, and morpheus moves to com.adobe.testing:s3mock 5.2.0 instead. So
asterix-app keeps io.findify with its pinned akka-http-core, and
AwsS3ExternalDatasetTest keeps S3Mock.Builder -- the merge had taken
morpheus's S3MockApplication import and field type while leaving this
branch's construction, which would not have compiled. The
NoSuchBucketException message in testsuite_external_dataset_s3.xml keeps
its form without the trailing full stop, that wording having come from
adobe's mock rather than from the sdk.
What is kept from the incoming test change is the bucket emptying before
delete in createBucket, which is plain awssdk and holds for either mock.
Co-Authored-By: Claude Opus 5 <no...@anthropic.com>
Change-Id: I3b204222e1499be3779dbbcc7e248ae2ae14a4c8
Ext-ref: MB-73268
.../test/external_dataset/aws/AwsS3ExternalDatasetTest.java | 6 ++++++
asterixdb/pom.xml | 10 +++++-----
2 files changed, 11 insertions(+), 5 deletions(-)
diff --cc asterixdb/asterix-app/src/test/java/org/apache/asterix/test/external_dataset/aws/AwsS3ExternalDatasetTest.java
index edd20bf63b,738e19f1f0..d341800db8
--- a/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/external_dataset/aws/AwsS3ExternalDatasetTest.java
+++ b/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/external_dataset/aws/AwsS3ExternalDatasetTest.java
@@@ -518,9 -293,16 +520,13 @@@ public class AwsS3ExternalDatasetTest
}
}
- private static void dropRecreateBucket(String bucketName, String definition, String files) {
- String definitionPath = definition + (definition.endsWith("/") ? "" : "/");
- String[] fileSplits = files.split(",");
-
+ private static void createBucket(String bucketName) {
LOGGER.info("Dropping bucket " + bucketName);
try {
+ // the bucket has to be emptied first, a bucket with content cannot be deleted
+ client.listObjectsV2Paginator(ListObjectsV2Request.builder().bucket(bucketName).build()).contents()
+ .forEach(object -> client
+ .deleteObject(DeleteObjectRequest.builder().bucket(bucketName).key(object.key()).build()));
client.deleteBucket(DELETE_BUCKET_BUILDER.bucket(bucketName).build());
} catch (NoSuchBucketException e) {
// ignore
diff --cc asterixdb/pom.xml
index 5fe76ca8c9,19ef647cda..7d8400661a
--- a/asterixdb/pom.xml
+++ b/asterixdb/pom.xml
@@@ -81,42 -81,31 +81,42 @@@
<shim.stage>none</shim.stage>
<pytestlib.stage>none</pytestlib.stage>
<skip-npm-touch.stage>none</skip-npm-touch.stage>
+ <azurite.npm.install.stage>none</azurite.npm.install.stage>
+ <azurite.install.stage>none</azurite.install.stage>
+ <azurite.stage>none</azurite.stage>
+ <gcs.download.stage>none</gcs.download.stage>
+ <gcs.install.stage>none</gcs.install.stage>
+ <gcs.stage>none</gcs.stage>
<!-- Versions under dependencymanagement or used in many projects via properties -->
- <algebricks.version>0.3.8.3-SNAPSHOT</algebricks.version>
- <hyracks.version>0.3.8.3-SNAPSHOT</hyracks.version>
- <!-- hadoop's S3A initialises against the awssdk it was built against (3.4.3 reads
- ChecksumAlgorithm.CRC64_NVME, added in awssdk 2.30), so the two must move
- together: advancing hadoop alone throws NoSuchFieldError on the thread
- compiling the query, which halts the JVM. -->
- <!-- @pinned-with: ${awsjavasdk.version} -->
- <hadoop.version>3.4.3</hadoop.version>
+ <!-- @locked: keep in sync across all testcontainers modules; 2.x not yet available for kafka/postgresql/azure -->
+ <testcontainers.version>1.21.4</testcontainers.version>
+ <algebricks.version>0.3.10-SNAPSHOT</algebricks.version>
+ <hyracks.version>0.3.10-SNAPSHOT</hyracks.version>
+ <!-- When moving past 3.5.0, check whether the release includes HADOOP-19906 (fixed in 3.4.4,
+ 3.5.1 and 3.6.0), which restores Subject propagation to child threads on JDK 22+. The
+ single-threaded input listing forced in HDFSUtils#configureHadoopAuthentication is a
+ workaround for its absence and should be removed at that point. -->
+ <hadoop.version>3.5.0</hadoop.version>
<jacoco.version>0.7.6.201602180812</jacoco.version>
- <log4j.version>2.25.5</log4j.version>
- <awsjavasdk.version>2.54.7</awsjavasdk.version>
- <s3mock.version>5.2.0</s3mock.version>
- <parquet.version>1.18.0</parquet.version> <!-- NOTICE: please update transitives from parquet below on any change -->
+ <log4j.version>2.26.1</log4j.version>
+ <avro.version>1.12.1</avro.version>
+ <parquet.version>1.17.1</parquet.version> <!-- NOTICE: please update transitives from parquet below on any change -->
<hadoop-awsjavasdk.version>1.12.797</hadoop-awsjavasdk.version>
- <awsjavasdk.version>2.50.3</awsjavasdk.version>
++ <awsjavasdk.version>2.54.7</awsjavasdk.version>
+ <!-- @derived-from: software.amazon.awssdk:aws-sdk-java-pom:${awsjavasdk.version}#awscrt.version -->
- <awsjavasdk.crt.version>0.48.2</awsjavasdk.crt.version>
++ <awsjavasdk.crt.version>0.48.4</awsjavasdk.crt.version>
+
<azureblobjavasdk.version>12.35.1</azureblobjavasdk.version>
<azurecommonjavasdk.version>12.34.1</azurecommonjavasdk.version>
<azureidentity.version>1.18.5</azureidentity.version>
<azuredatalakejavasdk.version>12.28.1</azuredatalakejavasdk.version>
- <azurecore.version>1.59.0</azurecore.version>
- <azurecorehttpnetty.version>1.16.6</azurecorehttpnetty.version>
+ <azurecore.version>1.59.1</azurecore.version>
+ <azurecorehttpnetty.version>1.16.7</azurecorehttpnetty.version>
- <gcsjavasdk.version>2.72.0</gcsjavasdk.version>
+ <!-- keep in sync with hyracks-fullstack/pom.xml -->
+ <google-cloud-storage.version>2.72.0</google-cloud-storage.version>
<hadoop-azuresdk.version>8.6.6</hadoop-azuresdk.version>
<hadoop-gcs.version>hadoop3-2.2.25</hadoop-gcs.version>
<io.opencensus.version>0.31.1</io.opencensus.version>