You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by ma...@apache.org on 2024/08/15 16:27:44 UTC
(cassandra) 01/02: Merge branch 'cassandra-4.1' into cassandra-5.0.0
This is an automated email from the ASF dual-hosted git repository.
maedhroz pushed a commit to branch cassandra-5.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit c91ffe8a9fedc20a71de4601a6969a63ee3e8e9d
Merge: 3afa2585d9 e5fc9b3e14
Author: Caleb Rackliffe <ca...@gmail.com>
AuthorDate: Thu Aug 15 10:24:12 2024 -0500
Merge branch 'cassandra-4.1' into cassandra-5.0.0
* cassandra-4.1:
Use default commitlog settings in test YAMLs
CHANGES.txt | 2 ++
test/conf/cassandra-mtls-backward-compatibility.yaml | 3 ++-
test/conf/cassandra-mtls.yaml | 3 ++-
test/conf/cassandra-murmur.yaml | 3 ++-
test/conf/cassandra-old.yaml | 3 ++-
test/conf/cassandra-pem-jks-sslcontextfactory.yaml | 3 ++-
...assandra-pem-sslcontextfactory-invalidconfiguration.yaml | 3 ++-
...ssandra-pem-sslcontextfactory-mismatching-passwords.yaml | 3 ++-
.../cassandra-pem-sslcontextfactory-unencryptedkeys.yaml | 3 ++-
test/conf/cassandra-pem-sslcontextfactory.yaml | 3 ++-
test/conf/cassandra-seeds.yaml | 3 ++-
.../cassandra-sslcontextfactory-invalidconfiguration.yaml | 3 ++-
test/conf/cassandra-sslcontextfactory.yaml | 3 ++-
test/conf/cassandra.yaml | 3 ++-
test/conf/unit-test-conf/test-native-port.yaml | 3 ++-
.../apache/cassandra/distributed/impl/InstanceConfig.java | 3 ++-
.../distributed/test/ReplicaFilteringProtectionTest.java | 3 +--
.../org/apache/cassandra/simulator/ClusterSimulation.java | 13 ++++++++++++-
.../cassandra/config/FailStartupDuplicateParamsTest.java | 3 ++-
.../config/LoadOldYAMLBackwardCompatibilityTest.java | 2 +-
.../apache/cassandra/config/ParseAndConvertUnitsTest.java | 2 +-
test/unit/org/apache/cassandra/cql3/CQLTester.java | 11 ++++++++---
.../apache/cassandra/db/commitlog/CommitLogReaderTest.java | 9 ++++++++-
23 files changed, 65 insertions(+), 25 deletions(-)
diff --cc CHANGES.txt
index 7e061bda27,0f8d48e83b..061fa7cd50
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,51 -1,4 +1,53 @@@
-4.1.6
+5.0-rc2
+ * Fix direct IO support always being evaluated to false upon the first start of a node (CASSANDRA-19779)
+ * Memtable allocation type unslabbed_heap_buffers_logged will cause an assertion error for TrieMemtables and SegmentedTrieMemtables (CASSANDRA-19835)
++Merged from 4.0:
++ * Use default commitlog settings in test YAMLs (CASSANDRA-19830)
+
+
+5.0-rc1
+ * Move bcpkix-jdk18on to build dependencies, update to 1.78 and explicitly enumerate transitive dependencies (CASSANDRA-19739)
+ * Avoid streams in the common case for UpdateTransaction creation (CASSANDRA-19675)
+ * Only wait until native_transport_timeout for dispatcher to finish (CASSANDRA-19697)
+ * Disable chronicle analytics (CASSANDRA-19656)
+ * Replace Stream iteration with for-loop for StorageProxy::updateCoordinatorWriteLatencyTableMetric (CASSANDRA-19676)
+ * Enforce metric naming contract if scope is used in a metric name (CASSANDRA-19619)
+ * Avoid reading of the same IndexInfo from disk many times for a large partition (CASSANDRA-19557)
+ * Resolve the oldest hints just from descriptors and current writer if available (CASSANDRA-19600)
+ * Optionally fail writes when SAI refuses to index a term value exceeding configured term max size (CASSANDRA-19493)
+ * Vector search can restrict on clustering keys when filtering isn't required (CASSANDRA-19544)
+ * Fix FBUtilities' parsing of gcp cos_containerd kernel versions (CASSANDRA-18594)
+ * Clean up KeyRangeIterator classes (CASSANDRA-19428)
+ * Warn clients about possible consistency violations for filtering queries against multiple mutable columns (CASSANDRA-19489)
+ * Align buffer with commitlog segment size (CASSANDRA-19471)
+ * Ensure SAI indexes empty byte buffers for types that allow them as a valid input (CASSANDRA-19461)
+ * Deprecate Python 3.7 and earlier, but allow cqlsh to run with Python 3.6-3.11 (CASSANDRA-19467)
+ * Set uuid_sstable_identifiers_enabled to true in cassandra-latest.yaml (CASSANDRA-19460)
+ * Revert switching to approxTime in Dispatcher (CASSANDRA-19454)
+ * Add an optimized default configuration to tests and make it available for new users (CASSANDRA-18753)
+ * Fix remote JMX under Java17 (CASSANDRA-19453)
+ * Avoid consistency violations for SAI intersections over unrepaired data at consistency levels requiring reconciliation (CASSANDRA-19018)
+ * Fix NullPointerException in ANN+WHERE when adding rows in another partition (CASSANDRA-19404)
+ * Record latencies for SAI post-filtering reads against local storage (CASSANDRA-18940)
+ * Fix VectorMemoryIndex#update logic to compare vectors. Fix Index view (CASSANDRA-19168)
+ * Deprecate native_transport_port_ssl (CASSANDRA-19392)
+ * Update packaging shell includes (CASSANDRA-19283)
+ * Fix data corruption in VectorCodec when using heap buffers (CASSANDRA-19167)
+ * Avoid over-skipping of key iterators from static column indexes during mixed intersections (CASSANDRA-19278)
+ * Make concurrent_index_builders configurable at runtime (CASSANDRA-19266)
+ * Fix storage_compatibility_mode for streaming (CASSANDRA-19126)
+ * Add support of vector type to cqlsh COPY command (CASSANDRA-19118)
+ * Make CQLSSTableWriter to support building of SAI indexes (CASSANDRA-18714)
+ * Append additional JVM options when using JDK17+ (CASSANDRA-19001)
+ * Upgrade Python driver to 3.29.0 (CASSANDRA-19245)
+ * Creating a SASI index after creating an SAI index does not break secondary index queries (CASSANDRA-18939)
+ * Optionally fail when a non-partition-restricted query is issued against an index (CASSANDRA-18796)
+ * Add a startup check to fail startup when using invalid configuration with certain Kernel and FS type (CASSANDRA-19196)
+ * UCS min_sstable_size should not be lower than target_sstable_size lower bound (CASSANDRA-19112)
+ * Fix the correspondingMessagingVersion of SSTable format and improve TTL overflow tests coverage (CASSANDRA-19197)
+ * Fix resource cleanup after SAI query timeouts (CASSANDRA-19177)
+ * Suppress CVE-2023-6481 (CASSANDRA-19184)
+Merged from 4.1:
* Add native transport deadline, an ultimate deadline for all tasks related to a specific request (CASSANDRA-19534)
* Do not create a role if ALTER ROLE IF EXISTS operates on non-existing role (CASSANDRA-19749)
* Use OpOrder in repairIterator to ensure we don't lose memtables mid-paxos repair (Cassandra-19668)
diff --cc test/conf/cassandra-mtls-backward-compatibility.yaml
index 7e5a2fd29d,0000000000..66c8b21d2e
mode 100644,000000..100644
--- a/test/conf/cassandra-mtls-backward-compatibility.yaml
+++ b/test/conf/cassandra-mtls-backward-compatibility.yaml
@@@ -1,75 -1,0 +1,76 @@@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+#
+# This file is used for testing mTLS authenticators
+#
+cluster_name: Test Cluster
+memtable_allocation_type: offheap_objects
- commitlog_sync: batch
++commitlog_sync: periodic
++commitlog_sync_period: 10s
+commitlog_segment_size: 5MiB
+commitlog_directory: build/test/cassandra/commitlog
+cdc_raw_directory: build/test/cassandra/cdc_raw
+cdc_enabled: false
+hints_directory: build/test/cassandra/hints
+partitioner: org.apache.cassandra.dht.ByteOrderedPartitioner
+listen_address: 127.0.0.1
+storage_port: 7012
+ssl_storage_port: 17012
+start_native_transport: true
+native_transport_port: 9042
+column_index_size: 4KiB
+saved_caches_directory: build/test/cassandra/saved_caches
+data_file_directories:
+ - build/test/cassandra/data
+disk_access_mode: mmap_index_only
+seed_provider:
+ - class_name: org.apache.cassandra.locator.SimpleSeedProvider
+ parameters:
+ - seeds: "127.0.0.1:7012"
+endpoint_snitch: org.apache.cassandra.locator.SimpleSnitch
+dynamic_snitch: true
+incremental_backups: true
+concurrent_compactors: 4
+compaction_throughput: 0MiB/s
+row_cache_class_name: org.apache.cassandra.cache.OHCProvider
+row_cache_size: 16MiB
+user_defined_functions_enabled: true
+scripted_user_defined_functions_enabled: true
+prepared_statements_cache_size: 1MiB
+corrupted_tombstone_strategy: exception
+stream_entire_sstables: true
+stream_throughput_outbound: 23841858MiB/s
+sasi_indexes_enabled: true
+materialized_views_enabled: true
+drop_compact_storage_enabled: true
+file_cache_enabled: true
+auto_hints_cleanup_enabled: true
+default_keyspace_rf: 1
+
+server_encryption_options:
+ internode_encryption: none
+ keystore: test/conf/cassandra_ssl_test.keystore
+ keystore_password: cassandra
+ outbound_keystore: test/conf/cassandra_ssl_test_outbound.keystore
+ outbound_keystore_password: cassandra
+ truststore: test/conf/cassandra_ssl_test.truststore
+ truststore_password: cassandra
+ require_client_auth: true
+internode_authenticator: org.apache.cassandra.auth.AllowAllInternodeAuthenticator
+authenticator: org.apache.cassandra.auth.AllowAllAuthenticator
diff --cc test/conf/cassandra-mtls.yaml
index 84c7f9b2b6,0000000000..d6f1b3e52c
mode 100644,000000..100644
--- a/test/conf/cassandra-mtls.yaml
+++ b/test/conf/cassandra-mtls.yaml
@@@ -1,88 -1,0 +1,89 @@@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+#
+# This file is used for testing mTLS authenticators
+#
+cluster_name: Test Cluster
+memtable_allocation_type: offheap_objects
- commitlog_sync: batch
++commitlog_sync: periodic
++commitlog_sync_period: 10s
+commitlog_segment_size: 5MiB
+commitlog_directory: build/test/cassandra/commitlog
+cdc_raw_directory: build/test/cassandra/cdc_raw
+cdc_enabled: false
+hints_directory: build/test/cassandra/hints
+partitioner: org.apache.cassandra.dht.ByteOrderedPartitioner
+listen_address: 127.0.0.1
+storage_port: 7012
+ssl_storage_port: 17012
+start_native_transport: true
+native_transport_port: 9042
+column_index_size: 4KiB
+saved_caches_directory: build/test/cassandra/saved_caches
+data_file_directories:
+ - build/test/cassandra/data
+disk_access_mode: mmap_index_only
+seed_provider:
+ - class_name: org.apache.cassandra.locator.SimpleSeedProvider
+ parameters:
+ - seeds: "127.0.0.1:7012"
+endpoint_snitch: org.apache.cassandra.locator.SimpleSnitch
+dynamic_snitch: true
+incremental_backups: true
+concurrent_compactors: 4
+compaction_throughput: 0MiB/s
+row_cache_class_name: org.apache.cassandra.cache.OHCProvider
+row_cache_size: 16MiB
+prepared_statements_cache_size: 1MiB
+corrupted_tombstone_strategy: exception
+stream_entire_sstables: true
+stream_throughput_outbound: 23841858MiB/s
+sasi_indexes_enabled: true
+materialized_views_enabled: true
+drop_compact_storage_enabled: true
+file_cache_enabled: true
+auto_hints_cleanup_enabled: true
+default_keyspace_rf: 1
+
+client_encryption_options:
+ enabled: true
+ require_client_auth: true
+ keystore: test/conf/cassandra_ssl_test.keystore
+ keystore_password: cassandra
+ truststore: test/conf/cassandra_ssl_test.truststore
+ truststore_password: cassandra
+
+server_encryption_options:
+ internode_encryption: all
+ enabled: true
+ keystore: test/conf/cassandra_ssl_test.keystore
+ keystore_password: cassandra
+ outbound_keystore: test/conf/cassandra_ssl_test_outbound.keystore
+ outbound_keystore_password: cassandra
+ truststore: test/conf/cassandra_ssl_test.truststore
+ truststore_password: cassandra
+ require_client_auth: true
+internode_authenticator:
+ class_name : org.apache.cassandra.auth.MutualTlsInternodeAuthenticator
+ parameters :
+ validator_class_name: org.apache.cassandra.auth.SpiffeCertificateValidator
+authenticator:
+ class_name : org.apache.cassandra.auth.MutualTlsAuthenticator
+ parameters :
+ validator_class_name: org.apache.cassandra.auth.SpiffeCertificateValidator
diff --cc test/conf/cassandra.yaml
index d04fa43dc7,156267e935..e9ba02c441
--- a/test/conf/cassandra.yaml
+++ b/test/conf/cassandra.yaml
@@@ -5,10 -5,10 +5,11 @@@
cluster_name: Test Cluster
memtable_allocation_type: heap_buffers
# memtable_allocation_type: offheap_objects
- commitlog_sync: batch
+ commitlog_sync: periodic
+ commitlog_sync_period: 10s
commitlog_segment_size: 5MiB
commitlog_directory: build/test/cassandra/commitlog
+commitlog_disk_access_mode: legacy
# commitlog_compression:
# - class_name: LZ4Compressor
cdc_raw_directory: build/test/cassandra/cdc_raw
diff --cc test/simulator/main/org/apache/cassandra/simulator/ClusterSimulation.java
index 92e483487f,0efa0ab93e..f7f9346385
--- a/test/simulator/main/org/apache/cassandra/simulator/ClusterSimulation.java
+++ b/test/simulator/main/org/apache/cassandra/simulator/ClusterSimulation.java
@@@ -51,9 -53,9 +51,10 @@@ import org.apache.cassandra.distributed
import org.apache.cassandra.distributed.api.IIsolatedExecutor.SerializableConsumer;
import org.apache.cassandra.distributed.api.IIsolatedExecutor.SerializableRunnable;
import org.apache.cassandra.distributed.impl.DirectStreamingConnectionFactory;
+ import org.apache.cassandra.distributed.impl.InstanceConfig;
import org.apache.cassandra.distributed.impl.IsolatedExecutor;
import org.apache.cassandra.io.compress.LZ4Compressor;
+import org.apache.cassandra.io.util.FileSystems;
import org.apache.cassandra.service.paxos.BallotGenerator;
import org.apache.cassandra.service.paxos.PaxosPrepare;
import org.apache.cassandra.simulator.RandomSource.Choices;
@@@ -685,10 -684,19 +686,20 @@@ public class ClusterSimulation<S extend
.set("memtable_allocation_type", builder.memoryListener != null ? "unslabbed_heap_buffers_logged" : "heap_buffers")
.set("file_cache_size", "16MiB")
.set("use_deterministic_table_id", true)
- .set("disk_access_mode", "standard")
+ .set("disk_access_mode", disk_access_mode)
- .set("failure_detector", SimulatedFailureDetector.Instance.class.getName());
+ .set("failure_detector", SimulatedFailureDetector.Instance.class.getName())
- .set("commitlog_compression", new ParameterizedClass(LZ4Compressor.class.getName(), emptyMap()))
+ .set("commitlog_sync", "batch");
+
+ // TODO: Add remove() to IInstanceConfig
+ if (config instanceof InstanceConfig)
+ {
+ InstanceConfig instanceConfig = (InstanceConfig) config;
+ instanceConfig.remove("commitlog_sync_period_in_ms");
+ instanceConfig.remove("commitlog_sync_period");
+ }
+
+ if (commitlogCompressed)
+ config.set("commitlog_compression", new ParameterizedClass(LZ4Compressor.class.getName(), emptyMap()));
configUpdater.accept(threadAllocator.update(config));
})
.withInstanceInitializer(new IInstanceInitializer()
diff --cc test/unit/org/apache/cassandra/cql3/CQLTester.java
index be3aa01dff,61bb4839b8..6eade9333e
--- a/test/unit/org/apache/cassandra/cql3/CQLTester.java
+++ b/test/unit/org/apache/cassandra/cql3/CQLTester.java
@@@ -399,9 -316,16 +399,17 @@@ public abstract class CQLTeste
@BeforeClass
public static void setUpClass()
+ {
+ prePrepareServer();
+
+ // Once per-JVM is enough
+ prepareServer();
+ }
+
+ protected static void prePrepareServer()
{
- System.setProperty("cassandra.superuser_setup_delay_ms", "0");
+ CassandraRelevantProperties.SUPERUSER_SETUP_DELAY_MS.setLong(0);
+ ServerTestUtils.daemonInitialization();
if (ROW_CACHE_SIZE_IN_MIB > 0)
DatabaseDescriptor.setRowCacheSizeInMiB(ROW_CACHE_SIZE_IN_MIB);
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org