You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by ma...@apache.org on 2022/02/17 09:49:11 UTC
[cassandra] branch trunk updated (9a8dc6d -> c69e37e)
This is an automated email from the ASF dual-hosted git repository.
marcuse pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git.
from 9a8dc6d Merge branch 'cassandra-4.0' into trunk
new b58a5c8 LeveledCompactionStrategy disk space check improvements
new e53ad64 Merge branch 'cassandra-3.0' into cassandra-3.11
new 702824a Merge branch 'cassandra-3.11' into cassandra-4.0
new c69e37e Merge branch 'cassandra-4.0' into trunk
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
CHANGES.txt | 2 +
.../cassandra/db/compaction/CompactionTask.java | 6 +-
.../db/compaction/LeveledCompactionTask.java | 45 +++++-
.../compaction/writers/CompactionAwareWriter.java | 7 +-
.../writers/MajorLeveledCompactionWriter.java | 6 +
.../compaction/writers/MaxSSTableSizeWriter.java | 6 +
.../SplittingSizeTieredCompactionWriter.java | 8 +-
.../compaction/LeveledCompactionStrategyTest.java | 161 ++++++++++++++++++++-
.../{ => writers}/CompactionAwareWriterTest.java | 48 +++++-
9 files changed, 271 insertions(+), 18 deletions(-)
rename test/unit/org/apache/cassandra/db/compaction/{ => writers}/CompactionAwareWriterTest.java (84%)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-4.0' into trunk
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
marcuse pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit c69e37eca76d96c331807ea23280945afecad1e9
Merge: 9a8dc6d 702824a
Author: Marcus Eriksson <ma...@apache.org>
AuthorDate: Thu Feb 17 10:39:33 2022 +0100
Merge branch 'cassandra-4.0' into trunk
CHANGES.txt | 2 +
.../cassandra/db/compaction/CompactionTask.java | 6 +-
.../db/compaction/LeveledCompactionTask.java | 45 +++++-
.../compaction/writers/CompactionAwareWriter.java | 7 +-
.../writers/MajorLeveledCompactionWriter.java | 6 +
.../compaction/writers/MaxSSTableSizeWriter.java | 6 +
.../SplittingSizeTieredCompactionWriter.java | 8 +-
.../compaction/LeveledCompactionStrategyTest.java | 161 ++++++++++++++++++++-
.../{ => writers}/CompactionAwareWriterTest.java | 48 +++++-
9 files changed, 271 insertions(+), 18 deletions(-)
diff --cc CHANGES.txt
index 6a43be1,4fa9ba3..85204fc
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,98 -1,7 +1,100 @@@
+4.1
+ * Fix improper CDC commit log segments deletion in non-blocking mode (CASSANDRA-17233)
+ * Add support for string concatenations through the + operator (CASSANDRA-17190)
+ * Limit the maximum hints size per host (CASSANDRA-17142)
+ * Add a virtual table for exposing batch metrics (CASSANDRA-17225)
+ * Flatten guardrails config (CASSANDRA-17353)
+ * Instance failed to start up due to NPE in StartupClusterConnectivityChecker (CASSANDRA-17347)
+ * add the shorter version of version flag (-v) in cqlsh (CASSANDRA-17236)
+ * Make vtables accessible via internode messaging (CASSANDRA-17295)
+ * Add support for PEM based key material for SSL (CASSANDRA-17031)
+ * Standardize storage configuration parameters' names. Support unit suffixes. (CASSANDRA-15234)
+ * Remove support for Windows (CASSANDRA-16956)
+ * Runtime-configurable YAML option to prohibit USE statements (CASSANDRA-17318)
+ * When streaming sees a ClosedChannelException this triggers the disk failure policy (CASSANDRA-17116)
+ * Add a virtual table for exposing prepared statements metrics (CASSANDRA-17224)
+ * Remove python 2.x support from cqlsh (CASSANDRA-17242)
+ * Prewarm role and credential caches to avoid timeouts at startup (CASSANDRA-16958)
+ * Make capacity/validity/updateinterval/activeupdate for Auth Caches configurable via nodetool (CASSANDRA-17063)
+ * Added startup check for read_ahead_kb setting (CASSANDRA-16436)
+ * Avoid unecessary array allocations and initializations when performing query checks (CASSANDRA-17209)
+ * Add guardrail for list operations that require read before write (CASSANDRA-17154)
+ * Migrate thresholds for number of keyspaces and tables to guardrails (CASSANDRA-17195)
+ * Remove self-reference in SSTableTidier (CASSANDRA-17205)
+ * Add guardrail for query page size (CASSANDRA-17189)
+ * Allow column_index_size_in_kb to be configurable through nodetool (CASSANDRA-17121)
+ * Emit a metric for number of local read and write calls
+ * Add non-blocking mode for CDC writes (CASSANDRA-17001)
+ * Add guardrails framework (CASSANDRA-17147)
+ * Harden resource management on SSTable components to prevent future leaks (CASSANDRA-17174)
+ * Make nodes more resilient to local unrelated files during startup (CASSANDRA-17082)
+ * repair prepare message would produce a wrong error message if network timeout happened rather than reply wait timeout (CASSANDRA-16992)
+ * Log queries that fail on timeout or unavailable errors up to once per minute by default (CASSANDRA-17159)
+ * Refactor normal/preview/IR repair to standardize repair cleanup and error handling of failed RepairJobs (CASSANDRA-17069)
+ * Log missing peers in StartupClusterConnectivityChecker (CASSANDRA-17130)
+ * Introduce separate rate limiting settings for entire SSTable streaming (CASSANDRA-17065)
+ * Implement Virtual Tables for Auth Caches (CASSANDRA-16914)
+ * Actively update auth cache in the background (CASSANDRA-16957)
+ * Add unix time conversion functions (CASSANDRA-17029)
+ * JVMStabilityInspector.forceHeapSpaceOomMaybe should handle all non-heap OOMs rather than only supporting direct only (CASSANDRA-17128)
+ * Forbid other Future implementations with checkstyle (CASSANDRA-17055)
+ * commit log was switched from non-daemon to daemon threads, which causes the JVM to exit in some case as no non-daemon threads are active (CASSANDRA-17085)
+ * Add a Denylist to block reads and writes on specific partition keys (CASSANDRA-12106)
+ * v4+ protocol did not clean up client warnings, which caused leaking the state (CASSANDRA-17054)
+ * Remove duplicate toCQLString in ReadCommand (CASSANDRA-17023)
+ * Ensure hint window is persistent across restarts of a node (CASSANDRA-14309)
+ * Allow to GRANT or REVOKE multiple permissions in a single statement (CASSANDRA-17030)
+ * Allow to grant permission for all tables in a keyspace (CASSANDRA-17027)
+ * Log time spent writing keys during compaction (CASSANDRA-17037)
+ * Make nodetool compactionstats and sstable_tasks consistent (CASSANDRA-16976)
+ * Add metrics and logging around index summary redistribution (CASSANDRA-17036)
+ * Add configuration options for minimum allowable replication factor and default replication factor (CASSANDRA-14557)
+ * Expose information about stored hints via a nodetool command and a virtual table (CASSANDRA-14795)
+ * Add broadcast_rpc_address to system.local (CASSANDRA-11181)
+ * Add support for type casting in WHERE clause components and in the values of INSERT/UPDATE statements (CASSANDRA-14337)
+ * add credentials file support to CQLSH (CASSANDRA-16983)
+ * Skip remaining bytes in the Envelope buffer when a ProtocolException is thrown to avoid double decoding (CASSANDRA-17026)
+ * Allow reverse iteration of resources during permissions checking (CASSANDRA-17016)
+ * Add feature to verify correct ownership of attached locations on disk at startup (CASSANDRA-16879)
+ * Make SSLContext creation pluggable/extensible (CASSANDRA-16666)
+ * Add soft/hard limits to local reads to protect against reading too much data in a single query (CASSANDRA-16896)
+ * Avoid token cache invalidation for removing a non-member node (CASSANDRA-15290)
+ * Allow configuration of consistency levels on auth operations (CASSANDRA-12988)
+ * Add number of sstables in a compaction to compactionstats output (CASSANDRA-16844)
+ * Upgrade Caffeine to 2.9.2 (CASSANDRA-15153)
+ * Allow DELETE and TRUNCATE to work on Virtual Tables if the implementation allows it (CASSANDRA-16806)
+ * Include SASI components to snapshots (CASSANDRA-15134)
+ * Fix missed wait latencies in the output of `nodetool tpstats -F` (CASSANDRA-16938)
+ * Reduce native transport max frame size to 16MB (CASSANDRA-16886)
+ * Add support for filtering using IN restrictions (CASSANDRA-14344)
+ * Provide a nodetool command to invalidate auth caches (CASSANDRA-16404)
+ * Catch read repair timeout exceptions and add metric (CASSANDRA-16880)
+ * Exclude Jackson 1.x transitive dependency of hadoop* provided dependencies (CASSANDRA-16854)
+ * Add client warnings and abort to tombstone and coordinator reads which go past a low/high watermark (CASSANDRA-16850)
+ * Add TTL support to nodetool snapshots (CASSANDRA-16789)
+ * Allow CommitLogSegmentReader to optionally skip sync marker CRC checks (CASSANDRA-16842)
+ * allow blocking IPs from updating metrics about traffic (CASSANDRA-16859)
+ * Request-Based Native Transport Rate-Limiting (CASSANDRA-16663)
+ * Implement nodetool getauditlog command (CASSANDRA-16725)
+ * Clean up repair code (CASSANDRA-13720)
+ * Background schedule to clean up orphaned hints files (CASSANDRA-16815)
+ * Modify SecondaryIndexManager#indexPartition() to retrieve only columns for which indexes are actually being built (CASSANDRA-16776)
+ * Batch the token metadata update to improve the speed (CASSANDRA-15291)
+ * Reduce the log level on "expected" repair exceptions (CASSANDRA-16775)
+ * Make JMXTimer expose attributes using consistent time unit (CASSANDRA-16760)
+ * Remove check on gossip status from DynamicEndpointSnitch::updateScores (CASSANDRA-11671)
+ * Fix AbstractReadQuery::toCQLString not returning valid CQL (CASSANDRA-16510)
+ * Log when compacting many tombstones (CASSANDRA-16780)
+ * Display bytes per level in tablestats for LCS tables (CASSANDRA-16799)
+ * Add isolated flush timer to CommitLogMetrics and ensure writes correspond to single WaitingOnCommit data points (CASSANDRA-16701)
+ * Add a system property to set hostId if not yet initialized (CASSANDRA-14582)
+ * GossiperTest.testHasVersion3Nodes didn't take into account trunk version changes, fixed to rely on latest version (CASSANDRA-16651)
+ * Update JNA library to 5.9.0 and snappy-java to version 1.1.8.4 (CASSANDRA-17040)
+
4.0.4
+ Merged from 3.0:
* Lazy transaction log replica creation allows incorrect replica content divergence during anticompaction (CASSANDRA-17273)
+ * LeveledCompactionStrategy disk space check improvements (CASSANDRA-17272)
4.0.3
* Deprecate otc_coalescing_strategy, otc_coalescing_window_us, otc_coalescing_enough_coalesced_messages,
diff --cc src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index b0ec8da,3b0e172..ec730c6
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@@ -87,10 -83,10 +87,10 @@@ public class CompactionTask extends Abs
if (partialCompactionsAcceptable() && transaction.originals().size() > 1)
{
// Try again w/o the largest one.
- logger.warn("insufficient space to compact all requested files. {}MiB required, {}",
- logger.warn("insufficient space to compact all requested files. {}MB required, {} for compaction {}",
++ logger.warn("insufficient space to compact all requested files. {}MiB required, {} for compaction {}",
(float) expectedSize / 1024 / 1024,
- StringUtils.join(transaction.originals(), ", "));
-
+ StringUtils.join(transaction.originals(), ", "),
+ transaction.opId());
// Note that we have removed files that are still marked as compacting.
// This suboptimal but ok since the caller will unmark all the sstables at the end.
SSTableReader removedSSTable = cfs.getMaxSizeFile(nonExpiredSSTables);
diff --cc test/unit/org/apache/cassandra/db/compaction/writers/CompactionAwareWriterTest.java
index 68936f5,5e127dd..9e29772
--- a/test/unit/org/apache/cassandra/db/compaction/writers/CompactionAwareWriterTest.java
+++ b/test/unit/org/apache/cassandra/db/compaction/writers/CompactionAwareWriterTest.java
@@@ -15,8 -15,9 +15,8 @@@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
- package org.apache.cassandra.db.compaction;
+ package org.apache.cassandra.db.compaction.writers;
-import java.io.File;
import java.nio.ByteBuffer;
import java.util.*;
@@@ -26,13 -27,13 +26,14 @@@ import org.junit.*
import org.apache.cassandra.cql3.CQLTester;
import org.apache.cassandra.cql3.QueryProcessor;
import org.apache.cassandra.db.*;
- import org.apache.cassandra.db.compaction.writers.CompactionAwareWriter;
- import org.apache.cassandra.db.compaction.writers.DefaultCompactionWriter;
- import org.apache.cassandra.db.compaction.writers.MajorLeveledCompactionWriter;
- import org.apache.cassandra.db.compaction.writers.MaxSSTableSizeWriter;
- import org.apache.cassandra.db.compaction.writers.SplittingSizeTieredCompactionWriter;
+ import org.apache.cassandra.db.compaction.AbstractCompactionStrategy;
+ import org.apache.cassandra.db.compaction.CompactionController;
+ import org.apache.cassandra.db.compaction.CompactionIterator;
+ import org.apache.cassandra.db.compaction.OperationType;
import org.apache.cassandra.db.lifecycle.LifecycleTransaction;
import org.apache.cassandra.io.sstable.format.SSTableReader;
++import org.apache.cassandra.io.util.File;
+ import org.apache.cassandra.schema.MockSchema;
import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.UUIDGen;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org