You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@horaedb.apache.org by "dracoooooo (via GitHub)" <gi...@apache.org> on 2024/09/08 07:13:25 UTC
[PR] feat: Implement delete operation for WAL based on local storage [horaedb]
dracoooooo opened a new pull request, #1566:
URL: https://github.com/apache/horaedb/pull/1566
## Rationale
Currently the WAL based on the local disk does not support the delete function. This PR implements that functionality.
This is a follow-up task of #1552 and #1556.
## Detailed Changes
1. For each `Segment`, add a hashmap to record the minimum and maximum sequence numbers of all tables within that segment. During `delete` and `write` operations, this hashmap will be updated. During read operations, logs will be filtered based on this hashmap.
2. During the `delete` operation, based on the aforementioned hashmap, if all logs of all tables in a read-only segment (a segment that is not currently being written to) are marked as deleted, the segment file will be physically deleted from the disk.
## Test Plan
Unit test, TSBS and running a script locally that repeatedly inserts data, forcibly kills, and restarts the database process to test persistence.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For additional commands, e-mail: commits-help@horaedb.apache.org
Re: [PR] feat: Implement delete operation for WAL based on local storage [horaedb]
Posted by "jiacai2050 (via GitHub)" <gi...@apache.org>.
jiacai2050 commented on code in PR #1566:
URL: https://github.com/apache/horaedb/pull/1566#discussion_r1749740827
##########
src/wal/src/local_storage_impl/segment.rs:
##########
@@ -370,9 +401,44 @@ impl Segment {
self.record_position.append(positions);
// Update min and max sequence number
- self.min_seq = self.min_seq.min(prev_sequence_num);
- self.max_seq = self.max_seq.max(next_sequence_num - 1);
+ self.min_seq = min(self.min_seq, prev_sequence_num);
+ self.max_seq = max(self.max_seq, next_sequence_num - 1);
+
+ // Update sequence range
+ self.table_ranges
+ .entry(table_id)
+ .and_modify(|seq_range| {
+ seq_range.0 = min(seq_range.0, prev_sequence_num);
+ seq_range.1 = max(seq_range.1, next_sequence_num - 1);
+ })
+ .or_insert((prev_sequence_num, next_sequence_num - 1));
+
+ Ok(())
+ }
+
+ fn mark_deleted(&mut self, table_id: TableId, sequence_num: SequenceNumber) {
+ if let Some(range) = self.table_ranges.get_mut(&table_id) {
+ // If sequence number is MAX, remove the range directly to prevent overflow
Review Comment:
👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For additional commands, e-mail: commits-help@horaedb.apache.org
Re: [PR] feat: Implement delete operation for WAL based on local storage [horaedb]
Posted by "jiacai2050 (via GitHub)" <gi...@apache.org>.
jiacai2050 commented on code in PR #1566:
URL: https://github.com/apache/horaedb/pull/1566#discussion_r1749683028
##########
src/wal/src/local_storage_impl/segment.rs:
##########
@@ -454,6 +515,76 @@ impl SegmentManager {
}
Ok(())
}
+
+ pub fn mark_delete_entries_up_to(
+ &self,
+ location: WalLocation,
+ sequence_num: SequenceNumber,
+ ) -> Result<()> {
+ let current_segment_id = self.current_segment.lock().unwrap().lock().unwrap().id;
+ let mut segments_to_remove: Vec<u64> = Vec::new();
+ let mut all_segments = self.all_segments.lock().unwrap();
+
+ for (_, segment) in all_segments.iter() {
+ let mut segment = segment.lock().unwrap();
+
+ segment.mark_deleted(location.table_id, sequence_num);
Review Comment:
Since the seq in once segment is increasing among all tables, so we can just compare the `sequence_num` with its min/max segment of one segment?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For additional commands, e-mail: commits-help@horaedb.apache.org
Re: [PR] feat: Implement delete operation for WAL based on local storage [horaedb]
Posted by "dracoooooo (via GitHub)" <gi...@apache.org>.
dracoooooo commented on code in PR #1566:
URL: https://github.com/apache/horaedb/pull/1566#discussion_r1757289636
##########
src/wal/src/local_storage_impl/segment.rs:
##########
@@ -454,6 +516,89 @@ impl SegmentManager {
}
Ok(())
}
+
+ pub fn mark_delete_entries_up_to(
+ &self,
+ location: WalLocation,
+ sequence_num: SequenceNumber,
+ ) -> Result<()> {
+ let current_segment_id = self.current_segment.lock().unwrap().lock().unwrap().id;
+ let mut all_segments = self.all_segments.lock().unwrap();
+ let mut segments_to_remove = Vec::new();
+
+ for (_, segment) in all_segments.iter() {
+ let mut guard = segment.lock().unwrap();
+
+ guard.mark_deleted(location.table_id, sequence_num);
+
+ // Delete this segment if it is empty
+ if guard.is_empty() && guard.id != current_segment_id {
+ let mut cache = self.cache.lock().unwrap();
+
+ // Check if segment is already in cache
+ if let Some(index) = cache.iter().position(|(id, _)| *id == guard.id) {
+ cache.remove(index);
+ }
+
+ segments_to_remove.push((guard.id, segment.clone()));
+ }
+ }
+
+ // Delete segments in all_segments
+ for (segment_id, _) in segments_to_remove.iter() {
Review Comment:
`segments_to_remove` is `Vec<(u64, Arc<...>)>`, so there is no `keys()` for this type. Do I need to change it to a hashmap?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For additional commands, e-mail: commits-help@horaedb.apache.org
Re: [PR] feat: Implement delete operation for WAL based on local storage [horaedb]
Posted by "jiacai2050 (via GitHub)" <gi...@apache.org>.
jiacai2050 commented on code in PR #1566:
URL: https://github.com/apache/horaedb/pull/1566#discussion_r1749680761
##########
src/wal/src/local_storage_impl/segment.rs:
##########
@@ -454,6 +515,76 @@ impl SegmentManager {
}
Ok(())
}
+
+ pub fn mark_delete_entries_up_to(
+ &self,
+ location: WalLocation,
+ sequence_num: SequenceNumber,
+ ) -> Result<()> {
+ let current_segment_id = self.current_segment.lock().unwrap().lock().unwrap().id;
+ let mut segments_to_remove: Vec<u64> = Vec::new();
+ let mut all_segments = self.all_segments.lock().unwrap();
+
+ for (_, segment) in all_segments.iter() {
+ let mut segment = segment.lock().unwrap();
+
+ segment.mark_deleted(location.table_id, sequence_num);
+
+ // Delete this segment if it is empty
+ if segment.is_empty() && segment.id != current_segment_id {
+ let mut cache = self.cache.lock().unwrap();
+
+ // Check if segment is already in cache
+ if let Some(index) = cache.iter().position(|id| *id == segment.id) {
+ cache.remove(index);
+ }
+
+ segments_to_remove.push(segment.id);
+ segment.delete()?;
Review Comment:
This is a heavy task, I think we can do this outside all_segment's lock.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For additional commands, e-mail: commits-help@horaedb.apache.org
Re: [PR] feat: Implement delete operation for WAL based on local storage [horaedb]
Posted by "dracoooooo (via GitHub)" <gi...@apache.org>.
dracoooooo commented on code in PR #1566:
URL: https://github.com/apache/horaedb/pull/1566#discussion_r1757319079
##########
src/wal/src/local_storage_impl/segment.rs:
##########
@@ -383,10 +465,13 @@ pub struct SegmentManager {
all_segments: Mutex<HashMap<u64, Arc<Mutex<Segment>>>>,
/// Cache for opened segments
- cache: Mutex<VecDeque<u64>>,
+ cache: Mutex<VecDeque<(u64, Arc<Mutex<Segment>>)>>,
Review Comment:
The `u64` in this tuple represents the segment ID. The reason for storing it is to determine if a segment exists in the cache without needing to acquire the lock for that segment. The newly added `Arc<Mutex<Segment>>` is to remove the original `SegmentManager::get_segment` method, which is redundant and may cause deadlock.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For additional commands, e-mail: commits-help@horaedb.apache.org
Re: [PR] feat: Implement delete operation for WAL based on local storage [horaedb]
Posted by "jiacai2050 (via GitHub)" <gi...@apache.org>.
jiacai2050 commented on code in PR #1566:
URL: https://github.com/apache/horaedb/pull/1566#discussion_r1749687359
##########
src/wal/src/local_storage_impl/segment.rs:
##########
@@ -454,6 +515,76 @@ impl SegmentManager {
}
Ok(())
}
+
+ pub fn mark_delete_entries_up_to(
+ &self,
+ location: WalLocation,
+ sequence_num: SequenceNumber,
+ ) -> Result<()> {
+ let current_segment_id = self.current_segment.lock().unwrap().lock().unwrap().id;
+ let mut segments_to_remove: Vec<u64> = Vec::new();
+ let mut all_segments = self.all_segments.lock().unwrap();
+
+ for (_, segment) in all_segments.iter() {
+ let mut segment = segment.lock().unwrap();
+
+ segment.mark_deleted(location.table_id, sequence_num);
+
+ // Delete this segment if it is empty
+ if segment.is_empty() && segment.id != current_segment_id {
+ let mut cache = self.cache.lock().unwrap();
+
+ // Check if segment is already in cache
+ if let Some(index) = cache.iter().position(|id| *id == segment.id) {
+ cache.remove(index);
+ }
+
+ segments_to_remove.push(segment.id);
+ segment.delete()?;
+ }
+ }
+
+ for segment_id in segments_to_remove {
+ all_segments.remove(&segment_id);
+ }
+
+ Ok(())
+ }
+
+ pub fn get_relevant_segments(
+ &self,
+ table_id: Option<TableId>,
+ start: SequenceNumber,
+ end: SequenceNumber,
+ ) -> Result<Vec<u64>> {
+ // Find all segments that contain the requested sequence numbers
+ let mut relevant_segments = Vec::new();
+
+ let all_segments = self.all_segments.lock().unwrap();
+
+ for (_, segment) in all_segments.iter() {
Review Comment:
```suggestion
for segment in all_segments.values() {
```
##########
src/wal/src/local_storage_impl/segment.rs:
##########
@@ -454,6 +515,76 @@ impl SegmentManager {
}
Ok(())
}
+
+ pub fn mark_delete_entries_up_to(
+ &self,
+ location: WalLocation,
+ sequence_num: SequenceNumber,
+ ) -> Result<()> {
+ let current_segment_id = self.current_segment.lock().unwrap().lock().unwrap().id;
+ let mut segments_to_remove: Vec<u64> = Vec::new();
+ let mut all_segments = self.all_segments.lock().unwrap();
+
+ for (_, segment) in all_segments.iter() {
+ let mut segment = segment.lock().unwrap();
+
+ segment.mark_deleted(location.table_id, sequence_num);
+
+ // Delete this segment if it is empty
+ if segment.is_empty() && segment.id != current_segment_id {
+ let mut cache = self.cache.lock().unwrap();
+
+ // Check if segment is already in cache
+ if let Some(index) = cache.iter().position(|id| *id == segment.id) {
+ cache.remove(index);
+ }
+
+ segments_to_remove.push(segment.id);
+ segment.delete()?;
+ }
+ }
+
+ for segment_id in segments_to_remove {
+ all_segments.remove(&segment_id);
+ }
+
+ Ok(())
+ }
+
+ pub fn get_relevant_segments(
+ &self,
+ table_id: Option<TableId>,
+ start: SequenceNumber,
+ end: SequenceNumber,
+ ) -> Result<Vec<u64>> {
+ // Find all segments that contain the requested sequence numbers
+ let mut relevant_segments = Vec::new();
+
+ let all_segments = self.all_segments.lock().unwrap();
+
+ for (_, segment) in all_segments.iter() {
Review Comment:
```suggestion
for segment in all_segments.values() {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For additional commands, e-mail: commits-help@horaedb.apache.org
Re: [PR] feat: Implement delete operation for WAL based on local storage [horaedb]
Posted by "jiacai2050 (via GitHub)" <gi...@apache.org>.
jiacai2050 commented on code in PR #1566:
URL: https://github.com/apache/horaedb/pull/1566#discussion_r1756527410
##########
src/wal/src/local_storage_impl/segment.rs:
##########
@@ -454,6 +516,89 @@ impl SegmentManager {
}
Ok(())
}
+
+ pub fn mark_delete_entries_up_to(
+ &self,
+ location: WalLocation,
+ sequence_num: SequenceNumber,
+ ) -> Result<()> {
+ let current_segment_id = self.current_segment.lock().unwrap().lock().unwrap().id;
+ let mut all_segments = self.all_segments.lock().unwrap();
+ let mut segments_to_remove = Vec::new();
+
+ for (_, segment) in all_segments.iter() {
+ let mut guard = segment.lock().unwrap();
+
+ guard.mark_deleted(location.table_id, sequence_num);
+
+ // Delete this segment if it is empty
+ if guard.is_empty() && guard.id != current_segment_id {
+ let mut cache = self.cache.lock().unwrap();
+
+ // Check if segment is already in cache
+ if let Some(index) = cache.iter().position(|(id, _)| *id == guard.id) {
+ cache.remove(index);
+ }
+
+ segments_to_remove.push((guard.id, segment.clone()));
+ }
+ }
+
+ // Delete segments in all_segments
+ for (segment_id, _) in segments_to_remove.iter() {
+ all_segments.remove(segment_id);
+ }
+
+ drop(all_segments);
+
+ // Delete segments on disk
+ for (_, segment) in segments_to_remove.iter() {
Review Comment:
values()
##########
src/wal/src/local_storage_impl/segment.rs:
##########
@@ -1300,4 +1464,63 @@ mod tests {
let runtime = Arc::new(Builder::default().build().unwrap());
runtime.block_on(test_multi_segment_write_and_read_inner(runtime.clone()));
}
+
+ #[test]
+ fn test_region_mark_delete_entries_up_to() {
+ const SEGMENT_SIZE: usize = 4096;
+
+ let dir = tempdir().unwrap();
+ let runtime = Arc::new(Builder::default().build().unwrap());
+
+ // Create a new region
+ let region = Region::new(
+ 1,
+ 2,
+ SEGMENT_SIZE,
+ dir.path().to_str().unwrap().to_string(),
+ runtime.clone(),
+ )
+ .unwrap();
+ let region = Arc::new(region);
+
+ // Write some log entries
+ let location = WalLocation::new(1, 1); // region_id = 1, table_id = 1
+ let mut sequence = MIN_SEQUENCE_NUMBER + 1;
+
+ for _i in 0..10 {
+ let log_entries = 0..100;
+ let log_batch_encoder = LogBatchEncoder::create(location);
+ let log_batch = log_batch_encoder
+ .encode_batch(log_entries.clone().map(|v| MemoryPayload { val: v }))
+ .expect("should succeed to encode payloads");
+
+ let write_ctx = WriteContext::default();
+ region.write(&write_ctx, &log_batch).unwrap();
Review Comment:
We need to check the returned seq is expected.
##########
src/wal/src/local_storage_impl/segment.rs:
##########
@@ -454,6 +516,89 @@ impl SegmentManager {
}
Ok(())
}
+
+ pub fn mark_delete_entries_up_to(
+ &self,
+ location: WalLocation,
+ sequence_num: SequenceNumber,
+ ) -> Result<()> {
+ let current_segment_id = self.current_segment.lock().unwrap().lock().unwrap().id;
+ let mut all_segments = self.all_segments.lock().unwrap();
+ let mut segments_to_remove = Vec::new();
+
+ for (_, segment) in all_segments.iter() {
+ let mut guard = segment.lock().unwrap();
+
+ guard.mark_deleted(location.table_id, sequence_num);
+
+ // Delete this segment if it is empty
+ if guard.is_empty() && guard.id != current_segment_id {
Review Comment:
Since current segment may changed, the check here should be `guard.id < current_segment_id`.
##########
src/wal/src/local_storage_impl/segment.rs:
##########
@@ -1300,4 +1464,63 @@ mod tests {
let runtime = Arc::new(Builder::default().build().unwrap());
runtime.block_on(test_multi_segment_write_and_read_inner(runtime.clone()));
}
+
+ #[test]
+ fn test_region_mark_delete_entries_up_to() {
+ const SEGMENT_SIZE: usize = 4096;
+
+ let dir = tempdir().unwrap();
+ let runtime = Arc::new(Builder::default().build().unwrap());
+
+ // Create a new region
+ let region = Region::new(
+ 1,
+ 2,
+ SEGMENT_SIZE,
+ dir.path().to_str().unwrap().to_string(),
+ runtime.clone(),
+ )
+ .unwrap();
+ let region = Arc::new(region);
+
+ // Write some log entries
+ let location = WalLocation::new(1, 1); // region_id = 1, table_id = 1
+ let mut sequence = MIN_SEQUENCE_NUMBER + 1;
+
+ for _i in 0..10 {
+ let log_entries = 0..100;
+ let log_batch_encoder = LogBatchEncoder::create(location);
+ let log_batch = log_batch_encoder
+ .encode_batch(log_entries.clone().map(|v| MemoryPayload { val: v }))
+ .expect("should succeed to encode payloads");
+
+ let write_ctx = WriteContext::default();
+ region.write(&write_ctx, &log_batch).unwrap();
+ sequence += 100;
+ }
+
+ // Expect more than one segment
+ {
+ let all_segments = region.segment_manager.all_segments.lock().unwrap();
+ assert!(
+ all_segments.len() > 1,
+ "Expected multiple segments, but got {}",
+ all_segments.len()
+ );
+ }
+
+ // Mark delete entries up to sequence - 1, so only the last segment should
+ // remain
+ let mark_delete_sequence = sequence - 1;
+ region
+ .mark_delete_entries_up_to(location, mark_delete_sequence)
+ .unwrap();
+
+ {
+ let all_segments = region.segment_manager.all_segments.lock().unwrap();
+ assert_eq!(all_segments.len(), 1);
Review Comment:
We also need to check file inside wal directory is 1.
##########
src/wal/src/local_storage_impl/segment.rs:
##########
@@ -454,6 +516,89 @@ impl SegmentManager {
}
Ok(())
}
+
+ pub fn mark_delete_entries_up_to(
+ &self,
+ location: WalLocation,
+ sequence_num: SequenceNumber,
+ ) -> Result<()> {
+ let current_segment_id = self.current_segment.lock().unwrap().lock().unwrap().id;
+ let mut all_segments = self.all_segments.lock().unwrap();
+ let mut segments_to_remove = Vec::new();
+
+ for (_, segment) in all_segments.iter() {
+ let mut guard = segment.lock().unwrap();
+
+ guard.mark_deleted(location.table_id, sequence_num);
+
+ // Delete this segment if it is empty
+ if guard.is_empty() && guard.id != current_segment_id {
+ let mut cache = self.cache.lock().unwrap();
+
+ // Check if segment is already in cache
+ if let Some(index) = cache.iter().position(|(id, _)| *id == guard.id) {
+ cache.remove(index);
+ }
+
+ segments_to_remove.push((guard.id, segment.clone()));
+ }
+ }
+
+ // Delete segments in all_segments
+ for (segment_id, _) in segments_to_remove.iter() {
Review Comment:
keys()
##########
src/wal/src/local_storage_impl/segment.rs:
##########
@@ -383,10 +465,13 @@ pub struct SegmentManager {
all_segments: Mutex<HashMap<u64, Arc<Mutex<Segment>>>>,
/// Cache for opened segments
- cache: Mutex<VecDeque<u64>>,
+ cache: Mutex<VecDeque<(u64, Arc<Mutex<Segment>>)>>,
Review Comment:
Why add seq here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For additional commands, e-mail: commits-help@horaedb.apache.org
Re: [PR] feat: Implement delete operation for WAL based on local storage [horaedb]
Posted by "jiacai2050 (via GitHub)" <gi...@apache.org>.
jiacai2050 merged PR #1566:
URL: https://github.com/apache/horaedb/pull/1566
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For additional commands, e-mail: commits-help@horaedb.apache.org
Re: [PR] feat: Implement delete operation for WAL based on local storage [horaedb]
Posted by "dracoooooo (via GitHub)" <gi...@apache.org>.
dracoooooo commented on code in PR #1566:
URL: https://github.com/apache/horaedb/pull/1566#discussion_r1753341964
##########
src/wal/src/local_storage_impl/segment.rs:
##########
@@ -454,6 +515,76 @@ impl SegmentManager {
}
Ok(())
}
+
+ pub fn mark_delete_entries_up_to(
+ &self,
+ location: WalLocation,
+ sequence_num: SequenceNumber,
+ ) -> Result<()> {
+ let current_segment_id = self.current_segment.lock().unwrap().lock().unwrap().id;
+ let mut segments_to_remove: Vec<u64> = Vec::new();
Review Comment:
I think no if we move `segment.delete()` outside all_segment's lock,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For additional commands, e-mail: commits-help@horaedb.apache.org
Re: [PR] feat: Implement delete operation for WAL based on local storage [horaedb]
Posted by "jiacai2050 (via GitHub)" <gi...@apache.org>.
jiacai2050 commented on code in PR #1566:
URL: https://github.com/apache/horaedb/pull/1566#discussion_r1749686599
##########
src/wal/src/local_storage_impl/segment.rs:
##########
@@ -454,6 +515,76 @@ impl SegmentManager {
}
Ok(())
}
+
+ pub fn mark_delete_entries_up_to(
+ &self,
+ location: WalLocation,
+ sequence_num: SequenceNumber,
+ ) -> Result<()> {
+ let current_segment_id = self.current_segment.lock().unwrap().lock().unwrap().id;
+ let mut segments_to_remove: Vec<u64> = Vec::new();
Review Comment:
Can we avoid this vec with `HashMap::retain`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@horaedb.apache.org
For additional commands, e-mail: commits-help@horaedb.apache.org