[GitHub] [skywalking-banyandb] wu-sheng commented on a diff in pull request #249: Write-ahead Logging design doc

["wu-sheng (via GitHub)" <gi...@apache.org>]

wu-sheng commented on code in PR #249:
URL: https://github.com/apache/skywalking-banyandb/pull/249#discussion_r1109128798


##########
docs/concept/wal.md:
##########
@@ -0,0 +1,47 @@
+# Background
+
+Write Ahead Logging (WAL) is a technique used in databases to ensure that data is not lost due to system crashes or other failures. The basic idea of WAL is to log changes to a database in a separate file before applying them to the database itself. This way, if there is a system failure, the database can be recovered by replaying the log of changes from the WAL file.
+BanyanDB leverages the WAL to enhance the data buffer for schema resource writing. In such a system, write operations are first written to the WAL file before being applied to the interval buffer. This ensures that the log is written to disk before the actual data is written. Hence the term "write ahead".
+
+# Format
+
+![](https://github.com/apache/skywalking-website/tree/master/static/doc-graph/banyandb/v0.4.0/wal-format.png)
+
+A segment refers to a block of data in the WAL file that contains a sequence of database changes. Once `rotate` is invoked, a new segment is created to continue logging subsequent changes.
+A "WALEntry" is a data unit representing a series of changes to a Series. Each WALEntry is written to a segment.
+
+WAlEntry contains as follows:
+- Length:8 bytes, which means the length of a WalEntry.
+- Series ID:8 bytes, the same as request Series ID.
+- Count:4 bytes, how many binary/timestamps in one WalEntry.
+- Timestamp:8 bytes.
+- Binary Length:2 bytes.
+- Binary: value in the write request.
+
+# Write process
+
+![](https://github.com/apache/skywalking-website/tree/master/static/doc-graph/banyandb/v0.4.0/wal.png)

Review Comment:
   @hanahmily Is the graph good? If so, we should merge the website side PR, and update the link pointing to skywalking.a.o like this
   
   https://skywalking.apache.org/doc-graph/banyandb/v0.2.0/structure.png



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org