You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2020/06/03 07:31:42 UTC
[GitHub] [incubator-doris] vagetablechicken edited a comment on issue #3469: [Memory Engine] Write Pipeline: internal data processing and storage
vagetablechicken edited a comment on issue #3469:
URL: https://github.com/apache/incubator-doris/issues/3469#issuecomment-631289944
### WAL file format - need to rename
WAL file will save multiple partial rows. One write TX's WAL may be split by size => generates multiple {txn_id}_{segment_id}.wal
For the related data structs, such as ParitalRowBatch, refer to https://github.com/decster/choco/blob/b644430e4e540da8ce0a6d13f07223c818706425/src/choco/partial_row_batch.h#L11
A WAL file has the same struct as PartialRowBatch, shown below. WAL file size could be determined by config::write_buffer_size.
The schema part, we could reuse the definition in segment proto.
https://github.com/apache/incubator-doris/blob/d2d95bfa841245d734bcd416b4c429ebe98b8321/gensrc/proto/segment_v2.proto#L158
- [ x ] add magic code(version)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org