[GitHub] [iotdb] MiniSho opened a new pull request, #11142: [IOTDB 6154] Pipe: Switch log/tsfile mode timing in Hybrid mode

["MiniSho (via GitHub)" <gi...@apache.org>]

MiniSho opened a new pull request, #11142:
URL: https://github.com/apache/iotdb/pull/11142

   In this PR, I've make two improvements：
   1. HybridExtractor will first try to do extraction in log mode, and then choose log or tsfile mode to continue extracting, but if (leader data regions num * Wal size) > (maximum size of wal buffer) will cause OOM, so we need to use tsfile mode if the size exceeds the threshold.
   2. PipeRealtimeDataRegionHybridExtractor don't need to use log mode until the historical extractor has consumed all the hitorical data.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [iotdb] SteveYurongSu merged pull request #11142: [IOTDB-6154] Pipe: better algorithm for hybrid mode to switch log/tsfile extraction in iotdb-extractor

["SteveYurongSu (via GitHub)" <gi...@apache.org>]

SteveYurongSu merged PR #11142:
URL: https://github.com/apache/iotdb/pull/11142


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org