[jira] [Commented] (TUBEMQ-125) Add memory pool storage

["Guocheng Zhang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TUBEMQ-125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17147369#comment-17147369 ] 

Guocheng Zhang commented on TUBEMQ-125:
---------------------------------------

A round of changes has been made recently, and I found that if multiple blocks are to be implemented and can be dynamically adjusted, at least 2 locks need to be added to the memory storage block: one lock solves the operation conflict of the increase and decrease of the memory blocks, and one lock solves the memory read write the indexs, and adjust the value of the data flush indexs. Compared with the previous two blocks of memory, the memory pool operation is more complicated and more difficult to manage. 

I will temporarily suspend the change here.

> Add memory pool storage
> -----------------------
>
>                 Key: TUBEMQ-125
>                 URL: https://issues.apache.org/jira/browse/TUBEMQ-125
>             Project: Apache TubeMQ
>          Issue Type: Sub-task
>            Reporter: Guocheng Zhang
>            Assignee: Guocheng Zhang
>            Priority: Major
>         Attachments: screenshot-1.png
>
>
> 3. The number of memory cache blocks should be configurable: the current memory cache is managed according to the fixed configuration of 2 memory blocks per topic. We should allow the business to build more memory cache space based on actual resource conditions;
> ------------------------------------------------------------------------
> I plan to improve the content of this piece like this:
>  !screenshot-1.png! 
> I plan to change the memory for receiving messages from the current fixed 2 blocks to multiple blocks and use them recyclely:
> The number of blocks and  the size of each block in the memory pool is configurable, with a minimum of 2 blocks, with a minimum of 1M; after the memory block is full or triggered, it will switch to the next block of memory to continue reading and writing; memory pool storage combined with improved TUBEMQ-120 and TUBEMQ-123 to write the data in the memory to the disk in an asynchronous batch flushing mode. The overflow failures will be return to the requests if there is no free memory block currently available for switching.
> After this adjustment, it is very convenient for the system where the number of topics in the system is very small but the memory is very rich, and the system's short-term anti-peak ability is also increased.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)