[GitHub] [doris] xiaokang commented on a diff in pull request #10924: [Enhancement] [Memory] Limit memory usage use process actual physical memory

[GitBox <gi...@apache.org>]

xiaokang commented on code in PR #10924:
URL: https://github.com/apache/doris/pull/10924#discussion_r922917682


##########
be/src/util/perf_counters.h:
##########
@@ -95,6 +97,26 @@ class PerfCounters {
     PerfCounters();
     ~PerfCounters();
 
+    // Refactor
+
+    struct ProcStatus {
+        int64_t vm_size = 0;
+        int64_t vm_peak = 0;
+        int64_t vm_rss = 0;
+    };
+
+    static int parser_int(const std::string& state_key);

Review Comment:
   parse_xxx is better



##########
be/src/runtime/exec_env_init.cpp:
##########
@@ -183,7 +183,6 @@ Status ExecEnv::_init_mem_tracker() {
                      << ". Using physical memory instead";
         global_memory_limit_bytes = MemInfo::physical_mem();
     }
-    MemTracker::get_process_tracker()->set_limit(global_memory_limit_bytes);

Review Comment:
   so we do NOT limit total memory by process tracker, and it's just used to track virtual memory of process, right?



##########
be/src/runtime/mem_tracker.h:
##########
@@ -112,7 +113,11 @@ class MemTracker {
     static std::shared_ptr<MemTracker> get_temporary_mem_tracker(const std::string& label);
 
     Status check_sys_mem_info(int64_t bytes) {

Review Comment:
   is bytes still virtual memory?



##########
be/src/olap/rowset/segment_v2/segment.cpp:
##########
@@ -48,9 +48,10 @@ Status Segment::open(io::FileSystem* fs, const std::string& path, uint32_t segme
 Segment::Segment(uint32_t segment_id, const TabletSchema* tablet_schema)
         : _segment_id(segment_id), _tablet_schema(*tablet_schema) {
 #ifndef BE_TEST
-    _mem_tracker = StorageEngine::instance()->tablet_mem_tracker();
+    _mem_tracker = MemTracker::create_virtual_tracker(
+            -1, "Segment", StorageEngine::instance()->tablet_mem_tracker());
 #else
-    _mem_tracker = MemTracker::get_process_tracker();

Review Comment:
   should process_tracker be used in other places?



##########
be/src/http/action/compaction_action.h:
##########
@@ -75,8 +70,6 @@ class CompactionAction : public HttpHandler {
     static std::mutex _compaction_running_mutex;
     /// whether there is manual compaction running
     static bool _is_compaction_running;
-    /// memory tracker
-    std::shared_ptr<MemTracker> _compaction_mem_tracker;

Review Comment:
   why remove _compaction_mem_tracker?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org