[jira] [Created] (HBASE-29384) Async Profiler occasionally fails to capture profiling data

["JinHyuk Kim (Jira)" <ji...@apache.org>]

JinHyuk Kim created HBASE-29384:
-----------------------------------

             Summary: Async Profiler occasionally fails to capture profiling data
                 Key: HBASE-29384
                 URL: https://issues.apache.org/jira/browse/HBASE-29384
             Project: HBase
          Issue Type: Bug
          Components: master, regionserver
            Reporter: JinHyuk Kim
            Assignee: JinHyuk Kim


h3. *Problem*

When using the HBase Web UI to start async-profiler, the profiler works correctly on some RegionServers, but on others, the SVG result is never rendered and the page keeps refreshing indefinitely.
h3. *Root Cause*

Currently, the profiler output (SVG) is written to the {{/tmp/prof-output-hbase}} directory, which is created only once when the RegionServer starts. However, since {{/tmp}} is often subject to periodic cleanup by the OS (e.g., via {{systemd-tmpfiles-clean}} on Linux), this directory can be removed after some time.

As a result, if async-profiler is not used for a while after the RegionServer starts, the output directory may no longer exist when profiling is requested. In that case, the profiler fails with an error message, and no SVG result is generated.

The error message looks like
{code:java}
Could not open /tmp/prof-output-hbase/async-prof-pid-xxxxxx-cpu-1.svg{code}
related code: https://github.com/async-profiler/async-profiler/blob/v1.8.8/src/profiler.cpp#L1246
h3. *Solution*

We can ensure that the profiler output directory exists every time profiling is requested.
{{ProfilerServlet}} now checks for the existence of the output directory and creates it if it has been deleted.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)