[jira] [Resolved] (HBASE-29384) Async Profiler occasionally fails to capture profiling data

["Junegunn Choi (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-29384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Junegunn Choi resolved HBASE-29384.
-----------------------------------
    Fix Version/s: 2.7.0
                   3.0.0-beta-2
                   2.6.3
                   2.5.12
       Resolution: Fixed

> Async Profiler occasionally fails to capture profiling data
> -----------------------------------------------------------
>
>                 Key: HBASE-29384
>                 URL: https://issues.apache.org/jira/browse/HBASE-29384
>             Project: HBase
>          Issue Type: Bug
>          Components: master, regionserver
>            Reporter: JinHyuk Kim
>            Assignee: JinHyuk Kim
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 2.7.0, 3.0.0-beta-2, 2.6.3, 2.5.12
>
>
> h3. *Problem*
> When using the HBase Web UI to start async-profiler, the profiler works correctly on some RegionServers, but on others, the SVG result is never rendered and the page keeps refreshing indefinitely.
> h3. *Root Cause*
> Currently, the profiler output (SVG) is written to the {{/tmp/prof-output-hbase}} directory, which is created only once when the RegionServer starts. However, since {{/tmp}} is often subject to periodic cleanup by the OS (e.g., via {{systemd-tmpfiles-clean}} on Linux), this directory can be removed after some time.
> As a result, if async-profiler is not used for a while after the RegionServer starts, the output directory may no longer exist when profiling is requested. In that case, the profiler fails with an error message, and no SVG result is generated.
> The error message looks like
> {code:java}
> Could not open /tmp/prof-output-hbase/async-prof-pid-xxxxxx-cpu-1.svg{code}
> related code: https://github.com/async-profiler/async-profiler/blob/v1.8.8/src/profiler.cpp#L1246
> h3. *Solution*
> We can ensure that the profiler output directory exists every time profiling is requested.
> {{ProfilerServlet}} now checks for the existence of the output directory and creates it if it has been deleted.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)