[jira] [Resolved] (IMPALA-12745) dump_breakpad_symbols.py's parallelism doesn't work with RPM/DEBs

["Joe McDonnell (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-12745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joe McDonnell resolved IMPALA-12745.
------------------------------------
    Fix Version/s: Impala 4.4.0
       Resolution: Fixed

> dump_breakpad_symbols.py's parallelism doesn't work with RPM/DEBs
> -----------------------------------------------------------------
>
>                 Key: IMPALA-12745
>                 URL: https://issues.apache.org/jira/browse/IMPALA-12745
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Infrastructure
>    Affects Versions: Impala 4.4.0
>            Reporter: Joe McDonnell
>            Assignee: Joe McDonnell
>            Priority: Critical
>             Fix For: Impala 4.4.0
>
>
> When using the "-r" or "--pkg" option, dump_breakpad_symbols.py is extracting the RPM/DEB into a temporary directory. The lifetime of that temporary directory is maintained by yielding tuples from enumerate_pkg_files(). When using parallelism (added in IMPALA-11511), the yield doesn't keep the temporary directory around while the parallel threads are processing, so they fail with:
>  
> {noformat}
> Found debugging info in /tmp/tmpqfZ9MZ/usr/lib/debug/usr/lib/impala/sbin-retail/impalad.debug
> Failed to open ELF file '/tmp/tmpqfZ9MZ/usr/lib/debug/usr/lib/impala/sbin-retail/impalad.debug': No such file or directory
> Failed to write symbol file.
> {noformat}
>  
> Testing shows that this is still a problem with num_processes=1, so there should also be a change to be able to turn off the ThreadPool entirely. Processing OS packages can force the parallelism off for now as they don't benefit much from parallelism.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)