You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/03/07 13:50:00 UTC
[jira] [Commented] (IMPALA-4080) Share codegen work between fragment instances
[ https://issues.apache.org/jira/browse/IMPALA-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697441#comment-17697441 ]
ASF subversion and git services commented on IMPALA-4080:
---------------------------------------------------------
Commit d98ab986a6a2218523ec147f70110300f019150b in impala's branch refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=d98ab986a ]
IMPALA-11223: Use unique id to create codegen instances
When startup flag asm_module_dir is set, impalad will dump the codegen
disassembly to files under that folder. The file name is "id.asm" in
which "id" is the codegen instance id. Before IMPALA-4080 (f2837e9), we
used fragment instance id as the codegen id. After that, since codegen
is done in fragment level (shared by fragment instances), we use query
id instead. This introduces conflicts between different fragments. The
asm files will be overwritten.
The same conflict happens in dumping IR modules (when unopt_module_dir
or opt_module_dir is set).
This changes the codegen instance id to be "QueryID_FragmentName_PID".
The PID suffix is needed since we usually have several impalads running
together on our dev box.
Also adds logs when IR or disassembly are dumped to files. It helps to
know which instance performs the codegen.
Tests:
- Manually verified the asm file names are expected.
Change-Id: I7672906365c916bbe750eeb9906cab38573e6c31
Reviewed-on: http://gerrit.cloudera.org:8080/19505
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Share codegen work between fragment instances
> ---------------------------------------------
>
> Key: IMPALA-4080
> URL: https://issues.apache.org/jira/browse/IMPALA-4080
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.8.0
> Reporter: Tim Armstrong
> Assignee: Bikramjeet Vig
> Priority: Major
> Labels: codegen, multithreading
> Fix For: Impala 4.0.0
>
>
> With multithreading, we should find a way to share codegen modules between fragment instances to avoid redundant compilation, optimisation, etc.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org