You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/02/04 18:28:00 UTC
[jira] [Commented] (IMPALA-10274) Move impala-python initialization
into the CMake build
[ https://issues.apache.org/jira/browse/IMPALA-10274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17279038#comment-17279038 ]
ASF subversion and git services commented on IMPALA-10274:
----------------------------------------------------------
Commit 60f8f87b09a27618df2ac73c1cc6dcd052f8c60d in impala's branch refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=60f8f87 ]
IMPALA-10274: Initialize impala-python as part of the CMake build
Initializing the impala-python virtualenv takes a couple minutes,
so it is useful to do that in parallel to the rest of the build.
This moves the impala-python initialization to its own step
in the CMake build. It stops using impala-python for commands
invoked from buildall.sh or the CMake build to avoid premature
or concurrent initializations of impala-python. Then, it adds
a dedicated step to initialize impala-python.
Testing:
- Ran a core job and a couple builds
- Rebuilt and verified that impala-python is not reinitialized
if it is already initialized
Change-Id: Ieff51263c55bd234028fed7101c94b4a928590f0
Reviewed-on: http://gerrit.cloudera.org:8080/16607
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Move impala-python initialization into the CMake build
> ------------------------------------------------------
>
> Key: IMPALA-10274
> URL: https://issues.apache.org/jira/browse/IMPALA-10274
> Project: IMPALA
> Issue Type: Improvement
> Components: Infrastructure
> Affects Versions: Impala 4.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Major
>
> Currently, the impala-python virtualenv gets initialized on the first invocation of bin/impala-python. Most of the time, this happens when running bin/gen_build_version.py from buildall.sh. This is a serial part of the build, and the virtualenv initialization takes more than two minutes.
> It would be better to use ordinary python for bin/gen_build_version.py (and any other python commands during the build) and initialize the virtualenv as a specific step in the CMake build. This would allow the virtualenv initialization to happen in parallel to the rest of the build, which saves multiple minutes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org