You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2022/08/30 12:09:00 UTC
[jira] [Commented] (IMPALA-11457) Ozone parallelism reduced when backends are co-located
[ https://issues.apache.org/jira/browse/IMPALA-11457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597821#comment-17597821 ]
ASF subversion and git services commented on IMPALA-11457:
----------------------------------------------------------
Commit 15fd47ed1a5151bd620e404612c82cf7fe94e9b0 in impala's branch refs/heads/branch-4.1.1 from Michael Smith
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=15fd47ed1 ]
IMPALA-11457 Fix regression with unknown disk id
Fixes an issue where disk ID is set to 65536 incorrectly when storage ID
is unknown due to mixed usage of short and ushort (introduced by
d6b5f82e31732c355af3f3d1a8e5da94ba9c1349). The incorrect unknown disk ID
would result in all local reads going to a single disk queue and
reducing parallelism, which specifically happens with co-located Ozone
endpoints.
This issue caused the test_admission_control_with_multiple_coords to
fail with Ozone.
Change-Id: I571ac0669ceb6a42561594c3f96723d5ed293902
Reviewed-on: http://gerrit.cloudera.org:8080/18792
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-on: http://gerrit.cloudera.org:8080/18914
Tested-by: Quanlong Huang <hu...@gmail.com>
Reviewed-by: Michael Smith <mi...@cloudera.com>
Reviewed-by: Joe McDonnell <jo...@cloudera.com>
> Ozone parallelism reduced when backends are co-located
> ------------------------------------------------------
>
> Key: IMPALA-11457
> URL: https://issues.apache.org/jira/browse/IMPALA-11457
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 4.1.0
> Reporter: Michael Smith
> Assignee: Michael Smith
> Priority: Critical
> Fix For: Impala 4.2.0
>
>
> When Impala and Ozone backends are co-located (at the same IP address), access to Ozone will use a local filesystem access path. It's supposed to assign each backend a separate disk ID so access happens in parallel, but a bug in assigning queues when storage ID is unknown results in all Ozone backends sharing the same queue. This results in reduced parallelism as the queue uses a single I/O thread.
> To make Impala see Ozone as co-located in a dev environment, start Impala with {{--hostname=localhost}} (which is already the default for Ozone).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org