You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Tim Armstrong (Jira)" <ji...@apache.org> on 2020/04/01 18:31:00 UTC
[jira] [Assigned] (IMPALA-7138) Fix detection and handling of
Device Mapper volumes
[ https://issues.apache.org/jira/browse/IMPALA-7138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong reassigned IMPALA-7138:
-------------------------------------
Assignee: Tim Armstrong
> Fix detection and handling of Device Mapper volumes
> ---------------------------------------------------
>
> Key: IMPALA-7138
> URL: https://issues.apache.org/jira/browse/IMPALA-7138
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 3.0, Impala 2.12.0
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Labels: resource-management
>
> I've heard a couple of instances of Impala doing the wrong thing when multiple scratch directories are present on a single DeviceMapper logical volume. E.g. in the following setup, we only use one directory as a disk.
> {noformat}
> >> mount list
> /dev/mapper/xx01-lv_data01 on /data01 type ext4 (rw,relatime,data=ordered)
> /dev/mapper/xx02-lv_data02 on /data02 type ext4 (rw,relatime,data=ordered)
> /dev/mapper/xx03-lv_data03 on /data03 type ext4 (rw,relatime,data=ordered)
> /dev/mapper/xx04-lv_data04 on /data04 type ext4 (rw,relatime,data=ordered)
> /dev/mapper/xx05-lv_data05 on /data05 type ext4 (rw,relatime,data=ordered)
> /dev/mapper/xx06-lv_data06 on /data06 type ext4 (rw,relatime,data=ordered)
> /dev/mapper/xx07-lv_data07 on /data07 type ext4 (rw,relatime,data=ordered)
> /dev/mapper/xx08-lv_data08 on /data08 type ext4 (rw,relatime,data=ordered)
> /dev/mapper/xx09-lv_data09 on /data09 type ext4 (rw,relatime,data=ordered)
> /dev/mapper/xx10-lv_data10 on /data10 type ext4 (rw,relatime,data=ordered)
> /dev/mapper/xx11-lv_data11 on /data11 type ext4 (rw,relatime,data=ordered)
> /dev/mapper/xx12-lv_data12 on /data12 type ext4 (rw,relatime,data=ordered)
> Scratch dirs are:
> /data01/impala/impalad,/data02/impala/impalad,/data03/impala/impalad,/data04/impala/impalad,/data05/impala/impalad,/data06/impala/impalad,/data07/impala/impalad,/data08/impala/impalad,/data09/impala/impalad,/data10/impala/impalad,/data11/impala/impalad,/data12/impala/impalad
> ~~~ tmp-file-mgr.cc:122] Using scratch directory /data01/impala/impalad/impala-scratch on disk13.
> {noformat}
> A workaround is to set --allow_multiple_scratch_dirs_per_device=true
> There are a few questions here:
> # Does the deduplication logic even make sense to have?
> # Do we need to fix how we treat Device mapper volumes?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org