You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "liang yu (Jira)" <ji...@apache.org> on 2024/08/23 03:06:00 UTC
[jira] [Updated] (HIVE-28473) INSERT OVERWRITE LOCAL DIRECTORY writes staging files to wrong hdfs directory
[ https://issues.apache.org/jira/browse/HIVE-28473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
liang yu updated HIVE-28473:
----------------------------
Description:
using HIVE 3.1.3 ; mr engine; HADOOP 3.3.4
It can be reproduced by:
{code:java}
INSERT OVERWRITE LOCAL DIRECTORY "/path/to/local/dir"
select a
from table
group by a; {code}
when I run this sql in hive, it throws exception:
{code:java}
RuntimeException: cannot create staging directory "hdfs:/path/to/local/dir/.hive-stagingxxxx":
Permission denied: user=aaa, access=WRITE, inode="/":hdfs:hdfs:drwxr-xr-x {code}
It tried to write the staging file to a hdfs path which is generated from LOCAL DIRECTORY.
but the staging file should be written to a temp hdfs directory which is created from hiveSessionPath and executionId
was:
using HIVE 3.1.3 ; mr engine; HADOOP 3.3.4
It can be reproduced by:
{code:java}
INSERT OVERWRITE LOCAL DIRECTORY "/path/to/local/dir"
select a
from table
group by a; {code}
when I run this sql in hive, it throws exception:
{code:java}
RuntimeException: cannot create staging directory "hdfs://nameservice/path/to/local/dir/.hive-stagingxxxx":
Permission denied: user=aaa, access=WRITE, inode="/":hdfs:hdfs:drwxr-xr-x {code}
It tried to write the staging file to a hdfs path which is generated from LOCAL DIRECTORY.
but the staging file should be written to a temp hdfs directory which is created from hiveSessionPath and executionId
> INSERT OVERWRITE LOCAL DIRECTORY writes staging files to wrong hdfs directory
> -----------------------------------------------------------------------------
>
> Key: HIVE-28473
> URL: https://issues.apache.org/jira/browse/HIVE-28473
> Project: Hive
> Issue Type: Bug
> Affects Versions: 3.1.3
> Environment: Hadoop 3.3.4
> HIVE 3.1.3
> mapreduce engine
> Reporter: liang yu
> Priority: Major
>
> using HIVE 3.1.3 ; mr engine; HADOOP 3.3.4
>
> It can be reproduced by:
> {code:java}
> INSERT OVERWRITE LOCAL DIRECTORY "/path/to/local/dir"
> select a
> from table
> group by a; {code}
>
> when I run this sql in hive, it throws exception:
> {code:java}
> RuntimeException: cannot create staging directory "hdfs:/path/to/local/dir/.hive-stagingxxxx":
> Permission denied: user=aaa, access=WRITE, inode="/":hdfs:hdfs:drwxr-xr-x {code}
> It tried to write the staging file to a hdfs path which is generated from LOCAL DIRECTORY.
> but the staging file should be written to a temp hdfs directory which is created from hiveSessionPath and executionId
--
This message was sent by Atlassian Jira
(v8.20.10#820010)