You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Michael Smith (Jira)" <ji...@apache.org> on 2022/11/17 00:18:00 UTC
[jira] [Updated] (IMPALA-11730) Add support for spilling to Ozone
[ https://issues.apache.org/jira/browse/IMPALA-11730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Smith updated IMPALA-11730:
-----------------------------------
Description:
Attempting to use Ozone as a scratch dir via something like
{code:java}
-scratch_dirs=/tmp/scratch,ofs://localhost:9862/impala/tmp'{code}
fails to use the Ozone path with
{code:java}
I1116 16:14:03.494846 919843 status.cc:129] Malformed scratch directory capacity configuration 'ofs://localhost:9862/impala/tmp'
@ 0xf9189f impala::Status::Status()
@ 0x1377524 impala::TmpDir::ParseByteLimit()
@ 0x138e4ba impala::TmpDir::ParseTokens()
@ 0x138e609 impala::TmpDir::Parse()
@ 0x13901bd impala::TmpFileMgr::InitCustom()
@ 0x13916f3 impala::TmpFileMgr::InitCustom()
@ 0x13917dd impala::TmpFileMgr::Init()
@ 0x15a295f impala::ImpalaServer::ImpalaServer()
@ 0x156df01 ImpaladMain()
@ 0xe7cc4c main
@ 0x7f0ac6a74083 __libc_start_main
@ 0xe7caae _start
W1116 16:14:03.508049 919843 tmp-file-mgr.cc:309] Directory ofs://localhost:9862/impala/tmp is not used because Malformed scratch directory capacity configuration 'ofs://localhost:9862/impala/tmp'{code}
We're working on feature parity between HDFS and Ozone, so we should support this.
was:
Attempting to use Ozone as a scratch dir via something like
{code:java}
-scratch_dirs=/tmp/scratch,ofs://localhost:9862/impala/tmp'{code}
fails to use the Ozone path with
{code:java}
I1116 16:14:03.494846 919843 status.cc:129] Malformed scratch directory capacity configuration 'ofs://localhost:9862/impala/tmp'
@ 0xf9189f impala::Status::Status()
@ 0x1377524 impala::TmpDir::ParseByteLimit()
@ 0x138e4ba impala::TmpDir::ParseTokens()
@ 0x138e609 impala::TmpDir::Parse()
@ 0x13901bd impala::TmpFileMgr::InitCustom()
@ 0x13916f3 impala::TmpFileMgr::InitCustom()
@ 0x13917dd impala::TmpFileMgr::Init()
@ 0x15a295f impala::ImpalaServer::ImpalaServer()
@ 0x156df01 ImpaladMain()
@ 0xe7cc4c main
@ 0x7f0ac6a74083 __libc_start_main
@ 0xe7caae _start
W1116 16:14:03.508049 919843 tmp-file-mgr.cc:309] Directory ofs://localhost:9862/impala/tmp is not used because Malformed scratch directory capacity configuration 'ofs://localhost:9862/impala/tmp'{code}
> Add support for spilling to Ozone
> ---------------------------------
>
> Key: IMPALA-11730
> URL: https://issues.apache.org/jira/browse/IMPALA-11730
> Project: IMPALA
> Issue Type: New Feature
> Components: Backend
> Reporter: Michael Smith
> Assignee: Michael Smith
> Priority: Major
> Labels: ozone
>
> Attempting to use Ozone as a scratch dir via something like
> {code:java}
> -scratch_dirs=/tmp/scratch,ofs://localhost:9862/impala/tmp'{code}
> fails to use the Ozone path with
> {code:java}
> I1116 16:14:03.494846 919843 status.cc:129] Malformed scratch directory capacity configuration 'ofs://localhost:9862/impala/tmp'
> @ 0xf9189f impala::Status::Status()
> @ 0x1377524 impala::TmpDir::ParseByteLimit()
> @ 0x138e4ba impala::TmpDir::ParseTokens()
> @ 0x138e609 impala::TmpDir::Parse()
> @ 0x13901bd impala::TmpFileMgr::InitCustom()
> @ 0x13916f3 impala::TmpFileMgr::InitCustom()
> @ 0x13917dd impala::TmpFileMgr::Init()
> @ 0x15a295f impala::ImpalaServer::ImpalaServer()
> @ 0x156df01 ImpaladMain()
> @ 0xe7cc4c main
> @ 0x7f0ac6a74083 __libc_start_main
> @ 0xe7caae _start
> W1116 16:14:03.508049 919843 tmp-file-mgr.cc:309] Directory ofs://localhost:9862/impala/tmp is not used because Malformed scratch directory capacity configuration 'ofs://localhost:9862/impala/tmp'{code}
> We're working on feature parity between HDFS and Ozone, so we should support this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org