You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues-all@impala.apache.org by "Yida Wu (Jira)" <ji...@apache.org> on 2021/12/21 00:51:00 UTC

[jira] [Closed] (IMPALA-10945) Unfavorable change on the S3 scratch path behavior

     [ https://issues.apache.org/jira/browse/IMPALA-10945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yida Wu closed IMPALA-10945.
----------------------------
    Resolution: Fixed

> Unfavorable change on the S3 scratch path behavior
> --------------------------------------------------
>
>                 Key: IMPALA-10945
>                 URL: https://issues.apache.org/jira/browse/IMPALA-10945
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Backend
>    Affects Versions: Impala 4.1.0
>            Reporter: Yida Wu
>            Assignee: Yida Wu
>            Priority: Major
>
> IMPALA-10429 "Support Spill to HDFS" introduces a new behavior to S3 scratch path.
> It added a connection verification logic for the S3 path (for optimization), but the HdfsFsCache requires the path to have a directory behind, like "s3a://host/dir", otherwise it will throw an error. 
> [https://github.com/apache/impala/blob/1a61a8025c87c37921a1bba4c49f754d8bd10bcc/be/src/runtime/tmp-file-mgr.cc#L333]
> The change brings a problem, that may require the user to have a different s3a scratch path. It would be good to be compatible with the path we support in the previous version.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org