[jira] [Updated] (IMPALA-9437) Cannot get Ozone file block size after opening the file

["Michael Smith (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-9437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Smith updated IMPALA-9437:
----------------------------------
    Affects Version/s: Impala 4.2.0

> Cannot get Ozone file block size after opening the file
> -------------------------------------------------------
>
>                 Key: IMPALA-9437
>                 URL: https://issues.apache.org/jira/browse/IMPALA-9437
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Backend
>    Affects Versions: Impala 4.2.0
>            Reporter: Sahil Takiar
>            Assignee: Michael Smith
>            Priority: Minor
>
> When creating a tmp file on HDFS, {{HdfsTableSink::CreateNewTmpFile}} first opens the file, and then stats the file ({{hdfsGetPathInfo}}) before actually writing any data to the file or closing the file. HDFS seems to allow this behavior. However, Ozone, S3A, and ABFS do not. Impala does this for HDFS in order to get the block size of the opened file. According to {{HdfsTableSink}} it is possible for HDFS to create a block size with a different one than requested by Impala. So in order to track the correct block size for a file, the file needs to be stat'ed after opening it. For S3A and ABFS this isn't a big deal, because they aren't block based filesystem, but Ozone is. So we should investigate the impact of not having this capability and consider adding it to the Ozone client.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org