You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2018/09/10 22:14:01 UTC
[jira] [Commented] (IMPALA-6442) Misleading file offset reporting
in error messages
[ https://issues.apache.org/jira/browse/IMPALA-6442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16609847#comment-16609847 ]
ASF subversion and git services commented on IMPALA-6442:
---------------------------------------------------------
Commit dbe0f86e005b07133ef38972905ff1dd2ef35efc in impala's branch refs/heads/master from [~yzhangal]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=dbe0f86 ]
IMPALA-6442: Misleading file offset reporting in error messages.
The error message described in IMPALA-6442 incorrectly reported the file offset where the
Parquet footer starts, as if the offset is counted from the file end instead of from the
file beginning. The fix changed the reported file offset to be counted from the beginning
of the Parquet file.
Testing:
Create a small table that contains one row of data with a single column that's bigint and
store it as Parquet. Manually changed the footer size field to be
1) smaller than the original footer size by 1, to trigger the error message fixed by
this jira to be printed, to verify that the fix functions correctly;
2) bigger than the file size, thus to trigger another related error message to be
printed.
Change-Id: I35235e99ea9ceb0d31961dd3b8069f7194f5a2de
Reviewed-on: http://gerrit.cloudera.org:8080/11379
Reviewed-by: Lars Volker <lv...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Misleading file offset reporting in error messages
> --------------------------------------------------
>
> Key: IMPALA-6442
> URL: https://issues.apache.org/jira/browse/IMPALA-6442
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.11.0
> Reporter: Zoltan Ivanfi
> Assignee: Yongjun Zhang
> Priority: Minor
> Labels: parquet
>
> {{HdfsParquetScanner::ProcessFooter}} has an error message "File $0 has invalid file metadata at file offset $1." However, the value reported as "file offset" is an offset from the _end_ of the file, not from its _beginning_. This is very misleading, since without explicitly stating that the offset is from the end, it is usually understood to be counted from the beginning.
> Additionally, although the function name is clearly about a "footer", two comments explicitly mention processing the "header". This falsely suggests that metadata is at the beginning of the file, when in reality it is at the end.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org