You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2022/03/12 07:25:00 UTC
[jira] [Commented] (IMPALA-6636) Use async IO in ORC scanner
[ https://issues.apache.org/jira/browse/IMPALA-6636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17505205#comment-17505205 ]
ASF subversion and git services commented on IMPALA-6636:
---------------------------------------------------------
Commit 4aeffe71355a003de8c1d625c1443a79dba431e5 in impala's branch refs/heads/master from Riza Suminto
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=4aeffe7 ]
IMPALA-11177: Remove DCHECK in useAsyncIoForStream
TestScannersFuzzing::test_fuzz_alltypes tests how impala handles
corruption in data files. IMPALA-6636 adds support for ORC async IO
where it put DCHECK within method useAsyncIoForStream to verify correct
orc::StreamKind. If this part of the ORC test file is corrupted, impalad
will crash in this DCHECK. We should remove this DCHECK and let ORC lib
deals with the corrupted orc::StreamKind itself.
Testing:
- Run and pass TestScannersFuzzing::test_fuzz_alltypes
Change-Id: I70b21badf39869091feff202e7a08a8460abf9db
Reviewed-on: http://gerrit.cloudera.org:8080/18311
Reviewed-by: Quanlong Huang <hu...@gmail.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Use async IO in ORC scanner
> ---------------------------
>
> Key: IMPALA-6636
> URL: https://issues.apache.org/jira/browse/IMPALA-6636
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Quanlong Huang
> Assignee: Riza Suminto
> Priority: Critical
> Fix For: Impala 4.1.0
>
>
> Though ORC-262 has no progress, we can still prefech data and let the ORC lib reading from an in-memory InputStream.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org