You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@ozone.apache.org by "mingchao zhao (Jira)" <ji...@apache.org> on 2022/09/19 13:14:00 UTC

[jira] [Comment Edited] (HDDS-7235) EOFException occurs when executing TPC-DS using o3fs

    [ https://issues.apache.org/jira/browse/HDDS-7235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17606593#comment-17606593 ] 

mingchao zhao edited comment on HDDS-7235 at 9/19/22 1:13 PM:
--------------------------------------------------------------

Looking at the stack is the problem with using SequenceFile. The implementation of this part in Hive is as follows.
 !screenshot-2.png! 
I first verified the logic in a test case of HDFS and found it to be normal.
 !screenshot-3.png! 

But using the above test in Ozone throws an EOFException.  So the behavior of Ozone O3Fs here should be consistent with that of HDFS.
The problem mainly lies in the check of POS in BlockInputStream and ChunkInputStream. BlockInputStream and ChunkInputStream need to be consistent with KeyInputStream.
 !image-2022-09-19-21-12-04-228.png! 


was (Author: micahzhao):
Looking at the stack is the problem with using SequenceFile. The implementation of this part in Hive is as follows.
 !screenshot-2.png! 
I first verified the logic in a test case of HDFS and found it to be normal.
 !screenshot-3.png! 

But using the above test in Ozone throws an EOFException.  So the behavior of Ozone O3Fs here should be consistent with that of HDFS.

> EOFException occurs when executing TPC-DS using o3fs
> ----------------------------------------------------
>
>                 Key: HDDS-7235
>                 URL: https://issues.apache.org/jira/browse/HDDS-7235
>             Project: Apache Ozone
>          Issue Type: Bug
>            Reporter: mingchao zhao
>            Assignee: mingchao zhao
>            Priority: Major
>         Attachments: image-2022-09-19-21-12-04-228.png, screenshot-1.png, screenshot-2.png, screenshot-3.png, syslog
>
>
> When use o3fs to generate TPCDS test data, the following exceptions will occur during the process of converting text to ORC 
> !screenshot-1.png! 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org