You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Wei-Chiu Chuang (JIRA)" <ji...@apache.org> on 2019/03/01 00:37:00 UTC
[jira] [Updated] (HDFS-14083) libhdfs logs errors when opened FS
doesn't support ByteBufferReadable
[ https://issues.apache.org/jira/browse/HDFS-14083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wei-Chiu Chuang updated HDFS-14083:
-----------------------------------
Resolution: Won't Fix
Status: Resolved (was: Patch Available)
Close as won't fix. Let's use HDFS-14111 to resolve this bug altogether.
> libhdfs logs errors when opened FS doesn't support ByteBufferReadable
> ---------------------------------------------------------------------
>
> Key: HDFS-14083
> URL: https://issues.apache.org/jira/browse/HDFS-14083
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: libhdfs, native
> Affects Versions: 3.0.3
> Reporter: Pranay Singh
> Assignee: Pranay Singh
> Priority: Minor
> Attachments: HADOOP-15928.001.patch, HADOOP-15928.002.patch, HDFS-14083.003.patch, HDFS-14083.004.patch, HDFS-14083.005.patch, HDFS-14083.006.patch, HDFS-14083.007.patch, HDFS-14083.008.patch, HDFS-14083.009.patch
>
>
> Problem:
> ------------
> There is excessive error logging when a file is opened by libhdfs (DFSClient/HDFS) in S3 environment, this issue is caused because buffered read is not supported in S3 environment, HADOOP-14603 "S3A input stream to support ByteBufferReadable"
> The following message is printed repeatedly in the error log/ to STDERR:
> {code}
> --------------------------------------------------------------------------------------------------
> UnsupportedOperationException: Byte-buffer read unsupported by input streamjava.lang.UnsupportedOperationException: Byte-buffer read unsupported by input stream
> at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:150)
> {code}
> h3. Root cause
> After investigating the issue, it appears that the above exception is printed because
> when a file is opened via {{hdfsOpenFileImpl()}} calls {{readDirect()}} which is hitting this
> exception.
> h3. Fix:
> Since the hdfs client is not initiating the byte buffered read but is happening in a implicit manner, we should not be generating the error log during open of a file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org