You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Xiao Chen (JIRA)" <ji...@apache.org> on 2017/09/27 20:29:00 UTC
[jira] [Comment Edited] (HADOOP-14872) CryptoInputStream should
implement unbuffer
[ https://issues.apache.org/jira/browse/HADOOP-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183202#comment-16183202 ]
Xiao Chen edited comment on HADOOP-14872 at 9/27/17 8:28 PM:
-------------------------------------------------------------
Thanks [~jzhuge] and [~stevel@apache.org]!
Looks good to me too. My belated review nits:
- Please improve javadoc of {{StreamCapabilities$StreamCapability#isEqual()}}. 'only in case' could be clearer. This is what the underlying {{equalsIgnoreCase}} javadoc says:
{noformat}
Compares this {@code String} to another {@code String}, ignoring case
* considerations. Two strings are considered equal ignoring case if they
* are of the same length and corresponding characters in the two strings
* are equal ignoring case.
{noformat}
- Name of above method - how about just {{equalsIgnoreCase}} too?
- Unnecessary line break at javadoc of {{DROPBEHIND}}.
- {{CanUnbuffer}} javadoc could use some improvement. 'Implement this interface to indicate that they can clear their buffers on request.'
- Please remember to remove the test line from {{TestUnbuffer}} when commit. :)
Thank you.
was (Author: xiaochen):
Thanks [~jzhuge] and [~stevel@apache.org]!
Looks good to me too. My belated review nits:
- Please improve javadoc of {{StreamCapabilities$StreamCapability#isEqual()}}. 'only in case' could be clearer. This is what the underlying {{equalsIgnoreCase}} javadoc says:
{quote}
Compares this {@code String} to another {@code String}, ignoring case
* considerations. Two strings are considered equal ignoring case if they
* are of the same length and corresponding characters in the two strings
* are equal ignoring case.
{quote}
- Name of above method - how about just {{equalsIgnoreCase}} too?
- Unnecessary line break at javadoc of {{DROPBEHIND}}.
- {{CanUnbuffer}} javadoc could use some improvement. 'Implement this interface to indicate that they can clear their buffers on request.'
- Please remember to remove the test line from {{TestUnbuffer}} when commit. :)
Thank you.
> CryptoInputStream should implement unbuffer
> -------------------------------------------
>
> Key: HADOOP-14872
> URL: https://issues.apache.org/jira/browse/HADOOP-14872
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.6.4
> Reporter: John Zhuge
> Assignee: John Zhuge
> Attachments: HADOOP-14872.001.patch, HADOOP-14872.002.patch, HADOOP-14872.003.patch, HADOOP-14872.004.patch, HADOOP-14872.005.patch
>
>
> Discovered in IMPALA-5909.
> Opening an encrypted HDFS file returns a chain of wrapped input streams:
> {noformat}
> HdfsDataInputStream
> CryptoInputStream
> DFSInputStream
> {noformat}
> If an application such as Impala or HBase calls HdfsDataInputStream#unbuffer, FSDataInputStream#unbuffer will be called:
> {code:java}
> try {
> ((CanUnbuffer)in).unbuffer();
> } catch (ClassCastException e) {
> throw new UnsupportedOperationException("this stream does not " +
> "support unbuffering.");
> }
> {code}
> If the {{in}} class does not implement CanUnbuffer, UOE will be thrown. If the application is not careful, tons of UOEs will show up in logs.
> In comparison, opening an non-encrypted HDFS file returns this chain:
> {noformat}
> HdfsDataInputStream
> DFSInputStream
> {noformat}
> DFSInputStream implements CanUnbuffer.
> It is good for CryptoInputStream to implement CanUnbuffer for 2 reasons:
> * Release buffer, cache, or any other resource when instructed
> * Able to call its wrapped DFSInputStream unbuffer
> * Avoid the UOE described above. Applications may not handle the UOE very well.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org