[jira] [Commented] (HBASE-27670) Improve FSUtils to directly obtain FSDataOutputStream

["Duo Zhang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-27670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17693985#comment-17693985 ] 

Duo Zhang commented on HBASE-27670:
-----------------------------------

This is because we still need to support old versions of hadoop... I can not recall the which version hadoop adds the builder support, but maybe for newer hbase versions where we drop some old hadoop support we can change to use the API directly.

> Improve FSUtils to directly obtain FSDataOutputStream
> -----------------------------------------------------
>
>                 Key: HBASE-27670
>                 URL: https://issues.apache.org/jira/browse/HBASE-27670
>             Project: HBase
>          Issue Type: Improvement
>         Environment: HBase version: 2.2.3
>            Reporter: alan.zhao
>            Priority: Major
>
> hbase interacts with hdfs and obtains FSDataOutputStream to generate HFiles. In order to support favoredNodes, reflection is used. The DistributedFileSystem has a more direct way to get the FSDataOutputStream，for example:dfs.createFile(path).permission(perm).create()...;  this API allows you to set various parameters, including favoredNodes. I think avoiding reflection can improve performance, and if you agree with me, I can optimize this part of the code;
> Model:hbase-server
> class:FSUtils
>  
> {code:java}
> public static FSDataOutputStream create(Configuration conf, FileSystem fs, Path path,
> FsPermission perm, InetSocketAddress[] favoredNodes) throws IOException {
> if (fs instanceof HFileSystem) {
> FileSystem backingFs = ((HFileSystem) fs).getBackingFs();
> if (backingFs instanceof DistributedFileSystem) {
> // Try to use the favoredNodes version via reflection to allow backwards-
> // compatibility.
> short replication = Short.parseShort(conf.get(ColumnFamilyDescriptorBuilder.DFS_REPLICATION,
> String.valueOf(ColumnFamilyDescriptorBuilder.DEFAULT_DFS_REPLICATION)));
> try {
> return (FSDataOutputStream) (DistributedFileSystem.class
> .getDeclaredMethod("create", Path.class, FsPermission.class, boolean.class, int.class,
> short.class, long.class, Progressable.class, InetSocketAddress[].class)
> .invoke(backingFs, path, perm, true, CommonFSUtils.getDefaultBufferSize(backingFs),
> replication > 0 ? replication : CommonFSUtils.getDefaultReplication(backingFs, path),
> CommonFSUtils.getDefaultBlockSize(backingFs, path), null, favoredNodes));{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)