[jira] [Commented] (HBASE-28955) Improve lease renew for FanOutOneBlockAsyncDFSOutput

["Istvan Toth (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-28955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17896000#comment-17896000 ] 

Istvan Toth commented on HBASE-28955:
-------------------------------------

I'm not sure I get it, [~zhangduo] .
DFSOutputStream is coming from Hadoop.

Can't we use our usual reflection tricks to search for the Hadoop 3 specific constuctor when creating the dummy DFSOutputStream, and fall back to passing null if it is not available (Since we don't need to pass it it in that case) ?

> Improve lease renew for FanOutOneBlockAsyncDFSOutput
> ----------------------------------------------------
>
>                 Key: HBASE-28955
>                 URL: https://issues.apache.org/jira/browse/HBASE-28955
>             Project: HBase
>          Issue Type: Bug
>          Components: Filesystem Integration, io
>            Reporter: Duo Zhang
>            Assignee: Duo Zhang
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 3.0.0-beta-2
>
>
> When working with hadoop 3.4.x, we saw this in the stdout file
> {noformat}
> Exception in thread "LeaseRenewer:zhangduo@home" java.lang.NullPointerException: Cannot invoke "org.apache.hadoop.hdfs.DFSOutputStream.getNamespace()" because "outputStream" is null
>         at org.apache.hadoop.hdfs.DFSClient.getNamespaces(DFSClient.java:596)
>         at org.apache.hadoop.hdfs.DFSClient.renewLease(DFSClient.java:618)
>         at org.apache.hadoop.hdfs.client.impl.LeaseRenewer.renew(LeaseRenewer.java:425)
>         at org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:445)
>         at org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$800(LeaseRenewer.java:77)
>         at org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:336)
>         at java.base/java.lang.Thread.run(Thread.java:840)
> {noformat}
> This is because in  newer DFSClient implementation, we need to pass namespace when renewer lease so we can not just pass null as DFSOutputStream when calling DFSClient.beginFileLease. We should find a way to deal with it.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)