You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2019/02/22 16:04:00 UTC
[jira] [Commented] (HADOOP-16138) hadoop fs mkdir of nonexistent
abfs container raises NPE
[ https://issues.apache.org/jira/browse/HADOOP-16138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16775313#comment-16775313 ]
Steve Loughran commented on HADOOP-16138:
-----------------------------------------
{code}
2019-02-22 16:00:05,804 [main] DEBUG azurebfs.AzureBlobFileSystem (AzureBlobFileSystem.java:initialize(101)) - Initializing AzureBlobFileSystem for abfs://container2@abfswales1.dfs.core.windows.net/
2019-02-22 16:00:06,082 [main] DEBUG services.AbfsClientThrottlingIntercept (AbfsClientThrottlingIntercept.java:initializeSingleton(62)) - Client-side throttling is enabled for the ABFS file system.
2019-02-22 16:00:06,102 [main] DEBUG azurebfs.AzureBlobFileSystem (AzureBlobFileSystem.java:getFileStatus(434)) - AzureBlobFileSystem.getFileStatus path: abfs://container2@abfswales1.dfs.core.windows.net/
2019-02-22 16:00:06,102 [main] DEBUG azurebfs.AzureBlobFileSystem (AzureBlobFileSystem.java:performAbfsAuthCheck(1101)) - ABFS authorizer is not initialized. No authorization check will be performed.
2019-02-22 16:00:06,102 [main] DEBUG azurebfs.AzureBlobFileSystemStore (AzureBlobFileSystemStore.java:getIsNamespaceEnabled(200)) - Get root ACL status
2019-02-22 16:00:06,132 [main] DEBUG services.AbfsClient (AbfsRestOperation.java:executeHttpOperation(152)) - Signing request with shared key
2019-02-22 16:00:06,136 [main] DEBUG services.SharedKeyCredentials (SharedKeyCredentials.java:signRequest(96)) - Signing request with timestamp of Fri, 22 Feb 2019 16:00:06 GMT and signature SharedKey /* snip */
2019-02-22 16:00:06,136 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(51)) - Response Headers
2019-02-22 16:00:06,137 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Accept-Charset=utf-8
2019-02-22 16:00:06,137 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - x-ms-version=2018-11-09
2019-02-22 16:00:06,137 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Accept=application/json, application/octet-stream
2019-02-22 16:00:06,137 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - x-ms-date=Fri, 22 Feb 2019 16:00:06 GMT
2019-02-22 16:00:06,137 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - User-Agent=Azure Blob FS/3.3.0-SNAPSHOT (JavaJRE 1.8.0_121; MacOSX 10.14; openssl-1.0) unknown
2019-02-22 16:00:06,137 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - x-ms-client-request-id=d4484925-a273-482e-8aa9-e0fdef6cdf82
2019-02-22 16:00:06,138 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Content-Type=
2019-02-22 16:00:06,277 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(51)) - Request Headers
2019-02-22 16:00:06,279 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - HTTP Response=HTTP/1.1 404 The specified filesystem does not exist.
2019-02-22 16:00:06,279 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - x-ms-version=2018-11-09
2019-02-22 16:00:06,279 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Server=Windows-Azure-HDFS/1.0 Microsoft-HTTPAPI/2.0
2019-02-22 16:00:06,279 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - x-ms-error-code=FilesystemNotFound
2019-02-22 16:00:06,279 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - x-ms-request-id=d4459a80-901f-0015-41c7-ca9890000000
2019-02-22 16:00:06,279 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Date=Fri, 22 Feb 2019 16:00:05 GMT
2019-02-22 16:00:06,279 [main] DEBUG services.AbfsClient (AbfsRestOperation.java:executeHttpOperation(202)) - HttpRequest: 404,,cid=d4484925-a273-482e-8aa9-e0fdef6cdf82,rid=d4459a80-901f-0015-41c7-ca9890000000,connMs=26,sendMs=0,recvMs=138,sent=0,recv=0,HEAD,https://abfswales1.dfs.core.windows.net/container2//?upn=false&action=getAccessControl&timeout=90
2019-02-22 16:00:06,285 [main] DEBUG azurebfs.AzureBlobFileSystem (AzureBlobFileSystem.java:getFileStatus(434)) - AzureBlobFileSystem.getFileStatus path: null
2019-02-22 16:00:06,286 [main] DEBUG fs.FsShell (FsShell.java:run(344)) - Error
java.lang.NullPointerException
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:712)
at org.apache.hadoop.fs.FileSystem.makeQualified(FileSystem.java:602)
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.makeQualified(AzureBlobFileSystem.java:466)
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.getFileStatus(AzureBlobFileSystem.java:436)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1695)
at org.apache.hadoop.fs.shell.Mkdir.processNonexistentPath(Mkdir.java:76)
at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:288)
at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:270)
at org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:120)
at org.apache.hadoop.fs.shell.Command.run(Command.java:177)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:327)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:390)
-mkdir: Fatal internal error
java.lang.NullPointerException
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:712)
at org.apache.hadoop.fs.FileSystem.makeQualified(FileSystem.java:602)
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.makeQualified(AzureBlobFileSystem.java:466)
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.getFileStatus(AzureBlobFileSystem.java:436)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1695)
at org.apache.hadoop.fs.shell.Mkdir.processNonexistentPath(Mkdir.java:76)
at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:288)
at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:270)
at org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:120)
at org.apache.hadoop.fs.shell.Command.run(Command.java:177)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:327)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:390)
2019-02-22 16:00:06,290 [shutdown-hook-0] DEBUG azurebfs.AzureBlobFileSystem (AzureBlobFileSystem.java:close(427)) - AzureBlobFileSystem.close
{code}
> hadoop fs mkdir of nonexistent abfs container raises NPE
> --------------------------------------------------------
>
> Key: HADOOP-16138
> URL: https://issues.apache.org/jira/browse/HADOOP-16138
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.2.0
> Reporter: Steve Loughran
> Priority: Minor
>
> If you try to do a mkdir on the root of a nonexistent container, you get an NPE
> {code}
> hadoop fs -mkdir abfs://container2@abfswales1.dfs.core.windows.net/
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org