You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2019/02/25 14:31:00 UTC
[jira] [Comment Edited] (HADOOP-16068) ABFS Auth and DT plugins to
be bound to specific URI of the FS
[ https://issues.apache.org/jira/browse/HADOOP-16068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16776930#comment-16776930 ]
Steve Loughran edited comment on HADOOP-16068 at 2/25/19 2:30 PM:
------------------------------------------------------------------
log on oauth returns HTML for next patch. Doesn't fix problem, but helps
{code}
bin/hadoop fs -ls abfs://container@abfswales1.dfs.core.windows.net/ 14:22:01
2019-02-25 14:23:07,975 [main] DEBUG azurebfs.AzureBlobFileSystem (AzureBlobFileSystem.java:initialize(101)) - Initializing AzureBlobFileSystem for abfs://container@abfswales1.dfs.core.windows.net/
2019-02-25 14:23:08,182 [main] DEBUG services.AbfsClientThrottlingIntercept (AbfsClientThrottlingIntercept.java:initializeSingleton(62)) - Client-side throttling is enabled for the ABFS file system.
2019-02-25 14:23:08,204 [main] DEBUG azurebfs.AzureBlobFileSystem (AzureBlobFileSystem.java:getFileStatus(434)) - AzureBlobFileSystem.getFileStatus path: abfs://container@abfswales1.dfs.core.windows.net/
2019-02-25 14:23:08,204 [main] DEBUG azurebfs.AzureBlobFileSystem (AzureBlobFileSystem.java:performAbfsAuthCheck(1101)) - ABFS authorizer is not initialized. No authorization check will be performed.
2019-02-25 14:23:08,204 [main] DEBUG azurebfs.AzureBlobFileSystemStore (AzureBlobFileSystemStore.java:getIsNamespaceEnabled(200)) - Get root ACL status
2019-02-25 14:23:08,272 [main] DEBUG oauth2.AccessTokenProvider (AccessTokenProvider.java:isTokenAboutToExpire(77)) - AADToken: no token. Returning expiring=true
2019-02-25 14:23:08,272 [main] DEBUG oauth2.AccessTokenProvider (AccessTokenProvider.java:getToken(48)) - AAD Token is missing or expired: Calling refresh-token from abstract base class
2019-02-25 14:23:08,272 [main] DEBUG oauth2.AccessTokenProvider (ClientCredsTokenProvider.java:refreshToken(57)) - AADToken: refreshing client-credential based token
2019-02-25 14:23:08,274 [main] DEBUG oauth2.AzureADAuthenticator (AzureADAuthenticator.java:getTokenUsingClientCreds(94)) - AADToken: starting to fetch token using client creds for client ID 40c8b4e5-865a-4297-bbbd-195df2a8a806
2019-02-25 14:23:08,274 [main] DEBUG oauth2.AzureADAuthenticator (AzureADAuthenticator.java:getTokenSingleCall(297)) - Requesting an OAuth token by POST to https://login.microsoftonline.com/b60c9401-2154-40aa-9cff-5e3d1a20085d/oauth2/authorize
2019-02-25 14:23:08,275 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(51)) - Request Headers
2019-02-25 14:23:08,275 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Connection=close
2019-02-25 14:23:08,872 [main] DEBUG oauth2.AzureADAuthenticator (AzureADAuthenticator.java:getTokenSingleCall(319)) - Response 200
2019-02-25 14:23:08,872 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(51)) - Response Headers
2019-02-25 14:23:08,873 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - HTTP Response=HTTP/1.1 200 OK
2019-02-25 14:23:08,873 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Server=Microsoft-IIS/10.0
2019-02-25 14:23:08,874 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - X-Content-Type-Options=nosniff
2019-02-25 14:23:08,874 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Connection=close
2019-02-25 14:23:08,874 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Pragma=no-cache
2019-02-25 14:23:08,874 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - P3P=CP="DSP CUR OTPi IND OTRi ONL FIN"
2019-02-25 14:23:08,875 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Date=Mon, 25 Feb 2019 14:23:08 GMT
2019-02-25 14:23:08,875 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - X-Frame-Options=DENY
2019-02-25 14:23:08,875 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Strict-Transport-Security=max-age=31536000; includeSubDomains
2019-02-25 14:23:08,875 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Cache-Control=no-cache, no-store
2019-02-25 14:23:08,875 [main] DEBUG services.AbfsIoUtils
2019-02-25 14:23:08,875 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - X-DNS-Prefetch-Control=on
2019-02-25 14:23:08,875 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Expires=-1
2019-02-25 14:23:08,876 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Content-Length=27304
2019-02-25 14:23:08,876 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - x-ms-request-id=a48ce162-4579-4ae7-ac93-97e304b05400
2019-02-25 14:23:08,876 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Link=<https://aadcdn.msauth.net>; rel=dns-prefetch;<https://aadcdn.msftauth.net>; rel=dns-prefetch;<https://aadcdn.msftauth.net>; rel=preconnect; crossorigin
2019-02-25 14:23:08,876 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Content-Type=text/html; charset=utf-8
2019-02-25 14:23:08,876 [main] DEBUG oauth2.AzureADAuthenticator (AzureADAuthenticator.java:getTokenSingleCall(356)) - AADToken: HTTP connection to https://login.microsoftonline.com/b60c9401-2154-40aa-9cff-5e3d1a20085d/oauth2/authorize failed for getting token from AzureAD. Http response: 200 OK
Content-Type: text/html; charset=utf-8 Content-Length: 27304 Request ID: a48ce162-4579-4ae7-ac93-97e304b05400 Proxies: none
First 1K of Body:
<!DOCTYPE html>
<html dir="ltr" class="" lang="en">
<head>
<title>Sign in to your account</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=2.0, user-scalable=yes">
<meta http-equiv="Pragma" content="no-cache">
<meta http-equiv="Expires" content="-1">
<link rel="preconnect" href="https://aadcdn.msftauth.net" crossorigin>
<meta http-equiv="x-dns-prefetch-control" content="on">
<link rel="dns-prefetch" href="//aadcdn.msftauth.net">
<link rel="dns-prefetch" href="//aadcdn.msauth.net">
<meta name="PageID" content="ConvergedSignIn" />
<meta name="SiteID" content="" />
<meta name="ReqLC" content="1033" />
<meta name="LocLC" content="en-US" />
<noscript>
<meta http-equiv="Refresh" content="0; URL=https://login.microsoftonline.com/jsdisabled" />
</noscript>
2019-02-25 14:23:08,879 [main] DEBUG services.AbfsClient (AbfsRestOperation.java:executeHttpOperation(180)) - HttpRequestFailure: 0,,cid=95bbc863-1bcc-483e-b2c4-625bfb2f55fa,rid=,connMs=65,sendMs=0,recvMs=0,sent=0,recv=0,HEAD,https://abfswales1.dfs.core.windows.net/container//?upn=false&action=getAccessControl&timeout=90
org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator$UnexpectedResponseException: HTTP Error 200 ; url='https://login.microsoftonline.com/b60c9401-2154-40aa-9cff-5e3d1a20085d/oauth2/authorize'OK; requestId='a48ce162-4579-4ae7-ac93-97e304b05400'; contentType='text/html; charset=utf-8'; payload'AADToken: HTTP connection to https://login.microsoftonline.com/b60c9401-2154-40aa-9cff-5e3d1a20085d/oauth2/authorize failed for getting token from AzureAD. Http response: 200 OK
Content-Type: text/html; charset=utf-8 Content-Length: 27304 Request ID: a48ce162-4579-4ae7-ac93-97e304b05400 Proxies: none
First 1K of Body:
<!DOCTYPE html>
<html dir="ltr" class="" lang="en">
<head>
<title>Sign in to your account</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=2.0, user-scalable=yes">
<meta http-equiv="Pragma" content="no-cache">
<meta http-equiv="Expires" content="-1">
<link rel="preconnect" href="https://aadcdn.msftauth.net" crossorigin>
<meta http-equiv="x-dns-prefetch-control" content="on">
<link rel="dns-prefetch" href="//aadcdn.msftauth.net">
<link rel="dns-prefetch" href="//aadcdn.msauth.net">
<meta name="PageID" content="ConvergedSignIn" />
<meta name="SiteID" content="" />
<meta name="ReqLC" content="1033" />
<meta name="LocLC" content="en-US" />
<noscript>
<meta http-equiv="Refresh" content="0; URL=https://login.microsoftonline.com/jsdisabled" />
</noscript>
'
at org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.getTokenSingleCall(AzureADAuthenticator.java:362)
at org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.getTokenCall(AzureADAuthenticator.java:267)
at org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.getTokenUsingClientCreds(AzureADAuthenticator.java:96)
at org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider.refreshToken(ClientCredsTokenProvider.java:58)
at org.apache.hadoop.fs.azurebfs.oauth2.AccessTokenProvider.getToken(AccessTokenProvider.java:50)
at org.apache.hadoop.fs.azurebfs.services.AbfsClient.getAccessToken(AbfsClient.java:563)
at org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.executeHttpOperation(AbfsRestOperation.java:151)
at org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.execute(AbfsRestOperation.java:125)
at org.apache.hadoop.fs.azurebfs.services.AbfsClient.getAclStatus(AbfsClient.java:515)
at org.apache.hadoop.fs.azurebfs.services.AbfsClient.getAclStatus(AbfsClient.java:498)
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.getIsNamespaceEnabled(AzureBlobFileSystemStore.java:202)
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.getFileStatus(AzureBlobFileSystemStore.java:467)
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.getFileStatus(AzureBlobFileSystem.java:440)
at org.apache.hadoop.fs.Globber.getFileStatus(Globber.java:65)
at org.apache.hadoop.fs.Globber.doGlob(Globber.java:294)
at org.apache.hadoop.fs.Globber.glob(Globber.java:149)
at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:2027)
at org.apache.hadoop.fs.shell.PathData.expandAsGlob(PathData.java:353)
at org.apache.hadoop.fs.shell.Command.expandArgument(Command.java:250)
at org.apache.hadoop.fs.shell.Command.expandArguments(Command.java:233)
at org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:104)
at org.apache.hadoop.fs.shell.Command.run(Command.java:177)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:327)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:390)
ls: HTTP Error 200 ; url='https://login.microsoftonline.com/b60c9401-2154-40aa-9cff-5e3d1a20085d/oauth2/authorize'OK; requestId='a48ce162-4579-4ae7-ac93-97e304b05400'; contentType='text/html; charset=utf-8'; payload'AADToken: HTTP connection to https://login.microsoftonline.com/b60c9401-2154-40aa-9cff-5e3d1a20085d/oauth2/authorize failed for getting token from AzureAD. Http response: 200 OK
2019-02-25 14:23:08,887 [shutdown-hook-0] DEBUG azurebfs.AzureBlobFileSystem (AzureBlobFileSystem.java:close(427)) - AzureBlobFileSystem.close
{code}
was (Author: stevel@apache.org):
log on oauth returns HTML for next patch. Doesn't fix problem, but helps
{code}
bin/hadoop fs -ls abfs://container@abfswales1.dfs.core.windows.net/ 14:22:01
2019-02-25 14:23:07,975 [main] DEBUG azurebfs.AzureBlobFileSystem (AzureBlobFileSystem.java:initialize(101)) - Initializing AzureBlobFileSystem for abfs://container@abfswales1.dfs.core.windows.net/
2019-02-25 14:23:08,182 [main] DEBUG services.AbfsClientThrottlingIntercept (AbfsClientThrottlingIntercept.java:initializeSingleton(62)) - Client-side throttling is enabled for the ABFS file system.
2019-02-25 14:23:08,204 [main] DEBUG azurebfs.AzureBlobFileSystem (AzureBlobFileSystem.java:getFileStatus(434)) - AzureBlobFileSystem.getFileStatus path: abfs://container@abfswales1.dfs.core.windows.net/
2019-02-25 14:23:08,204 [main] DEBUG azurebfs.AzureBlobFileSystem (AzureBlobFileSystem.java:performAbfsAuthCheck(1101)) - ABFS authorizer is not initialized. No authorization check will be performed.
2019-02-25 14:23:08,204 [main] DEBUG azurebfs.AzureBlobFileSystemStore (AzureBlobFileSystemStore.java:getIsNamespaceEnabled(200)) - Get root ACL status
2019-02-25 14:23:08,272 [main] DEBUG oauth2.AccessTokenProvider (AccessTokenProvider.java:isTokenAboutToExpire(77)) - AADToken: no token. Returning expiring=true
2019-02-25 14:23:08,272 [main] DEBUG oauth2.AccessTokenProvider (AccessTokenProvider.java:getToken(48)) - AAD Token is missing or expired: Calling refresh-token from abstract base class
2019-02-25 14:23:08,272 [main] DEBUG oauth2.AccessTokenProvider (ClientCredsTokenProvider.java:refreshToken(57)) - AADToken: refreshing client-credential based token
2019-02-25 14:23:08,274 [main] DEBUG oauth2.AzureADAuthenticator (AzureADAuthenticator.java:getTokenUsingClientCreds(94)) - AADToken: starting to fetch token using client creds for client ID 40c8b4e5-865a-4297-bbbd-195df2a8a806
2019-02-25 14:23:08,274 [main] DEBUG oauth2.AzureADAuthenticator (AzureADAuthenticator.java:getTokenSingleCall(297)) - Requesting an OAuth token by POST to https://login.microsoftonline.com/b60c9401-2154-40aa-9cff-5e3d1a20085d/oauth2/authorize
2019-02-25 14:23:08,275 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(51)) - Request Headers
2019-02-25 14:23:08,275 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Connection=close
2019-02-25 14:23:08,872 [main] DEBUG oauth2.AzureADAuthenticator (AzureADAuthenticator.java:getTokenSingleCall(319)) - Response 200
2019-02-25 14:23:08,872 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(51)) - Response Headers
2019-02-25 14:23:08,873 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - HTTP Response=HTTP/1.1 200 OK
2019-02-25 14:23:08,873 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Server=Microsoft-IIS/10.0
2019-02-25 14:23:08,874 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - X-Content-Type-Options=nosniff
2019-02-25 14:23:08,874 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Connection=close
2019-02-25 14:23:08,874 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Pragma=no-cache
2019-02-25 14:23:08,874 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - P3P=CP="DSP CUR OTPi IND OTRi ONL FIN"
2019-02-25 14:23:08,875 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Date=Mon, 25 Feb 2019 14:23:08 GMT
2019-02-25 14:23:08,875 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - X-Frame-Options=DENY
2019-02-25 14:23:08,875 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Strict-Transport-Security=max-age=31536000; includeSubDomains
2019-02-25 14:23:08,875 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Cache-Control=no-cache, no-store
2019-02-25 14:23:08,875 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Set-Cookie=stsservicecookie=ests; path=/; secure; HttpOnly;x-ms-gateway-slice=prod; path=/; secure; HttpOnly;esctx=AQABAAAAAACEfexXxjamQb3OeGQ4GugvJnqCleb5fTHXUNtG2e2vTfvF4cV_3pPI9WICNEPT-85W2F8bAZgHI_L1btZuYPY5pru5civf8jCRgzmUuU5mfcyeeYApPXDdetus1cHPtPEe7cNjOewm_S0dVIvGvA3pS87ggExUYAXwnRzRj55Z1T7ftN444Vg3QHApA6jYvewgAA; domain=.login.microsoftonline.com; path=/; secure; HttpOnly;fpc=AqAbRMgiP0pMldA4zkrmUY36f-C2AQAAAEzxBdQOAAAA; expires=Wed, 27-Mar-2019 14:23:08 GMT; path=/; secure; HttpOnly;buid=AQABAAEAAACEfexXxjamQb3OeGQ4Gugva-v_Nlyn-Tpf02RlcpUSwNRWwiDsQefnZ8f2H7B0EiRdWHy-J7zGXejZ1macMiJmALkHcFBkDYfFXzyxWAw6OPIK7OicwCF7wJjiblzZe5MgAA; expires=Wed, 27-Mar-2019 14:23:08 GMT; path=/; secure; HttpOnly
2019-02-25 14:23:08,875 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - X-DNS-Prefetch-Control=on
2019-02-25 14:23:08,875 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Expires=-1
2019-02-25 14:23:08,876 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Content-Length=27304
2019-02-25 14:23:08,876 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - x-ms-request-id=a48ce162-4579-4ae7-ac93-97e304b05400
2019-02-25 14:23:08,876 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Link=<https://aadcdn.msauth.net>; rel=dns-prefetch;<https://aadcdn.msftauth.net>; rel=dns-prefetch;<https://aadcdn.msftauth.net>; rel=preconnect; crossorigin
2019-02-25 14:23:08,876 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54)) - Content-Type=text/html; charset=utf-8
2019-02-25 14:23:08,876 [main] DEBUG oauth2.AzureADAuthenticator (AzureADAuthenticator.java:getTokenSingleCall(356)) - AADToken: HTTP connection to https://login.microsoftonline.com/b60c9401-2154-40aa-9cff-5e3d1a20085d/oauth2/authorize failed for getting token from AzureAD. Http response: 200 OK
Content-Type: text/html; charset=utf-8 Content-Length: 27304 Request ID: a48ce162-4579-4ae7-ac93-97e304b05400 Proxies: none
First 1K of Body:
<!DOCTYPE html>
<html dir="ltr" class="" lang="en">
<head>
<title>Sign in to your account</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=2.0, user-scalable=yes">
<meta http-equiv="Pragma" content="no-cache">
<meta http-equiv="Expires" content="-1">
<link rel="preconnect" href="https://aadcdn.msftauth.net" crossorigin>
<meta http-equiv="x-dns-prefetch-control" content="on">
<link rel="dns-prefetch" href="//aadcdn.msftauth.net">
<link rel="dns-prefetch" href="//aadcdn.msauth.net">
<meta name="PageID" content="ConvergedSignIn" />
<meta name="SiteID" content="" />
<meta name="ReqLC" content="1033" />
<meta name="LocLC" content="en-US" />
<noscript>
<meta http-equiv="Refresh" content="0; URL=https://login.microsoftonline.com/jsdisabled" />
</noscript>
2019-02-25 14:23:08,879 [main] DEBUG services.AbfsClient (AbfsRestOperation.java:executeHttpOperation(180)) - HttpRequestFailure: 0,,cid=95bbc863-1bcc-483e-b2c4-625bfb2f55fa,rid=,connMs=65,sendMs=0,recvMs=0,sent=0,recv=0,HEAD,https://abfswales1.dfs.core.windows.net/container//?upn=false&action=getAccessControl&timeout=90
org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator$UnexpectedResponseException: HTTP Error 200 ; url='https://login.microsoftonline.com/b60c9401-2154-40aa-9cff-5e3d1a20085d/oauth2/authorize'OK; requestId='a48ce162-4579-4ae7-ac93-97e304b05400'; contentType='text/html; charset=utf-8'; payload'AADToken: HTTP connection to https://login.microsoftonline.com/b60c9401-2154-40aa-9cff-5e3d1a20085d/oauth2/authorize failed for getting token from AzureAD. Http response: 200 OK
Content-Type: text/html; charset=utf-8 Content-Length: 27304 Request ID: a48ce162-4579-4ae7-ac93-97e304b05400 Proxies: none
First 1K of Body:
<!DOCTYPE html>
<html dir="ltr" class="" lang="en">
<head>
<title>Sign in to your account</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=2.0, user-scalable=yes">
<meta http-equiv="Pragma" content="no-cache">
<meta http-equiv="Expires" content="-1">
<link rel="preconnect" href="https://aadcdn.msftauth.net" crossorigin>
<meta http-equiv="x-dns-prefetch-control" content="on">
<link rel="dns-prefetch" href="//aadcdn.msftauth.net">
<link rel="dns-prefetch" href="//aadcdn.msauth.net">
<meta name="PageID" content="ConvergedSignIn" />
<meta name="SiteID" content="" />
<meta name="ReqLC" content="1033" />
<meta name="LocLC" content="en-US" />
<noscript>
<meta http-equiv="Refresh" content="0; URL=https://login.microsoftonline.com/jsdisabled" />
</noscript>
'
at org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.getTokenSingleCall(AzureADAuthenticator.java:362)
at org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.getTokenCall(AzureADAuthenticator.java:267)
at org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.getTokenUsingClientCreds(AzureADAuthenticator.java:96)
at org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider.refreshToken(ClientCredsTokenProvider.java:58)
at org.apache.hadoop.fs.azurebfs.oauth2.AccessTokenProvider.getToken(AccessTokenProvider.java:50)
at org.apache.hadoop.fs.azurebfs.services.AbfsClient.getAccessToken(AbfsClient.java:563)
at org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.executeHttpOperation(AbfsRestOperation.java:151)
at org.apache.hadoop.fs.azurebfs.services.AbfsRestOperation.execute(AbfsRestOperation.java:125)
at org.apache.hadoop.fs.azurebfs.services.AbfsClient.getAclStatus(AbfsClient.java:515)
at org.apache.hadoop.fs.azurebfs.services.AbfsClient.getAclStatus(AbfsClient.java:498)
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.getIsNamespaceEnabled(AzureBlobFileSystemStore.java:202)
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.getFileStatus(AzureBlobFileSystemStore.java:467)
at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.getFileStatus(AzureBlobFileSystem.java:440)
at org.apache.hadoop.fs.Globber.getFileStatus(Globber.java:65)
at org.apache.hadoop.fs.Globber.doGlob(Globber.java:294)
at org.apache.hadoop.fs.Globber.glob(Globber.java:149)
at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:2027)
at org.apache.hadoop.fs.shell.PathData.expandAsGlob(PathData.java:353)
at org.apache.hadoop.fs.shell.Command.expandArgument(Command.java:250)
at org.apache.hadoop.fs.shell.Command.expandArguments(Command.java:233)
at org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:104)
at org.apache.hadoop.fs.shell.Command.run(Command.java:177)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:327)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:390)
ls: HTTP Error 200 ; url='https://login.microsoftonline.com/b60c9401-2154-40aa-9cff-5e3d1a20085d/oauth2/authorize'OK; requestId='a48ce162-4579-4ae7-ac93-97e304b05400'; contentType='text/html; charset=utf-8'; payload'AADToken: HTTP connection to https://login.microsoftonline.com/b60c9401-2154-40aa-9cff-5e3d1a20085d/oauth2/authorize failed for getting token from AzureAD. Http response: 200 OK
2019-02-25 14:23:08,887 [shutdown-hook-0] DEBUG azurebfs.AzureBlobFileSystem (AzureBlobFileSystem.java:close(427)) - AzureBlobFileSystem.close
{code}
> ABFS Auth and DT plugins to be bound to specific URI of the FS
> --------------------------------------------------------------
>
> Key: HADOOP-16068
> URL: https://issues.apache.org/jira/browse/HADOOP-16068
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.2.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Attachments: HADOOP-16068-001.patch, HADOOP-16068-002.patch, HADOOP-16068-003.patch, HADOOP-16068-004.patch, HADOOP-16068-005.patch, HADOOP-16068-006.patch, HADOOP-16068-007.patch, HADOOP-16068-008.patch, HADOOP-16068-009.patch
>
>
> followup from HADOOP-15692: pass in the URI & conf of the owner FS to bind the plugins to the specific FS instance. Without that you can't have per FS auth
> +add a stub DT plugin for testing, verify that DTs are collected.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org