You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Mingliang Liu (JIRA)" <ji...@apache.org> on 2017/06/01 21:18:04 UTC
[jira] [Comment Edited] (HADOOP-14461) Azure: handle failure
gracefully in case of missing account access key
[ https://issues.apache.org/jira/browse/HADOOP-14461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16033716#comment-16033716 ]
Mingliang Liu edited comment on HADOOP-14461 at 6/1/17 9:17 PM:
----------------------------------------------------------------
[~snayak], thanks for the suggestion. I want to make it clear, what else can we do here to handle failure in {{AzureNativeFileSystemStore::getAccountKeyFromConfiguration()}}. Its current usage is like: if the account access key is configured, we will pick it up and connect; otherwise we use anonymous credentials, and will complain "container not found" in {{connectUsingAnonymousCredentials()}} if the container is not public. Sampel exception messages are like:
{code}
org.apache.hadoop.fs.azure.AzureException: org.apache.hadoop.fs.azure.AzureException: Container test in account liuml07.blob.core.windows.net not found, and we can't create it using anonymous credentials, and no credentials found for them in the configuration.
at org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.createAzureStorageSession(AzureNativeFileSystemStore.java:1027)
at org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.initialize(AzureNativeFileSystemStore.java:487)
at org.apache.hadoop.fs.azure.NativeAzureFileSystem.initialize(NativeAzureFileSystem.java:1267)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3258)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:123)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3307)
{code}
Actually here the container does exist, and the problem is that the account access key was not provided. The exception message was not clear about this. From the source code:
{code:title=connectUsingAnonymousCredentials}
// Check for container existence, and our ability to access it.
try {
if (!container.exists(getInstrumentedContext())) {
throw new AzureException("Container " + containerName + " in account "
+ accountName + " not found, and we can't create"
+ " it using anoynomous credentials, and no credentials found for them"
+ " in the configuration.");
}
} catch (StorageException ex) {
throw new AzureException("Unable to access container " + containerName
+ " in account " + accountName
+ " using anonymous credentials, and no credentials found for them "
+ " in the configuration.", ex);
}
{code}
It's not triggering the {{catch (StorageException ex)}} clause. Maybe we should change the code here, or is there any other mechanism to probe the access ability of a counter@account?
The v1 patch added a logging message, but that's not enough at all.
was (Author: liuml07):
[~snayak], thanks for the suggestion. I want to make it clear, what else can we do here to handle failure in {{AzureNativeFileSystemStore::getAccountKeyFromConfiguration()}}. Its current usage is like: if the account access key is configured, we will pick it up and connect; otherwise we use anonymous credentials, and will complain "container not found" in {{connectUsingAnonymousCredentials()}}, exception messages like:
{code}
org.apache.hadoop.fs.azure.AzureException: org.apache.hadoop.fs.azure.AzureException: Container test in account liuml07.blob.core.windows.net not found, and we can't create it using anonymous credentials, and no credentials found for them in the configuration.
at org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.createAzureStorageSession(AzureNativeFileSystemStore.java:1027)
at org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.initialize(AzureNativeFileSystemStore.java:487)
at org.apache.hadoop.fs.azure.NativeAzureFileSystem.initialize(NativeAzureFileSystem.java:1267)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3258)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:123)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3307)
{code}
Actually here the container does exist, and the problem is that the account access key was not provided. The exception message was not clear about this. From the source code:
{code:title=connectUsingAnonymousCredentials}
// Check for container existence, and our ability to access it.
try {
if (!container.exists(getInstrumentedContext())) {
throw new AzureException("Container " + containerName + " in account "
+ accountName + " not found, and we can't create"
+ " it using anoynomous credentials, and no credentials found for them"
+ " in the configuration.");
}
} catch (StorageException ex) {
throw new AzureException("Unable to access container " + containerName
+ " in account " + accountName
+ " using anonymous credentials, and no credentials found for them "
+ " in the configuration.", ex);
}
{code}
It's not triggering the {{catch (StorageException ex)}} clause. Maybe we should change the code here, or is there any other mechanism to probe the access ability of a counter@account?
The v1 patch added a logging message, but that's not enough at all.
> Azure: handle failure gracefully in case of missing account access key
> ----------------------------------------------------------------------
>
> Key: HADOOP-14461
> URL: https://issues.apache.org/jira/browse/HADOOP-14461
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
> Reporter: Mingliang Liu
> Assignee: Mingliang Liu
> Attachments: HADOOP-14461.000.patch, HADOOP-14461.001.patch
>
>
> Currently if the {{fs.azure.account.key.youraccount}} is missing, we will get error stack like this:
> {code}
> java.lang.IllegalArgumentException: The String is not a valid Base64-encoded string.
> at com.microsoft.azure.storage.core.Base64.decode(Base64.java:63)
> at com.microsoft.azure.storage.StorageCredentialsAccountAndKey.<init>(StorageCredentialsAccountAndKey.java:81)
> at org.apache.hadoop.fs.azure.AzureBlobStorageTestAccount.createStorageAccount(AzureBlobStorageTestAccount.java:464)
> at org.apache.hadoop.fs.azure.AzureBlobStorageTestAccount.createTestAccount(AzureBlobStorageTestAccount.java:501)
> at org.apache.hadoop.fs.azure.AzureBlobStorageTestAccount.create(AzureBlobStorageTestAccount.java:522)
> at org.apache.hadoop.fs.azure.AzureBlobStorageTestAccount.create(AzureBlobStorageTestAccount.java:451)
> at org.apache.hadoop.fs.azure.TestNativeAzureFileSystemAuthorization.createTestAccount(TestNativeAzureFileSystemAuthorization.java:50)
> at org.apache.hadoop.fs.azure.AbstractWasbTestBase.setUp(AbstractWasbTestBase.java:47)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:168)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
> at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
> at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> {code}
> Actually, this error message is not very helpful. I'd admire you if you can immediately find the root cause of this failure.
> Currently if the test *account* is missing, we simply skip the test with meaningful error message. Here if the *account key* is missing, we should do the same thing: skip the test, and tell the user why we skip it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org