You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Yaniv Ironi (Jira)" <ji...@apache.org> on 2021/03/14 11:48:00 UTC
[jira] [Created] (HADOOP-17589) Support Custom Endpoint For Hadoop
Azure working with Storage Emulator
Yaniv Ironi created HADOOP-17589:
------------------------------------
Summary: Support Custom Endpoint For Hadoop Azure working with Storage Emulator
Key: HADOOP-17589
URL: https://issues.apache.org/jira/browse/HADOOP-17589
Project: Hadoop Common
Issue Type: Bug
Components: hadoop-thirdparty
Affects Versions: 3.4.0
Reporter: Yaniv Ironi
When using the Hadoop Azure library to write files using the wasb:// schema, and using the configuration for Hadoop Azure storage emulator (fs.azure.storage.emulator.account.name), The client assumes that the emulator will run on localhost with the default port.
When writing ITs and using Test Containers, the emulator will run on a custom endpoint but there is no way to tell that for the client.
The problem is in AzureNativeFileSystemStore.java, in connectUsingCredentials method,
which runs:
CloudStorageAccount account = CloudStorageAccount.getDevelopmentStorageAccount()
Which will use the default localhost address.
The fix is easy (And I already fixed it and tested it on my own environment, and I will PR the fix)
CloudStorageAccount.getDevelopmentStorageAccounts has already an overload that accepts URI for a custom endpont, and we can use it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org