[jira] [Commented] (HADOOP-17725) Improve error message for token providers in ABFS

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-17725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17638658#comment-17638658 ] 

ASF GitHub Bot commented on HADOOP-17725:
-----------------------------------------

CLevasseur commented on PR #4262:
URL: https://github.com/apache/hadoop/pull/4262#issuecomment-1327466165

   Hi @pranavsaxena-microsoft, I've tried to follow[ this section ](https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-azure/src/site/markdown/testing_azure.md#generating-test-run-configurations-and-test-triggers-over-various-config-combinations)of the documentation that you mentioned, but it's outdated:
   
   - The folder `./src/test/resources/accountSettings/` doesn't exist, nor the template file that I should use to create my account settings file
   - `dev-support/testrun-scripts/runtests.sh` should prompt a menu, but in my case it runs `AppendBlob-HNS-OAuth`  and gives me no choice
   
   I followed the rest of the documentation by copying  `./src/test/resources/azure-auth-keys.xml.template` to `./src/test/resources/azure-auth-keys.xml` and replacing those variables by the right values in the xml file:
   - `{ABFS_ACCOUNT_NAME}`
   - `{ACCOUNT_ACCESS_KEY}`
   - `{TENANTID}`
   - `{WASB_ACCOUNT_NAME}`
   - `{WASB_FILESYSTEM}`
   - `{CONTAINER_NAME}`
   - `{ACCOUNT_NAME}`
   
   Then I ran
   ```
   dev-support/testrun-scripts/runtests.sh -c "NonHNS-SharedKey"
   [...]
   [INFO] ------------------------------------------------------------------------
   [INFO] BUILD SUCCESS
   [INFO] ------------------------------------------------------------------------
   [INFO] Total time:  11.030 s
   [INFO] Finished at: 2022-11-25T13:08:18Z
   [INFO] ------------------------------------------------------------------------
   
   Running the combination: NonHNS-SharedKey...
   # Then it terminates without running anything
   ```
   
   So it doesn't run the tests, same for `HNS-OAuth` and `HNS-SharedKey`. Do you know what I am missing for those tests to run ?




> Improve error message for token providers in ABFS
> -------------------------------------------------
>
>                 Key: HADOOP-17725
>                 URL: https://issues.apache.org/jira/browse/HADOOP-17725
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs/azure, hadoop-thirdparty
>    Affects Versions: 3.3.0
>            Reporter: Ivan Sadikov
>            Assignee: Viraj Jasani
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 3.3.2
>
>          Time Spent: 8h
>  Remaining Estimate: 0h
>
> It would be good to improve error messages for token providers in ABFS. Currently, when a configuration key is not found or mistyped, the error is not very clear on what went wrong. It would be good to indicate that the key was required but not found in Hadoop configuration when creating a token provider.
> For example, when running the following code:
> {code:java}
> import org.apache.hadoop.conf._
> import org.apache.hadoop.fs._
> val conf = new Configuration()
> conf.set("fs.azure.account.auth.type", "OAuth")
> conf.set("fs.azure.account.oauth.provider.type", "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider")
> conf.set("fs.azure.account.oauth2.client.id", "my-client-id")
> // conf.set("fs.azure.account.oauth2.client.secret.my-account.dfs.core.windows.net", "my-secret")
> conf.set("fs.azure.account.oauth2.client.endpoint", "my-endpoint")
> val path = new Path("abfss://container@my-account.dfs.core.windows.net/")
> val fs = path.getFileSystem(conf)
> fs.getFileStatus(path){code}
> The following exception is thrown:
> {code:java}
> TokenAccessProviderException: Unable to load OAuth token provider class.
> ...
> Caused by: UncheckedExecutionException: java.lang.NullPointerException: clientSecret
> ...
> Caused by: NullPointerException: clientSecret {code}
> which does not tell what configuration key was not loaded.
>  
> IMHO, it would be good if the exception was something like this:
> {code:java}
> TokenAccessProviderException: Unable to load OAuth token provider class.
> ...
> Caused by: ConfigurationPropertyNotFoundException: Configuration property fs.azure.account.oauth2.client.secret not found. {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org