[jira] [Commented] (HADOOP-15808) Harden Token service loader use

["Steve Loughran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16683781#comment-16683781 ] 

Steve Loughran commented on HADOOP-15808:
-----------------------------------------

bq. I don't fully understand why the Iterator#next() triggers the exception but service loading has lazy behaviors so I guess that must be it.

Exactly. They build a list of classes to load, but the load is only in the next() call. And ~everything which uses the service loader API doesn't expect failures here.

w.r.t those test options

#  Create a fake type that triggers this by not having the dependencies.
# Spy some of the types to trigger the exception always.

I thought of #1 but felt it was too risky; imagine if it was in hadoop-common-test and somehow that JAR ended up on the CP of a hadoop version which didn't have the hardening. You'd not be able to load any tokens.

The other option would be factor out the service loading into some templated class for use everywhere: you pass in the details needed for the load, get back the list of successfully loaded classes. We could also take an arg about log level of errors (debug, info, error). For FS loading, debug is all we need. 

With this factored out, the test would be to load some explicitly failing service purely for hadoop-common-test, so it wouldn't interfere with token loading -and we'd have something we knew worked everywhere.



> Harden Token service loader use
> -------------------------------
>
>                 Key: HADOOP-15808
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15808
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: security
>    Affects Versions: 2.9.1, 3.1.2
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Major
>         Attachments: HADOOP-15808-001.patch, HADOOP-15808-002.patch, HADOOP-15808-003.patch
>
>
> The Hadoop token service loading (identifiers, renewers...) works provided there's no problems loading any registered implementation. If there's a classloading or classcasting problem, the exception raised will stop all token support working; possibly the application not starting.
> This matters for S3A/HADOOP-14556 as things may not load if aws-sdk isn't on the classpath. It probably lurks in the wasb/abfs support too, but things have worked there because the installations with DT support there have always had correctly set up classpaths.
> Fix: do what we did for the FS service loader. Catch failures to instantiate a service provider impl and skip it



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org