You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Gergely Pollák (Jira)" <ji...@apache.org> on 2021/09/20 12:33:00 UTC

[jira] [Commented] (HDFS-16129) HttpFS signature secret file misusage

    [ https://issues.apache.org/jira/browse/HDFS-16129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17417621#comment-17417621 ] 

Gergely Pollák commented on HDFS-16129:
---------------------------------------

LGTM +1, merged to trunk.
@tdomok We have backported YARN-10814 to 3.2 and 3.3, please check if we need to backport this as well, and reopen the jira if we need to.

> HttpFS signature secret file misusage
> -------------------------------------
>
>                 Key: HDFS-16129
>                 URL: https://issues.apache.org/jira/browse/HDFS-16129
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: httpfs
>    Affects Versions: 3.4.0
>            Reporter: Tamas Domok
>            Assignee: Tamas Domok
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 8h
>  Remaining Estimate: 0h
>
> I started to work on the YARN-10814 issue, and found this bug in the HttpFS. I investigated the problem and I already have some fix for it.
>  
> If the deprecated *httpfs.authentication.signature.secret.file* is not set in the configuration (e.g.: httpfs-site.xml) then the new *hadoop.http.authentication.signature.secret.file* config option won't be used, it will fallback to the random secret provider silently.
> The _HttpFSServerWebServer_ sets an _authFilterConfigurationPrefix_ when building the server for the old path (*httpfs.authentication.*). Later the _AuthenticationFilter.constructSecretProvider_ will immediately fallback to +random+, because the config won't contain the file. If the old path was set too, then it handled the file, and the provider was set to +file+ type.
> The configuration should be based on both the old and the new prefix filter, merging the two. The new config option should win in my opinion.
>  
> There is another issue in the _HttpFSAuthenticationFilter_, it is closely related.
> If both config option is set then the _HttpFSAuthenticationFilter_ will fail with an impossible file path (e.g.: *${httpfs.config.dir}/httpfs-signature.secret*).
> _HttpFSAuthenticationFilter_ constructs the configuration, filtering first the new config prefix then the old prefix. The old prefix code works correctly, it uses the _conf.get(key)_
> instead of the _entry.getValue()_ which gives back the file path mentioned earlier. The code duplication can be eliminated and I think it would be better to change the order, first adding the config options from the old path then the new, and the new should overwrite the old values, with a warning log message.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org