You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by "Bhavik Patel (Jira)" <ji...@apache.org> on 2022/03/08 02:57:00 UTC
[jira] [Resolved] (RANGER-3595) Tar of KMS contains rubbish files
[ https://issues.apache.org/jira/browse/RANGER-3595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bhavik Patel resolved RANGER-3595.
----------------------------------
Resolution: Fixed
close the review request related to this jira
> Tar of KMS contains rubbish files
> ---------------------------------
>
> Key: RANGER-3595
> URL: https://issues.apache.org/jira/browse/RANGER-3595
> Project: Ranger
> Issue Type: Improvement
> Components: kms
> Affects Versions: 3.0.0, 2.3.0
> Reporter: kirby zhou
> Assignee: kirby zhou
> Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> There are lots of .class files under ews/webapp/. They wont be loaded by any classpath. And they are duplicated against files inside ews/webapp/lib/ranger-kms-3.0.0-SNAPSHOT.jar.
> It seems dirty and may cause some security problem.
> {code:bash}
> #] tar tf target/ranger-3.0.0-SNAPSHOT-kms.tar.gz ranger-3.0.0-SNAPSHOT-kms/ | egrep 'ews/webapp/org' | head
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/ranger/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/ranger/kms/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/ranger/kms/biz/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/ranger/kms/dao/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/ranger/entity/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/hadoop/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/hadoop/crypto/
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/hadoop/crypto/key/
> //代码占位符
> {code}
> * The reason is that:
> distro/src/main/assembly/kms.xml
>
> {code:java}
> <moduleSet>
> <useAllReactorProjects>true</useAllReactorProjects>
> <includes>
> <include>org.apache.ranger:ranger-kms</include>
> </includes>
> <binaries>
> <outputDirectory>ews/webapp</outputDirectory>
> <includeDependencies>false</includeDependencies>
> <unpack>true</unpack>
> </binaries>
> </moduleSet> {code}
> Why ?
>
> The secret is in kms/scripts/setup.sh:
>
> {code:java}
> setup_kms(){
> #copying ranger kms provider
> oldP=${PWD}
> cd $PWD/ews/webapp
> log "[I] Adding ranger kms provider as services in hadoop-common jar"
> for f in lib/hadoop-common*.jar
> do
> ${JAVA_HOME}/bin/jar -uf ${f} META-INF/services/org.apache.hadoop.crypto.key.KeyProviderFactory
> chown ${unix_user}:${unix_group} ${f}
> done
> cd ${oldP}
> }
> {code}
>
>
> The code above is VERY VERY DIRTY!
> It hacks into hadoop-common.jar., Overwrite resource "META-INF/services/org.apache.hadoop.crypto.key.KeyProviderFactory". Ensure the following code can load 'org.apache.hadoop.crypto.key.RangerKeyStoreProvider$Factory' by 'META-INF/.../KeyProviderFactory'.
>
>
> {code:java}
> // org.apache.hadoop.crypto.key: KeyProviderFactory.java
> private static final ServiceLoader<KeyProviderFactory> serviceLoader = ServiceLoader.load(KeyProviderFactory.class, KeyProviderFactory.class.getClassLoader());
> {code}
>
>
> But this is unnecessary.
> ServiceLoader will read all resources with the same name using the ClassLoader of KeyProviderFactory. We just need to put a jar contains that property side by side of hadoop-common.jar ( ews/webapp/lib/ ). And ranger-kms-3.0.0-SNAPSHOT.jar already here.
> {code:java}
> % tar tf ../target/ranger-*-kms.tar.gz | egrep 'kms[^/]*\.jar|hadoop-common'
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/lib/ranger-kms-3.0.0-SNAPSHOT.jar
> ranger-3.0.0-SNAPSHOT-kms/ews/webapp/lib/hadoop-common-3.3.0.jar
> ...
> % tar tf target/ranger-kms-3.0.0-SNAPSHOT.jar | fgrep ProviderFactory
> META-INF/services/org.apache.hadoop.crypto.key.KeyProviderFactory
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)