You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Yubi Lee <eu...@gmail.com> on 2023/02/04 07:13:58 UTC

Review Request 74294: RANGER-4068: Ranger usersync needs dnsjava library

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74294/
-----------------------------------------------------------

Review request for ranger.


Bugs: RANGER-4068
    https://issues.apache.org/jira/browse/RANGER-4068


Repository: ranger


Description
-------

After I set `hadoop.security.token.service.use_ip=false` in core-site.xml, I found `java.lang.NoClassDefFoundError: org/xbill/DNS/ResolverConfig` in usersync log.
I found that it is caused by missing dnsjava library in ranger usersync jar as dependency.
Ranger usersync should have `dnsjava` library.

```
28 Jan 2023 00:50:18 DEBUG o.a.h.s.SecurityUtil [UnixUserSyncThread] - Setting hadoop.security.token.service.use_ip to false
28 Jan 2023 00:50:18 ERROR o.a.r.u.UserGroupSync [UnixUserSyncThread] - Failed to initialize UserGroup source/sink. Will retry after 60000 milliseconds. Error details:
java.lang.NoClassDefFoundError: org/xbill/DNS/ResolverConfig
        at org.apache.hadoop.security.SecurityUtil$QualifiedHostResolver.<init>(SecurityUtil.java:592)
        at org.apache.hadoop.security.SecurityUtil.setTokenServiceUseIp(SecurityUtil.java:129)
        at org.apache.hadoop.security.SecurityUtil.setConfigurationInternal(SecurityUtil.java:102)
        at org.apache.hadoop.security.SecurityUtil.<clinit>(SecurityUtil.java:88)
        at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:312)
        at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:300)
        at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:567)
        at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3614)
        at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3604)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3441)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:524)
        at org.apache.hadoop.fs.Path.getFileSystem(Path.java:365)
        at org.apache.hadoop.security.alias.JavaKeyStoreProvider.initFileSystem(JavaKeyStoreProvider.java:89)
        at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.<init>(AbstractJavaKeyStoreProvider.java:85)
        at org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:49)
        at org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:41)
        at org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:100)
        at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:73)
        at org.apache.ranger.credentialapi.CredentialReader.getDecryptedString(CredentialReader.java:74)
        at org.apache.ranger.unixusersync.config.UserGroupSyncConfig.getSSLKeyStorePathPassword(UserGroupSyncConfig.java:439)
        at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.init(PolicyMgrUserGroupBuilder.java:216)
        at org.apache.ranger.usergroupsync.UserGroupSync.run(UserGroupSync.java:49)
```


Diffs
-----

  distro/src/main/assembly/usersync.xml 187d535e4 
  ugsync/pom.xml 12801d4e2 


Diff: https://reviews.apache.org/r/74294/diff/1/


Testing
-------


Thanks,

Yubi Lee

Re: Review Request 74294: RANGER-4068: Ranger usersync needs dnsjava library

Posted by Yubi Lee <eu...@gmail.com>.

> On 4 1, 2023, 11:07 오후, Selvamohan Neethiraj wrote:
> > Can you please document the testing and also the errors (before the fix) and the after-fix results ?

I already mentioned the errors on description. And after this patch, I can start ranger usersync process with `hadoop.security.token.service.use_ip=false` without an error. 


after-fix results:


```
07 Apr 2023 13:30:02 DEBUG o.a.h.s.SecurityUtil [UnixUserSyncThread] - Setting hadoop.security.token.service.use_ip to false
07 Apr 2023 13:30:02 DEBUG o.a.h.s.Groups [UnixUserSyncThread] -  Creating new Groups object
```


- Yubi


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74294/#review225330
-----------------------------------------------------------


On 2 4, 2023, 7:13 오전, Yubi Lee wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74294/
> -----------------------------------------------------------
> 
> (Updated 2 4, 2023, 7:13 오전)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-4068
>     https://issues.apache.org/jira/browse/RANGER-4068
> 
> 
> Repository: ranger
> 
> 
> Description
> -------
> 
> After I set `hadoop.security.token.service.use_ip=false` in core-site.xml, I found `java.lang.NoClassDefFoundError: org/xbill/DNS/ResolverConfig` in usersync log.
> I found that it is caused by missing dnsjava library in ranger usersync jar as dependency.
> Ranger usersync should have `dnsjava` library.
> 
> ```
> 28 Jan 2023 00:50:18 DEBUG o.a.h.s.SecurityUtil [UnixUserSyncThread] - Setting hadoop.security.token.service.use_ip to false
> 28 Jan 2023 00:50:18 ERROR o.a.r.u.UserGroupSync [UnixUserSyncThread] - Failed to initialize UserGroup source/sink. Will retry after 60000 milliseconds. Error details:
> java.lang.NoClassDefFoundError: org/xbill/DNS/ResolverConfig
>         at org.apache.hadoop.security.SecurityUtil$QualifiedHostResolver.<init>(SecurityUtil.java:592)
>         at org.apache.hadoop.security.SecurityUtil.setTokenServiceUseIp(SecurityUtil.java:129)
>         at org.apache.hadoop.security.SecurityUtil.setConfigurationInternal(SecurityUtil.java:102)
>         at org.apache.hadoop.security.SecurityUtil.<clinit>(SecurityUtil.java:88)
>         at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:312)
>         at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:300)
>         at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:567)
>         at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3614)
>         at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3604)
>         at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3441)
>         at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:524)
>         at org.apache.hadoop.fs.Path.getFileSystem(Path.java:365)
>         at org.apache.hadoop.security.alias.JavaKeyStoreProvider.initFileSystem(JavaKeyStoreProvider.java:89)
>         at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.<init>(AbstractJavaKeyStoreProvider.java:85)
>         at org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:49)
>         at org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:41)
>         at org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:100)
>         at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:73)
>         at org.apache.ranger.credentialapi.CredentialReader.getDecryptedString(CredentialReader.java:74)
>         at org.apache.ranger.unixusersync.config.UserGroupSyncConfig.getSSLKeyStorePathPassword(UserGroupSyncConfig.java:439)
>         at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.init(PolicyMgrUserGroupBuilder.java:216)
>         at org.apache.ranger.usergroupsync.UserGroupSync.run(UserGroupSync.java:49)
> ```
> 
> 
> Diffs
> -----
> 
>   distro/src/main/assembly/usersync.xml 187d535e4 
>   ugsync/pom.xml 12801d4e2 
> 
> 
> Diff: https://reviews.apache.org/r/74294/diff/1/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Yubi Lee
> 
>

Re: Review Request 74294: RANGER-4068: Ranger usersync needs dnsjava library

Posted by Selvamohan Neethiraj <sn...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74294/#review225330
-----------------------------------------------------------



Can you please document the testing and also the errors (before the fix) and the after-fix results ?

- Selvamohan Neethiraj


On Feb. 4, 2023, 2:13 a.m., Yubi Lee wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74294/
> -----------------------------------------------------------
> 
> (Updated Feb. 4, 2023, 2:13 a.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-4068
>     https://issues.apache.org/jira/browse/RANGER-4068
> 
> 
> Repository: ranger
> 
> 
> Description
> -------
> 
> After I set `hadoop.security.token.service.use_ip=false` in core-site.xml, I found `java.lang.NoClassDefFoundError: org/xbill/DNS/ResolverConfig` in usersync log.
> I found that it is caused by missing dnsjava library in ranger usersync jar as dependency.
> Ranger usersync should have `dnsjava` library.
> 
> ```
> 28 Jan 2023 00:50:18 DEBUG o.a.h.s.SecurityUtil [UnixUserSyncThread] - Setting hadoop.security.token.service.use_ip to false
> 28 Jan 2023 00:50:18 ERROR o.a.r.u.UserGroupSync [UnixUserSyncThread] - Failed to initialize UserGroup source/sink. Will retry after 60000 milliseconds. Error details:
> java.lang.NoClassDefFoundError: org/xbill/DNS/ResolverConfig
>         at org.apache.hadoop.security.SecurityUtil$QualifiedHostResolver.<init>(SecurityUtil.java:592)
>         at org.apache.hadoop.security.SecurityUtil.setTokenServiceUseIp(SecurityUtil.java:129)
>         at org.apache.hadoop.security.SecurityUtil.setConfigurationInternal(SecurityUtil.java:102)
>         at org.apache.hadoop.security.SecurityUtil.<clinit>(SecurityUtil.java:88)
>         at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:312)
>         at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:300)
>         at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:567)
>         at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3614)
>         at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3604)
>         at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3441)
>         at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:524)
>         at org.apache.hadoop.fs.Path.getFileSystem(Path.java:365)
>         at org.apache.hadoop.security.alias.JavaKeyStoreProvider.initFileSystem(JavaKeyStoreProvider.java:89)
>         at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.<init>(AbstractJavaKeyStoreProvider.java:85)
>         at org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:49)
>         at org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:41)
>         at org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:100)
>         at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:73)
>         at org.apache.ranger.credentialapi.CredentialReader.getDecryptedString(CredentialReader.java:74)
>         at org.apache.ranger.unixusersync.config.UserGroupSyncConfig.getSSLKeyStorePathPassword(UserGroupSyncConfig.java:439)
>         at org.apache.ranger.unixusersync.process.PolicyMgrUserGroupBuilder.init(PolicyMgrUserGroupBuilder.java:216)
>         at org.apache.ranger.usergroupsync.UserGroupSync.run(UserGroupSync.java:49)
> ```
> 
> 
> Diffs
> -----
> 
>   distro/src/main/assembly/usersync.xml 187d535e4 
>   ugsync/pom.xml 12801d4e2 
> 
> 
> Diff: https://reviews.apache.org/r/74294/diff/1/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Yubi Lee
> 
>