You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ranger.apache.org by Don Bosco Durai <bo...@apache.org> on 2015/12/01 00:04:29 UTC
Re: access audit logs are not being shown on ranger ui
I see the following errors:
2015-12-01 02:56:30,065 FATAL org.apache.ranger.audit.destination.SolrAuditDestination: Can't connect to Solr server. URL=[http://192.168.23.105:6083/solr/ranger_audits]
java.lang.NoClassDefFoundError: org/apache/http/entity/mime/content/ContentBody
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.makeSolrClient(LBHttpSolrClient.java:265)
I had similar issues previously with incompatible http-client and http-mime jar files. I am not sure whether you are hitting the sample.
Can you check the hadoop lib folders for *http*.jar files? Mine is httpclient-4.2.5.jar and we package httpmime-4.2.5.jar with HDFS plugin.
Thanks
Bosco
From: Hafiz Mujadid <ha...@gmail.com>
Reply-To: <us...@ranger.incubator.apache.org>
Date: Monday, November 30, 2015 at 2:22 PM
To: <us...@ranger.incubator.apache.org>
Subject: Re: access audit logs are not being shown on ranger ui
Hi Bosco,
Attached is the content of namenode log file.
Solr is working fine, i checked from the web UI
Also /var/log/hadoop/hdfs/audit/solr/spool contains following contents:
archive
index_batch_batch.solr_hdfs.json
index_batch_batch.solr_hdfs_closed.json
and all of these files and folders are empty
Hi Madhan,
I have used same URL but still it does not work.
On Tue, Dec 1, 2015 at 12:26 AM, Madhan Neethiraj <ma...@apache.org> wrote:
Hafiz,
Different Solr URLs are specified in configuration of HDFS plugin and Ranger Admin. Can you please review and update such that they both point to the same Solr instance?
properties for hdfs plugin:
XAAUDIT.SOLR.URL=http://192.168.23.105:6083/solr/ranger_audits
properties in ranger-admin:
audit_solr_urls=http://122.129.79.70:6083/solr/ranger_audits
Madhan
From: Don Bosco Durai <bo...@apache.org>
Reply-To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Date: Monday, November 30, 2015 at 10:48 AM
To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
Subject: Re: access audit logs are not being shown on ranger ui
The properties seems correct. Best way to test is to :
Check the HDFS logs.
Check Solr from web interface (use the same url you configured)
See if there are any unprocessed files in /var/log/hadoop/hdfs/audit/solr/spool
Bosco
From: Hafiz Mujadid <ha...@gmail.com>
Reply-To: <us...@ranger.incubator.apache.org>
Date: Monday, November 30, 2015 at 7:22 AM
To: <us...@ranger.incubator.apache.org>
Subject: access audit logs are not being shown on ranger ui
hi,
I have enabled ranger audits in solr, but access audits are not being shown.
properties for hdfs plugin
XAAUDIT.SOLR.ENABLE=true
XAAUDIT.SOLR.URL=http://192.168.23.105:6083/solr/ranger_audits
XAAUDIT.SOLR.USER=ranger_solr
XAAUDIT.SOLR.PASSWORD=NONE
XAAUDIT.SOLR.ZOOKEEPER=NONE
XAAUDIT.SOLR.FILE_SPOOL_DIR=/var/log/hadoop/hdfs/audit/solr/spool
properties in ranger-admin
audit_store=solr
audit_solr_urls=http://122.129.79.70:6083/solr/ranger_audits
audit_solr_user=ranger_solr
audit_solr_password=NONE
audit_solr_zookeepers=
But no audits logs are shown in ranger-admin
how can i check the issue ?
thanks
--
Regards: HAFIZ MUJADID
Re: access audit logs are not being shown on ranger ui
Posted by Hafiz Mujadid <ha...@gmail.com>.
When i hit some hdfs command following logs are appended in hadoop log file
2015-12-01 13:24:43,662 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 0 on 9000, call
org.apache.hadoop.hdfs.protocol.ClientProtocol.getFileInfo from
192.168.23.105:56699 Call#0 Retry#0:
org.apache.hadoop.security.AccessControlException: Permission denied:
user=asma, access=EXECUTE, inode="/pg/m":hduser:hadoop:d---------
2015-12-01 13:25:17,369 WARN
org.apache.hadoop.security.UserGroupInformation: No groups available for
user hduser
2015-12-01 13:25:17,384 WARN
org.apache.hadoop.security.UserGroupInformation: No groups available for
user hduser
2015-12-01 13:25:17,384 WARN
org.apache.hadoop.security.UserGroupInformation: No groups available for
user hduser
2015-12-01 13:25:17,384 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from
192.168.23.105
2015-12-01 13:25:17,385 INFO
org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs
2015-12-01 13:25:17,385 INFO
org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 4569
2015-12-01 13:25:17,385 INFO
org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2
Total time for transactions(ms): 1 Number of transactions batched in Syncs:
0 Number of syncs: 2 SyncTimes(ms): 57
2015-12-01 13:25:17,399 INFO
org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2
Total time for transactions(ms): 1 Number of transactions batched in Syncs:
0 Number of syncs: 3 SyncTimes(ms): 70
2015-12-01 13:25:17,400 INFO
org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits
file /app/hadoop/tmp/dfs/name/current/edits_inprogress_0000000000000004569
->
/app/hadoop/tmp/dfs/name/current/edits_0000000000000004569-0000000000000004570
2015-12-01 13:25:17,400 INFO
org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at
4571
2015-12-01 13:26:07,565 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 2 on 9000, call
org.apache.hadoop.hdfs.protocol.ClientProtocol.getFileInfo from
192.168.23.105:56703 Call#0 Retry#0:
org.apache.hadoop.security.AccessControlException: Permission denied:
user=asma, access=EXECUTE, inode="/pg/m":hduser:hadoop:d---------
On Tue, Dec 1, 2015 at 1:18 PM, Hafiz Mujadid <ha...@gmail.com>
wrote:
> Hi Bosco,
>
> I resolved this issue by placing httpclient jar in ranger-hdfs-plugin/lib
> and hadoop/lib and hadoop/share/hadoop/hdfs/lib
>
> But audits are still not being shown
>
> latest hadoop log file is attached. Please have a look and suggest me next
> step. thanks
>
>
>
> On Tue, Dec 1, 2015 at 12:50 PM, Hafiz Mujadid <ha...@gmail.com>
> wrote:
>
>> Hi Bosco,
>> I checked the hadoop/lib folder there was no httpclient jar file so i
>> downloaded and placed 4.2.5 version of httpclient. httpmime is 4.2.5
>> version already. so both the jars are of same version now. But when i start
>> hadoop, face following exception as you mentioned
>>
>> java.lang.NoClassDefFoundError:
>> org/apache/http/entity/mime/content/ContentBody
>> at
>> org.apache.solr.client.solrj.impl.LBHttpSolrClient.makeSolrClient(LBHttpSolrClient.java:265)
>> at
>> org.apache.solr.client.solrj.impl.LBHttpSolrClient.<init>(LBHttpSolrClient.java:237)
>> at
>> org.apache.solr.client.solrj.impl.LBHttpSolrClient.<init>(LBHttpSolrClient.java:217)
>> at
>> org.apache.solr.client.solrj.impl.LBHttpSolrClient.<init>(LBHttpSolrClient.java:212)
>> at
>> org.apache.ranger.audit.destination.SolrAuditDestination.connect(SolrAuditDestination.java:92)
>> at
>> org.apache.ranger.audit.destination.SolrAuditDestination.init(SolrAuditDestination.java:56)
>> at
>> org.apache.ranger.audit.provider.AuditProviderFactory.init(AuditProviderFactory.java:163)
>> at
>> org.apache.ranger.authorization.hadoop.config.RangerConfiguration.initAudit(RangerConfiguration.java:127)
>> at
>> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:89)
>> at
>> org.apache.ranger.authorization.hadoop.RangerHdfsPlugin.init(RangerHdfsAuthorizer.java:400)
>> at
>> org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.start(RangerHdfsAuthorizer.java:84)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startCommonServices(FSNamesystem.java:1074)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.startCommonServices(NameNode.java:678)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:664)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:811)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:795)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1488)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.http.entity.mime.content.ContentBody
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>
>>
>> On Tue, Dec 1, 2015 at 4:04 AM, Don Bosco Durai <bo...@apache.org> wrote:
>>
>>> I see the following errors:
>>> 2015-12-01 02:56:30,065 FATAL
>>> org.apache.ranger.audit.destination.SolrAuditDestination: Can't connect to
>>> Solr server. URL=[http://192.168.23.105:6083/solr/ranger_audits]
>>> java.lang.NoClassDefFoundError:
>>> org/apache/http/entity/mime/content/ContentBody
>>> at
>>> org.apache.solr.client.solrj.impl.LBHttpSolrClient.makeSolrClient(LBHttpSolrClient.java:265)
>>>
>>> I had similar issues previously with incompatible http-client and
>>> http-mime jar files. I am not sure whether you are hitting the sample.
>>>
>>> Can you check the hadoop lib folders for *http*.jar files? Mine
>>> is httpclient-4.2.5.jar and we package httpmime-4.2.5.jar with HDFS
>>> plugin.
>>>
>>> Thanks
>>>
>>> Bosco
>>>
>>>
>>> From: Hafiz Mujadid <ha...@gmail.com>
>>> Reply-To: <us...@ranger.incubator.apache.org>
>>> Date: Monday, November 30, 2015 at 2:22 PM
>>> To: <us...@ranger.incubator.apache.org>
>>>
>>> Subject: Re: access audit logs are not being shown on ranger ui
>>>
>>> Hi Bosco,
>>>
>>> Attached is the content of namenode log file.
>>> Solr is working fine, i checked from the web UI
>>> Also /var/log/hadoop/hdfs/audit/solr/spool contains following contents:
>>>
>>> archive
>>> index_batch_batch.solr_hdfs.json
>>> index_batch_batch.solr_hdfs_closed.json
>>>
>>> and all of these files and folders are empty
>>>
>>>
>>> Hi Madhan,
>>>
>>> I have used same URL but still it does not work.
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Dec 1, 2015 at 12:26 AM, Madhan Neethiraj <ma...@apache.org>
>>> wrote:
>>>
>>>> Hafiz,
>>>>
>>>> Different Solr URLs are specified in configuration of HDFS plugin and
>>>> Ranger Admin. Can you please review and update such that they both point to
>>>> the same Solr instance?
>>>>
>>>> properties for hdfs plugin:
>>>>
>>>> *XAAUDIT.SOLR.URL=http://192.168.23.105:6083/solr/ranger_audits
>>>> <http://192.168.23.105:6083/solr/ranger_audits>*
>>>>
>>>>
>>>> properties in ranger-admin:
>>>>
>>>>
>>>> *audit_solr_urls=http://122.129.79.70:6083/solr/ranger_audits
>>>> <http://122.129.79.70:6083/solr/ranger_audits>*
>>>>
>>>>
>>>> Madhan
>>>>
>>>>
>>>> From: Don Bosco Durai <bo...@apache.org>
>>>> Reply-To: "user@ranger.incubator.apache.org" <
>>>> user@ranger.incubator.apache.org>
>>>> Date: Monday, November 30, 2015 at 10:48 AM
>>>> To: "user@ranger.incubator.apache.org" <
>>>> user@ranger.incubator.apache.org>
>>>> Subject: Re: access audit logs are not being shown on ranger ui
>>>>
>>>> The properties seems correct. Best way to test is to :
>>>>
>>>> 1. Check the HDFS logs.
>>>> 2. Check Solr from web interface (use the same url you configured)
>>>> 3. See if there are any unprocessed files in
>>>> */var/log/hadoop/hdfs/audit/solr/spool*
>>>>
>>>> Bosco
>>>>
>>>> From: Hafiz Mujadid <ha...@gmail.com>
>>>> Reply-To: <us...@ranger.incubator.apache.org>
>>>> Date: Monday, November 30, 2015 at 7:22 AM
>>>> To: <us...@ranger.incubator.apache.org>
>>>> Subject: access audit logs are not being shown on ranger ui
>>>>
>>>> hi,
>>>>
>>>> I have enabled ranger audits in solr, but access audits are not being
>>>> shown.
>>>> properties for hdfs plugin
>>>> *XAAUDIT.SOLR.ENABLE=true*
>>>> *XAAUDIT.SOLR.URL=http://192.168.23.105:6083/solr/ranger_audits
>>>> <http://192.168.23.105:6083/solr/ranger_audits>*
>>>> *XAAUDIT.SOLR.USER=ranger_solr*
>>>> *XAAUDIT.SOLR.PASSWORD=NONE*
>>>> *XAAUDIT.SOLR.ZOOKEEPER=NONE*
>>>> *XAAUDIT.SOLR.FILE_SPOOL_DIR=/var/log/hadoop/hdfs/audit/solr/spool*
>>>>
>>>> properties in ranger-admin
>>>>
>>>>
>>>> *audit_store=solr*
>>>>
>>>> *audit_solr_urls=http://122.129.79.70:6083/solr/ranger_audits
>>>> <http://122.129.79.70:6083/solr/ranger_audits>*
>>>> *audit_solr_user=ranger_solr*
>>>> *audit_solr_password=NONE*
>>>> *audit_solr_zookeepers=*
>>>>
>>>> But no audits logs are shown in ranger-admin
>>>>
>>>>
>>>> how can i check the issue ?
>>>>
>>>> thanks
>>>>
>>>>
>>>
>>>
>>> --
>>> Regards: HAFIZ MUJADID
>>>
>>>
>>
>>
>> --
>> Regards: HAFIZ MUJADID
>>
>
>
>
> --
> Regards: HAFIZ MUJADID
>
--
Regards: HAFIZ MUJADID
Re: access audit logs are not being shown on ranger ui
Posted by Hafiz Mujadid <ha...@gmail.com>.
Hi Bosco,
I resolved this issue by placing httpclient jar in ranger-hdfs-plugin/lib
and hadoop/lib and hadoop/share/hadoop/hdfs/lib
But audits are still not being shown
latest hadoop log file is attached. Please have a look and suggest me next
step. thanks
On Tue, Dec 1, 2015 at 12:50 PM, Hafiz Mujadid <ha...@gmail.com>
wrote:
> Hi Bosco,
> I checked the hadoop/lib folder there was no httpclient jar file so i
> downloaded and placed 4.2.5 version of httpclient. httpmime is 4.2.5
> version already. so both the jars are of same version now. But when i start
> hadoop, face following exception as you mentioned
>
> java.lang.NoClassDefFoundError:
> org/apache/http/entity/mime/content/ContentBody
> at
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.makeSolrClient(LBHttpSolrClient.java:265)
> at
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.<init>(LBHttpSolrClient.java:237)
> at
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.<init>(LBHttpSolrClient.java:217)
> at
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.<init>(LBHttpSolrClient.java:212)
> at
> org.apache.ranger.audit.destination.SolrAuditDestination.connect(SolrAuditDestination.java:92)
> at
> org.apache.ranger.audit.destination.SolrAuditDestination.init(SolrAuditDestination.java:56)
> at
> org.apache.ranger.audit.provider.AuditProviderFactory.init(AuditProviderFactory.java:163)
> at
> org.apache.ranger.authorization.hadoop.config.RangerConfiguration.initAudit(RangerConfiguration.java:127)
> at
> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:89)
> at
> org.apache.ranger.authorization.hadoop.RangerHdfsPlugin.init(RangerHdfsAuthorizer.java:400)
> at
> org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.start(RangerHdfsAuthorizer.java:84)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startCommonServices(FSNamesystem.java:1074)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.startCommonServices(NameNode.java:678)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:664)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:811)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:795)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1488)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.http.entity.mime.content.ContentBody
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>
>
> On Tue, Dec 1, 2015 at 4:04 AM, Don Bosco Durai <bo...@apache.org> wrote:
>
>> I see the following errors:
>> 2015-12-01 02:56:30,065 FATAL
>> org.apache.ranger.audit.destination.SolrAuditDestination: Can't connect to
>> Solr server. URL=[http://192.168.23.105:6083/solr/ranger_audits]
>> java.lang.NoClassDefFoundError:
>> org/apache/http/entity/mime/content/ContentBody
>> at
>> org.apache.solr.client.solrj.impl.LBHttpSolrClient.makeSolrClient(LBHttpSolrClient.java:265)
>>
>> I had similar issues previously with incompatible http-client and
>> http-mime jar files. I am not sure whether you are hitting the sample.
>>
>> Can you check the hadoop lib folders for *http*.jar files? Mine
>> is httpclient-4.2.5.jar and we package httpmime-4.2.5.jar with HDFS
>> plugin.
>>
>> Thanks
>>
>> Bosco
>>
>>
>> From: Hafiz Mujadid <ha...@gmail.com>
>> Reply-To: <us...@ranger.incubator.apache.org>
>> Date: Monday, November 30, 2015 at 2:22 PM
>> To: <us...@ranger.incubator.apache.org>
>>
>> Subject: Re: access audit logs are not being shown on ranger ui
>>
>> Hi Bosco,
>>
>> Attached is the content of namenode log file.
>> Solr is working fine, i checked from the web UI
>> Also /var/log/hadoop/hdfs/audit/solr/spool contains following contents:
>>
>> archive
>> index_batch_batch.solr_hdfs.json
>> index_batch_batch.solr_hdfs_closed.json
>>
>> and all of these files and folders are empty
>>
>>
>> Hi Madhan,
>>
>> I have used same URL but still it does not work.
>>
>>
>>
>>
>>
>> On Tue, Dec 1, 2015 at 12:26 AM, Madhan Neethiraj <ma...@apache.org>
>> wrote:
>>
>>> Hafiz,
>>>
>>> Different Solr URLs are specified in configuration of HDFS plugin and
>>> Ranger Admin. Can you please review and update such that they both point to
>>> the same Solr instance?
>>>
>>> properties for hdfs plugin:
>>>
>>> *XAAUDIT.SOLR.URL=http://192.168.23.105:6083/solr/ranger_audits
>>> <http://192.168.23.105:6083/solr/ranger_audits>*
>>>
>>>
>>> properties in ranger-admin:
>>>
>>>
>>> *audit_solr_urls=http://122.129.79.70:6083/solr/ranger_audits
>>> <http://122.129.79.70:6083/solr/ranger_audits>*
>>>
>>>
>>> Madhan
>>>
>>>
>>> From: Don Bosco Durai <bo...@apache.org>
>>> Reply-To: "user@ranger.incubator.apache.org" <
>>> user@ranger.incubator.apache.org>
>>> Date: Monday, November 30, 2015 at 10:48 AM
>>> To: "user@ranger.incubator.apache.org" <user@ranger.incubator.apache.org
>>> >
>>> Subject: Re: access audit logs are not being shown on ranger ui
>>>
>>> The properties seems correct. Best way to test is to :
>>>
>>> 1. Check the HDFS logs.
>>> 2. Check Solr from web interface (use the same url you configured)
>>> 3. See if there are any unprocessed files in
>>> */var/log/hadoop/hdfs/audit/solr/spool*
>>>
>>> Bosco
>>>
>>> From: Hafiz Mujadid <ha...@gmail.com>
>>> Reply-To: <us...@ranger.incubator.apache.org>
>>> Date: Monday, November 30, 2015 at 7:22 AM
>>> To: <us...@ranger.incubator.apache.org>
>>> Subject: access audit logs are not being shown on ranger ui
>>>
>>> hi,
>>>
>>> I have enabled ranger audits in solr, but access audits are not being
>>> shown.
>>> properties for hdfs plugin
>>> *XAAUDIT.SOLR.ENABLE=true*
>>> *XAAUDIT.SOLR.URL=http://192.168.23.105:6083/solr/ranger_audits
>>> <http://192.168.23.105:6083/solr/ranger_audits>*
>>> *XAAUDIT.SOLR.USER=ranger_solr*
>>> *XAAUDIT.SOLR.PASSWORD=NONE*
>>> *XAAUDIT.SOLR.ZOOKEEPER=NONE*
>>> *XAAUDIT.SOLR.FILE_SPOOL_DIR=/var/log/hadoop/hdfs/audit/solr/spool*
>>>
>>> properties in ranger-admin
>>>
>>>
>>> *audit_store=solr*
>>>
>>> *audit_solr_urls=http://122.129.79.70:6083/solr/ranger_audits
>>> <http://122.129.79.70:6083/solr/ranger_audits>*
>>> *audit_solr_user=ranger_solr*
>>> *audit_solr_password=NONE*
>>> *audit_solr_zookeepers=*
>>>
>>> But no audits logs are shown in ranger-admin
>>>
>>>
>>> how can i check the issue ?
>>>
>>> thanks
>>>
>>>
>>
>>
>> --
>> Regards: HAFIZ MUJADID
>>
>>
>
>
> --
> Regards: HAFIZ MUJADID
>
--
Regards: HAFIZ MUJADID
Re: access audit logs are not being shown on ranger ui
Posted by Hafiz Mujadid <ha...@gmail.com>.
Hi Bosco,
I checked the hadoop/lib folder there was no httpclient jar file so i
downloaded and placed 4.2.5 version of httpclient. httpmime is 4.2.5
version already. so both the jars are of same version now. But when i start
hadoop, face following exception as you mentioned
java.lang.NoClassDefFoundError:
org/apache/http/entity/mime/content/ContentBody
at
org.apache.solr.client.solrj.impl.LBHttpSolrClient.makeSolrClient(LBHttpSolrClient.java:265)
at
org.apache.solr.client.solrj.impl.LBHttpSolrClient.<init>(LBHttpSolrClient.java:237)
at
org.apache.solr.client.solrj.impl.LBHttpSolrClient.<init>(LBHttpSolrClient.java:217)
at
org.apache.solr.client.solrj.impl.LBHttpSolrClient.<init>(LBHttpSolrClient.java:212)
at
org.apache.ranger.audit.destination.SolrAuditDestination.connect(SolrAuditDestination.java:92)
at
org.apache.ranger.audit.destination.SolrAuditDestination.init(SolrAuditDestination.java:56)
at
org.apache.ranger.audit.provider.AuditProviderFactory.init(AuditProviderFactory.java:163)
at
org.apache.ranger.authorization.hadoop.config.RangerConfiguration.initAudit(RangerConfiguration.java:127)
at
org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:89)
at
org.apache.ranger.authorization.hadoop.RangerHdfsPlugin.init(RangerHdfsAuthorizer.java:400)
at
org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.start(RangerHdfsAuthorizer.java:84)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startCommonServices(FSNamesystem.java:1074)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.startCommonServices(NameNode.java:678)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:664)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:811)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:795)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1488)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554)
Caused by: java.lang.ClassNotFoundException:
org.apache.http.entity.mime.content.ContentBody
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
On Tue, Dec 1, 2015 at 4:04 AM, Don Bosco Durai <bo...@apache.org> wrote:
> I see the following errors:
> 2015-12-01 02:56:30,065 FATAL
> org.apache.ranger.audit.destination.SolrAuditDestination: Can't connect to
> Solr server. URL=[http://192.168.23.105:6083/solr/ranger_audits]
> java.lang.NoClassDefFoundError:
> org/apache/http/entity/mime/content/ContentBody
> at
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.makeSolrClient(LBHttpSolrClient.java:265)
>
> I had similar issues previously with incompatible http-client and
> http-mime jar files. I am not sure whether you are hitting the sample.
>
> Can you check the hadoop lib folders for *http*.jar files? Mine
> is httpclient-4.2.5.jar and we package httpmime-4.2.5.jar with HDFS
> plugin.
>
> Thanks
>
> Bosco
>
>
> From: Hafiz Mujadid <ha...@gmail.com>
> Reply-To: <us...@ranger.incubator.apache.org>
> Date: Monday, November 30, 2015 at 2:22 PM
> To: <us...@ranger.incubator.apache.org>
>
> Subject: Re: access audit logs are not being shown on ranger ui
>
> Hi Bosco,
>
> Attached is the content of namenode log file.
> Solr is working fine, i checked from the web UI
> Also /var/log/hadoop/hdfs/audit/solr/spool contains following contents:
>
> archive
> index_batch_batch.solr_hdfs.json
> index_batch_batch.solr_hdfs_closed.json
>
> and all of these files and folders are empty
>
>
> Hi Madhan,
>
> I have used same URL but still it does not work.
>
>
>
>
>
> On Tue, Dec 1, 2015 at 12:26 AM, Madhan Neethiraj <ma...@apache.org>
> wrote:
>
>> Hafiz,
>>
>> Different Solr URLs are specified in configuration of HDFS plugin and
>> Ranger Admin. Can you please review and update such that they both point to
>> the same Solr instance?
>>
>> properties for hdfs plugin:
>>
>> *XAAUDIT.SOLR.URL=http://192.168.23.105:6083/solr/ranger_audits
>> <http://192.168.23.105:6083/solr/ranger_audits>*
>>
>>
>> properties in ranger-admin:
>>
>>
>> *audit_solr_urls=http://122.129.79.70:6083/solr/ranger_audits
>> <http://122.129.79.70:6083/solr/ranger_audits>*
>>
>>
>> Madhan
>>
>>
>> From: Don Bosco Durai <bo...@apache.org>
>> Reply-To: "user@ranger.incubator.apache.org" <
>> user@ranger.incubator.apache.org>
>> Date: Monday, November 30, 2015 at 10:48 AM
>> To: "user@ranger.incubator.apache.org" <us...@ranger.incubator.apache.org>
>> Subject: Re: access audit logs are not being shown on ranger ui
>>
>> The properties seems correct. Best way to test is to :
>>
>> 1. Check the HDFS logs.
>> 2. Check Solr from web interface (use the same url you configured)
>> 3. See if there are any unprocessed files in
>> */var/log/hadoop/hdfs/audit/solr/spool*
>>
>> Bosco
>>
>> From: Hafiz Mujadid <ha...@gmail.com>
>> Reply-To: <us...@ranger.incubator.apache.org>
>> Date: Monday, November 30, 2015 at 7:22 AM
>> To: <us...@ranger.incubator.apache.org>
>> Subject: access audit logs are not being shown on ranger ui
>>
>> hi,
>>
>> I have enabled ranger audits in solr, but access audits are not being
>> shown.
>> properties for hdfs plugin
>> *XAAUDIT.SOLR.ENABLE=true*
>> *XAAUDIT.SOLR.URL=http://192.168.23.105:6083/solr/ranger_audits
>> <http://192.168.23.105:6083/solr/ranger_audits>*
>> *XAAUDIT.SOLR.USER=ranger_solr*
>> *XAAUDIT.SOLR.PASSWORD=NONE*
>> *XAAUDIT.SOLR.ZOOKEEPER=NONE*
>> *XAAUDIT.SOLR.FILE_SPOOL_DIR=/var/log/hadoop/hdfs/audit/solr/spool*
>>
>> properties in ranger-admin
>>
>>
>> *audit_store=solr*
>>
>> *audit_solr_urls=http://122.129.79.70:6083/solr/ranger_audits
>> <http://122.129.79.70:6083/solr/ranger_audits>*
>> *audit_solr_user=ranger_solr*
>> *audit_solr_password=NONE*
>> *audit_solr_zookeepers=*
>>
>> But no audits logs are shown in ranger-admin
>>
>>
>> how can i check the issue ?
>>
>> thanks
>>
>>
>
>
> --
> Regards: HAFIZ MUJADID
>
>
--
Regards: HAFIZ MUJADID