You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Bryan Bende <bb...@gmail.com> on 2016/04/22 19:14:09 UTC
Auditing in Solr
Hi all,
I'm trying to test sending audit logs to Solr from my plugin...
I verified I can get to Solr admin on
http://localhost:6083/solr/#/ranger_audits
In my ranger-nifi-audit.xml I have:
<property>
<name>xasecure.audit.solr.is.enabled</name>
<value>true</value>
</property>
<property>
<name>xasecure.audit.solr.async.max.queue.size</name>
<value>1</value>
</property>
<property>
<name>xasecure.audit.solr.async.max.flush.interval.ms</name>
<value>1000</value>
</property>
<property>
<name>xasecure.audit.solr.solr_url</name>
<value>http://localhost:6083/solr/ranger_audits</value>
</property>
Using the debugger I have verified the SolrAuditProvider connect() method
is getting called and successfully creating an HttpSolrClient.
Then I simulate an authorization request, but the log methods on
SolrAuditProvider never get called, and nothing gets sent to Solr. There
are no errors in the log.
The console prints this during initialization:
[INFO]: No v3 audit configuration found. Trying v2 audit configurations
[INFO]: SolrAuditProvider is enabled
[INFO]: AuditDestination() enter
[INFO]: init() called
[INFO]: BaseAuditProvider.init()
[INFO]: propPrefix=xasecure.audit.provider
[INFO]: Using providerName from property prefix. providerName=provider
[INFO]: providerName=provider
[INFO]: MultiDestAuditProvider: creating..
[INFO]: AsyncAuditProvider(MySolrAuditProvider): creating..
[INFO]:
MultiDestAuditProvider.addAuditProvider(providerType=org.apache.ranger.audit.provider.solr.SolrAuditProvider)
[INFO]: AsyncAuditProvider(MySolrAuditProvider).init()
[INFO]: MultiDestAuditProvider.init()
[INFO]: BaseAuditProvider.init()
[INFO]: propPrefix=xasecure.audit.provider
[INFO]: providerName=multi_dest
[INFO]: init() called
[INFO]: BaseAuditProvider.init()
[INFO]: propPrefix=xasecure.audit.provider
[INFO]: providerName=multi_dest.provider
[INFO]: ==> AsyncAuditProvider.run()
Is there anything I am missing that would stop the events from being sent
to Solr?
Thanks,
Bryan
Re: Auditing in Solr
Posted by Bryan Bende <bb...@gmail.com>.
Balaji/Bosco,
The policy did have auditing turned on. With the configuration Bosco
provided I was able to figure it out...
When I created RangerBasePlugin I never set a RangerAccessResultProcessor,
and then I was calling isAccessAllowed(request) also without a result
processor.
Once I set the result processor as RangerDefaultAuditHandler the events
started hitting Solr and now I can see them through Ranger.
Thanks for the help!
-Bryan
On Fri, Apr 22, 2016 at 3:59 PM, Don Bosco Durai <bo...@apache.org> wrote:
>
> >[INFO]: No v3 audit configuration found. Trying v2 audit configurations
> This doesn’t seem correct. Here are the properties for enabling Solr:
>
> <property>
> <name>xasecure.audit.destination.solr</name>
> <value>true</value>
> </property>
>
> <property>
> <name>xasecure.audit.destination.solr.batch.filespool.dir</name>
> <value>/var/log/hadoop/hdfs/audit/solr/spool</value> (Update path to
> your local spool. Will be used if solr is not available)
> </property>
>
> <property>
> <name>xasecure.audit.destination.solr.zookeepers</name>
> <value>myzookeepr:2181/ranger_audits</value>
> </property>
>
>
>
> If you are using standalone URL:
> <property>
> <name>xasecure.audit.destination.solr.urls</name>
> <value>http://localhost:8666/rangeraudits</value>
> </property>
>
>
>
> Thanks
>
> Bosco
>
>
>
>
>
>
>
> On 4/22/16, 12:14 PM, "Balaji Ganesan" <ba...@gmail.com> wrote:
>
> >Do you have a Ranger policy with auditing on ?
> >
> >On Fri, Apr 22, 2016 at 10:14 AM, Bryan Bende <bb...@gmail.com> wrote:
> >
> >> Hi all,
> >>
> >> I'm trying to test sending audit logs to Solr from my plugin...
> >>
> >> I verified I can get to Solr admin on
> >> http://localhost:6083/solr/#/ranger_audits
> >>
> >> In my ranger-nifi-audit.xml I have:
> >>
> >> <property>
> >> <name>xasecure.audit.solr.is.enabled</name>
> >> <value>true</value>
> >> </property>
> >>
> >> <property>
> >> <name>xasecure.audit.solr.async.max.queue.size</name>
> >> <value>1</value>
> >> </property>
> >>
> >> <property>
> >> <name>xasecure.audit.solr.async.max.flush.interval.ms</name>
> >> <value>1000</value>
> >> </property>
> >>
> >> <property>
> >> <name>xasecure.audit.solr.solr_url</name>
> >> <value>http://localhost:6083/solr/ranger_audits</value>
> >> </property>
> >>
> >> Using the debugger I have verified the SolrAuditProvider connect()
> method
> >> is getting called and successfully creating an HttpSolrClient.
> >>
> >> Then I simulate an authorization request, but the log methods on
> >> SolrAuditProvider never get called, and nothing gets sent to Solr. There
> >> are no errors in the log.
> >>
> >> The console prints this during initialization:
> >>
> >> [INFO]: No v3 audit configuration found. Trying v2 audit configurations
> >> [INFO]: SolrAuditProvider is enabled
> >> [INFO]: AuditDestination() enter
> >> [INFO]: init() called
> >> [INFO]: BaseAuditProvider.init()
> >> [INFO]: propPrefix=xasecure.audit.provider
> >> [INFO]: Using providerName from property prefix. providerName=provider
> >> [INFO]: providerName=provider
> >> [INFO]: MultiDestAuditProvider: creating..
> >> [INFO]: AsyncAuditProvider(MySolrAuditProvider): creating..
> >> [INFO]:
> >>
> >>
> MultiDestAuditProvider.addAuditProvider(providerType=org.apache.ranger.audit.provider.solr.SolrAuditProvider)
> >> [INFO]: AsyncAuditProvider(MySolrAuditProvider).init()
> >> [INFO]: MultiDestAuditProvider.init()
> >> [INFO]: BaseAuditProvider.init()
> >> [INFO]: propPrefix=xasecure.audit.provider
> >> [INFO]: providerName=multi_dest
> >> [INFO]: init() called
> >> [INFO]: BaseAuditProvider.init()
> >> [INFO]: propPrefix=xasecure.audit.provider
> >> [INFO]: providerName=multi_dest.provider
> >> [INFO]: ==> AsyncAuditProvider.run()
> >>
> >> Is there anything I am missing that would stop the events from being
> sent
> >> to Solr?
> >>
> >> Thanks,
> >>
> >> Bryan
> >>
>
>
Re: Auditing in Solr
Posted by Don Bosco Durai <bo...@apache.org>.
>[INFO]: No v3 audit configuration found. Trying v2 audit configurations
This doesn’t seem correct. Here are the properties for enabling Solr:
<property>
<name>xasecure.audit.destination.solr</name>
<value>true</value>
</property>
<property>
<name>xasecure.audit.destination.solr.batch.filespool.dir</name>
<value>/var/log/hadoop/hdfs/audit/solr/spool</value> (Update path to your local spool. Will be used if solr is not available)
</property>
<property>
<name>xasecure.audit.destination.solr.zookeepers</name>
<value>myzookeepr:2181/ranger_audits</value>
</property>
If you are using standalone URL:
<property>
<name>xasecure.audit.destination.solr.urls</name>
<value>http://localhost:8666/rangeraudits</value>
</property>
Thanks
Bosco
On 4/22/16, 12:14 PM, "Balaji Ganesan" <ba...@gmail.com> wrote:
>Do you have a Ranger policy with auditing on ?
>
>On Fri, Apr 22, 2016 at 10:14 AM, Bryan Bende <bb...@gmail.com> wrote:
>
>> Hi all,
>>
>> I'm trying to test sending audit logs to Solr from my plugin...
>>
>> I verified I can get to Solr admin on
>> http://localhost:6083/solr/#/ranger_audits
>>
>> In my ranger-nifi-audit.xml I have:
>>
>> <property>
>> <name>xasecure.audit.solr.is.enabled</name>
>> <value>true</value>
>> </property>
>>
>> <property>
>> <name>xasecure.audit.solr.async.max.queue.size</name>
>> <value>1</value>
>> </property>
>>
>> <property>
>> <name>xasecure.audit.solr.async.max.flush.interval.ms</name>
>> <value>1000</value>
>> </property>
>>
>> <property>
>> <name>xasecure.audit.solr.solr_url</name>
>> <value>http://localhost:6083/solr/ranger_audits</value>
>> </property>
>>
>> Using the debugger I have verified the SolrAuditProvider connect() method
>> is getting called and successfully creating an HttpSolrClient.
>>
>> Then I simulate an authorization request, but the log methods on
>> SolrAuditProvider never get called, and nothing gets sent to Solr. There
>> are no errors in the log.
>>
>> The console prints this during initialization:
>>
>> [INFO]: No v3 audit configuration found. Trying v2 audit configurations
>> [INFO]: SolrAuditProvider is enabled
>> [INFO]: AuditDestination() enter
>> [INFO]: init() called
>> [INFO]: BaseAuditProvider.init()
>> [INFO]: propPrefix=xasecure.audit.provider
>> [INFO]: Using providerName from property prefix. providerName=provider
>> [INFO]: providerName=provider
>> [INFO]: MultiDestAuditProvider: creating..
>> [INFO]: AsyncAuditProvider(MySolrAuditProvider): creating..
>> [INFO]:
>>
>> MultiDestAuditProvider.addAuditProvider(providerType=org.apache.ranger.audit.provider.solr.SolrAuditProvider)
>> [INFO]: AsyncAuditProvider(MySolrAuditProvider).init()
>> [INFO]: MultiDestAuditProvider.init()
>> [INFO]: BaseAuditProvider.init()
>> [INFO]: propPrefix=xasecure.audit.provider
>> [INFO]: providerName=multi_dest
>> [INFO]: init() called
>> [INFO]: BaseAuditProvider.init()
>> [INFO]: propPrefix=xasecure.audit.provider
>> [INFO]: providerName=multi_dest.provider
>> [INFO]: ==> AsyncAuditProvider.run()
>>
>> Is there anything I am missing that would stop the events from being sent
>> to Solr?
>>
>> Thanks,
>>
>> Bryan
>>
Re: Auditing in Solr
Posted by Balaji Ganesan <ba...@gmail.com>.
Do you have a Ranger policy with auditing on ?
On Fri, Apr 22, 2016 at 10:14 AM, Bryan Bende <bb...@gmail.com> wrote:
> Hi all,
>
> I'm trying to test sending audit logs to Solr from my plugin...
>
> I verified I can get to Solr admin on
> http://localhost:6083/solr/#/ranger_audits
>
> In my ranger-nifi-audit.xml I have:
>
> <property>
> <name>xasecure.audit.solr.is.enabled</name>
> <value>true</value>
> </property>
>
> <property>
> <name>xasecure.audit.solr.async.max.queue.size</name>
> <value>1</value>
> </property>
>
> <property>
> <name>xasecure.audit.solr.async.max.flush.interval.ms</name>
> <value>1000</value>
> </property>
>
> <property>
> <name>xasecure.audit.solr.solr_url</name>
> <value>http://localhost:6083/solr/ranger_audits</value>
> </property>
>
> Using the debugger I have verified the SolrAuditProvider connect() method
> is getting called and successfully creating an HttpSolrClient.
>
> Then I simulate an authorization request, but the log methods on
> SolrAuditProvider never get called, and nothing gets sent to Solr. There
> are no errors in the log.
>
> The console prints this during initialization:
>
> [INFO]: No v3 audit configuration found. Trying v2 audit configurations
> [INFO]: SolrAuditProvider is enabled
> [INFO]: AuditDestination() enter
> [INFO]: init() called
> [INFO]: BaseAuditProvider.init()
> [INFO]: propPrefix=xasecure.audit.provider
> [INFO]: Using providerName from property prefix. providerName=provider
> [INFO]: providerName=provider
> [INFO]: MultiDestAuditProvider: creating..
> [INFO]: AsyncAuditProvider(MySolrAuditProvider): creating..
> [INFO]:
>
> MultiDestAuditProvider.addAuditProvider(providerType=org.apache.ranger.audit.provider.solr.SolrAuditProvider)
> [INFO]: AsyncAuditProvider(MySolrAuditProvider).init()
> [INFO]: MultiDestAuditProvider.init()
> [INFO]: BaseAuditProvider.init()
> [INFO]: propPrefix=xasecure.audit.provider
> [INFO]: providerName=multi_dest
> [INFO]: init() called
> [INFO]: BaseAuditProvider.init()
> [INFO]: propPrefix=xasecure.audit.provider
> [INFO]: providerName=multi_dest.provider
> [INFO]: ==> AsyncAuditProvider.run()
>
> Is there anything I am missing that would stop the events from being sent
> to Solr?
>
> Thanks,
>
> Bryan
>