You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Fatima Khan <fa...@gmail.com> on 2017/09/30 06:26:22 UTC
Review Request 62709: RANGER-1817 : Audit to Solr fails to log when
the number of columns are in large number
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62709/
-----------------------------------------------------------
Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.
Bugs: RANGER-1817
https://issues.apache.org/jira/browse/RANGER-1817
Repository: ranger
Description
-------
Audit to Solr fails to log when the number of columns are in large number. This is due to Solr has a hard limit on solr.StrField and if this string is exceeding max length 32766, it throws exception which causes the audit to fail. To overcome this we need to trip this in Audit records and the best place to do it is in solr schema for ranger-audits.
For this we need to change the file managed_schema in ranger and commit it to zookeeper.
Change required in the managed_schema file is, find the following in the managed_schema file and add this param to limit the length to 2500 max.
Diffs
-----
security-admin/contrib/solr_for_audit_setup/conf/managed-schema ee1d894
Diff: https://reviews.apache.org/r/62709/diff/1/
Testing
-------
1.Tested Ranger installation
2.Junit test was successful
Thanks,
Fatima Khan
Re: Review Request 62709: RANGER-1817 : Audit to Solr fails to log
when the number of columns are in large number
Posted by Mehul Parikh <me...@freestoneinfotech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62709/#review187053
-----------------------------------------------------------
Ship it!
Ship It!
- Mehul Parikh
On Sept. 30, 2017, 6:26 a.m., Fatima Khan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62709/
> -----------------------------------------------------------
>
> (Updated Sept. 30, 2017, 6:26 a.m.)
>
>
> Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1817
> https://issues.apache.org/jira/browse/RANGER-1817
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Audit to Solr fails to log when the number of columns are in large number. This is due to Solr has a hard limit on solr.StrField and if this string is exceeding max length 32766, it throws exception which causes the audit to fail. To overcome this we need to trip this in Audit records and the best place to do it is in solr schema for ranger-audits.
>
> For this we need to change the file managed_schema in ranger and commit it to zookeeper.
> Change required in the managed_schema file is, find the following in the managed_schema file and add this param to limit the length to 2500 max.
>
>
> Diffs
> -----
>
> security-admin/contrib/solr_for_audit_setup/conf/managed-schema ee1d894
>
>
> Diff: https://reviews.apache.org/r/62709/diff/1/
>
>
> Testing
> -------
>
> 1.Tested Ranger installation
> 2.Junit test was successful
>
>
> Thanks,
>
> Fatima Khan
>
>
Re: Review Request 62709: RANGER-1817 : Audit to Solr fails to log
when the number of columns are in large number
Posted by Fatima Khan <fa...@gmail.com>.
> On Sept. 30, 2017, 4:24 p.m., Ramesh Mani wrote:
> > security-admin/contrib/solr_for_audit_setup/conf/managed-schema
> > Lines 32 (patched)
> > <https://reviews.apache.org/r/62709/diff/1/?file=1841268#file1841268line32>
> >
> > We need to have a default value for this config in ranger, which can be overridden.
> >
> > Default can be a much less value say 100, which can be rendered on UI without any performance issue.
Currently we are not exposing any properties in Ranger. This will be a new feature.
- Fatima
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62709/#review186786
-----------------------------------------------------------
On Sept. 30, 2017, 6:26 a.m., Fatima Khan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62709/
> -----------------------------------------------------------
>
> (Updated Sept. 30, 2017, 6:26 a.m.)
>
>
> Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1817
> https://issues.apache.org/jira/browse/RANGER-1817
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Audit to Solr fails to log when the number of columns are in large number. This is due to Solr has a hard limit on solr.StrField and if this string is exceeding max length 32766, it throws exception which causes the audit to fail. To overcome this we need to trip this in Audit records and the best place to do it is in solr schema for ranger-audits.
>
> For this we need to change the file managed_schema in ranger and commit it to zookeeper.
> Change required in the managed_schema file is, find the following in the managed_schema file and add this param to limit the length to 2500 max.
>
>
> Diffs
> -----
>
> security-admin/contrib/solr_for_audit_setup/conf/managed-schema ee1d894
>
>
> Diff: https://reviews.apache.org/r/62709/diff/1/
>
>
> Testing
> -------
>
> 1.Tested Ranger installation
> 2.Junit test was successful
>
>
> Thanks,
>
> Fatima Khan
>
>
Re: Review Request 62709: RANGER-1817 : Audit to Solr fails to log
when the number of columns are in large number
Posted by Ramesh Mani <rm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62709/#review186786
-----------------------------------------------------------
security-admin/contrib/solr_for_audit_setup/conf/managed-schema
Lines 32 (patched)
<https://reviews.apache.org/r/62709/#comment263620>
We need to have a default value for this config in ranger, which can be overridden.
Default can be a much less value say 100, which can be rendered on UI without any performance issue.
- Ramesh Mani
On Sept. 30, 2017, 6:26 a.m., Fatima Khan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62709/
> -----------------------------------------------------------
>
> (Updated Sept. 30, 2017, 6:26 a.m.)
>
>
> Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1817
> https://issues.apache.org/jira/browse/RANGER-1817
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Audit to Solr fails to log when the number of columns are in large number. This is due to Solr has a hard limit on solr.StrField and if this string is exceeding max length 32766, it throws exception which causes the audit to fail. To overcome this we need to trip this in Audit records and the best place to do it is in solr schema for ranger-audits.
>
> For this we need to change the file managed_schema in ranger and commit it to zookeeper.
> Change required in the managed_schema file is, find the following in the managed_schema file and add this param to limit the length to 2500 max.
>
>
> Diffs
> -----
>
> security-admin/contrib/solr_for_audit_setup/conf/managed-schema ee1d894
>
>
> Diff: https://reviews.apache.org/r/62709/diff/1/
>
>
> Testing
> -------
>
> 1.Tested Ranger installation
> 2.Junit test was successful
>
>
> Thanks,
>
> Fatima Khan
>
>
Re: Review Request 62709: RANGER-1817 : Audit to Solr fails to log
when the number of columns are in large number
Posted by Ramesh Mani <rm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62709/#review187051
-----------------------------------------------------------
Ship it!
Ship It!
- Ramesh Mani
On Sept. 30, 2017, 6:26 a.m., Fatima Khan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62709/
> -----------------------------------------------------------
>
> (Updated Sept. 30, 2017, 6:26 a.m.)
>
>
> Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1817
> https://issues.apache.org/jira/browse/RANGER-1817
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Audit to Solr fails to log when the number of columns are in large number. This is due to Solr has a hard limit on solr.StrField and if this string is exceeding max length 32766, it throws exception which causes the audit to fail. To overcome this we need to trip this in Audit records and the best place to do it is in solr schema for ranger-audits.
>
> For this we need to change the file managed_schema in ranger and commit it to zookeeper.
> Change required in the managed_schema file is, find the following in the managed_schema file and add this param to limit the length to 2500 max.
>
>
> Diffs
> -----
>
> security-admin/contrib/solr_for_audit_setup/conf/managed-schema ee1d894
>
>
> Diff: https://reviews.apache.org/r/62709/diff/1/
>
>
> Testing
> -------
>
> 1.Tested Ranger installation
> 2.Junit test was successful
>
>
> Thanks,
>
> Fatima Khan
>
>