You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Abhay Kulkarni <ak...@hortonworks.com> on 2023/11/28 17:55:17 UTC
Review Request 74766: RANGER-4565: Enhance Ranger's performance tracing module to optionally collect statistical information
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74766/
-----------------------------------------------------------
Review request for ranger, Fateh Singh, madhan, Madhan Neethiraj, Mahesh Bandal, Pradeep Agrawal, Ramesh Mani, and Sailaja Polavarapu.
Repository: ranger
Description
-------
Ranger's performance tracing module supports tracking the time spent in specified API(s), as specified in the log4j configuration file. Sometimes it is useful to collect the performance data when an API is called frequently, and instead of tracking performance of a single invocation of the API it is more interesting to see how API performed over a large number of invocations. This Jira addresses this requirement.
The following configuration parameters are introduced to control the behavior of Ranger's performance tracking to enable collecting statistical data.
ranger.perf.aggregate.data - If set to true, the performance data is recorded for specified APIs in aggregate, else data for each API call is printed (default:false)
ranger.perf.aggregate.data.dump.interval - If set to value more than 0, aggregate performance data is printed with specified periodicity in seconds (default:0)
ranger.perf.aggregate.data.lock.enabled - If set to true, aggregate performance data is updated using R-W lock, otherwise locking is not used (default:false).
Diffs
-----
agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java 2f4af9763
agents-common/src/main/java/org/apache/ranger/plugin/util/PerfDataRecorder.java dce60b0ba
agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPerfCollectorTracer.java ebfebdae8
agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPerfTracer.java c8804739f
Diff: https://reviews.apache.org/r/74766/diff/1/
Testing
-------
Passes all unit tests. Verified that the aggregate performance is collected at specified intervals when properly configured.
Thanks,
Abhay Kulkarni
Re: Review Request 74766: RANGER-4565: Enhance Ranger's performance tracing module to optionally collect statistical information
Posted by Abhay Kulkarni <ak...@hortonworks.com>.
> On Nov. 28, 2023, 7:13 p.m., Madhan Neethiraj wrote:
> > agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPerfCollectorTracer.java
> > Line 40 (original), 40 (patched)
> > <https://reviews.apache.org/r/74766/diff/1/?file=2283489#file2283489line40>
> >
> > when ranger.perf.aggregate.data=false, perf log should be written to log file. PerfDataRecorder.recordStatistic() doesn't seem to do this. Please review.
When ranger.perf.aggregate.data=false, RangerPerfCollectorTracer instance is not created when perf logging for a module is enabled. Perf logging is done by RangerPerfTracer instance.
- Abhay
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74766/#review226015
-----------------------------------------------------------
On Nov. 28, 2023, 5:55 p.m., Abhay Kulkarni wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74766/
> -----------------------------------------------------------
>
> (Updated Nov. 28, 2023, 5:55 p.m.)
>
>
> Review request for ranger, Fateh Singh, madhan, Madhan Neethiraj, Mahesh Bandal, Pradeep Agrawal, Ramesh Mani, and Sailaja Polavarapu.
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Ranger's performance tracing module supports tracking the time spent in specified API(s), as specified in the log4j configuration file. Sometimes it is useful to collect the performance data when an API is called frequently, and instead of tracking performance of a single invocation of the API it is more interesting to see how API performed over a large number of invocations. This Jira addresses this requirement.
>
> The following configuration parameters are introduced to control the behavior of Ranger's performance tracking to enable collecting statistical data.
>
> ranger.perf.aggregate.data - If set to true, the performance data is recorded for specified APIs in aggregate, else data for each API call is printed (default:false)
>
> ranger.perf.aggregate.data.dump.interval - If set to value more than 0, aggregate performance data is printed with specified periodicity in seconds (default:0)
>
> ranger.perf.aggregate.data.lock.enabled - If set to true, aggregate performance data is updated using R-W lock, otherwise locking is not used (default:false).
>
>
> Diffs
> -----
>
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java 2f4af9763
> agents-common/src/main/java/org/apache/ranger/plugin/util/PerfDataRecorder.java dce60b0ba
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPerfCollectorTracer.java ebfebdae8
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPerfTracer.java c8804739f
>
>
> Diff: https://reviews.apache.org/r/74766/diff/1/
>
>
> Testing
> -------
>
> Passes all unit tests. Verified that the aggregate performance is collected at specified intervals when properly configured.
>
>
> Thanks,
>
> Abhay Kulkarni
>
>
Re: Review Request 74766: RANGER-4565: Enhance Ranger's performance tracing module to optionally collect statistical information
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74766/#review226015
-----------------------------------------------------------
agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPerfCollectorTracer.java
Line 40 (original), 40 (patched)
<https://reviews.apache.org/r/74766/#comment314366>
when ranger.perf.aggregate.data=false, perf log should be written to log file. PerfDataRecorder.recordStatistic() doesn't seem to do this. Please review.
- Madhan Neethiraj
On Nov. 28, 2023, 5:55 p.m., Abhay Kulkarni wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74766/
> -----------------------------------------------------------
>
> (Updated Nov. 28, 2023, 5:55 p.m.)
>
>
> Review request for ranger, Fateh Singh, madhan, Madhan Neethiraj, Mahesh Bandal, Pradeep Agrawal, Ramesh Mani, and Sailaja Polavarapu.
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Ranger's performance tracing module supports tracking the time spent in specified API(s), as specified in the log4j configuration file. Sometimes it is useful to collect the performance data when an API is called frequently, and instead of tracking performance of a single invocation of the API it is more interesting to see how API performed over a large number of invocations. This Jira addresses this requirement.
>
> The following configuration parameters are introduced to control the behavior of Ranger's performance tracking to enable collecting statistical data.
>
> ranger.perf.aggregate.data - If set to true, the performance data is recorded for specified APIs in aggregate, else data for each API call is printed (default:false)
>
> ranger.perf.aggregate.data.dump.interval - If set to value more than 0, aggregate performance data is printed with specified periodicity in seconds (default:0)
>
> ranger.perf.aggregate.data.lock.enabled - If set to true, aggregate performance data is updated using R-W lock, otherwise locking is not used (default:false).
>
>
> Diffs
> -----
>
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java 2f4af9763
> agents-common/src/main/java/org/apache/ranger/plugin/util/PerfDataRecorder.java dce60b0ba
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPerfCollectorTracer.java ebfebdae8
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPerfTracer.java c8804739f
>
>
> Diff: https://reviews.apache.org/r/74766/diff/1/
>
>
> Testing
> -------
>
> Passes all unit tests. Verified that the aggregate performance is collected at specified intervals when properly configured.
>
>
> Thanks,
>
> Abhay Kulkarni
>
>
Re: Review Request 74766: RANGER-4565: Enhance Ranger's performance tracing module to optionally collect statistical information
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74766/#review226018
-----------------------------------------------------------
Ship it!
Ship It!
- Madhan Neethiraj
On Nov. 28, 2023, 5:55 p.m., Abhay Kulkarni wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74766/
> -----------------------------------------------------------
>
> (Updated Nov. 28, 2023, 5:55 p.m.)
>
>
> Review request for ranger, Fateh Singh, madhan, Madhan Neethiraj, Mahesh Bandal, Pradeep Agrawal, Ramesh Mani, and Sailaja Polavarapu.
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Ranger's performance tracing module supports tracking the time spent in specified API(s), as specified in the log4j configuration file. Sometimes it is useful to collect the performance data when an API is called frequently, and instead of tracking performance of a single invocation of the API it is more interesting to see how API performed over a large number of invocations. This Jira addresses this requirement.
>
> The following configuration parameters are introduced to control the behavior of Ranger's performance tracking to enable collecting statistical data.
>
> ranger.perf.aggregate.data - If set to true, the performance data is recorded for specified APIs in aggregate, else data for each API call is printed (default:false)
>
> ranger.perf.aggregate.data.dump.interval - If set to value more than 0, aggregate performance data is printed with specified periodicity in seconds (default:0)
>
> ranger.perf.aggregate.data.lock.enabled - If set to true, aggregate performance data is updated using R-W lock, otherwise locking is not used (default:false).
>
>
> Diffs
> -----
>
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java 2f4af9763
> agents-common/src/main/java/org/apache/ranger/plugin/util/PerfDataRecorder.java dce60b0ba
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPerfCollectorTracer.java ebfebdae8
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPerfTracer.java c8804739f
>
>
> Diff: https://reviews.apache.org/r/74766/diff/1/
>
>
> Testing
> -------
>
> Passes all unit tests. Verified that the aggregate performance is collected at specified intervals when properly configured.
>
>
> Thanks,
>
> Abhay Kulkarni
>
>