You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@ranger.apache.org by Gautam Borad <gb...@gmail.com> on 2016/01/05 11:24:27 UTC

Review Request 41919: RANGER-798 : Handle different timezone issue while saving audit logs to Solr

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41919/
-----------------------------------------------------------

Review request for ranger, Alok Lal, Don Bosco Durai, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.


Bugs: RANGER-798
    https://issues.apache.org/jira/browse/RANGER-798


Repository: ranger


Description
-------

As a solution, sending new Date object value as audit event time to Solr Doc, if audit store is Solr.


Diffs
-----

  agents-audit/src/main/java/org/apache/ranger/audit/destination/SolrAuditDestination.java df471c3 

Diff: https://reviews.apache.org/r/41919/diff/


Testing
-------

Steps performed(after patch) :
1) Changed plugin system time zone to IST and restarted all components.
2) Initiated an HDFS audit event.
3) Checked event time in Solr of newly created audit log, Audit log event time  was matching with UTC.
4) Checked event time in Ranger UI, Newly generated Audit event is matching with current time.


Thanks,

Gautam Borad

Re: Review Request 41919: RANGER-798 : Handle different timezone issue while saving audit logs to Solr

Posted by Don Bosco Durai <bo...@apache.org>.


> On Jan. 5, 2016, 8:56 p.m., Don Bosco Durai wrote:
> > agents-audit/src/main/java/org/apache/ranger/audit/destination/SolrAuditDestination.java, line 203
> > <https://reviews.apache.org/r/41919/diff/1/?file=1181599#file1181599line203>
> >
> >     auditEvent.eventTime is already a date. Why are we overriding it?
> 
> Gautam Borad wrote:
>     Bosco, please check the updated description. Hope it clarifies the fix.

The eventTime is coming from the Request object and it should be the current server timezone and we shouldn't be needing to reset it. If we are resetting it, then we are breaking some assumptions, e.g. the original access might have happened in the past, the audit code might be getting it later (e.g. bufferred or delayed). So the time would be incorrect. So we have to make sure the time comes from the origin.

Can we investigate, why the original date is in UTC and who sets it?


- Don Bosco


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41919/#review112925
-----------------------------------------------------------


On Jan. 6, 2016, 4:06 a.m., Gautam Borad wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41919/
> -----------------------------------------------------------
> 
> (Updated Jan. 6, 2016, 4:06 a.m.)
> 
> 
> Review request for ranger, Alok Lal, Don Bosco Durai, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-798
>     https://issues.apache.org/jira/browse/RANGER-798
> 
> 
> Repository: ranger
> 
> 
> Description
> -------
> 
> As a solution, sending new Date object value as audit event time to Solr Doc, if audit store is Solr.
> 
> The current implementation of “auditEvent.getEventTime()” contains time in UTC and since it's a date object it will contain component machine's local timezone. When Solr receives this date object and timezone, it tries to convert it from given timezone to UTC timestamp, so their is double conversion of actual time before it get stored in Solr.
> 
> If we can provide server local time and timezone to Solr then Solr will convert the received time from given Timezone to UTC.
> 
> 
> Diffs
> -----
> 
>   agents-audit/src/main/java/org/apache/ranger/audit/destination/SolrAuditDestination.java df471c3 
> 
> Diff: https://reviews.apache.org/r/41919/diff/
> 
> 
> Testing
> -------
> 
> Steps performed(after patch) :
> 1) Changed plugin system time zone to IST and restarted all components.
> 2) Initiated an HDFS audit event.
> 3) Checked event time in Solr of newly created audit log, Audit log event time  was matching with UTC.
> 4) Checked event time in Ranger UI, Newly generated Audit event is matching with current time.
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>

Re: Review Request 41919: RANGER-798 : Handle different timezone issue while saving audit logs to Solr

Posted by Gautam Borad <gb...@gmail.com>.


> On Jan. 5, 2016, 8:56 p.m., Don Bosco Durai wrote:
> > agents-audit/src/main/java/org/apache/ranger/audit/destination/SolrAuditDestination.java, line 203
> > <https://reviews.apache.org/r/41919/diff/1/?file=1181599#file1181599line203>
> >
> >     auditEvent.eventTime is already a date. Why are we overriding it?

Bosco, please check the updated description. Hope it clarifies the fix.


- Gautam


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41919/#review112925
-----------------------------------------------------------


On Jan. 6, 2016, 4:06 a.m., Gautam Borad wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41919/
> -----------------------------------------------------------
> 
> (Updated Jan. 6, 2016, 4:06 a.m.)
> 
> 
> Review request for ranger, Alok Lal, Don Bosco Durai, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-798
>     https://issues.apache.org/jira/browse/RANGER-798
> 
> 
> Repository: ranger
> 
> 
> Description
> -------
> 
> As a solution, sending new Date object value as audit event time to Solr Doc, if audit store is Solr.
> 
> The current implementation of “auditEvent.getEventTime()” contains time in UTC and since it's a date object it will contain component machine's local timezone. When Solr receives this date object and timezone, it tries to convert it from given timezone to UTC timestamp, so their is double conversion of actual time before it get stored in Solr.
> 
> If we can provide server local time and timezone to Solr then Solr will convert the received time from given Timezone to UTC.
> 
> 
> Diffs
> -----
> 
>   agents-audit/src/main/java/org/apache/ranger/audit/destination/SolrAuditDestination.java df471c3 
> 
> Diff: https://reviews.apache.org/r/41919/diff/
> 
> 
> Testing
> -------
> 
> Steps performed(after patch) :
> 1) Changed plugin system time zone to IST and restarted all components.
> 2) Initiated an HDFS audit event.
> 3) Checked event time in Solr of newly created audit log, Audit log event time  was matching with UTC.
> 4) Checked event time in Ranger UI, Newly generated Audit event is matching with current time.
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>

Re: Review Request 41919: RANGER-798 : Handle different timezone issue while saving audit logs to Solr

Posted by Don Bosco Durai <bo...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41919/#review112925
-----------------------------------------------------------



agents-audit/src/main/java/org/apache/ranger/audit/destination/SolrAuditDestination.java (line 203)
<https://reviews.apache.org/r/41919/#comment173406>

    auditEvent.eventTime is already a date. Why are we overriding it?


- Don Bosco Durai


On Jan. 5, 2016, 10:24 a.m., Gautam Borad wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41919/
> -----------------------------------------------------------
> 
> (Updated Jan. 5, 2016, 10:24 a.m.)
> 
> 
> Review request for ranger, Alok Lal, Don Bosco Durai, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-798
>     https://issues.apache.org/jira/browse/RANGER-798
> 
> 
> Repository: ranger
> 
> 
> Description
> -------
> 
> As a solution, sending new Date object value as audit event time to Solr Doc, if audit store is Solr.
> 
> 
> Diffs
> -----
> 
>   agents-audit/src/main/java/org/apache/ranger/audit/destination/SolrAuditDestination.java df471c3 
> 
> Diff: https://reviews.apache.org/r/41919/diff/
> 
> 
> Testing
> -------
> 
> Steps performed(after patch) :
> 1) Changed plugin system time zone to IST and restarted all components.
> 2) Initiated an HDFS audit event.
> 3) Checked event time in Solr of newly created audit log, Audit log event time  was matching with UTC.
> 4) Checked event time in Ranger UI, Newly generated Audit event is matching with current time.
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>

Re: Review Request 41919: RANGER-798 - Approach 2 : Handle different timezone issue while saving audit logs to Solr

Posted by Gautam Borad <gb...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41919/
-----------------------------------------------------------

(Updated Jan. 27, 2016, 4:21 a.m.)

Review request for ranger, Alok Lal, Don Bosco Durai, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.

Changes
-------

Update patch will convert the received UTC timestamp value to Local timestamp and there will not be any difference in actual audit event time and event time inserted in solr.

Summary (updated)
-----------------

RANGER-798 - Approach 2 : Handle different timezone issue while saving audit logs to Solr

Bugs: RANGER-798
https://issues.apache.org/jira/browse/RANGER-798

Repository: ranger

Description
-------

As a solution, sending new Date object value as audit event time to Solr Doc, if audit store is Solr.

The current implementation of “auditEvent.getEventTime()” contains time in UTC and since it's a date object it will contain component machine's local timezone. When Solr receives this date object and timezone, it tries to convert it from given timezone to UTC timestamp, so their is double conversion of actual time before it get stored in Solr.

If we can provide server local time and timezone to Solr then Solr will convert the received time from given Timezone to UTC.

Diffs (updated)
-----

agents-audit/src/main/java/org/apache/ranger/audit/destination/SolrAuditDestination.java f875842
agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java 9586f73

Diff: https://reviews.apache.org/r/41919/diff/

Testing
-------

Steps performed(after patch) :
1) Changed plugin system time zone to IST and restarted all components.
2) Initiated an HDFS audit event.
3) Checked event time in Solr of newly created audit log, Audit log event time was matching with UTC.
4) Checked event time in Ranger UI, Newly generated Audit event is matching with current time.

Thanks,

Gautam Borad

Re: Review Request 41919: RANGER-798 : Handle different timezone issue while saving audit logs to Solr

Posted by Gautam Borad <gb...@gmail.com>.

(Updated Jan. 6, 2016, 4:06 a.m.)

Review request for ranger, Alok Lal, Don Bosco Durai, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.

Changes
-------

Updated the Desctiption to handle Boscos comment.

Bugs: RANGER-798
https://issues.apache.org/jira/browse/RANGER-798

Repository: ranger

Description (updated)
-------

As a solution, sending new Date object value as audit event time to Solr Doc, if audit store is Solr.

If we can provide server local time and timezone to Solr then Solr will convert the received time from given Timezone to UTC.

Diffs
-----

agents-audit/src/main/java/org/apache/ranger/audit/destination/SolrAuditDestination.java df471c3

Diff: https://reviews.apache.org/r/41919/diff/

Testing
-------

Thanks,

Gautam Borad