You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sentry.apache.org by Arjun Mishra <am...@cloudera.com> on 2017/09/08 19:23:38 UTC
Review Request 62192: SENTRY-1932: Improve logging for HMSPath
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62192/
-----------------------------------------------------------
Review request for sentry, Alexander Kolbasov, Na Li, and Vamsee Yarlagadda.
Repository: sentry
Description
-------
Currently HMSPath logs too much information when it skips paths which are outside the prefix:
//Begin of log snippet
2017-09-07 15:29:39,041 INFO org.apache.sentry.hdfs.HMSPaths: Path outside prefix
2017-09-07 15:29:39,040 INFO org.apache.sentry.hdfs.HMSPaths: Skipping to create authzObjPath as it is outside of prefix. authObj = l5_foo.customer_v1 pathElements=[data, blah, l5_blah_blah, CUSTOMER_V1, eap_part_no=12345]
//End of log snippet
This happens when Sentry sends a full snapshot and when NameNode receives delta snapshots.
Probably we should log information more clearly and log at DEBUG level.
Diffs
-----
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java 5a982637d
Diff: https://reviews.apache.org/r/62192/diff/1/
Testing
-------
Thanks,
Arjun Mishra
Re: Review Request 62192: SENTRY-1932: Improve logging for HMSPath
Posted by Alexander Kolbasov <ak...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62192/#review185026
-----------------------------------------------------------
Fix it, then Ship it!
Do we need both of these logs? May be one is sufficient?
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java
Line 423 (original), 423 (patched)
<https://reviews.apache.org/r/62192/#comment261257>
You are using LOG.debug so this check is useless anyway.
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java
Line 704 (original), 703 (patched)
<https://reviews.apache.org/r/62192/#comment261258>
Remove isDebugEnabled()
- Alexander Kolbasov
On Sept. 8, 2017, 9:19 p.m., Arjun Mishra wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62192/
> -----------------------------------------------------------
>
> (Updated Sept. 8, 2017, 9:19 p.m.)
>
>
> Review request for sentry, Alexander Kolbasov, Na Li, and Vamsee Yarlagadda.
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Currently HMSPath logs too much information when it skips paths which are outside the prefix:
>
> //Begin of log snippet
> 2017-09-07 15:29:39,041 INFO org.apache.sentry.hdfs.HMSPaths: Path outside prefix
> 2017-09-07 15:29:39,040 INFO org.apache.sentry.hdfs.HMSPaths: Skipping to create authzObjPath as it is outside of prefix. authObj = l5_foo.customer_v1 pathElements=[data, blah, l5_blah_blah, CUSTOMER_V1, eap_part_no=12345]
> //End of log snippet
>
> This happens when Sentry sends a full snapshot and when NameNode receives delta snapshots.
> Probably we should log information more clearly and log at DEBUG level.
>
>
> Diffs
> -----
>
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java 5a982637d
>
>
> Diff: https://reviews.apache.org/r/62192/diff/2/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Arjun Mishra
>
>
Re: Review Request 62192: SENTRY-1932: Improve logging for HMSPath
Posted by Alexander Kolbasov <ak...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62192/#review185033
-----------------------------------------------------------
Ship it!
Ship It!
- Alexander Kolbasov
On Sept. 8, 2017, 10:37 p.m., Arjun Mishra wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62192/
> -----------------------------------------------------------
>
> (Updated Sept. 8, 2017, 10:37 p.m.)
>
>
> Review request for sentry, Alexander Kolbasov, Na Li, and Vamsee Yarlagadda.
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Currently HMSPath logs too much information when it skips paths which are outside the prefix:
>
> //Begin of log snippet
> 2017-09-07 15:29:39,041 INFO org.apache.sentry.hdfs.HMSPaths: Path outside prefix
> 2017-09-07 15:29:39,040 INFO org.apache.sentry.hdfs.HMSPaths: Skipping to create authzObjPath as it is outside of prefix. authObj = l5_foo.customer_v1 pathElements=[data, blah, l5_blah_blah, CUSTOMER_V1, eap_part_no=12345]
> //End of log snippet
>
> This happens when Sentry sends a full snapshot and when NameNode receives delta snapshots.
> Probably we should log information more clearly and log at DEBUG level.
>
>
> Diffs
> -----
>
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java 5a982637d
>
>
> Diff: https://reviews.apache.org/r/62192/diff/3/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Arjun Mishra
>
>
Re: Review Request 62192: SENTRY-1932: Improve logging for HMSPath
Posted by Arjun Mishra <am...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62192/
-----------------------------------------------------------
(Updated Sept. 8, 2017, 10:37 p.m.)
Review request for sentry, Alexander Kolbasov, Na Li, and Vamsee Yarlagadda.
Changes
-------
Incorporated Sasha's suggestioins. Removed isDebugEnabled
Repository: sentry
Description
-------
Currently HMSPath logs too much information when it skips paths which are outside the prefix:
//Begin of log snippet
2017-09-07 15:29:39,041 INFO org.apache.sentry.hdfs.HMSPaths: Path outside prefix
2017-09-07 15:29:39,040 INFO org.apache.sentry.hdfs.HMSPaths: Skipping to create authzObjPath as it is outside of prefix. authObj = l5_foo.customer_v1 pathElements=[data, blah, l5_blah_blah, CUSTOMER_V1, eap_part_no=12345]
//End of log snippet
This happens when Sentry sends a full snapshot and when NameNode receives delta snapshots.
Probably we should log information more clearly and log at DEBUG level.
Diffs (updated)
-----
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java 5a982637d
Diff: https://reviews.apache.org/r/62192/diff/3/
Changes: https://reviews.apache.org/r/62192/diff/2-3/
Testing
-------
Thanks,
Arjun Mishra
Re: Review Request 62192: SENTRY-1932: Improve logging for HMSPath
Posted by Arjun Mishra <am...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62192/
-----------------------------------------------------------
(Updated Sept. 8, 2017, 9:19 p.m.)
Review request for sentry, Alexander Kolbasov, Na Li, and Vamsee Yarlagadda.
Changes
-------
Incorporated Lina's changes
Repository: sentry
Description
-------
Currently HMSPath logs too much information when it skips paths which are outside the prefix:
//Begin of log snippet
2017-09-07 15:29:39,041 INFO org.apache.sentry.hdfs.HMSPaths: Path outside prefix
2017-09-07 15:29:39,040 INFO org.apache.sentry.hdfs.HMSPaths: Skipping to create authzObjPath as it is outside of prefix. authObj = l5_foo.customer_v1 pathElements=[data, blah, l5_blah_blah, CUSTOMER_V1, eap_part_no=12345]
//End of log snippet
This happens when Sentry sends a full snapshot and when NameNode receives delta snapshots.
Probably we should log information more clearly and log at DEBUG level.
Diffs (updated)
-----
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java 5a982637d
Diff: https://reviews.apache.org/r/62192/diff/2/
Changes: https://reviews.apache.org/r/62192/diff/1-2/
Testing
-------
Thanks,
Arjun Mishra
Re: Review Request 62192: SENTRY-1932: Improve logging for HMSPath
Posted by Brian Towles <bt...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62192/#review185013
-----------------------------------------------------------
Ship it!
Ship It!
- Brian Towles
On Sept. 8, 2017, 2:23 p.m., Arjun Mishra wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62192/
> -----------------------------------------------------------
>
> (Updated Sept. 8, 2017, 2:23 p.m.)
>
>
> Review request for sentry, Alexander Kolbasov, Na Li, and Vamsee Yarlagadda.
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Currently HMSPath logs too much information when it skips paths which are outside the prefix:
>
> //Begin of log snippet
> 2017-09-07 15:29:39,041 INFO org.apache.sentry.hdfs.HMSPaths: Path outside prefix
> 2017-09-07 15:29:39,040 INFO org.apache.sentry.hdfs.HMSPaths: Skipping to create authzObjPath as it is outside of prefix. authObj = l5_foo.customer_v1 pathElements=[data, blah, l5_blah_blah, CUSTOMER_V1, eap_part_no=12345]
> //End of log snippet
>
> This happens when Sentry sends a full snapshot and when NameNode receives delta snapshots.
> Probably we should log information more clearly and log at DEBUG level.
>
>
> Diffs
> -----
>
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java 5a982637d
>
>
> Diff: https://reviews.apache.org/r/62192/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Arjun Mishra
>
>
Re: Review Request 62192: SENTRY-1932: Improve logging for HMSPath
Posted by Na Li <li...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62192/#review185015
-----------------------------------------------------------
sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java
Line 705 (original), 705 (patched)
<https://reviews.apache.org/r/62192/#comment261248>
You can use LOG.debug("Cannot create authz obj path for {} because it is outside of prefix", authzObj); It has better performance.
- Na Li
On Sept. 8, 2017, 7:23 p.m., Arjun Mishra wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62192/
> -----------------------------------------------------------
>
> (Updated Sept. 8, 2017, 7:23 p.m.)
>
>
> Review request for sentry, Alexander Kolbasov, Na Li, and Vamsee Yarlagadda.
>
>
> Repository: sentry
>
>
> Description
> -------
>
> Currently HMSPath logs too much information when it skips paths which are outside the prefix:
>
> //Begin of log snippet
> 2017-09-07 15:29:39,041 INFO org.apache.sentry.hdfs.HMSPaths: Path outside prefix
> 2017-09-07 15:29:39,040 INFO org.apache.sentry.hdfs.HMSPaths: Skipping to create authzObjPath as it is outside of prefix. authObj = l5_foo.customer_v1 pathElements=[data, blah, l5_blah_blah, CUSTOMER_V1, eap_part_no=12345]
> //End of log snippet
>
> This happens when Sentry sends a full snapshot and when NameNode receives delta snapshots.
> Probably we should log information more clearly and log at DEBUG level.
>
>
> Diffs
> -----
>
> sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/HMSPaths.java 5a982637d
>
>
> Diff: https://reviews.apache.org/r/62192/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Arjun Mishra
>
>