[jira] [Comment Edited] (SENTRY-2299) NPE In Sentry HDFS Sync Plugin

["Arjun Mishra (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SENTRY-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16877939#comment-16877939 ] 

Arjun Mishra edited comment on SENTRY-2299 at 7/8/19 2:25 PM:
--------------------------------------------------------------

Please see instructions to reproduce the issue.
# Create database _db1_ in non-sentry-managed location
## create database db1 location '/user/external/db1.db';
# Create another database _db2_ in managed location
## create database db2;
# Create table db1.tbl1 in non-sentry-managed location   
## create table db1.tbl1 (x INT) location '/user/external/db1.db/tbl1';
# Rename table _db1.tbl1 to db2.tbl2_ thereby both renaming it and moving locations from non-managed to managed paths
## alter table db1.tbl1 rename to db2.tbl2; 

This will cause result in an NPE. Then go ahead and create more tables/databases in sentry managed paths and you wont' see their ACLs being applied.

Test same with patch


was (Author: arjunmishra13):
Please see instructions to reproduce the issue.
# Create database _db1_ in non-sentry-managed location
## create database db1 location '/user/external/db1.db';
# Create another database _db2_ in managed location
## create database db2;
# Create external table db1.tbl1 in non-sentry-managed location   
## create external table db1.tbl1 (x INT) location '/user/external/db1.db/tbl1';
# Rename table _db1.tbl1 to db2.tbl2_ thereby both renaming it and moving locations from non-managed to managed paths
## alter table db1.tbl1 rename to db2.tbl2; 

This will cause result in an NPE. Then go ahead and create more tables/databases in sentry managed paths and you wont' see their ACLs being applied.

Test same with patch

> NPE In Sentry HDFS Sync Plugin
> ------------------------------
>
>                 Key: SENTRY-2299
>                 URL: https://issues.apache.org/jira/browse/SENTRY-2299
>             Project: Sentry
>          Issue Type: Bug
>          Components: Sentry
>    Affects Versions: 2.1.0
>            Reporter: Na Li
>            Assignee: Na Li
>            Priority: Critical
>         Attachments: SENTRY-2299.001.patch
>
>
> Sentry HDFS ACL synchronization stopped working and throws NullPointerException. The HDFS logs showed repeating errors like the following: 
> {code}
> 11:16:15.743 AM	WARN	SentryAuthorizationInfo	
> Failed to update, will retry in [30000]ms, error: 
> java.lang.NullPointerException
> at org.apache.sentry.hdfs.HMSPaths$Entry.access$200(HMSPaths.java:146)
> at org.apache.sentry.hdfs.HMSPaths.renameAuthzObject(HMSPaths.java:879)
> at org.apache.sentry.hdfs.UpdateableAuthzPaths.applyPartialUpdate(UpdateableAuthzPaths.java:118)
> at org.apache.sentry.hdfs.UpdateableAuthzPaths.updatePartial(UpdateableAuthzPaths.java:81)
> at org.apache.sentry.hdfs.SentryAuthorizationInfo.processUpdates(SentryAuthorizationInfo.java:211)
> at org.apache.sentry.hdfs.SentryAuthorizationInfo.update(SentryAuthorizationInfo.java:139)
> at org.apache.sentry.hdfs.SentryAuthorizationInfo.run(SentryAuthorizationInfo.java:232)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> The customer checked the Sentry logs and didn't see any corresponding errors. The issue stopped occurring, apparently not through any specific user intervention. (The customer tried manually failing over the active NameNode, with no change.)
> {code}
> Arjun mentioned the reason is that some delta update from sentry server was lost, so the oldEntry at line HMSPaths.java:879 was null. That caused null exception. 
> {code}
>   void renameAuthzObject(String oldName, List<List<String>> oldPathElems,
>       String newName, List<List<String>> newPathElems) {
>     if (LOG.isDebugEnabled()) {
>       LOG.debug(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})",
>         this, oldName, assemblePaths(oldPathElems), newName, assemblePaths(newPathElems)));
>     }
>     if (oldPathElems == null || oldPathElems.isEmpty() ||
>         newPathElems == null || newPathElems.isEmpty() ||
>         newName == null || newName.equals(oldName)) {
>       LOG.warn(String.format("%s renameAuthzObject({%s, %s} -> {%s, %s})" +
>         ": invalid inputs, skipping",
>         this, oldName, assemblePaths(oldPathElems), newName, assemblePaths(newPathElems)));
>       return;
>     }
>     // if oldPath == newPath, that is path has not changed as part of rename and hence new table
>     // needs to have old paths => new_table.add(old_table_partition_paths)
>     List<String> oldPathElements = oldPathElems.get(0);
>     List<String> newPathElements = newPathElems.get(0);
>     if (!oldPathElements.equals(newPathElements)) {
>       Entry oldEntry = root.find(oldPathElements.toArray(new String[0]), false);
>       Entry newParent = root.createParent(newPathElements);
>       oldEntry.moveTo(newParent, newPathElements.get(newPathElements.size() - 1)); -> oldEntry is null
>     }
> {code}
> There are several possible reasons why some delta changes are lost.
> {code}
> 1. Sentry server does not save the rename update as delta update. The chance is really low
> 2. The delta change is lost from sentry server to name node plugin. The chance is also low
> 3. When applying delta change about old entry, it is lost
> {code}
> The fix for this issue
> 1. Check if oldEntry is null. If so, don't call oldEntry.moveTo. Instead, throw exception, which will be caught by its caller and causes sentry client at name node plugin gets path full snapshot from sentry server. 
> 2. Find out why the oldEntry is null and fix it. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)