You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by "Bhavik Patel (Jira)" <ji...@apache.org> on 2023/04/11 13:08:00 UTC
[jira] [Commented] (RANGER-4178) NoClassDefFoundError: org/apache/hadoop/hive/ql/exec/vector/ColumnVector
[ https://issues.apache.org/jira/browse/RANGER-4178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17710943#comment-17710943 ]
Bhavik Patel commented on RANGER-4178:
--------------------------------------
[~rmani] Do we have any wiki where all the steps are listed to enable hdfs audit support in ORC file format?
I followed steps which are mentioned on RANGER-1837 but apart from that 2 more extra properties I added
* xasecure.audit.destination.hdfs.batch.filequeue.filetype=orc
* xasecure.audit.provider.filetype=orc
after adding above two properties I'm observing NullPointer Error
{code:java}
980 DEBUG sasl.DataTransferSaslUtil (DataTransferSaslUtil.java:getSaslPropertiesResolver(200)) - DataTransferProtocol using SaslPropertiesResolver, configured QOP dfs.data.transfer.protection = authentication,privacy, configured class dfs.data.transfer.saslproperties.resolver.class = class org.apache.hadoop.security.SaslPropertiesResolver
981 DEBUG unix.DomainSocketWatcher (DomainSocketWatcher.java:run(447)) - org.apache.hadoop.net.unix.DomainSocketWatcher$2@75f25ec5: starting with interruptCheckPeriodMs = 60000
984 ERROR utils.RangerORCAuditWriter (RangerORCAuditWriter.java:logAuditAsORC(87)) - Error while writing into ORC FileWriter
java.lang.NullPointerException
at org.apache.ranger.audit.utils.AbstractRangerAuditWriter.getFileSystemScheme(AbstractRangerAuditWriter.java:353)
at org.apache.ranger.audit.utils.AbstractRangerAuditWriter.createFileSystemFolders(AbstractRangerAuditWriter.java:105)
at org.apache.ranger.audit.utils.RangerORCAuditWriter.getORCFileWrite(RangerORCAuditWriter.java:149)
at org.apache.ranger.audit.utils.RangerORCAuditWriter$1.run(RangerORCAuditWriter.java:80)
at org.apache.ranger.audit.utils.RangerORCAuditWriter$1.run(RangerORCAuditWriter.java:77)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
at org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:541)
at org.apache.ranger.audit.utils.RangerORCAuditWriter.logAuditAsORC(RangerORCAuditWriter.java:77)
at org.apache.ranger.audit.utils.RangerORCAuditWriter.logAsORC(RangerORCAuditWriter.java:164)
at org.apache.ranger.audit.utils.RangerORCAuditWriter.log(RangerORCAuditWriter.java:116)
at org.apache.ranger.audit.destination.HDFSAuditDestination.logJSON(HDFSAuditDestination.java:81)
at org.apache.ranger.audit.destination.HDFSAuditDestination.log(HDFSAuditDestination.java:170)
at org.apache.ranger.audit.queue.AuditFileQueueSpool.sendEvent(AuditFileQueueSpool.java:943)
at org.apache.ranger.audit.queue.AuditFileQueueSpool.logEvent(AuditFileQueueSpool.java:930)
at org.apache.ranger.audit.queue.AuditFileQueueSpool.runLogAudit(AuditFileQueueSpool.java:861)
at org.apache.ranger.audit.queue.AuditFileQueueSpool.run(AuditFileQueueSpool.java:797)
at java.lang.Thread.run(Thread.java:750)
985 DEBUG utils.RangerORCAuditWriter (RangerORCAuditWriter.java:logAuditAsORC(91)) - Flushing HDFS audit in ORC Format. Event Size:7
,985 ERROR provider.BaseAuditHandler (BaseAuditHandler.java:logError(360)) - Error writing to log file.
java.lang.NullPointerException
at org.apache.ranger.audit.utils.AbstractRangerAuditWriter.getFileSystemScheme(AbstractRangerAuditWriter.java:353)
at org.apache.ranger.audit.utils.AbstractRangerAuditWriter.createFileSystemFolders(AbstractRangerAuditWriter.java:105)
at org.apache.ranger.audit.utils.RangerORCAuditWriter.getORCFileWrite(RangerORCAuditWriter.java:149)
at org.apache.ranger.audit.utils.RangerORCAuditWriter$1.run(RangerORCAuditWriter.java:80)
at org.apache.ranger.audit.utils.RangerORCAuditWriter$1.run(RangerORCAuditWriter.java:77)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
at org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:541)
at org.apache.ranger.audit.utils.RangerORCAuditWriter.logAuditAsORC(RangerORCAuditWriter.java:77)
at org.apache.ranger.audit.utils.RangerORCAuditWriter.logAsORC(RangerORCAuditWriter.java:164)
at org.apache.ranger.audit.utils.RangerORCAuditWriter.log(RangerORCAuditWriter.java:116)
at org.apache.ranger.audit.destination.HDFSAuditDestination.logJSON(HDFSAuditDestination.java:81)
at org.apache.ranger.audit.destination.HDFSAuditDestination.log(HDFSAuditDestination.java:170)
at org.apache.ranger.audit.queue.AuditFileQueueSpool.sendEvent(AuditFileQueueSpool.java:943)
at org.apache.ranger.audit.queue.AuditFileQueueSpool.logEvent(AuditFileQueueSpool.java:930)
at org.apache.ranger.audit.queue.AuditFileQueueSpool.runLogAudit(AuditFileQueueSpool.java:861)
at org.apache.ranger.audit.queue.AuditFileQueueSpool.run(AuditFileQueueSpool.java:797)
at java.lang.Thread.run(Thread.java:750) {code}
Let me know if any other extra properties are required to configure.
> NoClassDefFoundError: org/apache/hadoop/hive/ql/exec/vector/ColumnVector
> ------------------------------------------------------------------------
>
> Key: RANGER-4178
> URL: https://issues.apache.org/jira/browse/RANGER-4178
> Project: Ranger
> Issue Type: Bug
> Components: audit
> Affects Versions: 3.0.0, 2.2.0, 2.3.0
> Reporter: Bhavik Patel
> Assignee: Bhavik Patel
> Priority: Critical
> Attachments: 0001-RANGER-4178-NoClassDefFoundError-org-apache-hadoop-h.patch
>
>
> Observed below error when enabled audit type as ORC format.
> NoClassDefFoundError: org/apache/hadoop/hive/ql/exec/vector/ColumnVector
> https://issues.apache.org/jira/browse/RANGER-1837
> https://issues.apache.org/jira/browse/RANGER-3235
> cc: [~rmani]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)