You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Devaraj Das (JIRA)" <ji...@apache.org> on 2011/02/03 00:18:29 UTC
[jira] Created: (HIVE-1948) Have audit logging in the Metastore
Have audit logging in the Metastore
-----------------------------------
Key: HIVE-1948
URL: https://issues.apache.org/jira/browse/HIVE-1948
Project: Hive
Issue Type: Improvement
Components: Metastore
Reporter: Devaraj Das
Assignee: Devaraj Das
Fix For: 0.7.0
It would be good to have audit logging in the metastore, similar to Hadoop's NameNode audit logging. This would allow administrators to dig into details about which user performed metadata operations (like create/drop tables/partitions) and from where (IP address).
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1948) Have audit logging in the Metastore
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992345#comment-12992345 ]
Namit Jain commented on HIVE-1948:
----------------------------------
Can you regenerate the patch - I am getting some merge conflicts.
> Have audit logging in the Metastore
> -----------------------------------
>
> Key: HIVE-1948
> URL: https://issues.apache.org/jira/browse/HIVE-1948
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.7.0
>
> Attachments: audit-log-2.patch, audit-log.1.patch, audit-log.patch
>
>
> It would be good to have audit logging in the metastore, similar to Hadoop's NameNode audit logging. This would allow administrators to dig into details about which user performed metadata operations (like create/drop tables/partitions) and from where (IP address).
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1948) Have audit logging in the Metastore
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HIVE-1948:
------------------------------
Attachment: audit-log.1.patch
A slightly updated patch.
> Have audit logging in the Metastore
> -----------------------------------
>
> Key: HIVE-1948
> URL: https://issues.apache.org/jira/browse/HIVE-1948
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.7.0
>
> Attachments: audit-log.1.patch, audit-log.patch
>
>
> It would be good to have audit logging in the metastore, similar to Hadoop's NameNode audit logging. This would allow administrators to dig into details about which user performed metadata operations (like create/drop tables/partitions) and from where (IP address).
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1948) Have audit logging in the Metastore
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HIVE-1948:
------------------------------
Attachment: audit-log-2.patch
In this patch I made the logging to happen only when a secure shim is deployed (the case where such audit logging makes most sense).
> Have audit logging in the Metastore
> -----------------------------------
>
> Key: HIVE-1948
> URL: https://issues.apache.org/jira/browse/HIVE-1948
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.7.0
>
> Attachments: audit-log-2.patch, audit-log.1.patch, audit-log.patch
>
>
> It would be good to have audit logging in the metastore, similar to Hadoop's NameNode audit logging. This would allow administrators to dig into details about which user performed metadata operations (like create/drop tables/partitions) and from where (IP address).
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1948) Have audit logging in the Metastore
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12991189#comment-12991189 ]
Devaraj Das commented on HIVE-1948:
-----------------------------------
https://reviews.apache.org/r/398/ is the reviewboard URL
> Have audit logging in the Metastore
> -----------------------------------
>
> Key: HIVE-1948
> URL: https://issues.apache.org/jira/browse/HIVE-1948
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.7.0
>
> Attachments: audit-log.1.patch, audit-log.patch
>
>
> It would be good to have audit logging in the metastore, similar to Hadoop's NameNode audit logging. This would allow administrators to dig into details about which user performed metadata operations (like create/drop tables/partitions) and from where (IP address).
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1948) Have audit logging in the Metastore
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12991570#comment-12991570 ]
Namit Jain commented on HIVE-1948:
----------------------------------
What about the performance impact for this ?
I mean, there seems to be no way to turn it off - is reading the conf.
try {
ugi = ShimLoader.getHadoopShims().getUGIForConf(getConf());
} catch (Exception ex) {
throw new RuntimeException(ex);
}
for every audit operation acceptable ?
> Have audit logging in the Metastore
> -----------------------------------
>
> Key: HIVE-1948
> URL: https://issues.apache.org/jira/browse/HIVE-1948
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.7.0
>
> Attachments: audit-log.1.patch, audit-log.patch
>
>
> It would be good to have audit logging in the metastore, similar to Hadoop's NameNode audit logging. This would allow administrators to dig into details about which user performed metadata operations (like create/drop tables/partitions) and from where (IP address).
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1948) Have audit logging in the Metastore
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HIVE-1948:
------------------------------
Attachment: audit-log-3.patch
Regenerated patch
> Have audit logging in the Metastore
> -----------------------------------
>
> Key: HIVE-1948
> URL: https://issues.apache.org/jira/browse/HIVE-1948
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.7.0
>
> Attachments: audit-log-2.patch, audit-log-3.patch, audit-log.1.patch, audit-log.patch
>
>
> It would be good to have audit logging in the metastore, similar to Hadoop's NameNode audit logging. This would allow administrators to dig into details about which user performed metadata operations (like create/drop tables/partitions) and from where (IP address).
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HIVE-1948) Have audit logging in the Metastore
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992613#comment-12992613 ]
Namit Jain commented on HIVE-1948:
----------------------------------
+1
> Have audit logging in the Metastore
> -----------------------------------
>
> Key: HIVE-1948
> URL: https://issues.apache.org/jira/browse/HIVE-1948
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.7.0
>
> Attachments: audit-log-2.patch, audit-log-3.patch, audit-log.1.patch, audit-log.patch
>
>
> It would be good to have audit logging in the metastore, similar to Hadoop's NameNode audit logging. This would allow administrators to dig into details about which user performed metadata operations (like create/drop tables/partitions) and from where (IP address).
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1948) Have audit logging in the Metastore
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-1948:
-----------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed. Thanks Devaraj
> Have audit logging in the Metastore
> -----------------------------------
>
> Key: HIVE-1948
> URL: https://issues.apache.org/jira/browse/HIVE-1948
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.7.0
>
> Attachments: audit-log-2.patch, audit-log-3.patch, audit-log.1.patch, audit-log.patch
>
>
> It would be good to have audit logging in the metastore, similar to Hadoop's NameNode audit logging. This would allow administrators to dig into details about which user performed metadata operations (like create/drop tables/partitions) and from where (IP address).
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1948) Have audit logging in the Metastore
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HIVE-1948:
------------------------------
Status: Patch Available (was: Open)
> Have audit logging in the Metastore
> -----------------------------------
>
> Key: HIVE-1948
> URL: https://issues.apache.org/jira/browse/HIVE-1948
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.7.0
>
> Attachments: audit-log-2.patch, audit-log-3.patch, audit-log.1.patch, audit-log.patch
>
>
> It would be good to have audit logging in the metastore, similar to Hadoop's NameNode audit logging. This would allow administrators to dig into details about which user performed metadata operations (like create/drop tables/partitions) and from where (IP address).
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1948) Have audit logging in the Metastore
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HIVE-1948:
------------------------------
Attachment: audit-log.patch
Early patch. I am testing it.
> Have audit logging in the Metastore
> -----------------------------------
>
> Key: HIVE-1948
> URL: https://issues.apache.org/jira/browse/HIVE-1948
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.7.0
>
> Attachments: audit-log.patch
>
>
> It would be good to have audit logging in the metastore, similar to Hadoop's NameNode audit logging. This would allow administrators to dig into details about which user performed metadata operations (like create/drop tables/partitions) and from where (IP address).
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1948) Have audit logging in the Metastore
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-1948:
-----------------------------
Status: Open (was: Patch Available)
> Have audit logging in the Metastore
> -----------------------------------
>
> Key: HIVE-1948
> URL: https://issues.apache.org/jira/browse/HIVE-1948
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.7.0
>
> Attachments: audit-log-2.patch, audit-log.1.patch, audit-log.patch
>
>
> It would be good to have audit logging in the metastore, similar to Hadoop's NameNode audit logging. This would allow administrators to dig into details about which user performed metadata operations (like create/drop tables/partitions) and from where (IP address).
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1948) Have audit logging in the Metastore
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-1948:
---------------------------------
Component/s: Security
Logging
> Have audit logging in the Metastore
> -----------------------------------
>
> Key: HIVE-1948
> URL: https://issues.apache.org/jira/browse/HIVE-1948
> Project: Hive
> Issue Type: Improvement
> Components: Logging, Metastore, Security
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.7.0
>
> Attachments: audit-log-2.patch, audit-log-3.patch, audit-log.1.patch, audit-log.patch
>
>
> It would be good to have audit logging in the metastore, similar to Hadoop's NameNode audit logging. This would allow administrators to dig into details about which user performed metadata operations (like create/drop tables/partitions) and from where (IP address).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1948) Have audit logging in the Metastore
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HIVE-1948:
------------------------------
Status: Patch Available (was: Open)
Submitting patch for review. There is one caveat with this patch - it won't log the IP address of the remote clients when security is enabled in Hive. Making this work means a change in thrift. I have raised THRIFT-1053 for the same. Once THRIFT-1053 is addressed, I will provide a fix (in a different jira) to capture the IP address for the secure case too.
> Have audit logging in the Metastore
> -----------------------------------
>
> Key: HIVE-1948
> URL: https://issues.apache.org/jira/browse/HIVE-1948
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.7.0
>
> Attachments: audit-log.1.patch, audit-log.patch
>
>
> It would be good to have audit logging in the metastore, similar to Hadoop's NameNode audit logging. This would allow administrators to dig into details about which user performed metadata operations (like create/drop tables/partitions) and from where (IP address).
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira