You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@sentry.apache.org by "Alexander Kolbasov (JIRA)" <ji...@apache.org> on 2018/03/20 18:22:00 UTC
[jira] [Updated] (SENTRY-2185) Performance Issue: Saving
MAuthzPathsMapping should be done in batch for path full snapshot
[ https://issues.apache.org/jira/browse/SENTRY-2185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexander Kolbasov updated SENTRY-2185:
---------------------------------------
Description:
From the code below, for each MAuthzPathsMapping, there is a query to save each MAuthzPathsMapping instance. Need to save the changes in batch.
{code:java}
/**
* Persist an up-to-date HMS snapshot into Sentry DB in a single transaction with its latest
* notification ID
*
* @param authzPaths paths to be be persisted
* @param notificationID the latest notificationID associated with the snapshot
* @throws Exception
*/public void persistFullPathsImage(final Map<String, Collection<String>> authzPaths,final long notificationID) throws Exception {
tm.executeTransactionWithRetry(
pm -> {
pm.setDetachAllOnCommit(false); // No need to detach objectsdeleteNotificationsSince(pm, notificationID + 1);// persist the notidicationIDpm.makePersistent(new MSentryHmsNotification(notificationID));// persist the full snapshotlong snapshotID = getCurrentAuthzPathsSnapshotID(pm);long nextSnapshotID = snapshotID + 1;
pm.makePersistent(new MAuthzPathsSnapshotId(nextSnapshotID));
LOGGER.info("Attempting to commit new HMS snapshot with ID = {}", nextSnapshotID);for (Map.Entry<String, Collection<String>> authzPath : authzPaths.entrySet()) {
pm.makePersistent(new MAuthzPathsMapping(nextSnapshotID, authzPath.getKey(), authzPath.getValue()));
}return null;
});
}
{code}
was:
From the code below, for each MAuthzPathsMapping, there is a query to save each MAuthzPathsMapping instance. Need to save the changes in batch.
/**
* Persist an up-to-date HMS snapshot into Sentry DB in a single transaction with its latest
* notification ID
*
* @param authzPaths paths to be be persisted
* @param notificationID the latest notificationID associated with the snapshot
* @throws Exception
*/public void persistFullPathsImage(final Map<String, Collection<String>> authzPaths,final long notificationID) throws Exception {
tm.executeTransactionWithRetry(
pm -> {
pm.setDetachAllOnCommit(false); // No need to detach objectsdeleteNotificationsSince(pm, notificationID + 1);// persist the notidicationIDpm.makePersistent(new MSentryHmsNotification(notificationID));// persist the full snapshotlong snapshotID = getCurrentAuthzPathsSnapshotID(pm);long nextSnapshotID = snapshotID + 1;
pm.makePersistent(new MAuthzPathsSnapshotId(nextSnapshotID));
LOGGER.info("Attempting to commit new HMS snapshot with ID = {}", nextSnapshotID);for (Map.Entry<String, Collection<String>> authzPath : authzPaths.entrySet()) {
pm.makePersistent(new MAuthzPathsMapping(nextSnapshotID, authzPath.getKey(), authzPath.getValue()));
}return null;
});
}
> Performance Issue: Saving MAuthzPathsMapping should be done in batch for path full snapshot
> -------------------------------------------------------------------------------------------
>
> Key: SENTRY-2185
> URL: https://issues.apache.org/jira/browse/SENTRY-2185
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
> Affects Versions: 2.1.0
> Reporter: Na Li
> Priority: Major
>
> From the code below, for each MAuthzPathsMapping, there is a query to save each MAuthzPathsMapping instance. Need to save the changes in batch.
> {code:java}
> /**
> * Persist an up-to-date HMS snapshot into Sentry DB in a single transaction with its latest
> * notification ID
> *
> * @param authzPaths paths to be be persisted
> * @param notificationID the latest notificationID associated with the snapshot
> * @throws Exception
> */public void persistFullPathsImage(final Map<String, Collection<String>> authzPaths,final long notificationID) throws Exception {
> tm.executeTransactionWithRetry(
> pm -> {
> pm.setDetachAllOnCommit(false); // No need to detach objectsdeleteNotificationsSince(pm, notificationID + 1);// persist the notidicationIDpm.makePersistent(new MSentryHmsNotification(notificationID));// persist the full snapshotlong snapshotID = getCurrentAuthzPathsSnapshotID(pm);long nextSnapshotID = snapshotID + 1;
> pm.makePersistent(new MAuthzPathsSnapshotId(nextSnapshotID));
> LOGGER.info("Attempting to commit new HMS snapshot with ID = {}", nextSnapshotID);for (Map.Entry<String, Collection<String>> authzPath : authzPaths.entrySet()) {
> pm.makePersistent(new MAuthzPathsMapping(nextSnapshotID, authzPath.getKey(), authzPath.getValue()));
> }return null;
> });
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)