You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by "Madhan Neethiraj (Jira)" <ji...@apache.org> on 2024/06/21 15:17:00 UTC
[jira] [Resolved] (RANGER-4761) Reduce memory footprint of hbase plugin
[ https://issues.apache.org/jira/browse/RANGER-4761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Madhan Neethiraj resolved RANGER-4761.
--------------------------------------
Fix Version/s: 3.0.0
2.5.0
Resolution: Fixed
{noformat}
commit fa18ffb9e8195993f2c8f3289fbdffecafb2bd9a
Author: Fateh Singh <fa...@gmail.com>
Date: Mon May 6 13:15:49 2024 -0700
RANGER-4761: [hbase] make lazy memory allocation for family map (#307)
Map<String, Set> getColumnFamilies(Map<byte[], ? extends Collection<?>> families) becomes a bottleneck in multiget and multiput workloads wherein hundred/thousands of columns may be accessed together in a single request as it is a computatio
nally heavy function converting bytes to string and type-casting Collection to set of strings.
In the patch, make lazy memory allocation for family map instead of ahead of time memory allocation for family map of type Map<String, Set>.
{noformat}
> Reduce memory footprint of hbase plugin
> ---------------------------------------
>
> Key: RANGER-4761
> URL: https://issues.apache.org/jira/browse/RANGER-4761
> Project: Ranger
> Issue Type: Improvement
> Components: Ranger
> Reporter: Fateh Singh
> Assignee: Fateh Singh
> Priority: Major
> Fix For: 3.0.0, 2.5.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> * Map<String, Set> getColumnFamilies(Map<byte[], ? extends Collection<?>> families) becomes a bottleneck in multiget and multiput workloads wherein hundred/thousands of columns may be accessed together in a single request as it is a computationally heavy function converting bytes to string and type-casting Collection to set of strings.
> The implementation needs to be revisited to reduce memory footprint
> make lazy memory allocation for family map instead of ahead of time memory allocation for family map of type Map<String, Set>. With this approach, memory footprint will get reduced even further when enabling column auth optimization supported by RANGER-4670
--
This message was sent by Atlassian Jira
(v8.20.10#820010)