You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by "Madhan Neethiraj (Jira)" <ji...@apache.org> on 2024/06/21 15:17:00 UTC

[jira] [Resolved] (RANGER-4761) Reduce memory footprint of hbase plugin

     [ https://issues.apache.org/jira/browse/RANGER-4761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Madhan Neethiraj resolved RANGER-4761.
--------------------------------------
    Fix Version/s: 3.0.0
                   2.5.0
       Resolution: Fixed

{noformat}
commit fa18ffb9e8195993f2c8f3289fbdffecafb2bd9a
Author: Fateh Singh <fa...@gmail.com>
Date:   Mon May 6 13:15:49 2024 -0700

    RANGER-4761: [hbase] make lazy memory allocation for family map (#307)

    Map<String, Set> getColumnFamilies(Map<byte[], ? extends Collection<?>> families) becomes a bottleneck in multiget and multiput workloads wherein hundred/thousands of columns may be accessed together in a single request as it is a computatio
nally heavy function converting bytes to string and type-casting Collection to set of strings.

    In the patch, make lazy memory allocation for family map instead of ahead of time memory allocation for family map of type Map<String, Set>.
{noformat}

> Reduce memory footprint of hbase plugin
> ---------------------------------------
>
>                 Key: RANGER-4761
>                 URL: https://issues.apache.org/jira/browse/RANGER-4761
>             Project: Ranger
>          Issue Type: Improvement
>          Components: Ranger
>            Reporter: Fateh Singh
>            Assignee: Fateh Singh
>            Priority: Major
>             Fix For: 3.0.0, 2.5.0
>
>          Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> * Map<String, Set> getColumnFamilies(Map<byte[], ? extends Collection<?>> families) becomes a bottleneck in multiget and multiput workloads wherein hundred/thousands of columns may be accessed together in a single request as it is a computationally heavy function converting bytes to string and type-casting Collection to set of strings.
> The implementation needs to be revisited to reduce memory footprint
> make lazy memory allocation for family map instead of ahead of time memory allocation for family map of type Map<String, Set>. With this approach, memory footprint will get reduced even further when enabling column auth optimization supported by RANGER-4670



--
This message was sent by Atlassian Jira
(v8.20.10#820010)