[jira] [Created] (CALCITE-7666) Make regex pattern caches static to avoid repeated allocation in RexInterpreter

["Yu Xu (Jira)" <ji...@apache.org>]

Yu Xu created CALCITE-7666:
------------------------------

             Summary: Make regex pattern caches static to avoid repeated allocation in RexInterpreter
                 Key: CALCITE-7666
                 URL: https://issues.apache.org/jira/browse/CALCITE-7666
             Project: Calcite
          Issue Type: Improvement
          Components: core
    Affects Versions: 1.42.0
            Reporter: Yu Xu
            Assignee: Yu Xu
             Fix For: 1.43.0


Recently, during a performance test of Simpifly, I discovered a memory performance issue:
{code:java}
SELECT * FROM t
WHERE v0 AND (v0 OR v1 OR v2 OR ... OR v9)           -- 10 boolean expression
  AND (i0 = 0 OR i0 = 1 OR i0 = 2 OR i0 = 3)         -- 10 integer expression
  AND (s0 = 'foo' OR s0 = 'bar')                      -- 1 string expression {code}
Tests of this kind generally cannot complete successfully because the data volume is massive: 2^10 (Boolean) × 4^10 (Integer) × 2^1 (String) = 21,233,664 combinations

Based on this test, I analyzed the memory status of the process:

It is primarily caused by the following caches:
SqlFunctions.LikeFunction
SqlFunctions.SimilarFunction
 

SqlFunctions.SimilarEscapeFunction

 
{code:java}
private final LoadingCache<Key, Pattern> cache =
      CacheBuilder.newBuilder()
          .maximumSize(FUNCTION_LEVEL_CACHE_MAX_SIZE.value())
          .build(CacheLoader.from(Key::toPattern)); {code}
When tests frequently call `new RexInterpreter()`, a new `LikeFunction()` or `SimilarFunction()` is instantiated each time, and each of these constructors, in turn, creates a separate Guava `LoadingCache`.

When optimized the cache to be static, memory overhead was significantly reduced.The comparison results are as follows:
||metrics ||current||after change||ratio||
|LocalCache instance|134,586|0|↓100%|
|Memory usage|677M    |285M|↓58%|
|Memory compression|259M|36M|↓86%|

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)