You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by mi...@apache.org on 2023/12/19 20:47:23 UTC
(impala) branch master updated: IMPALA-11805: Use llvm ObjectCache for codegen caching
This is an automated email from the ASF dual-hosted git repository.
michaelsmith pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
The following commit(s) were added to refs/heads/master by this push:
new f93bd9862 IMPALA-11805: Use llvm ObjectCache for codegen caching
f93bd9862 is described below
commit f93bd986214e390375a34199c205537e440e2b25
Author: Yida Wu <yi...@cloudera.com>
AuthorDate: Sun Nov 26 00:49:34 2023 -0800
IMPALA-11805: Use llvm ObjectCache for codegen caching
Currently, we employ llvm::ExecutionEngine for codegen caching,
providing access to compiled functions within the cached engine.
However, the real challenge is the ExecutionEngine uses a lot of
memory which largely exceeds our memory estimates and it is very
hard to predict.
This patch addresses this issue by using llvm::ObjectCache for
codegen caching. In our case, each execution engine would have
only one module, and after the compilation of the module, the
compiled codegened functions of the module would be set to the
execution engine, therefore functions could be used by Impala.
During function compilation within the module, if an ObjectCache
is set to the execution engine, the compiled codegened functions
would be also written into the cache. This way, if we keep the
cache, when revisiting the same module (fragment), we can
efficiently reuse the specific ObjectCache, loading pre-compiled
codegened functions and saving time.
The tpch performance test indicates no significant regression
compared to the previous use of ExecutionEngine. Post-change,
the actual memory usage of each codegen caching entry is notably
reduced.
+----------+-----------------------+---------+------------+------------+----------------+
| Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) |
+----------+-----------------------+---------+------------+------------+----------------+
| TPCH(1) | parquet / none / none | 0.22 | -0.65% | 0.20 | -0.75% |
+----------+-----------------------+---------+------------+------------+----------------+
+----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------+----------------+---------+-------+
| Workload | Query | File Format | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%) | Base StdDev(%) | Iters | Median Diff(%) | MW Zval | Tval |
+----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------+----------------+---------+-------+
| TPCH(1) | TPCH-Q13 | parquet / none / none | 0.49 | 0.47 | +2.80% | 5.32% | 5.07% | 10 | +1.22% | 1.63 | 1.19 |
| TPCH(1) | TPCH-Q4 | parquet / none / none | 0.16 | 0.16 | +3.51% | 1.32% | * 10.38% * | 10 | +0.06% | 0.49 | 1.06 |
| TPCH(1) | TPCH-Q11 | parquet / none / none | 0.12 | 0.12 | +1.39% | 2.27% | 2.24% | 10 | +1.50% | 1.90 | 1.37 |
| TPCH(1) | TPCH-Q19 | parquet / none / none | 0.21 | 0.21 | +1.56% | * 10.02% * | * 11.42% * | 10 | +1.18% | 0.57 | 0.32 |
| TPCH(1) | TPCH-Q18 | parquet / none / none | 0.27 | 0.27 | +1.71% | 6.46% | 1.29% | 10 | -0.19% | -1.19 | 0.81 |
| TPCH(1) | TPCH-Q6 | parquet / none / none | 0.11 | 0.11 | +0.79% | 2.76% | 2.15% | 10 | +0.10% | 1.46 | 0.71 |
| TPCH(1) | TPCH-Q3 | parquet / none / none | 0.26 | 0.26 | +0.71% | 6.63% | 6.18% | 10 | +0.04% | 0.49 | 0.25 |
| TPCH(1) | TPCH-Q17 | parquet / none / none | 0.17 | 0.17 | +0.41% | * 14.66% * | * 13.01% * | 10 | +0.05% | 0.40 | 0.07 |
| TPCH(1) | TPCH-Q14 | parquet / none / none | 0.16 | 0.16 | +0.19% | 1.41% | 1.39% | 10 | +0.25% | 1.46 | 0.31 |
| TPCH(1) | TPCH-Q20 | parquet / none / none | 0.17 | 0.17 | +0.22% | 1.70% | 1.77% | 10 | -0.05% | -0.40 | 0.28 |
| TPCH(1) | TPCH-Q12 | parquet / none / none | 0.16 | 0.16 | -0.27% | 0.54% | 1.46% | 10 | +0.14% | 0.93 | -0.54 |
| TPCH(1) | TPCH-Q22 | parquet / none / none | 0.11 | 0.11 | -0.38% | 0.81% | 2.06% | 10 | +0.03% | 0.22 | -0.54 |
| TPCH(1) | TPCH-Q16 | parquet / none / none | 0.17 | 0.17 | -0.38% | 0.67% | 1.58% | 10 | -0.01% | -0.13 | -0.70 |
| TPCH(1) | TPCH-Q8 | parquet / none / none | 0.27 | 0.27 | -0.08% | 1.24% | 1.15% | 10 | -0.33% | -1.37 | -0.15 |
| TPCH(1) | TPCH-Q15 | parquet / none / none | 0.16 | 0.16 | -1.18% | * 16.61% * | * 10.25% * | 10 | +0.33% | 0.40 | -0.19 |
| TPCH(1) | TPCH-Q1 | parquet / none / none | 0.22 | 0.22 | -1.67% | 1.62% | 7.45% | 10 | +0.43% | 1.02 | -0.70 |
| TPCH(1) | TPCH-Q5 | parquet / none / none | 0.22 | 0.22 | -0.98% | 0.22% | 1.55% | 10 | -0.26% | -2.16 | -1.97 |
| TPCH(1) | TPCH-Q21 | parquet / none / none | 0.48 | 0.49 | -1.18% | 3.58% | 4.40% | 10 | -0.25% | -1.19 | -0.66 |
| TPCH(1) | TPCH-Q10 | parquet / none / none | 0.26 | 0.26 | -1.93% | 7.84% | 6.24% | 10 | -0.14% | -0.13 | -0.62 |
| TPCH(1) | TPCH-Q7 | parquet / none / none | 0.18 | 0.19 | -3.31% | * 11.47% * | * 12.47% * | 10 | -0.25% | -1.72 | -0.63 |
| TPCH(1) | TPCH-Q9 | parquet / none / none | 0.34 | 0.35 | -5.22% | 6.87% | * 10.03% * | 10 | -2.15% | -1.28 | -1.38 |
| TPCH(1) | TPCH-Q2 | parquet / none / none | 0.16 | 0.18 | -11.00% | * 16.07% * | 3.84% | 10 | -0.90% | -1.81 | -2.35 |
+----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------+----------------+---------+-------+
We are no longer using ExecutionEngine for caching, so we got rid of
the LlvmExecutionEngineWrapper class. Instead, we brought in a new
class CodeGenObjectCache to implement llvm::ObjectCache.
Testing:
Passed LlvmCodeGenCacheTest and custom_cluster/test_codegen_cache.py.
Change-Id: Ic3c1b46bb9018ed0320817141785a3bdc41fa677
Reviewed-on: http://gerrit.cloudera.org:8080/20733
Reviewed-by: Michael Smith <mi...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
be/src/codegen/CMakeLists.txt | 1 +
be/src/codegen/llvm-codegen-cache-test.cc | 279 +++++++++++++++++--------
be/src/codegen/llvm-codegen-cache.cc | 51 ++---
be/src/codegen/llvm-codegen-cache.h | 44 ++--
be/src/codegen/llvm-codegen-object-cache.cc | 81 +++++++
be/src/codegen/llvm-codegen-object-cache.h | 43 ++++
be/src/codegen/llvm-codegen.cc | 119 ++++++-----
be/src/codegen/llvm-codegen.h | 41 ++--
be/src/codegen/llvm-execution-engine-wrapper.h | 59 ------
be/src/runtime/test-env.h | 6 +-
tests/custom_cluster/test_codegen_cache.py | 15 +-
11 files changed, 468 insertions(+), 271 deletions(-)
diff --git a/be/src/codegen/CMakeLists.txt b/be/src/codegen/CMakeLists.txt
index 45f8ae0f1..327b7c4a4 100644
--- a/be/src/codegen/CMakeLists.txt
+++ b/be/src/codegen/CMakeLists.txt
@@ -38,6 +38,7 @@ add_library(CodeGen
codegen-util.cc
llvm-codegen.cc
llvm-codegen-cache.cc
+ llvm-codegen-object-cache.cc
instruction-counter.cc
${THIRDPARTY_LLVM_SRC_DIR}/SectionMemoryManager.cpp
${IR_O1_C_FILE}
diff --git a/be/src/codegen/llvm-codegen-cache-test.cc b/be/src/codegen/llvm-codegen-cache-test.cc
index e1976eeca..76597a1dd 100644
--- a/be/src/codegen/llvm-codegen-cache-test.cc
+++ b/be/src/codegen/llvm-codegen-cache-test.cc
@@ -33,10 +33,20 @@ DECLARE_bool(cache_force_single_shard);
DECLARE_string(codegen_cache_capacity);
namespace impala {
+
+// The object cache size for an engine containing a single compiled
+// function in our test case. The value is in bytes.
+const int ENGINE_CACHE_SIZE = 1100;
+// Capacity for large codegen cache. 256KB.
+const int64_t CODEGEN_CACHE_CAPACITY = 256 * 1024;
+
class LlvmCodeGenCacheTest : public testing::Test {
public:
virtual void SetUp() {
FLAGS_codegen_cache_capacity = "0";
+ // Using single shard makes the logic of scenarios simple for capacity and
+ // eviction-related behavior.
+ FLAGS_cache_force_single_shard = true;
metrics_.reset(new MetricGroup("codegen-cache-test"));
profile_ = RuntimeProfile::Create(&obj_pool_, "codegen-cache-test");
test_env_.reset(new TestEnv);
@@ -51,6 +61,7 @@ class LlvmCodeGenCacheTest : public testing::Test {
}
virtual void TearDown() {
+ FLAGS_cache_force_single_shard = false;
fragment_state_->ReleaseResources();
fragment_state_ = nullptr;
codegen_cache_.reset();
@@ -69,18 +80,12 @@ class LlvmCodeGenCacheTest : public testing::Test {
return codegen->AddFunctionToJitInternal(fn, fn_ptr);
}
- static Status FinalizeModule(LlvmCodeGen* codegen) { return codegen->FinalizeModule(); }
-
- static void CheckResult(CodeGenCacheEntry& entry, bool is_double = false) {
- ASSERT_TRUE(!entry.Empty());
- ASSERT_TRUE(entry.engine_pointer != nullptr);
- CheckResult(entry.engine_pointer, is_double);
+ static Status FinalizeModule(LlvmCodeGen* codegen, string* module_id = nullptr) {
+ return codegen->FinalizeModule(module_id);
}
static void CheckResult(LlvmCodeGen* codegen, bool is_double = false) {
- ASSERT_TRUE(codegen->execution_engine_wrapper_cached_ != nullptr);
- llvm::ExecutionEngine* cached_execution_engine =
- codegen->execution_engine_wrapper_cached_->execution_engine();
+ llvm::ExecutionEngine* cached_execution_engine = codegen->execution_engine();
ASSERT_TRUE(cached_execution_engine != nullptr);
CheckResult(cached_execution_engine, is_double);
}
@@ -101,9 +106,13 @@ class LlvmCodeGenCacheTest : public testing::Test {
}
}
- void AddLlvmCodegenEcho(LlvmCodeGen* codegen);
- void AddLlvmCodegenDouble(LlvmCodeGen* codegen);
- void GetLlvmEmptyFunction(LlvmCodeGen* codegen, llvm::Function**);
+ void CheckResult(
+ CodeGenCacheEntry& entry, const string& module_id, bool is_double = false);
+ shared_ptr<CodeGenObjectCache> CreateObjCache(bool is_double, string* module_id);
+ void AddLlvmCodegenEcho(LlvmCodeGen* codegen, string* module_id, bool doFinalizeModule);
+ void AddLlvmCodegenDouble(
+ LlvmCodeGen* codegen, string* module_id, bool doFinalizeModule);
+ void GetLlvmEmptyFunction(LlvmCodeGen* codegen, llvm::Function** func);
typedef int (*TestEcho)(int);
typedef int (*TestDouble)(int);
typedef void (*TestEmpty)();
@@ -113,13 +122,14 @@ class LlvmCodeGenCacheTest : public testing::Test {
void CheckMetrics(CodeGenCache*, int, int, int);
void CheckInUseMetrics(CodeGenCache*, int, int64_t);
void CheckEvictMetrics(CodeGenCache*, int);
+ void CheckObjCacheExists(LlvmCodeGen*);
int64_t GetMemCharge(LlvmCodeGen* codegen, string key_str, bool is_normal_mode);
- void CheckEngineCount(LlvmCodeGen*, int expect_count);
+ void CheckEngineCacheCount(LlvmCodeGen*, int expect_count);
void CheckToInsertMap();
void TestSwitchModeHelper(TCodeGenCacheMode::type mode, string key,
- int expect_entry_num, int expect_engine_num, llvm::ExecutionEngine** engine);
+ int expect_entry_num, int expect_engine_num, CodeGenObjectCache** cached_engine);
bool CheckKeyExist(TCodeGenCacheMode::type mode, string key);
- bool CheckEngineExist(llvm::ExecutionEngine* engine);
+ bool CheckEngineExist(CodeGenObjectCache* cached_engine);
void ExpectNumEngineSameAsEntry();
void StoreHelper(TCodeGenCacheMode::type mode, string key);
void TestConcurrentStore(int num_threads);
@@ -134,13 +144,68 @@ class LlvmCodeGenCacheTest : public testing::Test {
RuntimeProfile* profile_;
scoped_ptr<TestEnv> test_env_;
scoped_ptr<CodeGenCache> codegen_cache_;
- shared_ptr<LlvmExecutionEngineWrapper> exec_engine_wrapper_;
+ shared_ptr<CodeGenObjectCache> codegen_obj_cache_;
TQueryOptions query_options_;
TCodeGenOptLevel::type opt_level_ = TCodeGenOptLevel::O2;
};
-void LlvmCodeGenCacheTest::AddLlvmCodegenEcho(LlvmCodeGen* codegen) {
- ASSERT_TRUE(codegen != NULL);
+void LlvmCodeGenCacheTest::CheckResult(
+ CodeGenCacheEntry& entry, const string& module_id, bool is_double) {
+ ASSERT_TRUE(!entry.Empty());
+ ASSERT_TRUE(entry.cached_engine_pointer != nullptr);
+ scoped_ptr<LlvmCodeGen> codegen;
+ ASSERT_OK(
+ LlvmCodeGen::CreateImpalaCodegen(fragment_state_, nullptr, "test1", &codegen));
+ if (is_double) {
+ AddLlvmCodegenDouble(codegen.get(), nullptr, false /*doFinalizeModule*/);
+ } else {
+ AddLlvmCodegenEcho(codegen.get(), nullptr, false /*doFinalizeModule*/);
+ }
+ ASSERT_OK(codegen->FinalizeLazyMaterialization());
+ codegen->module_->setModuleIdentifier(module_id);
+ // Use the specific cache for finalizing the module and expect the compiled functions
+ // in the execution engine is from the cache, then verify the correctness.
+ codegen->execution_engine()->setObjectCache(entry.cached_engine_pointer);
+ codegen->execution_engine()->finalizeObject();
+ codegen->DestroyModule();
+ CheckResult(codegen.get(), is_double);
+ codegen->Close();
+}
+
+// The function is to create and return a CodeGenObjectCache which contains a specific
+// compiled codegened function.
+shared_ptr<CodeGenObjectCache> LlvmCodeGenCacheTest::CreateObjCache(
+ bool is_double, string* module_id) {
+ scoped_ptr<LlvmCodeGen> codegen;
+ shared_ptr<CodeGenObjectCache> engine_cache = make_shared<CodeGenObjectCache>();
+ EXPECT_TRUE(
+ LlvmCodeGen::CreateImpalaCodegen(fragment_state_, nullptr, "test", &codegen).ok());
+ string m_id;
+ if (is_double) {
+ AddLlvmCodegenDouble(codegen.get(), nullptr, false /*doFinalizeModule*/);
+ m_id = "module_double";
+ } else {
+ AddLlvmCodegenEcho(codegen.get(), nullptr, false /*doFinalizeModule*/);
+ m_id = "module_echo";
+ }
+ EXPECT_TRUE(codegen->FinalizeLazyMaterialization().ok());
+ codegen->PruneModule();
+ EXPECT_TRUE(codegen->OptimizeModule().ok());
+ codegen->module_->setModuleIdentifier(m_id);
+ codegen->engine_cache_ = engine_cache;
+ codegen->execution_engine()->setObjectCache(engine_cache.get());
+ codegen->execution_engine()->finalizeObject();
+ CheckObjCacheExists(codegen.get());
+ codegen->DestroyModule();
+ CheckResult(codegen.get(), is_double);
+ codegen->Close();
+ if (module_id != nullptr) *module_id = m_id;
+ return engine_cache;
+}
+
+void LlvmCodeGenCacheTest::AddLlvmCodegenEcho(
+ LlvmCodeGen* codegen, string* module_id = nullptr, bool doFinalizeModule = true) {
+ ASSERT_TRUE(codegen != nullptr);
LlvmCodeGen::FnPrototype prototype(codegen, "Echo", codegen->i32_type());
prototype.AddArgument(LlvmCodeGen::NamedVariable("n", codegen->i32_type()));
LlvmBuilder builder(codegen->context());
@@ -148,29 +213,32 @@ void LlvmCodeGenCacheTest::AddLlvmCodegenEcho(LlvmCodeGen* codegen) {
llvm::Function* fn = prototype.GeneratePrototype(&builder, args);
builder.CreateRet(args[0]);
fn = codegen->FinalizeFunction(fn);
- ASSERT_TRUE(fn != NULL);
+ ASSERT_TRUE(fn != nullptr);
CodegenFnPtr<TestEcho> jitted_fn;
AddFunctionToJit(codegen, fn, &jitted_fn);
- ASSERT_OK(FinalizeModule(codegen));
- ASSERT_TRUE(jitted_fn.load() != nullptr);
- TestEcho test_fn = jitted_fn.load();
- ASSERT_EQ(test_fn(1), 1);
+ if (doFinalizeModule) {
+ ASSERT_OK(FinalizeModule(codegen, module_id));
+ ASSERT_TRUE(jitted_fn.load() != nullptr);
+ TestEcho test_fn = jitted_fn.load();
+ ASSERT_EQ(test_fn(1), 1);
+ }
}
void LlvmCodeGenCacheTest::GetLlvmEmptyFunction(
LlvmCodeGen* codegen, llvm::Function** func) {
- ASSERT_TRUE(codegen != NULL);
+ ASSERT_TRUE(codegen != nullptr);
LlvmCodeGen::FnPrototype prototype(codegen, "TestEmpty", codegen->void_type());
LlvmBuilder builder(codegen->context());
llvm::Function* fn = prototype.GeneratePrototype(&builder, nullptr);
builder.CreateRetVoid();
fn = codegen->FinalizeFunction(fn);
- ASSERT_TRUE(fn != NULL);
+ ASSERT_TRUE(fn != nullptr);
*func = fn;
}
-void LlvmCodeGenCacheTest::AddLlvmCodegenDouble(LlvmCodeGen* codegen) {
- ASSERT_TRUE(codegen != NULL);
+void LlvmCodeGenCacheTest::AddLlvmCodegenDouble(
+ LlvmCodeGen* codegen, string* module_id = nullptr, bool doFinalizeModule = true) {
+ ASSERT_TRUE(codegen != nullptr);
LlvmCodeGen::FnPrototype prototype(codegen, "Double", codegen->i32_type());
prototype.AddArgument(LlvmCodeGen::NamedVariable("n", codegen->i32_type()));
LlvmBuilder builder(codegen->context());
@@ -180,59 +248,69 @@ void LlvmCodeGenCacheTest::AddLlvmCodegenDouble(LlvmCodeGen* codegen) {
args[0] = builder.CreateMul(args[0], mul);
builder.CreateRet(args[0]);
fn = codegen->FinalizeFunction(fn);
- ASSERT_TRUE(fn != NULL);
+ ASSERT_TRUE(fn != nullptr);
CodegenFnPtr<TestDouble> jitted_fn;
AddFunctionToJit(codegen, fn, &jitted_fn);
- ASSERT_OK(FinalizeModule(codegen));
- ASSERT_TRUE(jitted_fn.load() != nullptr);
- TestEcho test_fn = jitted_fn.load();
- ASSERT_EQ(test_fn(1), 2);
+ if (doFinalizeModule) {
+ ASSERT_OK(FinalizeModule(codegen, module_id));
+ ASSERT_TRUE(jitted_fn.load() != nullptr);
+ TestEcho test_fn = jitted_fn.load();
+ ASSERT_EQ(test_fn(1), 2);
+ }
}
/// Test the basic function of a codegen cache.
void LlvmCodeGenCacheTest::TestBasicFunction(TCodeGenCacheMode::type mode) {
- int64_t codegen_cache_capacity = 256 * 1024; // 256KB
- codegen_cache_.reset(new CodeGenCache(metrics_.get()));
bool is_normal_mode = !CodeGenCacheModeAnalyzer::is_optimal(mode);
- EXPECT_OK(codegen_cache_->Init(codegen_cache_capacity));
// Create a LlvmCodeGen containing a codegen function Echo.
- scoped_ptr<LlvmCodeGen> codegen;
- ASSERT_OK(LlvmCodeGen::CreateImpalaCodegen(fragment_state_, NULL, "test", &codegen));
- AddLlvmCodegenEcho(codegen.get());
+ scoped_ptr<LlvmCodeGen> codegen_echo;
+ ASSERT_OK(
+ LlvmCodeGen::CreateImpalaCodegen(fragment_state_, nullptr, "test", &codegen_echo));
+ string module_id_echo;
+ codegen_echo->engine_cache_ = CreateObjCache(false /*is_double*/, &module_id_echo);
// Create a LlvmCodeGen containing a codegen function Double.
scoped_ptr<LlvmCodeGen> codegen_double;
ASSERT_OK(LlvmCodeGen::CreateImpalaCodegen(
- fragment_state_, NULL, "test_double", &codegen_double));
- AddLlvmCodegenDouble(codegen_double.get());
+ fragment_state_, nullptr, "test_double", &codegen_double));
+ string module_id_double;
+ codegen_double->engine_cache_ = CreateObjCache(true /*is_double*/, &module_id_double);
+ CheckObjCacheExists(codegen_echo.get());
+ CheckObjCacheExists(codegen_double.get());
CodeGenCacheKey cache_key;
CodeGenCacheEntry entry;
string key = "key";
CodeGenCacheKeyConstructor::construct(key, &cache_key);
- int64_t mem_charge = GetMemCharge(codegen.get(), cache_key.data(), is_normal_mode);
+ int64_t mem_charge_echo =
+ GetMemCharge(codegen_echo.get(), cache_key.data(), is_normal_mode);
int64_t mem_charge_double =
GetMemCharge(codegen_double.get(), cache_key.data(), is_normal_mode);
+ scoped_ptr<MetricGroup> metrics;
+ metrics.reset(new MetricGroup("codegen-cache-test-basic"));
+ codegen_cache_.reset(new CodeGenCache(metrics.get()));
+ EXPECT_OK(codegen_cache_->Init(CODEGEN_CACHE_CAPACITY));
// Store and lookup the entry by the key.
- EXPECT_OK(codegen_cache_->Store(cache_key, codegen.get(), mode, opt_level_));
+ EXPECT_OK(codegen_cache_->Store(cache_key, codegen_echo.get(), mode, opt_level_));
CheckInUseMetrics(
- codegen_cache_.get(), 1 /*num_entry_in_use*/, mem_charge /*bytes_in_use*/);
- EXPECT_OK(codegen_cache_->Lookup(cache_key, mode, &entry, &exec_engine_wrapper_));
- CheckResult(entry);
- codegen->Close();
+ codegen_cache_.get(), 1 /*num_entry_in_use*/, mem_charge_echo /*bytes_in_use*/);
+ EXPECT_OK(codegen_cache_->Lookup(cache_key, mode, &entry, &codegen_obj_cache_));
+ CheckResult(entry, module_id_echo);
+ codegen_echo->Close();
// Close the LlvmCodeGen, but should not affect the stored cache.
- EXPECT_OK(codegen_cache_->Lookup(cache_key, mode, &entry, &exec_engine_wrapper_));
- CheckResult(entry);
+ EXPECT_OK(codegen_cache_->Lookup(cache_key, mode, &entry, &codegen_obj_cache_));
+ CheckResult(entry, module_id_echo);
// Override the entry with a different function, should be able to find the new
// function from the new entry.
EXPECT_OK(codegen_cache_->Store(cache_key, codegen_double.get(), mode, opt_level_));
CheckInUseMetrics(
codegen_cache_.get(), 1 /*num_entry_in_use*/, mem_charge_double /*bytes_in_use*/);
- EXPECT_OK(codegen_cache_->Lookup(cache_key, mode, &entry, &exec_engine_wrapper_));
- CheckResult(entry, true /*is_double*/);
+ EXPECT_OK(codegen_cache_->Lookup(cache_key, mode, &entry, &codegen_obj_cache_));
+ CheckResult(entry, module_id_double, true /*is_double*/);
EXPECT_EQ(codegen_cache_->codegen_cache_entries_evicted_->GetValue(), 1);
codegen_double->Close();
+ codegen_cache_.reset();
}
void LlvmCodeGenCacheTest::CheckMetrics(
@@ -254,44 +332,47 @@ void LlvmCodeGenCacheTest::CheckEvictMetrics(CodeGenCache* codegen_cache, int ev
EXPECT_EQ(codegen_cache->codegen_cache_entries_evicted_->GetValue(), evict);
}
+void LlvmCodeGenCacheTest::CheckObjCacheExists(LlvmCodeGen* codegen) {
+ // If a cache is written, the size of it should be larger than size of the struct.
+ EXPECT_GT(codegen->engine_cache()->objSize(), sizeof(CodeGenObjectCache));
+}
+
int64_t LlvmCodeGenCacheTest::GetMemCharge(
LlvmCodeGen* codegen, string key_str, bool is_normal_mode) {
if (is_normal_mode) {
- return codegen->memory_manager_->bytes_allocated() + key_str.size()
+ return codegen->engine_cache()->objSize() + key_str.size()
+ sizeof(CodeGenCacheEntry);
}
// Optimal mode would use hash code and length as the key.
- return codegen->memory_manager_->bytes_allocated() + CodeGenCacheKey::OptimalKeySize
+ return codegen->engine_cache()->objSize() + CodeGenCacheKey::OptimalKeySize
+ sizeof(CodeGenCacheEntry);
}
/// Test the situation that the codegen cache hits the limit of capacity, in this case,
/// eviction is needed when new insertion comes.
void LlvmCodeGenCacheTest::TestAtCapacity(TCodeGenCacheMode::type mode) {
- int64_t codegen_cache_capacity = 196; // 196B
+ // Allocates memory for small examples.
+ int64_t codegen_cache_capacity = ENGINE_CACHE_SIZE + sizeof(CodeGenObjectCache)
+ + sizeof(CodeGenCacheEntry) + CodeGenCacheKey::OptimalKeySize + sizeof(std::string);
bool is_normal_mode = !CodeGenCacheModeAnalyzer::is_optimal(mode);
- // 150B for optimal mode, or 164B on ARM systems
- if (!is_normal_mode) {
-#ifdef __aarch64__
- codegen_cache_capacity = 164;
-#else
- codegen_cache_capacity = 150;
-#endif
- }
- // Using single shard makes the logic of scenarios simple for capacity and
- // eviction-related behavior.
- FLAGS_cache_force_single_shard = true;
// Create two LlvmCodeGen objects containing a different codegen function separately.
scoped_ptr<LlvmCodeGen> codegen;
- ASSERT_OK(LlvmCodeGen::CreateImpalaCodegen(fragment_state_, NULL, "test", &codegen));
+
+ test_env_->ResetCodegenCache(metrics_.get());
+ CodeGenCache* cache = test_env_->codegen_cache();
+ EXPECT_OK(cache->Init(codegen_cache_capacity));
+
+ ASSERT_OK(LlvmCodeGen::CreateImpalaCodegen(fragment_state_, nullptr, "test", &codegen));
AddLlvmCodegenEcho(codegen.get());
codegen->GenerateFunctionNamesHashCode();
scoped_ptr<LlvmCodeGen> codegen_double;
ASSERT_OK(LlvmCodeGen::CreateImpalaCodegen(
- fragment_state_, NULL, "test_double", &codegen_double));
+ fragment_state_, nullptr, "test_double", &codegen_double));
AddLlvmCodegenDouble(codegen_double.get());
codegen_double->GenerateFunctionNamesHashCode();
+ CheckObjCacheExists(codegen.get());
+ CheckObjCacheExists(codegen_double.get());
CodeGenCacheKey cache_key_1;
CodeGenCacheKey cache_key_2;
@@ -307,10 +388,15 @@ void LlvmCodeGenCacheTest::TestAtCapacity(TCodeGenCacheMode::type mode) {
ASSERT_LE(mem_charge_2, codegen_cache_capacity);
ASSERT_GE(mem_charge_1 + mem_charge_2, codegen_cache_capacity);
- test_env_->ResetCodegenCache(metrics_.get());
- CodeGenCache* cache = test_env_->codegen_cache();
+ scoped_ptr<MetricGroup> metrics;
+ metrics.reset(new MetricGroup("codegen-test-capacity"));
+ test_env_->ResetCodegenCache(metrics.get());
+ cache = test_env_->codegen_cache();
EXPECT_OK(cache->Init(codegen_cache_capacity));
+ // Expect init metrics.
+ CheckMetrics(cache, 0 /*hit*/, 0 /*miss*/, 0 /*evict*/);
+
// Store key_1 and lookup.
EXPECT_OK(codegen->StoreCache(cache_key_1));
EXPECT_TRUE(codegen->LookupCache(cache_key_1));
@@ -340,6 +426,7 @@ void LlvmCodeGenCacheTest::TestAtCapacity(TCodeGenCacheMode::type mode) {
codegen->Close();
codegen_double->Close();
+ test_env_->ResetCodegenCache();
}
/// Test the case if we have a cache but doesn't contain the function
@@ -347,17 +434,17 @@ void LlvmCodeGenCacheTest::TestAtCapacity(TCodeGenCacheMode::type mode) {
void LlvmCodeGenCacheTest::TestSkipCache() {
// Initial a LlvmCodeGen object with a normal function.
scoped_ptr<LlvmCodeGen> codegen;
- ASSERT_OK(LlvmCodeGen::CreateImpalaCodegen(fragment_state_, NULL, "test", &codegen));
+ ASSERT_OK(LlvmCodeGen::CreateImpalaCodegen(fragment_state_, nullptr, "test", &codegen));
AddLlvmCodegenEcho(codegen.get());
// Create an empty function from other LlvmCodeGen to create the failure later.
scoped_ptr<LlvmCodeGen> codegen_empty;
ASSERT_OK(LlvmCodeGen::CreateImpalaCodegen(
- fragment_state_, NULL, "test_empty", &codegen_empty));
+ fragment_state_, nullptr, "test_empty", &codegen_empty));
llvm::Function* empty_func;
GetLlvmEmptyFunction(codegen_empty.get(), &empty_func);
test_env_->ResetCodegenCache(metrics_.get());
- EXPECT_OK(test_env_->codegen_cache()->Init(256 * 1024 /*capacity*/));
+ EXPECT_OK(test_env_->codegen_cache()->Init(CODEGEN_CACHE_CAPACITY));
CheckMetrics(test_env_->codegen_cache(), 0 /*hit*/, 0 /*miss*/, 0 /*evict*/);
CodeGenCacheKey cache_key;
@@ -421,12 +508,10 @@ TEST_F(LlvmCodeGenCacheTest, ModeAnalyzer) {
EXPECT_FALSE(CodeGenCacheModeAnalyzer::is_optimal(TCodeGenCacheMode::NORMAL_DEBUG));
}
-// Check the number of execution engine wrappers stored in the cache. Because the shared
-// pointer of the execution engine wrapper needs to be stored in the codegen cache while
-// the entry using the execution engine is stored in the cache.
-void LlvmCodeGenCacheTest::CheckEngineCount(LlvmCodeGen* codegen, int expect_count) {
+// Check the number of engine caches stored in the global cache.
+void LlvmCodeGenCacheTest::CheckEngineCacheCount(LlvmCodeGen* codegen, int expect_count) {
lock_guard<mutex> lock(codegen_cache_->cached_engines_lock_);
- auto engine_it = codegen_cache_->cached_engines_.find(codegen->execution_engine());
+ auto engine_it = codegen_cache_->cached_engines_.find(codegen->engine_cache());
EXPECT_TRUE(engine_it != codegen_cache_->cached_engines_.end());
EXPECT_EQ(codegen_cache_->cached_engines_.size(), expect_count);
}
@@ -441,13 +526,13 @@ bool LlvmCodeGenCacheTest::CheckKeyExist(TCodeGenCacheMode::type mode, string ke
CodeGenCacheKey cache_key;
CodeGenCacheEntry entry;
CodeGenCacheKeyConstructor::construct(key, &cache_key);
- EXPECT_OK(codegen_cache_->Lookup(cache_key, mode, &entry, &exec_engine_wrapper_));
+ EXPECT_OK(codegen_cache_->Lookup(cache_key, mode, &entry, &codegen_obj_cache_));
return !entry.Empty();
}
// Return true if the provided engine exists.
-bool LlvmCodeGenCacheTest::CheckEngineExist(llvm::ExecutionEngine* engine) {
- auto engine_it = codegen_cache_->cached_engines_.find(engine);
+bool LlvmCodeGenCacheTest::CheckEngineExist(CodeGenObjectCache* cached_engine) {
+ auto engine_it = codegen_cache_->cached_engines_.find(cached_engine);
return engine_it != codegen_cache_->cached_engines_.end();
}
@@ -462,11 +547,14 @@ void LlvmCodeGenCacheTest::ExpectNumEngineSameAsEntry() {
/// and mode.
void LlvmCodeGenCacheTest::TestSwitchModeHelper(TCodeGenCacheMode::type mode, string key,
int expect_entry_num = -1, int expect_engine_num = -1,
- llvm::ExecutionEngine** engine = nullptr) {
+ CodeGenObjectCache** engine_cache = nullptr) {
// Create a LlvmCodeGen containing a codegen function Echo.
scoped_ptr<LlvmCodeGen> codegen;
- ASSERT_OK(LlvmCodeGen::CreateImpalaCodegen(fragment_state_, NULL, "test", &codegen));
- AddLlvmCodegenEcho(codegen.get());
+ ASSERT_OK(LlvmCodeGen::CreateImpalaCodegen(fragment_state_, nullptr, "test", &codegen));
+ string module_id_echo;
+ codegen->engine_cache_ = CreateObjCache(false /*is_double*/, &module_id_echo);
+ ASSERT_TRUE(codegen->engine_cache_ != nullptr);
+ CheckObjCacheExists(codegen.get());
CodeGenCacheKey cache_key;
CodeGenCacheEntry entry;
@@ -478,17 +566,20 @@ void LlvmCodeGenCacheTest::TestSwitchModeHelper(TCodeGenCacheMode::type mode, st
CheckInUseMetrics(codegen_cache_.get(), expect_entry_num /*num_entry_in_use*/);
}
if (expect_engine_num != -1) {
- CheckEngineCount(codegen.get(), expect_engine_num);
+ CheckEngineCacheCount(codegen.get(), expect_engine_num);
}
- EXPECT_OK(codegen_cache_->Lookup(cache_key, mode, &entry, &exec_engine_wrapper_));
- CheckResult(entry);
- if (engine) *engine = codegen->execution_engine();
+ EXPECT_OK(codegen_cache_->Lookup(cache_key, mode, &entry, &codegen_obj_cache_));
+ CheckResult(entry, module_id_echo);
+ if (engine_cache) *engine_cache = codegen->engine_cache();
codegen->Close();
}
// Test to switch among different modes.
TEST_F(LlvmCodeGenCacheTest, SwitchMode) {
- int64_t codegen_cache_capacity = 512; // 512B
+ // Storage for 2 entries.
+ int64_t codegen_cache_capacity = 2
+ * (ENGINE_CACHE_SIZE + sizeof(CodeGenCacheEntry) + CodeGenCacheKey::OptimalKeySize
+ + sizeof(std::string));
codegen_cache_.reset(new CodeGenCache(metrics_.get()));
EXPECT_OK(codegen_cache_->Init(codegen_cache_capacity));
string key = "key";
@@ -503,7 +594,7 @@ TEST_F(LlvmCodeGenCacheTest, SwitchMode) {
TestSwitchModeHelper(TCodeGenCacheMode::NORMAL_DEBUG, key, 2, 2);
// Try again, the new insertion should replace the old ones, so the entry number and
// engine number won't change.
- llvm::ExecutionEngine *engine_opt, *engine_opt_dbg, *engine_normal, *engine_normal_dbg;
+ CodeGenObjectCache *engine_opt, *engine_opt_dbg, *engine_normal, *engine_normal_dbg;
TestSwitchModeHelper(TCodeGenCacheMode::OPTIMAL, key, 2, 2, &engine_opt);
TestSwitchModeHelper(TCodeGenCacheMode::OPTIMAL_DEBUG, key, 2, 2, &engine_opt_dbg);
// Expect the engines with the same key should be different, because the engine is
@@ -542,12 +633,15 @@ TEST_F(LlvmCodeGenCacheTest, SwitchMode) {
void LlvmCodeGenCacheTest::StoreHelper(TCodeGenCacheMode::type mode, string key) {
// Create a LlvmCodeGen containing a codegen function Echo.
scoped_ptr<LlvmCodeGen> codegen;
- ASSERT_OK(LlvmCodeGen::CreateImpalaCodegen(fragment_state_, NULL, "test", &codegen));
- AddLlvmCodegenEcho(codegen.get());
+ ASSERT_OK(LlvmCodeGen::CreateImpalaCodegen(fragment_state_, nullptr, "test", &codegen));
+ shared_ptr<CodeGenObjectCache> engine_cache =
+ CreateObjCache(true /*is_double*/, nullptr /*module_id*/);
+ codegen->engine_cache_ = engine_cache;
CodeGenCacheKey cache_key;
CodeGenCacheEntry entry;
CodeGenCacheKeyConstructor::construct(key, &cache_key);
EXPECT_OK(codegen_cache_->Store(cache_key, codegen.get(), mode, opt_level_));
+ CheckObjCacheExists(codegen.get());
codegen->Close();
}
@@ -579,7 +673,10 @@ void LlvmCodeGenCacheTest::TestConcurrentStore(int num_threads) {
}
TEST_F(LlvmCodeGenCacheTest, ConcurrentStore) {
- int64_t codegen_cache_capacity = 512; // 512B
+ // Storage for 2 entries.
+ int64_t codegen_cache_capacity = 2
+ * (ENGINE_CACHE_SIZE + sizeof(CodeGenCacheEntry) + CodeGenCacheKey::OptimalKeySize
+ + sizeof(std::string));
codegen_cache_.reset(new CodeGenCache(metrics_.get()));
EXPECT_OK(codegen_cache_->Init(codegen_cache_capacity));
TestConcurrentStore(8);
diff --git a/be/src/codegen/llvm-codegen-cache.cc b/be/src/codegen/llvm-codegen-cache.cc
index d2cf81414..4e089a8e5 100644
--- a/be/src/codegen/llvm-codegen-cache.cc
+++ b/be/src/codegen/llvm-codegen-cache.cc
@@ -41,7 +41,7 @@ void CodeGenCache::EvictionCallback::EvictedEntry(Slice key, Slice value) {
const CodeGenCacheEntry* cache_entry =
reinterpret_cast<const CodeGenCacheEntry*>(value.data());
DCHECK(cache_entry != nullptr);
- cache_->RemoveEngine(cache_entry->engine_pointer);
+ cache_->RemoveEngine(cache_entry->cached_engine_pointer);
LOG(INFO) << "Evict CodeGen Cache, key hash_code=" << CodeGenCacheKey::hash_code(key);
// For statistics.
if (codegen_cache_entries_evicted_ != nullptr) {
@@ -109,18 +109,20 @@ void CodeGenCache::ReleaseResources() {
Status CodeGenCache::Lookup(const CodeGenCacheKey& cache_key,
const TCodeGenCacheMode::type& mode, CodeGenCacheEntry* entry,
- shared_ptr<LlvmExecutionEngineWrapper>* execution_engine) {
+ shared_ptr<CodeGenObjectCache>* cached_engine) {
DCHECK(!is_closed_);
DCHECK(cache_ != nullptr);
DCHECK(entry != nullptr);
- DCHECK(execution_engine != nullptr);
+ DCHECK(cached_engine != nullptr);
// Use hash code and the total length as the key for optimal mode, because the whole
// key could be very large, using optimal mode could improve the performance and save
// memory consumption. However, it could lead to a collision, even though the chance
// is very small, in that case, we may switch to normal mode for the query or disable
// the codegen cache.
- Slice key = cache_key.optimal_key_slice();
- if (!CodeGenCacheModeAnalyzer::is_optimal(mode)) {
+ Slice key;
+ if (CodeGenCacheModeAnalyzer::is_optimal(mode)) {
+ key = cache_key.optimal_key_slice();
+ } else {
key = cache_key.data_slice();
}
Cache::UniqueHandle handle(cache_->Lookup(key));
@@ -131,8 +133,8 @@ Status CodeGenCache::Lookup(const CodeGenCacheKey& cache_key,
// because the shared pointer could be deleted in the eviction process.
// If can't find it, treat it as cache missing, because the engine is needed
// to look for jitted functions.
- if (LookupEngine(cached_entry->engine_pointer, execution_engine)) {
- entry->Reset(cached_entry->engine_pointer, cached_entry->num_functions,
+ if (LookupEngine(cached_entry->cached_engine_pointer, cached_engine)) {
+ entry->Reset(cached_entry->cached_engine_pointer, cached_entry->num_functions,
cached_entry->num_instructions, cached_entry->num_opt_functions,
cached_entry->num_opt_instructions, cached_entry->function_names_hashcode,
cached_entry->total_bytes_charge, cached_entry->opt_level);
@@ -148,12 +150,14 @@ Status CodeGenCache::StoreInternal(const CodeGenCacheKey& cache_key,
TCodeGenOptLevel::type opt_level) {
// In normal mode, we will store the whole key content to the cache.
// Otherwise, in optimal mode, we will only store the hash code and length of the key.
- Slice key = cache_key.optimal_key_slice();
- if (!CodeGenCacheModeAnalyzer::is_optimal(mode)) {
+ Slice key;
+ if (CodeGenCacheModeAnalyzer::is_optimal(mode)) {
+ key = cache_key.optimal_key_slice();
+ } else {
key = cache_key.data_slice();
}
// Memory charge includes both key and entry size.
- int64_t mem_charge = codegen->memory_manager_->bytes_allocated() + key.size()
+ int64_t mem_charge = codegen->engine_cache()->objSize() + key.size()
+ sizeof(CodeGenCacheEntry) + FLAGS_codegen_cache_entry_bytes_charge_overhead;
Cache::UniquePendingHandle pending_handle(
cache_->Allocate(key, sizeof(CodeGenCacheEntry), mem_charge));
@@ -164,7 +168,7 @@ Status CodeGenCache::StoreInternal(const CodeGenCacheKey& cache_key,
}
CodeGenCacheEntry* cache_entry =
reinterpret_cast<CodeGenCacheEntry*>(cache_->MutableValue(&pending_handle));
- cache_entry->Reset(codegen->execution_engine(), codegen->num_functions_->value(),
+ cache_entry->Reset(codegen->engine_cache(), codegen->num_functions_->value(),
codegen->num_instructions_->value(), codegen->num_opt_functions_->value(),
codegen->num_opt_instructions_->value(), codegen->function_names_hashcode_,
mem_charge, opt_level);
@@ -173,7 +177,7 @@ Status CodeGenCache::StoreInternal(const CodeGenCacheKey& cache_key,
Cache::UniqueHandle cache_handle =
cache_->Insert(move(pending_handle), evict_callback_.get());
if (cache_handle == nullptr) {
- RemoveEngine(codegen->execution_engine());
+ RemoveEngine(codegen->engine_cache());
return Status(Substitute("Couldn't insert codegen cache entry,"
" hash code:'$0', size: '$1'",
cache_key.hash_code().str(), mem_charge));
@@ -209,7 +213,6 @@ Status CodeGenCache::Store(const CodeGenCacheKey& cache_key, LlvmCodeGen* codege
// If hash code exists, return an okay immediately.
if (!key_to_insert_it.second) return status;
}
- // Do store the cache entry to the cache.
status = StoreInternal(cache_key, codegen, mode, opt_level);
// Remove the hash code of the key from the to_insert_keys set.
lock_guard<mutex> lock(to_insert_set_lock_);
@@ -220,25 +223,25 @@ Status CodeGenCache::Store(const CodeGenCacheKey& cache_key, LlvmCodeGen* codege
void CodeGenCache::StoreEngine(LlvmCodeGen* codegen) {
DCHECK(codegen != nullptr);
lock_guard<mutex> lock(cached_engines_lock_);
- cached_engines_.emplace(codegen->execution_engine(),
- codegen->execution_engine_wrapper_);
+ cached_engines_.emplace(codegen->engine_cache_.get(), codegen->engine_cache_);
}
-bool CodeGenCache::LookupEngine(const llvm::ExecutionEngine* engine,
- shared_ptr<LlvmExecutionEngineWrapper>* execution_engine_wrapper) {
- DCHECK(engine != nullptr);
- DCHECK(execution_engine_wrapper != nullptr);
+bool CodeGenCache::LookupEngine(const CodeGenObjectCache* cached_engine_raw_ptr,
+ shared_ptr<CodeGenObjectCache>* cached_engine) {
+ DCHECK(cached_engine_raw_ptr != nullptr);
lock_guard<mutex> lock(cached_engines_lock_);
- auto engine_it = cached_engines_.find(engine);
+ auto engine_it = cached_engines_.find(cached_engine_raw_ptr);
if (engine_it == cached_engines_.end()) return false;
- *execution_engine_wrapper = engine_it->second;
+ if (cached_engine != nullptr) {
+ *cached_engine = engine_it->second;
+ }
return true;
}
-void CodeGenCache::RemoveEngine(const llvm::ExecutionEngine* engine) {
- DCHECK(engine != nullptr);
+void CodeGenCache::RemoveEngine(const CodeGenObjectCache* cached_engine_raw_ptr) {
+ DCHECK(cached_engine_raw_ptr != nullptr);
lock_guard<mutex> lock(cached_engines_lock_);
- cached_engines_.erase(engine);
+ cached_engines_.erase(cached_engine_raw_ptr);
}
} // namespace impala
diff --git a/be/src/codegen/llvm-codegen-cache.h b/be/src/codegen/llvm-codegen-cache.h
index 6c58dd77f..d2e530390 100644
--- a/be/src/codegen/llvm-codegen-cache.h
+++ b/be/src/codegen/llvm-codegen-cache.h
@@ -161,14 +161,14 @@ class CodeGenCacheModeAnalyzer {
};
struct CodeGenCacheEntry {
- CodeGenCacheEntry() : engine_pointer(nullptr) {}
+ CodeGenCacheEntry() : cached_engine_pointer(nullptr) {}
// When Empty, no guarantees are made about the content of other fields.
- bool Empty() { return engine_pointer == nullptr; }
- void Reset() { engine_pointer = nullptr; }
- void Reset(llvm::ExecutionEngine* engine_ptr, int64_t num_funcs, int64_t num_instrucs,
- int64_t num_opt_funcs, int64_t num_opt_instrucs, uint64_t names_hashcode,
- int64_t charge, TCodeGenOptLevel::type opt) {
- engine_pointer = engine_ptr;
+ bool Empty() { return cached_engine_pointer == nullptr; }
+ void Reset() { cached_engine_pointer = nullptr; }
+ void Reset(CodeGenObjectCache* cached_engine_ptr, int64_t num_funcs,
+ int64_t num_instrucs, int64_t num_opt_funcs, int64_t num_opt_instrucs,
+ uint64_t names_hashcode, int64_t charge, TCodeGenOptLevel::type opt) {
+ cached_engine_pointer = cached_engine_ptr;
num_functions = num_funcs;
num_instructions = num_instrucs;
num_opt_functions = num_opt_funcs;
@@ -177,7 +177,7 @@ struct CodeGenCacheEntry {
total_bytes_charge = charge;
opt_level = opt;
}
- llvm::ExecutionEngine* engine_pointer;
+ CodeGenObjectCache* cached_engine_pointer;
/// Number of functions before optimization.
int64_t num_functions;
/// Number of instructions before optimization.
@@ -215,25 +215,25 @@ class CodeGenCache {
/// entry will be reset to empty.
/// Return Status::Okay unless there is any internal error to throw.
Status Lookup(const CodeGenCacheKey& key, const TCodeGenCacheMode::type& mode,
- CodeGenCacheEntry* entry,
- std::shared_ptr<LlvmExecutionEngineWrapper>* execution_engine);
+ CodeGenCacheEntry* entry, std::shared_ptr<CodeGenObjectCache>* cached_engine);
+ bool Lookup(const CodeGenCacheKey& key, const TCodeGenCacheMode::type& mode);
/// Store the cache entry with the specific cache key.
Status Store(const CodeGenCacheKey& key, LlvmCodeGen* codegen,
TCodeGenCacheMode::type mode, TCodeGenOptLevel::type opt_level);
- /// Store the shared pointer of llvm execution engine to the cache to keep all the
- /// jitted functions in that engine alive.
+ /// Store the shared pointer of CodeGenObjectCache to keep the compiled module cache
+ /// alive.
void StoreEngine(LlvmCodeGen* codegen);
- /// Look up the shared pointer of the LlvmExecutionEngineWrapper by the raw pointer of
- /// its execution engine field. If found, return true. Ohterwise, return false.
- bool LookupEngine(const llvm::ExecutionEngine* engine,
- std::shared_ptr<LlvmExecutionEngineWrapper>* execution_engine_wrapper);
+ /// Look up the shared pointer of the CodeGenObjectCache by its raw pointer.
+ /// If found, return true. Ohterwise, return false.
+ bool LookupEngine(const CodeGenObjectCache* cached_engine_raw_ptr,
+ std::shared_ptr<CodeGenObjectCache>* cached_engine = nullptr);
- /// Remove the shared pointer of llvm execution engine from the cache by its raw
- /// pointer address.
- void RemoveEngine(const llvm::ExecutionEngine* engine);
+ /// Remove the shared pointer of CodeGenObjectCache from the cache by its raw pointer
+ /// address.
+ void RemoveEngine(const CodeGenObjectCache* cached_engine_raw_ptr);
/// Increment a hit count or miss count.
void IncHitOrMissCount(bool hit) {
@@ -302,10 +302,10 @@ class CodeGenCache {
/// Protects to the map of cached engines.
std::mutex cached_engines_lock_;
- /// Stores shared pointers to LlvmExecutionEngineWrappers to keep the jitted functions
+ /// Stores shared pointers to CodeGenObjectCaches to keep the jitted functions
/// alive. The shared pointer entries could be removed when the cache entry is evicted
/// or when the whole cache is destructed.
- std::unordered_map<const llvm::ExecutionEngine*,
- std::shared_ptr<LlvmExecutionEngineWrapper>> cached_engines_;
+ std::unordered_map<const CodeGenObjectCache*, std::shared_ptr<CodeGenObjectCache>>
+ cached_engines_;
};
} // namespace impala
diff --git a/be/src/codegen/llvm-codegen-object-cache.cc b/be/src/codegen/llvm-codegen-object-cache.cc
new file mode 100644
index 000000000..ca211d75c
--- /dev/null
+++ b/be/src/codegen/llvm-codegen-object-cache.cc
@@ -0,0 +1,81 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "codegen/llvm-codegen-object-cache.h"
+
+#include <llvm/IR/Module.h>
+
+using std::string;
+using std::stringstream;
+
+namespace impala {
+
+void CodeGenObjectCache::notifyObjectCompiled(
+ const llvm::Module* M, llvm::MemoryBufferRef ObjBuffer) {
+ DCHECK(M != nullptr);
+ // Save the compiled object in the memory cache, with the module ID as the key.
+ // In our scenario, we anticipate having only one module per execution engine.
+ // Consequently, each CodeGenObjectCache should hold just one module.
+ // If a module with a different ID is encountered, a warning is logged for
+ // further investigation.
+ const string& module_id = M->getModuleIdentifier();
+ std::lock_guard<std::mutex> l(mutex_);
+ if (key_.empty()) {
+ LOG(INFO) << "Insert module id " << module_id << " in CodeGenObjectCache " << this
+ << " with size " << ObjBuffer.getBuffer().size() << " bytes";
+ key_ = module_id;
+ // The Buffer Id doesn't seem to serve a purpose in our case, and skipping it
+ // can conserve memory.
+ cached_obj_ =
+ llvm::MemoryBuffer::getMemBufferCopy(ObjBuffer.getBuffer(), "" /*Buffer Id*/);
+ } else if (key_ != module_id) {
+ stringstream ss;
+ ss << "Inserting a different module in CodeGenObjectCache " << this
+ << ". Old module id is " << module_id << ", and new module id is " << key_;
+ DCHECK(false) << ss.str();
+ LOG(WARNING) << ss.str();
+ }
+}
+
+std::unique_ptr<llvm::MemoryBuffer> CodeGenObjectCache::getObject(const llvm::Module* M) {
+ DCHECK(M != nullptr);
+ // Return a copy of the cached object containing the compiled module with the specified
+ // ID. If the ID doesn't match the one in the cached object, a warning is logged for
+ // further investigation.
+ const string& module_id = M->getModuleIdentifier();
+ std::lock_guard<std::mutex> l(mutex_);
+ if (module_id == key_) {
+ LOG(INFO) << "Object for module id " << module_id << " loaded from cache with size "
+ << cached_obj_->getMemBufferRef().getBufferSize() << " bytes";
+ return llvm::MemoryBuffer::getMemBufferCopy(
+ cached_obj_->getMemBufferRef().getBuffer());
+ } else if (!key_.empty()) {
+ stringstream ss;
+ ss << "Object for module id " << module_id << " can't be loaded from cache."
+ << " Existing module id is " << key_;
+ DCHECK(false) << ss.str();
+ LOG(WARNING) << ss.str();
+ }
+ return nullptr;
+}
+
+size_t CodeGenObjectCache::objSize() {
+ size_t sz = sizeof(CodeGenObjectCache);
+ if (cached_obj_ == nullptr) return sz;
+ return cached_obj_->getMemBufferRef().getBufferSize() + sz;
+}
+} // namespace impala
diff --git a/be/src/codegen/llvm-codegen-object-cache.h b/be/src/codegen/llvm-codegen-object-cache.h
new file mode 100644
index 000000000..c9ba2c6d0
--- /dev/null
+++ b/be/src/codegen/llvm-codegen-object-cache.h
@@ -0,0 +1,43 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include <memory>
+#include <mutex>
+
+#include <llvm/ExecutionEngine/ObjectCache.h>
+#include <llvm/Support/MemoryBuffer.h>
+
+#include "common/logging.h"
+
+namespace impala {
+
+class CodeGenObjectCache : public llvm::ObjectCache {
+ public:
+ CodeGenObjectCache() {}
+ virtual ~CodeGenObjectCache() {}
+ virtual void notifyObjectCompiled(const llvm::Module* M, llvm::MemoryBufferRef Obj);
+ virtual std::unique_ptr<llvm::MemoryBuffer> getObject(const llvm::Module* M);
+ size_t objSize();
+
+ private:
+ std::mutex mutex_;
+ std::string key_;
+ std::unique_ptr<llvm::MemoryBuffer> cached_obj_;
+};
+} // namespace impala
diff --git a/be/src/codegen/llvm-codegen.cc b/be/src/codegen/llvm-codegen.cc
index 424142527..b1d55f1bb 100644
--- a/be/src/codegen/llvm-codegen.cc
+++ b/be/src/codegen/llvm-codegen.cc
@@ -130,7 +130,6 @@ DEFINE_string_hidden(llvm_ir_opt, "Os",
DECLARE_bool(enable_legacy_avx_support);
namespace impala {
-
const string LlvmCodeGen::ASYNC_CODEGEN_THREAD_COUNTERS_PREFIX = "CodegenCompileThread";
bool LlvmCodeGen::llvm_initialized_ = false;
string LlvmCodeGen::cpu_name_;
@@ -477,9 +476,8 @@ Status LlvmCodeGen::Init(unique_ptr<llvm::Module> module) {
builder.setMAttrs(cpu_attrs_);
builder.setErrorStr(&error_string_);
- unique_ptr<llvm::ExecutionEngine> execution_engine =
- unique_ptr<llvm::ExecutionEngine>(builder.create());
- if (execution_engine == nullptr) {
+ execution_engine_ = unique_ptr<llvm::ExecutionEngine>(builder.create());
+ if (execution_engine_ == nullptr) {
module_ = nullptr; // module_ was owned by builder.
stringstream ss;
ss << "Could not create ExecutionEngine: " << error_string_;
@@ -487,18 +485,15 @@ Status LlvmCodeGen::Init(unique_ptr<llvm::Module> module) {
}
// The module data layout must match the one selected by the execution engine.
- module_->setDataLayout(execution_engine->getDataLayout());
+ module_->setDataLayout(execution_engine_->getDataLayout());
void_type_ = llvm::Type::getVoidTy(context());
ptr_type_ = llvm::PointerType::get(i8_type(), 0);
true_value_ = llvm::ConstantInt::get(context(), llvm::APInt(1, true, true));
false_value_ = llvm::ConstantInt::get(context(), llvm::APInt(1, false, true));
- unique_ptr<CodegenSymbolEmitter> symbol_emitter = SetupSymbolEmitter(
- execution_engine.get());
-
- execution_engine_wrapper_ = make_shared<LlvmExecutionEngineWrapper>(
- std::move(execution_engine), std::move(symbol_emitter));
+ symbol_emitter_ = SetupSymbolEmitter(execution_engine_.get());
+ engine_cache_ = make_shared<CodeGenObjectCache>();
RETURN_IF_ERROR(LoadIntrinsics());
@@ -522,7 +517,7 @@ unique_ptr<CodegenSymbolEmitter> LlvmCodeGen::SetupSymbolEmitter(
}
LlvmCodeGen::~LlvmCodeGen() {
- DCHECK(execution_engine_wrapper_ == nullptr) << "Must Close() before destruction";
+ DCHECK(execution_engine_ == nullptr) << "Must Close() before destruction";
}
void LlvmCodeGen::Close() {
@@ -533,9 +528,10 @@ void LlvmCodeGen::Close() {
memory_manager_ = nullptr;
}
if (mem_tracker_ != nullptr) mem_tracker_->Close();
-
- execution_engine_wrapper_.reset();
- execution_engine_wrapper_cached_.reset();
+ engine_cache_.reset();
+ engine_cache_cached_.reset();
+ execution_engine_.reset();
+ symbol_emitter_.reset();
module_ = nullptr;
}
@@ -1166,8 +1162,7 @@ bool LlvmCodeGen::LookupCache(CodeGenCacheKey& cache_key) {
CodeGenCache* cache = ExecEnv::GetInstance()->codegen_cache();
DCHECK(cache != nullptr);
Status lookup_status = cache->Lookup(cache_key,
- state_->query_options().codegen_cache_mode, &entry,
- &execution_engine_wrapper_cached_);
+ state_->query_options().codegen_cache_mode, &entry, &engine_cache_cached_);
bool entry_exists = lookup_status.ok() && !entry.Empty();
LOG(INFO) << DebugCacheEntryString(cache_key, true /*is_lookup*/,
CodeGenCacheModeAnalyzer::is_debug(state_->query_options().codegen_cache_mode),
@@ -1194,11 +1189,8 @@ bool LlvmCodeGen::LookupCache(CodeGenCacheKey& cache_key) {
return false;
}
- const bool fn_pointers_set_from_cache = SetFunctionPointers(cache, &cache_key);
- if (!fn_pointers_set_from_cache) return false;
-
- // Because we cache the entire execution engine, the cached number of functions should
- // be the same as the total optimized function number.
+ // Because we cache all the compiled codegened functions, the cached number of
+ // functions should be the same as the total optimized function number.
COUNTER_SET(num_cached_functions_, entry.num_opt_functions);
COUNTER_SET(num_functions_, entry.num_functions);
COUNTER_SET(num_instructions_, entry.num_instructions);
@@ -1259,7 +1251,7 @@ void LlvmCodeGen::PruneModule() {
module_pass_manager.run(*module_, module_analysis_manager);
}
-Status LlvmCodeGen::FinalizeModule() {
+Status LlvmCodeGen::FinalizeModule(string* module_id) {
DCHECK(!is_compiled_);
is_compiled_ = true;
@@ -1309,6 +1301,7 @@ Status LlvmCodeGen::FinalizeModule() {
bool codegen_cache_enabled = state_->codegen_cache_enabled() && codegen_cache_enabled_;
CodeGenCacheKey cache_key;
+ bool cache_hit = false;
if (codegen_cache_enabled) {
string bitcode;
SCOPED_TIMER(codegen_cache_lookup_timer_);
@@ -1323,25 +1316,29 @@ Status LlvmCodeGen::FinalizeModule() {
// in the cache store process if look up failed.
GenerateFunctionNamesHashCode();
DCHECK(!cache_key.empty());
- if (LookupCache(cache_key)) return Status::OK();
+ // Set the module id for the use of ObjectCache.
+ module_->setModuleIdentifier(cache_key.hash_code().str());
+ cache_hit = LookupCache(cache_key);
}
- // Update counters before final optimization, but after removing unused functions. This
- // gives us a rough measure of how much work the optimization and compilation must do.
- // If found in cache, counters will be restored from the cache entry.
- InstructionCounter counter;
- counter.visit(*module_);
- COUNTER_SET(num_functions_, counter.GetCount(InstructionCounter::TOTAL_FUNCTIONS));
- COUNTER_SET(num_instructions_, counter.GetCount(InstructionCounter::TOTAL_INSTS));
-
- if (optimizations_enabled_ && !FLAGS_disable_optimization_passes) {
- RETURN_IF_ERROR(OptimizeModule());
- counter.ResetCount();
+ if (!cache_hit) {
+ // Update counters before final optimization, but after removing unused functions.
+ // This gives us a rough measure of how much work the optimization and compilation
+ // must do. If found in cache, counters will be restored from the cache entry.
+ InstructionCounter counter;
counter.visit(*module_);
+ COUNTER_SET(num_functions_, counter.GetCount(InstructionCounter::TOTAL_FUNCTIONS));
+ COUNTER_SET(num_instructions_, counter.GetCount(InstructionCounter::TOTAL_INSTS));
+
+ if (optimizations_enabled_ && !FLAGS_disable_optimization_passes) {
+ RETURN_IF_ERROR(OptimizeModule());
+ counter.ResetCount();
+ counter.visit(*module_);
+ }
+ COUNTER_SET(
+ num_opt_functions_, counter.GetCount(InstructionCounter::TOTAL_FUNCTIONS));
+ COUNTER_SET(num_opt_instructions_, counter.GetCount(InstructionCounter::TOTAL_INSTS));
}
- COUNTER_SET(num_opt_functions_,
- counter.GetCount(InstructionCounter::TOTAL_FUNCTIONS));
- COUNTER_SET(num_opt_instructions_, counter.GetCount(InstructionCounter::TOTAL_INSTS));
if (FLAGS_opt_module_dir.size() != 0) {
string path = FLAGS_opt_module_dir + "/" + id_ + "_opt.ll";
@@ -1355,30 +1352,39 @@ Status LlvmCodeGen::FinalizeModule() {
}
}
+ if (codegen_cache_enabled) {
+ if (cache_hit) {
+ DCHECK(engine_cache_cached_ != nullptr);
+ execution_engine()->setObjectCache(engine_cache_cached_.get());
+ } else {
+ execution_engine()->setObjectCache(engine_cache_.get());
+ }
+ }
{
SCOPED_TIMER(compile_timer_);
// Finalize module, which compiles all functions.
execution_engine()->finalizeObject();
}
-
SetFunctionPointers();
Status store_cache_status;
- if (codegen_cache_enabled) {
+ if (codegen_cache_enabled && !cache_hit) {
SCOPED_TIMER(codegen_cache_save_timer_);
store_cache_status = StoreCache(cache_key);
}
- // If the codegen is stored to the cache successfully, the cache will be responsible to
- // track the memory consumption instead.
- if (!codegen_cache_enabled || !store_cache_status.ok()) {
- // Track the memory consumed by the compiled code.
- int64_t bytes_allocated = memory_manager_->bytes_allocated();
- if (!mem_tracker_->TryConsume(bytes_allocated)) {
- const string& msg = Substitute(
- "Failed to allocate '$0' bytes for compiled code module", bytes_allocated);
- return mem_tracker_->MemLimitExceeded(NULL, msg, bytes_allocated);
- }
- memory_manager_->set_bytes_tracked(bytes_allocated);
+
+ // Track the memory consumed by the runtime compiled code.
+ // If codegen cache is enabled, the part stored to the cache will be taken care by
+ // codegen cache to track the memory consumption.
+ int64_t bytes_allocated = memory_manager_->bytes_allocated();
+ if (!mem_tracker_->TryConsume(bytes_allocated)) {
+ const string& msg = Substitute(
+ "Failed to allocate '$0' bytes for compiled code module", bytes_allocated);
+ return mem_tracker_->MemLimitExceeded(NULL, msg, bytes_allocated);
}
+ memory_manager_->set_bytes_tracked(bytes_allocated);
+
+ // Get the module id before module destruction.
+ if (module_id != nullptr) *module_id = module_->getModuleIdentifier();
DestroyModule();
return Status::OK();
}
@@ -1490,18 +1496,15 @@ bool LlvmCodeGen::SetFunctionPointers(CodeGenCache* cache,
void* jitted_function = nullptr;
if (cache != nullptr) {
DCHECK(cache_key != nullptr);
- // execution_engine_wrapper_cached_ is used to keep the life of the jitted functions
- // in case the engine is evicted in the global cache.
- DCHECK(execution_engine_wrapper_cached_ != nullptr);
- llvm::ExecutionEngine* cached_execution_engine =
- execution_engine_wrapper_cached_->execution_engine();
-
+ // engine_cache_cached_ is used to keep the life of the object cache
+ // in case the object cache is evicted in the global cache.
+ DCHECK(engine_cache_cached_ != nullptr);
// Using the function getFunctionAddress() with a non-existent function name would
// hit an assertion during the test, could be a bug in llvm 5, need to review after
// upgrade llvm. But because we already checked the names hashcode for key collision
// cases, we expect all the functions should be in the cached execution engine.
- jitted_function = reinterpret_cast<void*>(
- cached_execution_engine->getFunctionAddress(function_name));
+ jitted_function =
+ reinterpret_cast<void*>(execution_engine()->getFunctionAddress(function_name));
if (jitted_function == nullptr) {
LOG(WARNING) << "Failed to get a jitted function from cache: "
<< function_name.data()
diff --git a/be/src/codegen/llvm-codegen.h b/be/src/codegen/llvm-codegen.h
index 23d170525..3f6de734e 100644
--- a/be/src/codegen/llvm-codegen.h
+++ b/be/src/codegen/llvm-codegen.h
@@ -35,10 +35,9 @@
#include <llvm/IR/Intrinsics.h>
#include <llvm/IR/LLVMContext.h>
#include <llvm/IR/Module.h>
-#include <llvm/Support/MemoryBuffer.h>
#include <llvm/Support/raw_ostream.h>
-#include "codegen/llvm-execution-engine-wrapper.h"
+#include "codegen/llvm-codegen-object-cache.h"
#include "exprs/scalar-expr.h"
#include "impala-ir/impala-ir-functions.h"
#include "runtime/types.h"
@@ -325,10 +324,11 @@ class LlvmCodeGen {
llvm::LLVMContext& context() { return *context_.get(); }
/// Returns execution engine interface
- llvm::ExecutionEngine* execution_engine() {
- if (execution_engine_wrapper_ == nullptr) return nullptr;
- return execution_engine_wrapper_->execution_engine();
- }
+ llvm::ExecutionEngine* execution_engine() { return execution_engine_.get(); }
+
+ /// Returns the cache which is for the execution engine to write the compiled functions
+ /// to.
+ CodeGenObjectCache* engine_cache() { return engine_cache_.get(); }
/// Register a expr function with unique id. It can be subsequently retrieved via
/// GetRegisteredExprFn with that id.
@@ -350,7 +350,9 @@ class LlvmCodeGen {
/// false, the module will not be optimized before compilation. After FinalizeModule()
/// is called, the LLVM module is destroyed and it is invalid to call any LlvmCodegen
/// functions.
- Status FinalizeModule();
+ /// During FinalizeModule(), a new module id might be assigned for caching storage and
+ /// retrieval. If module_id is not nullptr, the final module id is returned.
+ Status FinalizeModule(string* module_id = nullptr);
/// Start executing 'FinalizeModule' in a separate thread and return.
/// 'async_compile_thread_' is set to point to the new 'Thread' object.
@@ -915,15 +917,22 @@ class LlvmCodeGen {
std::unique_ptr<llvm::LLVMContext> context_;
/// Top level codegen object. Contains everything to jit one 'unit' of code. module_ is
- /// set by Init(). module_ is owned by the execution engine in
- /// execution_engine_wrapper_.
+ /// set by Init(). module_ is owned by the execution engine.
llvm::Module* module_;
- /// Execution/Jitting engine in a wrapper.
- std::shared_ptr<LlvmExecutionEngineWrapper> execution_engine_wrapper_;
+ // Execution/Jitting engine.
+ std::unique_ptr<llvm::ExecutionEngine> execution_engine_;
- /// Cached Execution/Jitting engine in a wrapper.
- std::shared_ptr<LlvmExecutionEngineWrapper> execution_engine_wrapper_cached_;
+ /// Object cache which is for the execution engine to write the compiled codegened
+ /// functions to. Would be used as a part of CodeGen caching.
+ std::shared_ptr<CodeGenObjectCache> engine_cache_;
+
+ /// Object cache from the global CodeGen Cache, storing compiled codegened functions
+ /// that align with the module of the current ExecutionEngine.
+ /// Not null only when there is a cache hit.
+ /// The purpose of it is to maintain the lifecycle of this CodeGenObjectCache in case
+ /// it gets evicted from the global cache while in use.
+ std::shared_ptr<CodeGenObjectCache> engine_cache_cached_;
/// The memory manager used by 'execution_engine_'. Owned by 'execution_engine_'.
ImpalaMCJITMemoryManager* memory_manager_;
@@ -991,6 +1000,12 @@ class LlvmCodeGen {
llvm::Constant* true_value_;
llvm::Constant* false_value_;
+ /// The symbol emitter associated with 'execution_engine_'. Methods on
+ /// 'symbol_emitter_' are called by 'execution_engine_' when code is emitted or
+ /// freed. The lifetime of the symbol emitter must be longer than
+ /// 'execution_engine_'.
+ std::unique_ptr<CodegenSymbolEmitter> symbol_emitter_;
+
/// Provides an implementation of a LLVM diagnostic handler and maintains the error
/// information from its callbacks.
class DiagnosticHandler {
diff --git a/be/src/codegen/llvm-execution-engine-wrapper.h b/be/src/codegen/llvm-execution-engine-wrapper.h
deleted file mode 100644
index 67e11179b..000000000
--- a/be/src/codegen/llvm-execution-engine-wrapper.h
+++ /dev/null
@@ -1,59 +0,0 @@
-// Licensed to the Apache Software Foundation (ASF) under one
-// or more contributor license agreements. See the NOTICE file
-// distributed with this work for additional information
-// regarding copyright ownership. The ASF licenses this file
-// to you under the Apache License, Version 2.0 (the
-// "License"); you may not use this file except in compliance
-// with the License. You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing,
-// software distributed under the License is distributed on an
-// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-// KIND, either express or implied. See the License for the
-// specific language governing permissions and limitations
-// under the License.
-
-#pragma once
-
-#include <memory>
-
-#include "codegen/codegen-symbol-emitter.h"
-
-#include <llvm/ExecutionEngine/ExecutionEngine.h>
-
-namespace impala {
-
-/// Class that encapsulates a non-NULL 'llvm::ExecutionEngine' and other objects the
-/// lifetimes of which are tied to the execution engine but are not owned by it.
-class LlvmExecutionEngineWrapper {
- public:
- LlvmExecutionEngineWrapper(std::unique_ptr<llvm::ExecutionEngine> execution_engine,
- std::unique_ptr<CodegenSymbolEmitter> symbol_emitter)
- : symbol_emitter_(std::move(symbol_emitter)),
- execution_engine_(std::move(execution_engine)) {
- DCHECK(execution_engine_ != nullptr);
- }
-
- ~LlvmExecutionEngineWrapper() {
- execution_engine_.reset();
- symbol_emitter_.reset();
- }
-
- llvm::ExecutionEngine* execution_engine() {
- return execution_engine_.get();
- }
-
- private:
- /// The symbol emitter associated with 'execution_engine_'. Methods on
- /// 'symbol_emitter_' are called by 'execution_engine_' when code is emitted or
- /// freed. The lifetime of the symbol emitter must be longer than
- /// 'execution_engine_'.
- std::unique_ptr<CodegenSymbolEmitter> symbol_emitter_;
-
- /// Execution/Jitting engine.
- std::unique_ptr<llvm::ExecutionEngine> execution_engine_;
-};
-
-} // namespace impala
diff --git a/be/src/runtime/test-env.h b/be/src/runtime/test-env.h
index 3b05e42f7..0f7d2a0a6 100644
--- a/be/src/runtime/test-env.h
+++ b/be/src/runtime/test-env.h
@@ -78,7 +78,11 @@ class TestEnv {
int64_t TotalQueryMemoryConsumption();
/// Reset the codegen cache.
- void ResetCodegenCache(MetricGroup* metrics) {
+ void ResetCodegenCache(MetricGroup* metrics = nullptr) {
+ if (metrics == nullptr) {
+ exec_env_->codegen_cache_.reset();
+ return;
+ }
exec_env_->codegen_cache_.reset(new CodeGenCache(metrics));
}
diff --git a/tests/custom_cluster/test_codegen_cache.py b/tests/custom_cluster/test_codegen_cache.py
index 560f847fc..2c55fa2a6 100644
--- a/tests/custom_cluster/test_codegen_cache.py
+++ b/tests/custom_cluster/test_codegen_cache.py
@@ -200,7 +200,12 @@ class TestCodegenCache(CustomClusterTestSuite):
When the --codegen_symbol_emitter_log_successful_destruction_test_only flag is set to
true, 'CodegenSymbolEmitter' will log a message when it is being destroyed correctly
(i.e. when use-after-free will not happen). If we don't have the expected message in
- the logs (after some timeout), the test fails."""
+ the logs (after some timeout), the test fails.
+
+ After IMPALA-11805, codegen caching is no longer using the 'llvm::ExecutionEngine',
+ instead we use 'CodeGenObjectCache'. While 'CodeGenObjectCache' doesn't impact the
+ lifecycle of 'CodegenSymbolEmitter's, the testcase in this context still verifies
+ the correct usage of 'CodegenSymbolEmitter's."""
exec_options = copy(vector.get_value('exec_option'))
exec_options['exec_single_node_rows_threshold'] = 0
@@ -225,19 +230,23 @@ class TestCodegenCache(CustomClusterTestSuite):
self.execute_query_expect_success(self.client, q1, exec_options)
cache_entries_in_use = self.get_metric('impala.codegen-cache.entries-in-use')
cache_entries_evicted = self.get_metric('impala.codegen-cache.entries-evicted')
+ # Query 1 contains 2 fragments.
+ fragments_ran = 2
assert cache_entries_in_use > 0
assert self.get_metric('impala.codegen-cache.hits') == 0
# Initialising the cross-compiled modules also consumes an LLVM executor engine.
- expected_num_msg = cache_entries_evicted + 1
+ expected_num_msg = fragments_ran + 1
self.assert_impalad_log_contains("INFO", symbol_emitter_ok_msg, expected_num_msg)
# ## Second query
self.execute_query_expect_success(self.client, q2, exec_options)
assert self.get_metric('impala.codegen-cache.hits') == 0
+ # Query 2 contains 4 fragments.
+ fragments_ran = fragments_ran + 4
cache_entries_evicted = self.get_metric('impala.codegen-cache.entries-evicted')
assert cache_entries_evicted >= cache_entries_in_use
# Initialising the cross-compiled modules also consumes an LLVM executor engine.
- expected_num_msg = cache_entries_evicted + 1
+ expected_num_msg = fragments_ran + 1
self.assert_impalad_log_contains("INFO", symbol_emitter_ok_msg, expected_num_msg)
@pytest.mark.execute_serially