You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Alexey Serbin (Code Review)" <ge...@cloudera.org> on 2026/01/31 22:00:14 UTC

[kudu-CR] WIP [codegen] fix SIGSEGV on RedHat9 with libgcc-11.5.0-10 and newer

Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/23925


Change subject: WIP [codegen] fix SIGSEGV on RedHat9 with libgcc-11.5.0-10 and newer
......................................................................

WIP [codegen] fix SIGSEGV on RedHat9 with libgcc-11.5.0-10 and newer

WIP:
  * better document the newly added code
  * add proper description, explaining what was going on
  * file a JIRA ticket and reference it in the commit message
  * address feedback, if any

Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
---
M src/kudu/codegen/jit_frame_manager.cc
M src/kudu/codegen/jit_frame_manager.h
M src/kudu/codegen/module_builder.cc
M thirdparty/download-thirdparty.sh
A thirdparty/patches/llvm-section-mm-extra-methods.patch
A thirdparty/patches/llvm-section-mm-memory-mapper.patch
6 files changed, 381 insertions(+), 60 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/25/23925/1
-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Patch Set 2:

(4 comments)

> (2 comments)
 > 
 > Thanks for working on this Alexey!
 > I haven't gone through the full change, I just have a clarification
 > question: Is this implementation susceptible to the same
 > scalability bottleneck that the b-tree was introduced to solve?

This implementation isn't susceptible to the scalability bottleneck in former implementation of libgcc, no.  This code isn't affecting that in any way.  Also, since Kudu always runs only a single MCJIT compilation at a time (no concurrency there), that scalability bottleneck is not relevant in any way to Kudu itself.

http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc
File src/kudu/codegen/jit_frame_manager.cc:

http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc@95
PS2, Line 95:   const size_t num_pages = (num_bytes + page_size - 1) / page_size;
> Are section allocation requests to the memory mapper guaranteed to be page-
Requests for section memory allocation do not come aligned to memory page boundary.  However, that logic takes into account the actually allocated size, so it utilizes all the free memory left in every allocated chunk (you can see that in the implementation of SectionMemoryManager::allocateSection() method in lib/ExecutionEngine/SectionMemoryManager.cpp).

With that, and the fact that there are just a few memory allocation requests for every data type section (RO, RW, Code), plus page-aligning the estimate for their sizes and doubling the size in JITFrameManager::reserveAllocationSpace(), there isn't a way to overrun the pre-allocated chunk.  Even if it did, there is JITFrameManager::runPostAllocationCheck() that takes care of handling such a condition.


http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc@115
PS2, Line 115:     ec = Memory::protectMappedMemory(result, protection_flags);
> Is it guaranteed that protectMappedMemory() always performs icache invalida
Yes, it is, and LLVM relies on this.  Also, as for applicable platforms where Kudu runs,  this is relevant only on aarch64.

This code doesn't deviate in this regard from what LLVM does -- as you could see from the comment on this method, it's modeled after LLVM's code in lib/Support/Unix/Memory.inc


http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc@138
PS2, Line 138: JITFrameManager::~JITFrameManager() {
> SectionMemoryManager’s destructor releases all AllocatedMem blocks. Here we
This is intentional, and it's necessary since not all the pre-allocated space might be utilized by all AllocatedMem blocks.


http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc@161
PS2, Line 161:   const size_t code_required_size_bytes =
             :       code_align * ((code_size + code_align - 1) / code_align);
             :   const size_t ro_data_required_size_bytes =
             :       ro_data_align * ((ro_data_size + ro_data_align - 1) / ro_data_align);
             :   const size_t rw_data_required_size_bytes =
             :       rw_data_align * ((rw_data_size + rw_data_align - 1) / rw_data_align);
> Do we rely on LLVM always passing non?zero alignment values here?
Yes, we do.  It cannot pass 0 for any of the alignment numbers here, but adding an extra DCHECK() wouldn't hurt.



-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Mon, 02 Feb 2026 20:22:12 +0000
Gerrit-HasComments: Yes

[kudu-CR] WIP [codegen] fix SIGSEGV on RedHat9 with libgcc-11.5.0-10 and newer

Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: WIP [codegen] fix SIGSEGV on RedHat9 with libgcc-11.5.0-10 and newer
......................................................................


Patch Set 1:

Build Started http://jenkins.kudu.apache.org/job/pre_commit/2238/


-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Sat, 31 Jan 2026 22:00:22 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Patch Set 3: Verified-1

Build Failed 

http://jenkins.kudu.apache.org/job/pre_commit/2243/ : FAILURE


-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Feb 2026 01:45:38 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has removed a vote on this change.

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Removed Verified-1 by Kudu Jenkins (120)
-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Zoltan Chovan (Code Review)" <ge...@cloudera.org>.
Zoltan Chovan has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Patch Set 2:

(2 comments)

Thanks for working on this Alexey! 
I haven't gone through the full change, I just have a clarification question: Is this implementation susceptible to the same scalability bottleneck that the b-tree was introduced to solve?

http://gerrit.cloudera.org:8080/#/c/23925/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/23925/2//COMMIT_MSG@13
PS2, Line 13: has
nit: has been


http://gerrit.cloudera.org:8080/#/c/23925/2//COMMIT_MSG@28
PS2, Line 28: assumes
            : assumes
nit: one too many assumes :)



-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Mon, 02 Feb 2026 09:28:30 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Kurt Deschler, Zoltan Martonka, Attila Bukor, Kudu Jenkins, Abhishek Chennaka, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/23925

to look at the new patch set (#2).

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................

KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Former libgcc versions used mutex-protected linked list to store
information on EH frames.  To address scalability bottlenecks of the
former implementation of the __{register,deregister}_frame() API,
the newer implementation found in unwind-fde-dw2.c [1] and related
headers has switched to a read-optimized b-tree [2].  The b-tree
uses the start address of the memory range as the key.

While the lock and the related bottleneck is now gone, the b-tree-based
implementation is heavily reliant on a few invariants regarding the
properties of frames/sections being (de)registered.  In particular,
there is now the range invariant (a.k.a. "the non-overlapping rule")
which the new implementation heavily relies upon when updating and
rebalancing the b-tree upon insertion/deletion of elements (at least,
this rule is referred to in the commit description at [3]):

  * There must not be two frames that have overlapping address ranges

While the old linked list implementation might have tolerated when this
rule wasn't honored, but the new b-tree implementation is susceptible to
undefined behavior (UB) in such a cases because the tree logic assumes
assumes the presence of clear range boundaries for its separators.

As it turns out, this applies not only to a pair of particular frame
description entries (FDEs), but also to the whole range span of FDEs
that come after corresponding CIE entry in the data structure that
is supplied to __{register,deregister}_frame() invocations. In
particular, in classify_object_over_fdes() the span is calculated by
going over each FDE, so get_pc_range() returns the range spanning from
min(beginning of all FDEs) up to max(end of all FDEs).

In its turn, the implementation of RuntimeDyld's SectionMemoryManager
in LLVM uses mmap() with MAP_ANON and MAP_PRIVATE flags to allocate
memory for jitted object sections, including the .eh_frame section.
That results on quite arbitrary placement of the mmap-ed/allocated
memory ranges since the placement of the allocated ranges isn't
controlled by the application beyond providing an 'address hint' for
the placement of the newly allocated memory range, and the kernel is
free to find _any_ range that it finds as appropriate given the size
of the newly requested range if its first attempt to establish the new
mapping at the closest memory page boundary fails because there is an
existing memory mapping at that address already [4].

Since the address space of a running process might become fragmented
when there are many jitted code references alive (their handles are kept
in the codegen's cache in addition to keeping references by scanners,
compaction operations, etc.), it's possible to end up in a memory layout
that's illustrated below:

  objA sections:         [.....][...][...]
  objB sections: [.....]                  [...][...]

The .eh_frame contents for the section layout above wouldn't comply
with the "non-overlapping rule" for the FDEs span, and the new libgcc
implementation could end up with receiving SIGSEGV in an attempt to
register .eh_frame section for one of the objects.

The situation described above manifests itself in Kudu tablet servers
crashing with SIGSEGV on RHEL9 with libgcc of 11.5.0-11 version.  The
updated libgcc package of 11.5.0-11 version came with updates pushed
into the RedHat's package repos along with releasing RHEL9.7 on
2025-11-12 (November 11, 2025).  From the libgcc's package changelog [5]
it's clear that RedHat's libgcc switched to the b-tree EH frame
implementation since 11.5.0-10, and most likely it's here to stay.

This patch addresses the issue of Kudu tablet servers crashing in
codegen by providing a custom implementation of the section memory
manager (JITFrameManager class) to be used by LLVM's MCJIT execution
engine.  In essence, it reserves the memory area for subsequent
allocations of codegenned object's sections, so they are all adjacent
and localized in the pre-allocated memory area.  This approach
guarantees that all the FDEs sent to libgcc's __register_frame() cannot
interleave with any other FDEs and spans of their boundary ranges
registered by Kudu tablet server during its life cycle.

[1] https://github.com/gcc-mirror/gcc/blob/1c0305d7aea53d788f3f74ca9a2bd9fb764c0109/libgcc/unwind-dw2-fde.c
[2] https://github.com/gcc-mirror/gcc/blob/1c0305d7aea53d788f3f74ca9a2bd9fb764c0109/libgcc/unwind-dw2-btree.h
[3] https://gcc.gnu.org/cgit/gcc/commit/libgcc/unwind-dw2-btree.h?id=21109b37e8585a7a1b27650fcbf1749380016108
[4] https://www.man7.org/linux/man-pages/man2/mmap.2.html
[5] https://rhel.pkgs.org/9/red-hat-ubi-baseos-x86_64/libgcc-11.5.0-11.el9.x86_64.rpm.html

Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
---
M src/kudu/codegen/jit_frame_manager.cc
M src/kudu/codegen/jit_frame_manager.h
M src/kudu/codegen/module_builder.cc
M thirdparty/download-thirdparty.sh
A thirdparty/patches/llvm-section-mm-extra-methods.patch
A thirdparty/patches/llvm-section-mm-memory-mapper.patch
6 files changed, 439 insertions(+), 58 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/25/23925/2
-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has removed a vote on this change.

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Removed Verified-1 by Kudu Jenkins (120)
-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Patch Set 2: Verified-1

Build Failed 

http://jenkins.kudu.apache.org/job/pre_commit/2239/ : FAILURE


-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Mon, 02 Feb 2026 05:22:29 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Marton Greber (Code Review)" <ge...@cloudera.org>.
Marton Greber has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Patch Set 2:

(4 comments)

Thanks for working on this, and for the detailed commit message Alexey!
Couple questions from my end while still trying to understand every bit.

Do we want any targeted tests for the new JIT memory manager behavior (e.g., accounting vs page‑rounded mappings and teardown order)?

http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc
File src/kudu/codegen/jit_frame_manager.cc:

http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc@95
PS2, Line 95:   const size_t num_pages = (num_bytes + page_size - 1) / page_size;
Are section allocation requests to the memory mapper guaranteed to be page-aligned/page-sized by LLVM? If not, the mapper’s internal page rounding + MAP_FIXED could potentially overshoot the reserved range unless we validate against the rounded size.


http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc@115
PS2, Line 115:     ec = Memory::protectMappedMemory(result, protection_flags);
Is it guaranteed that protectMappedMemory() always performs icache invalidation on this platform, and only when MF_EXEC is set? If not, do we need an explicit InvalidateInstructionCache call?


http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc@138
PS2, Line 138: JITFrameManager::~JITFrameManager() {
SectionMemoryManager’s destructor releases all AllocatedMem blocks. Here we also release preallocated_block_ in the derived destructor. Are we intentionally double releasing sub ranges of that prealloc region, or should we avoid releasing preallocated_block_ here and let the base class handle teardown?


http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc@161
PS2, Line 161:   const size_t code_required_size_bytes =
             :       code_align * ((code_size + code_align - 1) / code_align);
             :   const size_t ro_data_required_size_bytes =
             :       ro_data_align * ((ro_data_size + ro_data_align - 1) / ro_data_align);
             :   const size_t rw_data_required_size_bytes =
             :       rw_data_align * ((rw_data_size + rw_data_align - 1) / rw_data_align);
Do we rely on LLVM always passing non?zero alignment values here?



-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Mon, 02 Feb 2026 15:30:23 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Patch Set 2:

Build Started http://jenkins.kudu.apache.org/job/pre_commit/2239/


-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Mon, 02 Feb 2026 04:38:14 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Patch Set 3: Verified+1

(2 comments)

RELEASE: unrelated test of known flaky test (KUDU-3275)

http://gerrit.cloudera.org:8080/#/c/23925/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/23925/2//COMMIT_MSG@13
PS2, Line 13:  th
> nit: has been
Done


http://gerrit.cloudera.org:8080/#/c/23925/2//COMMIT_MSG@28
PS2, Line 28: mes
            : the pre
> nit: one too many assumes :)
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Feb 2026 02:39:50 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Patch Set 4:

Build Started http://jenkins.kudu.apache.org/job/pre_commit/2245/


-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 4
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Feb 2026 08:14:55 +0000
Gerrit-HasComments: No

[kudu-CR] WIP [codegen] fix SIGSEGV on RedHat9 with libgcc-11.5.0-10 and newer

Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: WIP [codegen] fix SIGSEGV on RedHat9 with libgcc-11.5.0-10 and newer
......................................................................


Patch Set 1: Verified+1

Build Successful 

http://jenkins.kudu.apache.org/job/pre_commit/2238/ : SUCCESS


-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Sat, 31 Jan 2026 23:03:43 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Kurt Deschler, Zoltan Martonka, Attila Bukor, Kudu Jenkins, Abhishek Chennaka, Michael Smith, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/23925

to look at the new patch set (#3).

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................

KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Former libgcc versions used a mutex-protected linked list to store
information on EH frames.  Addressing the scalability bottlenecks of
the __{register,deregister}_frame() API in the former implementation,
the new implementation in unwind-fde-dw2.c [1] and related headers
now uses the read-optimized b-tree approach [2].  The b-tree uses
the start address of a memory range as key.

While the lock and the related bottleneck are now gone, the b-tree-based
implementation is heavily reliant on a few invariants regarding the
properties of frames/sections being (de)registered.  In particular,
there is a range invariant (a.k.a. the "non-overlapping rule") which
the new implementation heavily relies upon when updating and rebalancing
the b-tree when inserting/removing elements.  At least, this rule is
referred to in the commit description at [3]:

  * There must not be two frames that have overlapping address ranges

While the former implementation might have tolerated when this rule
wasn't honored, the new b-tree implementation is susceptible to UB
(undefined behavior) in such a case because the tree logic assumes
the presence of clear range boundaries for its separators.

As it turns out, this applies not only to a pair of particular frame
description entries (FDEs), but also to the whole range span of FDEs
that come along with corresponding CIE entry in the data structure
that is supplied to __{register,deregister}_frame() invocations. In
particular, in classify_object_over_fdes() the span is calculated by
going over each FDE of the supplied CIE/FDE chunk, and, as a result,
get_pc_range() returns an address range spanning from
min(beginning address of all FDEs) up to max(end address of all FDEs).

In its turn, the implementation of RuntimeDyld's SectionMemoryManager
in LLVM uses mmap() with MAP_ANON and MAP_PRIVATE flags to allocate
memory for jitted object sections, including the .eh_frame section.
That results on quite an arbitrary placement of the mmap-ed/allocated
memory ranges since the placement of an allocated range isn't controlled
by MCJIT execution engine beyond providing an 'address hint' for the
placement of a newly allocated memory range.  The kernel is free to use
any range in the address space of the process that it finds appropriate
given the size of the requested memory range if its first attempt to
establish the new mapping at the closest memory page boundary fails
because there is an existing memory mapping at that address already [4].

The address space of a running Kudu tablet server process might become
fragmented when there are many jitted code references kept alive, while
other references might have been used and then purged out of the codegen
cache and other references dropped by already completed scanners,
compaction operations, etc.  Eventually, it's possible to end up in a
memory layout that's illustrated below:

  object A sections:         [.....][...][...]
  object B sections:  [.....]                 [...][...]

.eh_frame section contents for the layout above wouldn't comply with
the "non-overlapping rule" for the FDEs span, and the new libgcc
implementation could end up with receiving SIGSEGV in an attempt to
register .eh_frame section for one of those objects or updating the
b-tree upon de-registering corresponding .eh_frame section.

The situation described above manifested itself in Kudu tablet servers
crashing with SIGSEGV on RHEL9 with libgcc 11.5.0-11 installed.
The libgcc package of 11.5.0-11 version came with the updates pushed
into the RedHat package repos along with newly released RHEL 9.7 on
2025-11-12 (November 11, 2025).  From the libgcc's package changelog [5]
it's clear that RedHat's libgcc switched to the b-tree-based EH frame
implementation since version 11.5.0-10, and it's here to stay.

This patch addresses the issue of Kudu tablet servers crashing in the
codegen by providing a custom implementation of the section memory
manager for the LLVM's MCJIT execution engine.  In essence, it reserves
the memory area for subsequent allocations of all the codegenned
objects' sections, so they are now guaranteed to be adjacent and
localized in the pre-allocated memory area.  This approach ensures
that the ranges of all the FDEs provided to the libgcc's
__{register,deregister}_frame() API cannot interleave with ranges
of any other FDEs and range spans defined by their boundary entries
registered by a Kudu tablet server during its life cycle.

[1] https://github.com/gcc-mirror/gcc/blob/1c0305d7aea53d788f3f74ca9a2bd9fb764c0109/libgcc/unwind-dw2-fde.c
[2] https://github.com/gcc-mirror/gcc/blob/1c0305d7aea53d788f3f74ca9a2bd9fb764c0109/libgcc/unwind-dw2-btree.h
[3] https://gcc.gnu.org/cgit/gcc/commit/libgcc/unwind-dw2-btree.h?id=21109b37e8585a7a1b27650fcbf1749380016108
[4] https://www.man7.org/linux/man-pages/man2/mmap.2.html
[5] https://rhel.pkgs.org/9/red-hat-ubi-baseos-x86_64/libgcc-11.5.0-11.el9.x86_64.rpm.html

Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
---
M src/kudu/codegen/jit_frame_manager.cc
M src/kudu/codegen/jit_frame_manager.h
M src/kudu/codegen/module_builder.cc
M thirdparty/download-thirdparty.sh
A thirdparty/patches/llvm-section-mm-extra-methods.patch
A thirdparty/patches/llvm-section-mm-memory-mapper.patch
6 files changed, 439 insertions(+), 58 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/25/23925/3
-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Marton Greber (Code Review)" <ge...@cloudera.org>.
Marton Greber has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Patch Set 5: Verified+1 Code-Review+2

(3 comments)

Looks good to me!
unrelated test failure in the latest rev:
tsan: subprocess_server-test.0

http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc
File src/kudu/codegen/jit_frame_manager.cc:

http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc@95
PS2, Line 95:                                      near_block->allocatedSize()
> Requests for section memory allocation do not come aligned to memory page b
Thanks for explaining this, looks good!


http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc@115
PS2, Line 115:     start += page_size - (start % page_size);
> Yes, it is, and LLVM relies on this.  Also, as for applicable platforms whe
Just wanted to double check, hence my question. Thanks!


http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc@138
PS2, Line 138:       return {};
> As an alternative, we might override releaseMappedMemory() to be a no-op an
Thanks! Understood. Yes i think the current way is sound.



-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 5
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Feb 2026 11:14:50 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Marton Greber (Code Review)" <ge...@cloudera.org>.
Marton Greber has removed a vote on this change.

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Removed Verified-1 by Kudu Jenkins (120)
-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 5
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Patch Set 2: Verified+1

TSAN: unrelated test failures in known flaky tests (KUDU-3598)


-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Mon, 02 Feb 2026 05:33:40 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Patch Set 5: Verified-1

Build Failed 

http://jenkins.kudu.apache.org/job/pre_commit/2246/ : FAILURE


-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 5
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Feb 2026 10:26:56 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Patch Set 5:

Build Started http://jenkins.kudu.apache.org/job/pre_commit/2246/


-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 5
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Feb 2026 09:43:16 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Ashwani Raina (Code Review)" <ge...@cloudera.org>.
Ashwani Raina has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Patch Set 5: Code-Review+1

(3 comments)

Given the urgency of this fix, it would be great to have some unit tests added for this that verify  boundary, alignment, concurrency (wherever applicable) conditions in a separate patch.

http://gerrit.cloudera.org:8080/#/c/23925/5/src/kudu/codegen/jit_frame_manager.cc
File src/kudu/codegen/jit_frame_manager.cc:

http://gerrit.cloudera.org:8080/#/c/23925/5/src/kudu/codegen/jit_frame_manager.cc@76
PS5, Line 76: (if present)
When is it expected to be present?


http://gerrit.cloudera.org:8080/#/c/23925/5/src/kudu/codegen/jit_frame_manager.cc@110
PS5, Line 110: (num_bytes + page_size - 1) 
nit: do you think overflow check is required before this?


http://gerrit.cloudera.org:8080/#/c/23925/5/thirdparty/download-thirdparty.sh
File thirdparty/download-thirdparty.sh:

http://gerrit.cloudera.org:8080/#/c/23925/5/thirdparty/download-thirdparty.sh@349
PS5, Line 349: 10
nit: I think this is more of a logical level than number of patches applied. So, this can be 9.



-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 5
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Feb 2026 13:43:17 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Kurt Deschler, Zoltan Martonka, Attila Bukor, Kudu Jenkins, Abhishek Chennaka, Michael Smith, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/23925

to look at the new patch set (#4).

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................

KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Former libgcc versions used a mutex-protected linked list to store
information on EH frames.  Addressing the scalability bottlenecks of
the __{register,deregister}_frame() API in the former implementation,
the new implementation in unwind-fde-dw2.c [1] and related headers
now uses the read-optimized b-tree approach [2].  The b-tree uses
the start address of a memory range as key.

While the lock and the related bottleneck are now gone, the b-tree-based
implementation is heavily reliant on a few invariants regarding the
properties of frames/sections being (de)registered.  In particular,
there is a range invariant (a.k.a. the "non-overlapping rule") which
the new implementation heavily relies upon when updating and rebalancing
the b-tree when inserting/removing elements.  At least, this rule is
referred to in the commit description at [3]:

  * There must not be two frames that have overlapping address ranges

While the former implementation might have tolerated when this rule
wasn't honored, the new b-tree implementation is susceptible to UB
(undefined behavior) in such a case because the tree logic assumes
the presence of clear range boundaries for its separators.

As it turns out, this applies not only to a pair of particular frame
description entries (FDEs), but also to the whole range span of FDEs
that come along with corresponding CIE entry in the data structure
that is supplied to __{register,deregister}_frame() invocations. In
particular, in classify_object_over_fdes() the span is calculated by
going over each FDE of the supplied CIE/FDE chunk, and, as a result,
get_pc_range() returns an address range spanning from
min(beginning address of all FDEs) up to max(end address of all FDEs).

In its turn, the implementation of RuntimeDyld's SectionMemoryManager
in LLVM uses mmap() with MAP_ANON and MAP_PRIVATE flags to allocate
memory for jitted object sections, including the .eh_frame section.
That results on quite an arbitrary placement of the mmap-ed/allocated
memory ranges since the placement of an allocated range isn't controlled
by MCJIT execution engine beyond providing an 'address hint' for the
placement of a newly allocated memory range.  The kernel is free to use
any range in the address space of the process that it finds appropriate
given the size of the requested memory range if its first attempt to
establish the new mapping at the closest memory page boundary fails
because there is an existing memory mapping at that address already [4].

The address space of a running Kudu tablet server process might become
fragmented when there are many jitted code references kept alive, while
other references might have been used and then purged out of the codegen
cache and other references dropped by already completed scanners,
compaction operations, etc.  Eventually, it's possible to end up in a
memory layout that's illustrated below:

  object A sections:         [.....][...][...]
  object B sections:  [.....]                 [...][...]

.eh_frame section contents for the layout above wouldn't comply with
the "non-overlapping rule" for the FDEs span, and the new libgcc
implementation could end up with receiving SIGSEGV in an attempt to
register .eh_frame section for one of those objects or updating the
b-tree upon de-registering corresponding .eh_frame section.

The situation described above manifested itself in Kudu tablet servers
crashing with SIGSEGV on RHEL9 with libgcc 11.5.0-11 installed.
The libgcc package of 11.5.0-11 version came with the updates pushed
into the RedHat package repos along with newly released RHEL 9.7 on
2025-11-12 (November 11, 2025).  From the libgcc's package changelog [5]
it's clear that RedHat's libgcc switched to the b-tree-based EH frame
implementation since version 11.5.0-10, and it's here to stay.

This patch addresses the issue of Kudu tablet servers crashing in the
codegen by providing a custom implementation of the section memory
manager for the LLVM's MCJIT execution engine.  In essence, it reserves
the memory area for subsequent allocations of all the codegenned
objects' sections, so they are now guaranteed to be adjacent and
localized in the pre-allocated memory area.  This approach ensures
that the ranges of all the FDEs provided to the libgcc's
__{register,deregister}_frame() API cannot interleave with ranges
of any other FDEs and range spans defined by their boundary entries
registered by a Kudu tablet server during its life cycle.

[1] https://github.com/gcc-mirror/gcc/blob/1c0305d7aea53d788f3f74ca9a2bd9fb764c0109/libgcc/unwind-dw2-fde.c
[2] https://github.com/gcc-mirror/gcc/blob/1c0305d7aea53d788f3f74ca9a2bd9fb764c0109/libgcc/unwind-dw2-btree.h
[3] https://gcc.gnu.org/cgit/gcc/commit/libgcc/unwind-dw2-btree.h?id=21109b37e8585a7a1b27650fcbf1749380016108
[4] https://www.man7.org/linux/man-pages/man2/mmap.2.html
[5] https://rhel.pkgs.org/9/red-hat-ubi-baseos-x86_64/libgcc-11.5.0-11.el9.x86_64.rpm.html

Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
---
M src/kudu/codegen/jit_frame_manager.cc
M src/kudu/codegen/jit_frame_manager.h
M src/kudu/codegen/module_builder.cc
M thirdparty/download-thirdparty.sh
A thirdparty/patches/llvm-section-mm-extra-methods.patch
A thirdparty/patches/llvm-section-mm-memory-mapper.patch
6 files changed, 501 insertions(+), 69 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/25/23925/4
-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 4
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Patch Set 4: Verified-1

Build Failed 

http://jenkins.kudu.apache.org/job/pre_commit/2245/ : FAILURE


-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 4
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Feb 2026 08:57:58 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Patch Set 5:

(3 comments)

> (3 comments)
 > 
 > Given the urgency of this fix, it would be great to have some unit
 > tests added for this that verify  boundary, alignment, concurrency
 > (wherever applicable) conditions in a separate patch.

Thank you for the review.

Yep, it would be great to put together a few unit tests.

As for the testing that has been already performed, I can report that this code has passed many hours (even a few days) of many (30+) independent runs of CodegenTest.CodegenRandomSchemas stress test on x86_64 and aarch64 architectures on different OS versions.  So, I'm quite confident it works as intended, at least in the runtime environment the MCJIT execution engine as of LLVM 11.0.

http://gerrit.cloudera.org:8080/#/c/23925/5/src/kudu/codegen/jit_frame_manager.cc
File src/kudu/codegen/jit_frame_manager.cc:

http://gerrit.cloudera.org:8080/#/c/23925/5/src/kudu/codegen/jit_frame_manager.cc@76
PS5, Line 76: (if present)
> When is it expected to be present?
It's present for other than pre-allocation cases.  It's not documented in this in-line docs blurb as is, but it's clear if glancing over the in-line comments in the implementation below.


http://gerrit.cloudera.org:8080/#/c/23925/5/src/kudu/codegen/jit_frame_manager.cc@110
PS5, Line 110: (num_bytes + page_size - 1) 
> nit: do you think overflow check is required before this?
num_bytes is always greater than 0 here because of the if() condition at line 86, and page_size cannot be less than 1.  MCJIT does not request a lot of memory for this: we can add a check for overflow, but it would be superfluous.


http://gerrit.cloudera.org:8080/#/c/23925/5/thirdparty/download-thirdparty.sh
File thirdparty/download-thirdparty.sh:

http://gerrit.cloudera.org:8080/#/c/23925/5/thirdparty/download-thirdparty.sh@349
PS5, Line 349: 10
> nit: I think this is more of a logical level than number of patches applied
That's a good point, thanks.

However, I'd like to push this patch ASAP, so I'll leave this 10 as of PS5.



-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 5
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Feb 2026 15:57:42 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Zoltan Martonka (Code Review)" <ge...@cloudera.org>.
Zoltan Martonka has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Patch Set 2: Code-Review+1

(1 comment)

Looks good.
I think we should go ahead with this solution.

http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc
File src/kudu/codegen/jit_frame_manager.cc:

http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc@161
PS2, Line 161:   const size_t code_required_size_bytes =
             :       code_align * ((code_size + code_align - 1) / code_align);
             :   const size_t ro_data_required_size_bytes =
             :       ro_data_align * ((ro_data_size + ro_data_align - 1) / ro_data_align);
             :   const size_t rw_data_required_size_bytes =
             :       rw_data_align * ((rw_data_size + rw_data_align - 1) / rw_data_align);
             : 
             :   // Extra safety margin: pre-allocate 2 times more, aligning up the the memory
             :   // page size for each section type.
             :   const size_t required_size_bytes = 2 * (
             :       page_size * ((code_required_size_bytes + page_size - 1) / page_size) +
             :       page_size * ((ro_data_required_size_bytes + page_size - 1) / page_size) +
             :       page_size * ((rw_data_required_size_bytes + page_size - 1) / page_size));
[nit] I think making a labda auto align_up = [](int x, int alignment) { return (x + alignment -1)/ alignment * alignment; };  would increase readability for using it 6 places.
But it is just my subjective preference, so feel free to ignore this.



-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Mon, 02 Feb 2026 21:57:57 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Ashwani Raina (Code Review)" <ge...@cloudera.org>.
Ashwani Raina has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Patch Set 5:

I just noticed that you have revved the patch where most of my comments may not be applicable. I will take a look at the latest PS.


-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 5
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Feb 2026 09:54:17 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Kudu Jenkins (Code Review)" <ge...@cloudera.org>.
Kudu Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Patch Set 3:

Build Started http://jenkins.kudu.apache.org/job/pre_commit/2243/


-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Feb 2026 00:00:03 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Patch Set 2:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc
File src/kudu/codegen/jit_frame_manager.cc:

http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc@138
PS2, Line 138: JITFrameManager::~JITFrameManager() {
> This is intentional, and it's necessary since not all the pre-allocated spa
As an alternative, we might override releaseMappedMemory() to be a no-op and only release the pre-allocated block here.  munmap() should unmap all the internal sub-ranges.

However, with such an approach it would be quite natural to go even further and fully mock all the underlying memory management within the  pre-allocated block, doing all the internal accounting and call mprotect() instead of mmap() for the requested ranges just to provide the requested protection of the memory areas.

I guess this a good next step if we want to reduce the number of mmap()/unmap() calls, but it should be properly implemented and tested.  Since we want to fix this SIGSEGV crash issue ASAP, I'm not sure it makes sense to do it right now, but we can invest some time here, if necessary.  As for the bigger picture, the code generation isn't in a hot path: all the compilations are done in the background and the original requestor goes without JIT-optimized projection implementation if the code isn't in the codegen cache already, so I don't see high ROI on this topic, frankly.

One good trait of this current approach is that it is simple  enough and adds just an overhead of one extra mmap()/munmap() syscall pair, plus a few trivial checks per allocation of a data/code section.


http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc@161
PS2, Line 161:   const size_t code_required_size_bytes =
             :       code_align * ((code_size + code_align - 1) / code_align);
             :   const size_t ro_data_required_size_bytes =
             :       ro_data_align * ((ro_data_size + ro_data_align - 1) / ro_data_align);
             :   const size_t rw_data_required_size_bytes =
             :       rw_data_align * ((rw_data_size + rw_data_align - 1) / rw_data_align);
             : 
             :   // Extra safety margin: pre-allocate 2 times more, aligning up the the memory
             :   // page size for each section type.
             :   const size_t required_size_bytes = 2 * (
             :       page_size * ((code_required_size_bytes + page_size - 1) / page_size) +
             :       page_size * ((ro_data_required_size_bytes + page_size - 1) / page_size) +
             :       page_size * ((rw_data_required_size_bytes + page_size - 1) / page_size));
> [nit] I think making a labda auto align_up = [](int x, int alignment) { ret
That's a good point!

Done.



-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Feb 2026 08:15:00 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Ashwani Raina (Code Review)" <ge...@cloudera.org>.
Ashwani Raina has posted comments on this change. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................


Patch Set 4: Code-Review+1

(4 comments)

LGTM! Just a few nits.

http://gerrit.cloudera.org:8080/#/c/23925/3/src/kudu/codegen/jit_frame_manager.h
File src/kudu/codegen/jit_frame_manager.h:

http://gerrit.cloudera.org:8080/#/c/23925/3/src/kudu/codegen/jit_frame_manager.h@95
PS3, Line 95:         
remaining


http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc
File src/kudu/codegen/jit_frame_manager.cc:

http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc@221
PS2, Line 221: EQ(0, memory_range_.al
nit: Here and below. Why not call this only after null check for result to avoid unnecessary function call overhead?


http://gerrit.cloudera.org:8080/#/c/23925/2/src/kudu/codegen/jit_frame_manager.cc@230
PS2, Line 230:        memory_range_.allocatedSize() != 0;
             : }
             : 
             : // 
nit: Maybe consolidate this and above instance into one template helper function that takes corresponding function pointer (SectionMemoryManager::allocateDataSection , SectionMemoryManager::allocateCodeSection ) as an argument? The same helper can run post allocation check as well after that.


http://gerrit.cloudera.org:8080/#/c/23925/3/src/kudu/codegen/jit_frame_manager.cc
File src/kudu/codegen/jit_frame_manager.cc:

http://gerrit.cloudera.org:8080/#/c/23925/3/src/kudu/codegen/jit_frame_manager.cc@157
PS3, Line 157:   // ranges must increase.
Is it possible that there are some pre-allocated bytes left but not enough to satisfy the requirements?



-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 4
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>
Gerrit-Comment-Date: Tue, 03 Feb 2026 08:45:37 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/23925 )

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................

KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Former libgcc versions used a mutex-protected linked list to store
information on EH frames.  Addressing the scalability bottlenecks of
the __{register,deregister}_frame() API in the former implementation,
the new implementation in unwind-fde-dw2.c [1] and related headers
now uses the read-optimized b-tree approach [2].  The b-tree uses
the start address of a memory range as key.

While the lock and the related bottleneck are now gone, the b-tree-based
implementation is heavily reliant on a few invariants regarding the
properties of frames/sections being (de)registered.  In particular,
there is a range invariant (a.k.a. the "non-overlapping rule") which
the new implementation heavily relies upon when updating and rebalancing
the b-tree when inserting/removing elements.  At least, this rule is
referred to in the commit description at [3]:

  * There must not be two frames that have overlapping address ranges

While the former implementation might have tolerated when this rule
wasn't honored, the new b-tree implementation is susceptible to UB
(undefined behavior) in such a case because the tree logic assumes
the presence of clear range boundaries for its separators.

As it turns out, this applies not only to a pair of particular frame
description entries (FDEs), but also to the whole range span of FDEs
that come along with corresponding CIE entry in the data structure
that is supplied to __{register,deregister}_frame() invocations. In
particular, in classify_object_over_fdes() the span is calculated by
going over each FDE of the supplied CIE/FDE chunk, and, as a result,
get_pc_range() returns an address range spanning from
min(beginning address of all FDEs) up to max(end address of all FDEs).

In its turn, the implementation of RuntimeDyld's SectionMemoryManager
in LLVM uses mmap() with MAP_ANON and MAP_PRIVATE flags to allocate
memory for jitted object sections, including the .eh_frame section.
That results in quite an arbitrary placement of the mmap-ed/allocated
memory ranges since the placement of an allocated range isn't controlled
by MCJIT execution engine beyond providing an 'address hint' for the
placement of a newly allocated memory range.  The kernel is free to use
any range in the address space of the process that it finds appropriate
given the size of the requested memory range if its first attempt to
establish the new mapping at the closest memory page boundary fails
because there is an existing memory mapping at that address already [4].

The address space of a running Kudu tablet server process might become
fragmented when there are many jitted code references kept alive, while
other references might have been used and then purged out of the codegen
cache and other references dropped by already completed scanners,
compaction operations, etc.  Eventually, it's possible to end up in a
memory layout that's illustrated below:

  object A sections:         [.....][...][...]
  object B sections:  [.....]                 [...][...]

.eh_frame section contents for the layout above wouldn't comply with
the "non-overlapping rule" for the FDEs span, and the new libgcc
implementation could end up with receiving SIGSEGV in an attempt to
register .eh_frame section for one of those objects or updating the
b-tree upon de-registering corresponding .eh_frame section.

The situation described above manifested itself in Kudu tablet servers
crashing with SIGSEGV on RHEL9 with libgcc 11.5.0-11 installed.
The libgcc package of 11.5.0-11 version came with the updates pushed
into the RedHat package repos along with newly released RHEL 9.7 on
2025-11-12 (November 11, 2025).  From the libgcc's package changelog [5]
it's clear that RedHat's libgcc switched to the b-tree-based EH frame
implementation since version 11.5.0-10, and it's here to stay.

This patch addresses the issue of Kudu tablet servers crashing in the
codegen by providing a custom implementation of the section memory
manager for the LLVM's MCJIT execution engine.  In essence, it reserves
the memory area for subsequent allocations of all the codegenned
objects' sections, so they are now guaranteed to be adjacent and
localized in the pre-allocated memory area.  This approach ensures
that the ranges of all the FDEs provided to the libgcc's
__{register,deregister}_frame() API cannot interleave with ranges
of any other FDEs and range spans defined by their boundary entries
registered by a Kudu tablet server during its life cycle.

[1] https://github.com/gcc-mirror/gcc/blob/1c0305d7aea53d788f3f74ca9a2bd9fb764c0109/libgcc/unwind-dw2-fde.c
[2] https://github.com/gcc-mirror/gcc/blob/1c0305d7aea53d788f3f74ca9a2bd9fb764c0109/libgcc/unwind-dw2-btree.h
[3] https://gcc.gnu.org/cgit/gcc/commit/libgcc/unwind-dw2-btree.h?id=21109b37e8585a7a1b27650fcbf1749380016108
[4] https://www.man7.org/linux/man-pages/man2/mmap.2.html
[5] https://rhel.pkgs.org/9/red-hat-ubi-baseos-x86_64/libgcc-11.5.0-11.el9.x86_64.rpm.html

Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Reviewed-on: http://gerrit.cloudera.org:8080/23925
Reviewed-by: Marton Greber <gr...@gmail.com>
Tested-by: Marton Greber <gr...@gmail.com>
Reviewed-by: Ashwani Raina <ar...@gmail.com>
---
M src/kudu/codegen/jit_frame_manager.cc
M src/kudu/codegen/jit_frame_manager.h
M src/kudu/codegen/module_builder.cc
M thirdparty/download-thirdparty.sh
A thirdparty/patches/llvm-section-mm-extra-methods.patch
A thirdparty/patches/llvm-section-mm-memory-mapper.patch
6 files changed, 486 insertions(+), 69 deletions(-)

Approvals:
  Marton Greber: Looks good to me, approved; Verified
  Ashwani Raina: Looks good to me, but someone else must approve

-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 6
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>

[kudu-CR] KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Marton Greber, Zoltan Chovan, Kurt Deschler, Zoltan Martonka, Ashwani Raina, Attila Bukor, Kudu Jenkins, Abhishek Chennaka, Michael Smith, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/23925

to look at the new patch set (#5).

Change subject: KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+
......................................................................

KUDU-3736 fix SIGSEGV in codegen with libgcc-11.5.0-10+

Former libgcc versions used a mutex-protected linked list to store
information on EH frames.  Addressing the scalability bottlenecks of
the __{register,deregister}_frame() API in the former implementation,
the new implementation in unwind-fde-dw2.c [1] and related headers
now uses the read-optimized b-tree approach [2].  The b-tree uses
the start address of a memory range as key.

While the lock and the related bottleneck are now gone, the b-tree-based
implementation is heavily reliant on a few invariants regarding the
properties of frames/sections being (de)registered.  In particular,
there is a range invariant (a.k.a. the "non-overlapping rule") which
the new implementation heavily relies upon when updating and rebalancing
the b-tree when inserting/removing elements.  At least, this rule is
referred to in the commit description at [3]:

  * There must not be two frames that have overlapping address ranges

While the former implementation might have tolerated when this rule
wasn't honored, the new b-tree implementation is susceptible to UB
(undefined behavior) in such a case because the tree logic assumes
the presence of clear range boundaries for its separators.

As it turns out, this applies not only to a pair of particular frame
description entries (FDEs), but also to the whole range span of FDEs
that come along with corresponding CIE entry in the data structure
that is supplied to __{register,deregister}_frame() invocations. In
particular, in classify_object_over_fdes() the span is calculated by
going over each FDE of the supplied CIE/FDE chunk, and, as a result,
get_pc_range() returns an address range spanning from
min(beginning address of all FDEs) up to max(end address of all FDEs).

In its turn, the implementation of RuntimeDyld's SectionMemoryManager
in LLVM uses mmap() with MAP_ANON and MAP_PRIVATE flags to allocate
memory for jitted object sections, including the .eh_frame section.
That results in quite an arbitrary placement of the mmap-ed/allocated
memory ranges since the placement of an allocated range isn't controlled
by MCJIT execution engine beyond providing an 'address hint' for the
placement of a newly allocated memory range.  The kernel is free to use
any range in the address space of the process that it finds appropriate
given the size of the requested memory range if its first attempt to
establish the new mapping at the closest memory page boundary fails
because there is an existing memory mapping at that address already [4].

The address space of a running Kudu tablet server process might become
fragmented when there are many jitted code references kept alive, while
other references might have been used and then purged out of the codegen
cache and other references dropped by already completed scanners,
compaction operations, etc.  Eventually, it's possible to end up in a
memory layout that's illustrated below:

  object A sections:         [.....][...][...]
  object B sections:  [.....]                 [...][...]

.eh_frame section contents for the layout above wouldn't comply with
the "non-overlapping rule" for the FDEs span, and the new libgcc
implementation could end up with receiving SIGSEGV in an attempt to
register .eh_frame section for one of those objects or updating the
b-tree upon de-registering corresponding .eh_frame section.

The situation described above manifested itself in Kudu tablet servers
crashing with SIGSEGV on RHEL9 with libgcc 11.5.0-11 installed.
The libgcc package of 11.5.0-11 version came with the updates pushed
into the RedHat package repos along with newly released RHEL 9.7 on
2025-11-12 (November 11, 2025).  From the libgcc's package changelog [5]
it's clear that RedHat's libgcc switched to the b-tree-based EH frame
implementation since version 11.5.0-10, and it's here to stay.

This patch addresses the issue of Kudu tablet servers crashing in the
codegen by providing a custom implementation of the section memory
manager for the LLVM's MCJIT execution engine.  In essence, it reserves
the memory area for subsequent allocations of all the codegenned
objects' sections, so they are now guaranteed to be adjacent and
localized in the pre-allocated memory area.  This approach ensures
that the ranges of all the FDEs provided to the libgcc's
__{register,deregister}_frame() API cannot interleave with ranges
of any other FDEs and range spans defined by their boundary entries
registered by a Kudu tablet server during its life cycle.

[1] https://github.com/gcc-mirror/gcc/blob/1c0305d7aea53d788f3f74ca9a2bd9fb764c0109/libgcc/unwind-dw2-fde.c
[2] https://github.com/gcc-mirror/gcc/blob/1c0305d7aea53d788f3f74ca9a2bd9fb764c0109/libgcc/unwind-dw2-btree.h
[3] https://gcc.gnu.org/cgit/gcc/commit/libgcc/unwind-dw2-btree.h?id=21109b37e8585a7a1b27650fcbf1749380016108
[4] https://www.man7.org/linux/man-pages/man2/mmap.2.html
[5] https://rhel.pkgs.org/9/red-hat-ubi-baseos-x86_64/libgcc-11.5.0-11.el9.x86_64.rpm.html

Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
---
M src/kudu/codegen/jit_frame_manager.cc
M src/kudu/codegen/jit_frame_manager.h
M src/kudu/codegen/module_builder.cc
M thirdparty/download-thirdparty.sh
A thirdparty/patches/llvm-section-mm-extra-methods.patch
A thirdparty/patches/llvm-section-mm-memory-mapper.patch
6 files changed, 486 insertions(+), 69 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/25/23925/5
-- 
To view, visit http://gerrit.cloudera.org:8080/23925
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I691d2f442c3148f235847c4c8e56767577804b1a
Gerrit-Change-Number: 23925
Gerrit-PatchSet: 5
Gerrit-Owner: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Ashwani Raina <ar...@gmail.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Marton Greber <gr...@gmail.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>
Gerrit-Reviewer: Zoltan Martonka <zm...@cloudera.com>