You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by al...@apache.org on 2026/09/03 23:29:05 UTC

(kudu) branch master updated (12f7a3316 -> 2acbccb8c)

This is an automated email from the ASF dual-hosted git repository.

asf-gitbox-commits pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git


    from 12f7a3316 KUDU-3795 Fix ENOSPC on dist-test due to compaction tests
     new c317e6790 [tablet] Skip stable_sort for already sorted batch
     new 2acbccb8c KUDU-3803: Add support for building Kudu with GCC 15

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 src/kudu/tablet/tablet-test.cc                     | 198 ++++++++++++++++++++-
 src/kudu/tablet/tablet.cc                          |  49 +++--
 src/kudu/tablet/tablet_metrics.cc                  |  19 ++
 src/kudu/tablet/tablet_metrics.h                   |   5 +
 thirdparty/build-definitions.sh                    |   2 +-
 .../llvm-gcc15-fix-missing-cstdint-include.patch   |  31 ++++
 thirdparty/patches/rocksdb-gcc15-part1.patch       |  49 +++++
 thirdparty/patches/rocksdb-gcc15-part2.patch       |  35 ++++
 thirdparty/patches/rocksdb-gcc15-part3.patch       |  25 +++
 .../yaml-fix-missing-cstdint-for-GCC15.patch       |  38 ++++
 thirdparty/vars.sh                                 |  13 +-
 11 files changed, 448 insertions(+), 16 deletions(-)
 create mode 100644 thirdparty/patches/llvm-gcc15-fix-missing-cstdint-include.patch
 create mode 100644 thirdparty/patches/rocksdb-gcc15-part1.patch
 create mode 100644 thirdparty/patches/rocksdb-gcc15-part2.patch
 create mode 100644 thirdparty/patches/rocksdb-gcc15-part3.patch
 create mode 100644 thirdparty/patches/yaml-fix-missing-cstdint-for-GCC15.patch


(kudu) 01/02: [tablet] Skip stable_sort for already sorted batch

Posted by al...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

asf-gitbox-commits pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git

commit c317e67900300cb2e4f54dbf6227efb0e33b342b
Author: Ashwani <as...@apache.org>
AuthorDate: Tue Jul 21 22:44:18 2026 +0530

    [tablet] Skip stable_sort for already sorted batch
    
    Tablet::BulkCheckPresence unconditionally runs std::stable_sort over the
    encoded-key/index pairs for every write batch before deduplicating and
    issuing the RowSetTree lookup. When a client submits rows already in
    increasing key order such as bulk-ingest tools (e.g. 'kudu perf loadgen,
    Impala INSERT ... ORDER BY, Spark/Sqoop pipelines that pre-sort by PK)
    and workloads with monotonic primary keys typically do, this sort is
    pure overhead: the input is already a valid stable-sorted output.
    
    This patch tracks an 'is_sorted' flag while building 'keys_and_indexes'
    in the loop that already scans every op, flipping it to false on the
    first out-of-order pair. When the flag survives to the end of the loop,
    the stable_sort is skipped entirely, improving performance by a slight
    margin if not significant.
    
    Measured with 'kudu perf loadgen --use_upsert --num_rows_per_thread=5M'
    on a single-tablet cluster (RELEASE build), median of N=5 runs:
    
    Without this patch and sorted input batch:
    for i in 1 2 3 4 5; do ./build/latest/bin/kudu perf loadgen \
          127.0.0.1:8764 --use_upsert --num_threads=1 \
          --num_rows_per_thread=5000000 --table_num_hash_partitions=1 \
          --table_num_range_partitions=1 \
          --buffer_size_bytes=$((96 * 1024 * 1024)) \
          2>&1 | grep "time total"; done
        time total: 4419.16 ms
        time total: 4433.7 ms
        time total: 4430.2 ms
        time total: 4395.23 ms
        time total: 4346.59 ms
    Median: 4419 ms
    Mean: 4405 ms
    
    With this patch and sorted input batch:
    for i in 1 2 3 4 5; do ./build/latest/bin/kudu perf loadgen \
          127.0.0.1:8764 --use_upsert --num_threads=1 \
          --num_rows_per_thread=5000000 --table_num_hash_partitions=1 \
          --table_num_range_partitions=1 \
          --buffer_size_bytes=$((96 * 1024 * 1024)) \
          2>&1 | grep "time total"; done
        time total: 4181.83 ms
        time total: 4103.07 ms
        time total: 4183.71 ms
        time total: 4121.75 ms
        time total: 4133.22 ms
    Median: 4133 ms
    Mean: 4145 ms
    
    Median speedup: 6.5%
    Mean speedup: 5.9%
    
    Change-Id: I28d3f5ebdf7e3f13f04ffe8d35417a98c370fcf7
    Reviewed-on: http://gerrit.cloudera.org:8080/24615
    Reviewed-by: Alexey Serbin <al...@apache.org>
    Tested-by: Ashwani Raina <ar...@gmail.com>
---
 src/kudu/tablet/tablet-test.cc    | 198 +++++++++++++++++++++++++++++++++++++-
 src/kudu/tablet/tablet.cc         |  49 +++++++---
 src/kudu/tablet/tablet_metrics.cc |  19 ++++
 src/kudu/tablet/tablet_metrics.h  |   5 +
 4 files changed, 259 insertions(+), 12 deletions(-)

diff --git a/src/kudu/tablet/tablet-test.cc b/src/kudu/tablet/tablet-test.cc
index 5520fffa2..358bf266a 100644
--- a/src/kudu/tablet/tablet-test.cc
+++ b/src/kudu/tablet/tablet-test.cc
@@ -24,6 +24,7 @@
 #include <memory>
 #include <optional>
 #include <ostream>
+#include <random>
 #include <string>
 #include <type_traits>
 #include <utility>
@@ -48,8 +49,8 @@
 #include "kudu/fs/block_manager.h"
 #include "kudu/gutil/ref_counted.h"
 #include "kudu/gutil/stl_util.h"
-#include "kudu/gutil/strings/substitute.h"
 #include "kudu/gutil/strings/join.h"
+#include "kudu/gutil/strings/substitute.h"
 #include "kudu/tablet/delta_key.h"
 #include "kudu/tablet/delta_stats.h"
 #include "kudu/tablet/deltafile.h"
@@ -59,6 +60,7 @@
 #include "kudu/tablet/rowset.h"
 #include "kudu/tablet/rowset_metadata.h"
 #include "kudu/tablet/rowset_tree.h" // IWYU pragma: keep
+#include "kudu/tablet/tablet-harness.h"
 #include "kudu/tablet/tablet-test-base.h"
 #include "kudu/tablet/tablet-test-util.h"
 #include "kudu/tablet/tablet.pb.h"
@@ -1733,5 +1735,199 @@ TYPED_TEST(TestTablet, TestDiffScanUnobservableOperations) {
   NO_FATALS(diff_scan_no_rows());
 }
 
+// Regression coverage for the "skip stable_sort for already sorted batch"
+// optimization in Tablet::BulkCheckPresence. Its correctness rests on the fact
+// that a non-decreasing input is already a valid stable-sorted output, so a
+// pre-sorted batch and any permutation of it must resolve to identical presence
+// results. These tests lock that in and also assert that the pre_sorted /
+// needed_sort bookkeeping counters move as expected.
+//
+// The schema uses a non-negative INT32 key: a non-negative INT32 encodes into a
+// byte string whose ordering matches its numeric ordering, so a batch presented
+// in ascending key order is guaranteed to reach BulkCheckPresence already sorted
+// (fast path), while a descending/shuffled batch is guaranteed to require the
+// stable_sort (slow path). This lets the tests assert exactly which path was taken.
+class BulkCheckPresenceTest : public KuduTabletTest {
+ public:
+  BulkCheckPresenceTest()
+      : KuduTabletTest(Schema({ ColumnSchema("key", INT32),
+                                ColumnSchema("val", INT32) },
+                              /*key_columns=*/1)) {}
+
+ protected:
+  // A single row operation described independently of its position in a batch,
+  // so the same logical batch can be replayed in different orders.
+  struct RowSpec {
+    RowOperationsPB::Type type;
+    int32_t key;
+    int32_t val;
+  };
+
+  // Applies 'specs' as a single write batch (in the given vector order) against
+  // the current tablet. The backing KuduPartialRow objects are kept alive for
+  // the duration of the WriteBatch() call, matching LocalTabletWriter's
+  // pointer-into-caller semantics.
+  Status WriteBatchInOrder(const vector<RowSpec>& specs) {
+    LocalTabletWriter writer(tablet().get(), &client_schema_);
+    vector<KuduPartialRow> rows;
+    rows.reserve(specs.size());
+    for (const auto& spec : specs) {
+      KuduPartialRow row(&client_schema_);
+      RETURN_NOT_OK(row.SetInt32(0, spec.key));
+      RETURN_NOT_OK(row.SetInt32(1, spec.val));
+      rows.emplace_back(std::move(row));
+    }
+    vector<LocalTabletWriter::RowOp> ops;
+    ops.reserve(specs.size());
+    for (size_t i = 0; i < specs.size(); i++) {
+      ops.emplace_back(specs[i].type, &rows[i]);
+    }
+    return writer.WriteBatch(ops);
+  }
+
+  // Dumps the full tablet contents in primary-key order, using an ordered
+  // (merge) iterator — the same scan mode that production queries use when
+  // key order matters. Two independently-populated tablets that hold the same
+  // logical rows will produce identical output regardless of how they were
+  // written.
+  vector<string> DumpSortedRows() {
+    unique_ptr<RowwiseIterator> iter;
+    CHECK_OK(tablet()->NewOrderedRowIterator(client_schema_, &iter));
+    CHECK_OK(iter->Init(nullptr));
+    vector<string> lines;
+    CHECK_OK(IterateToStringList(iter.get(), &lines));
+    return lines;
+  }
+
+  int64_t PreSortedCount() {
+    return tablet()->metrics()->bulk_check_batches_pre_sorted->value();
+  }
+  int64_t NeededSortCount() {
+    return tablet()->metrics()->bulk_check_batches_needed_sort->value();
+  }
+};
+
+// A batch whose keys arrive already in ascending order must skip the
+// stable_sort (fast path), and the std::unique dedup that follows must
+// still retain the first op for a repeated key.
+TEST_F(BulkCheckPresenceTest, TestSortSkippedForAscendingBatch) {
+  const int kNumKeys = 200;
+
+  const int64_t pre_sorted_before = PreSortedCount();
+  const int64_t needed_sort_before = NeededSortCount();
+
+  // Ascending batch. Key 0 appears twice back-to-back: an INSERT followed by an
+  // INSERT_IGNORE with a different value. Equal adjacent keys keep the batch
+  // non-decreasing (so it stays on the fast path), and the first op (val 0)
+  // must win over the ignored second op (val 999).
+  vector<RowSpec> specs;
+  specs.push_back({ RowOperationsPB::INSERT, 0, 0 });
+  specs.push_back({ RowOperationsPB::INSERT_IGNORE, 0, 999 });
+  for (int k = 1; k < kNumKeys; k++) {
+    specs.push_back({ RowOperationsPB::INSERT, k, k * 10 });
+  }
+  ASSERT_OK(WriteBatchInOrder(specs));
+
+  // The sort was skipped: only the pre-sorted counter advanced.
+  ASSERT_EQ(pre_sorted_before + 1, PreSortedCount());
+  ASSERT_EQ(needed_sort_before, NeededSortCount());
+
+  // All distinct keys are present, and the first op for the repeated key won.
+  vector<string> rows = DumpSortedRows();
+  ASSERT_EQ(kNumKeys, rows.size());
+  ASSERT_EQ(R"((int32 key=0, int32 val=0))", rows[0]);
+}
+
+// A batch whose keys arrive out of order must take the stable_sort (slow path)
+// yet resolve to exactly the same presence results / tablet contents as the
+// equivalent pre-sorted batch. This is the core correctness invariant behind
+// the sort-skip optimization.
+TEST_F(BulkCheckPresenceTest, TestShuffledBatchMatchesSortedBatch) {
+  const int kNumKeys = 200;
+
+  // The canonical set of distinct rows, in ascending key order.
+  vector<RowSpec> sorted_specs;
+  sorted_specs.reserve(kNumKeys);
+  for (int k = 0; k < kNumKeys; k++) {
+    sorted_specs.push_back({ RowOperationsPB::INSERT, k, k * 10 });
+  }
+
+  // Apply the sorted batch to the fixture's tablet; it should take the fast
+  // path, and we record the resulting contents to compare against.
+  const int64_t pre_sorted_before = PreSortedCount();
+  const int64_t needed_sort_before = NeededSortCount();
+  ASSERT_OK(WriteBatchInOrder(sorted_specs));
+  ASSERT_EQ(pre_sorted_before + 1, PreSortedCount());
+  ASSERT_EQ(needed_sort_before, NeededSortCount());
+  const vector<string> sorted_rows = DumpSortedRows();
+  ASSERT_EQ(kNumKeys, sorted_rows.size());
+
+  // Build a shuffled permutation of the same set. A fixed seed keeps the test
+  // deterministic, and a 200-key shuffle is never already ascending, so the
+  // slow path is guaranteed.
+  vector<RowSpec> shuffled_specs = sorted_specs;
+  std::mt19937 rng(12345);
+  std::shuffle(shuffled_specs.begin(), shuffled_specs.end(), rng);
+
+  // Apply the shuffled batch to a brand-new, empty tablet so the same keys can
+  // be re-inserted without colliding with the first tablet's rows. Resetting
+  // 'harness_' forces a fresh filesystem layout to be formatted for the new
+  // root directory (rather than reopening the existing one).
+  harness_.reset();
+  SetUpTestTablet(GetTestPath("fs_root_shuffled"));
+  ASSERT_OK(WriteBatchInOrder(shuffled_specs));
+  ASSERT_EQ(1, NeededSortCount());
+  ASSERT_EQ(0, PreSortedCount());
+  const vector<string> shuffled_rows = DumpSortedRows();
+
+  // Same presence results regardless of input ordering.
+  ASSERT_EQ(sorted_rows, shuffled_rows);
+}
+
+// Regression test for the case where every op in a batch already has a result
+// (e.g. per-row decode validation failures) before BulkCheckPresence runs.
+// In that scenario the build loop that populates keys_and_indexes skips every
+// op, leaving the vector empty. The is_sorted flag retains its default value
+// of 'true', which must NOT cause bulk_check_batches_pre_sorted to be
+// incremented — there was no sorted batch; there was simply nothing to do.
+//
+// Trigger: UPDATE ops that specify only the primary-key column and omit all
+// value columns produce an empty row-changelist. The decoder records a per-row
+// InvalidArgument("No fields updated") failure on each such op, setting
+// has_result()=true before BulkCheckPresence is reached.
+TEST_F(BulkCheckPresenceTest, TestAllOpsPreFilledDoesNotIncrementSortCounter) {
+  const int64_t pre_before = PreSortedCount();
+  const int64_t needed_before = NeededSortCount();
+
+  // Build UPDATE ops that each omit the non-key column so the decoder marks
+  // every op failed ("No fields updated") before BulkCheckPresence runs.
+  LocalTabletWriter writer(tablet().get(), &client_schema_);
+  const int kNumOps = 10;
+  vector<KuduPartialRow> rows;
+  rows.reserve(kNumOps);
+  for (int k = 0; k < kNumOps; k++) {
+    KuduPartialRow row(&client_schema_);
+    ASSERT_OK(row.SetInt32(0, k));
+    rows.emplace_back(std::move(row));
+  }
+  vector<LocalTabletWriter::RowOp> ops;
+  ops.reserve(kNumOps);
+  for (int i = 0; i < kNumOps; i++) {
+    ops.emplace_back(RowOperationsPB::UPDATE, &rows[i]);
+  }
+
+  // Per-row decode failures do not abort the batch (DecodeWriteOperations still
+  // returns OK), so ApplyRowOperations — and therefore BulkCheckPresence — runs.
+  // WriteBatch surfaces the first per-row error as its own return status.
+  Status s = writer.WriteBatch(ops);
+  ASSERT_TRUE(s.IsInvalidArgument()) << "expected per-row decode error, got: "
+                                     << s.ToString();
+
+  // Neither counter must have moved: an all-skipped batch is not a "pre-sorted
+  // batch" and must not be counted as one.
+  ASSERT_EQ(pre_before, PreSortedCount());
+  ASSERT_EQ(needed_before, NeededSortCount());
+}
+
 } // namespace tablet
 } // namespace kudu
diff --git a/src/kudu/tablet/tablet.cc b/src/kudu/tablet/tablet.cc
index a5241400f..6b0eedb6e 100644
--- a/src/kudu/tablet/tablet.cc
+++ b/src/kudu/tablet/tablet.cc
@@ -1155,34 +1155,61 @@ Status Tablet::BulkCheckPresence(const IOContext* io_context, WriteOpState* op_s
   // so load it up top.
   RowOp* const * row_ops_base = op_state->row_ops().data();
 
-  // Run all of the ops through the RowSetTree.
+  // Run all of the ops through the RowSetTree. While building the vector,
+  // also detect whether the incoming batch of keys is already in ascending
+  // order so we can skip the stable_sort below on the common bulk-ingest
+  // case where the client already presents keys in sorted order.
   vector<pair<Slice, int>> keys_and_indexes;
   keys_and_indexes.reserve(num_ops);
+  bool is_sorted = true;
   for (int i = 0; i < num_ops; i++) {
     RowOp* op = row_ops_base[i];
     // If the op already failed in validation, or if we've got the original result
     // filled in already during replay, then we don't need to consult the RowSetTree.
     if (op->has_result() || op->orig_result_from_log) continue;
-    keys_and_indexes.emplace_back(op->key_probe->encoded_key_slice(), i);
+    const Slice& key = op->key_probe->encoded_key_slice();
+    // Once 'is_sorted' has flipped, short-circuit the compare: a well-predicted
+    // branch is cheaper than a Slice::compare (memcmp) per op on the slow path.
+    if (is_sorted && !keys_and_indexes.empty() &&
+        key < keys_and_indexes.back().first) {
+      is_sorted = false;
+    }
+    keys_and_indexes.emplace_back(key, i);
   }
 
+  // Every op in the batch either failed validation or already has a result
+  // from WAL replay, so there are no keys to look up in the RowSetTree.
+  // Treat this the same as the num_ops == 0 early return above: skip the
+  // sort, dedup, and metrics entirely rather than falsely incrementing
+  // bulk_check_batches_pre_sorted just because is_sorted was never flipped.
+  if (PREDICT_FALSE(keys_and_indexes.empty())) return Status::OK();
+
   // Sort the query points by their probe keys, retaining the equivalent indexes.
   //
   // It's important to do a stable-sort here so that the 'unique' call
   // below retains only the _first_ op the user specified, instead of
-  // an arbitrary one.
+  // an arbitrary one. When the batch was already in ascending order
+  // (detected above), the input is already a valid stable-sorted output
+  // (since ops with equal keys remain in their original relative order),
+  // so we can skip the sort entirely.
   //
   // TODO(todd): benchmark stable_sort vs using sort() and falling back to
   // comparing 'a.second' when a.first == b.first. Some microbenchmarks
   // seem to indicate stable_sort is actually faster.
-  // TODO(todd): could also consider weaving in a check in the loop above to
-  // see if the incoming batch is already totally-ordered and in that case
-  // skip this sort and std::unique call.
-  std::stable_sort(keys_and_indexes.begin(), keys_and_indexes.end(),
-                   [](const pair<Slice, int>& a,
-                      const pair<Slice, int>& b) {
-                     return a.first < b.first;
-                   });
+  if (!is_sorted) {
+    std::stable_sort(keys_and_indexes.begin(), keys_and_indexes.end(),
+                     [](const pair<Slice, int>& a,
+                        const pair<Slice, int>& b) {
+                       return a.first < b.first;
+                     });
+  }
+  if (metrics_) {
+    if (is_sorted) {
+      metrics_->bulk_check_batches_pre_sorted->Increment();
+    } else {
+      metrics_->bulk_check_batches_needed_sort->Increment();
+    }
+  }
   // If the batch has more than one operation for the same row, then we can't
   // use the up-front presence optimization on those operations, since the
   // first operation may change the result of the later presence-checks.
diff --git a/src/kudu/tablet/tablet_metrics.cc b/src/kudu/tablet/tablet_metrics.cc
index 096064e5e..b91f8dca0 100644
--- a/src/kudu/tablet/tablet_metrics.cc
+++ b/src/kudu/tablet/tablet_metrics.cc
@@ -174,6 +174,23 @@ METRIC_DEFINE_histogram(tablet, scan_duration_user_time,
                         kudu::MetricLevel::kDebug,
                         60000LU, 1);
 
+// These counters track the fast-path / slow-path split of the sort-skip
+// optimisation in Tablet::BulkCheckPresence.
+METRIC_DEFINE_counter(tablet, bulk_check_batches_pre_sorted,
+                      "BulkCheckPresence Pre-Sorted Batches",
+                      kudu::MetricUnit::kRequests,
+                      "Number of write batches whose keys arrived already in "
+                      "increasing order, allowing the stable_sort in "
+                      "Tablet::BulkCheckPresence to be skipped.",
+                      kudu::MetricLevel::kDebug);
+METRIC_DEFINE_counter(tablet, bulk_check_batches_needed_sort,
+                      "BulkCheckPresence Sorted Batches",
+                      kudu::MetricUnit::kRequests,
+                      "Number of write batches whose keys arrived out of "
+                      "order, requiring a stable_sort inside "
+                      "Tablet::BulkCheckPresence.",
+                      kudu::MetricLevel::kDebug);
+
 METRIC_DEFINE_counter(tablet, bloom_lookups, "Bloom Filter Lookups",
                       kudu::MetricUnit::kProbes,
                       "Number of times a bloom filter was consulted",
@@ -484,6 +501,8 @@ TabletMetrics::TabletMetrics(const scoped_refptr<MetricEntity>& entity)
     MINIT(key_file_lookups),
     MINIT(delta_file_lookups),
     MINIT(mrs_lookups),
+    MINIT(bulk_check_batches_pre_sorted),
+    MINIT(bulk_check_batches_needed_sort),
     MINIT(bytes_flushed),
     MINIT(deleted_rowset_gc_bytes_deleted),
     MINIT(undo_delta_block_gc_bytes_deleted),
diff --git a/src/kudu/tablet/tablet_metrics.h b/src/kudu/tablet/tablet_metrics.h
index 4153c910a..4ea07366b 100644
--- a/src/kudu/tablet/tablet_metrics.h
+++ b/src/kudu/tablet/tablet_metrics.h
@@ -75,6 +75,11 @@ struct TabletMetrics {
   scoped_refptr<Counter> delta_file_lookups;
   scoped_refptr<Counter> mrs_lookups;
 
+  // Fast-path / slow-path split for the sort-skip optimisation in
+  // Tablet::BulkCheckPresence.
+  scoped_refptr<Counter> bulk_check_batches_pre_sorted;
+  scoped_refptr<Counter> bulk_check_batches_needed_sort;
+
   // Operation stats.
   scoped_refptr<Counter> bytes_flushed;
   scoped_refptr<Counter> deleted_rowset_gc_bytes_deleted;


(kudu) 02/02: KUDU-3803: Add support for building Kudu with GCC 15

Posted by al...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

asf-gitbox-commits pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git

commit 2acbccb8cc0315a9fc9f27de90ffe32f482e1817
Author: Joe McDonnell <jo...@cloudera.com>
AuthorDate: Sat Nov 1 12:38:37 2025 -0700

    KUDU-3803: Add support for building Kudu with GCC 15
    
    This makes minor changes to the thirdparty build to
    support GCC 15:
     - Specify C17 for the postgresql build
     - Patch LLVM, Yaml, and RocksDB to add the cstdint header
    
    Testing:
     - Built Kudu with GCC 15
     - Built Kudu with an older system compiler
    
    Change-Id: I1e8a32837a1d9d6eb6b9dcc3c0369f42ce229ebc
    Reviewed-on: http://gerrit.cloudera.org:8080/24788
    Tested-by: Alexey Serbin <al...@apache.org>
    Reviewed-by: Alexey Serbin <al...@apache.org>
---
 thirdparty/build-definitions.sh                    |  2 +-
 .../llvm-gcc15-fix-missing-cstdint-include.patch   | 31 ++++++++++++++
 thirdparty/patches/rocksdb-gcc15-part1.patch       | 49 ++++++++++++++++++++++
 thirdparty/patches/rocksdb-gcc15-part2.patch       | 35 ++++++++++++++++
 thirdparty/patches/rocksdb-gcc15-part3.patch       | 25 +++++++++++
 .../yaml-fix-missing-cstdint-for-GCC15.patch       | 38 +++++++++++++++++
 thirdparty/vars.sh                                 | 13 ++++--
 7 files changed, 189 insertions(+), 4 deletions(-)

diff --git a/thirdparty/build-definitions.sh b/thirdparty/build-definitions.sh
index b68dda50b..992b42401 100644
--- a/thirdparty/build-definitions.sh
+++ b/thirdparty/build-definitions.sh
@@ -1227,7 +1227,7 @@ build_postgres() {
   #
   # [1] https://stackoverflow.com/questions/70931415/
   # [2] https://stackoverflow.com/questions/38058041/
-  CFLAGS="$EXTRA_CFLAGS" \
+  CFLAGS="$EXTRA_CFLAGS -std=c17" \
     LDFLAGS="$EXTRA_LDFLAGS" \
     $POSTGRES_SOURCE/configure \
     --prefix=$PREFIX \
diff --git a/thirdparty/patches/llvm-gcc15-fix-missing-cstdint-include.patch b/thirdparty/patches/llvm-gcc15-fix-missing-cstdint-include.patch
new file mode 100644
index 000000000..c6cb10ade
--- /dev/null
+++ b/thirdparty/patches/llvm-gcc15-fix-missing-cstdint-include.patch
@@ -0,0 +1,31 @@
+From 7abf44069aec61eee147ca67a6333fc34583b524 Mon Sep 17 00:00:00 2001
+From: Stephan Hageboeck <st...@cern.ch>
+Date: Mon, 20 Jan 2025 17:52:47 +0100
+Subject: [PATCH] Add missing include to X86MCTargetDesc.h (#123320)
+
+In gcc-15, explicit includes of `<cstdint>` are required when fixed-size
+integers are used. In this file, this include only happened as a side
+effect of including SmallVector.h
+
+Although llvm compiles fine, the root-project would benefit from
+explicitly including it here, so we can backport the patch.
+
+Maybe interesting for @hahnjo and @vgvassilev
+
+Backport Note: Adjusted to match LLVM 11 surrounding code
+---
+ llvm/lib/Target/X86/MCTargetDesc/X86MCTargetDesc.h | 1 +
+ 1 file changed, 1 insertion(+)
+
+diff --git a/llvm/lib/Target/X86/MCTargetDesc/X86MCTargetDesc.h b/llvm/lib/Target/X86/MCTargetDesc/X86MCTargetDesc.h
+index e166b68668d9b..0e0e13e896aea 100644
+--- a/llvm/lib/Target/X86/MCTargetDesc/X86MCTargetDesc.h
++++ b/llvm/lib/Target/X86/MCTargetDesc/X86MCTargetDesc.h
+@@ -14,6 +14,7 @@
+ #ifndef LLVM_LIB_TARGET_X86_MCTARGETDESC_X86MCTARGETDESC_H
+ #define LLVM_LIB_TARGET_X86_MCTARGETDESC_X86MCTARGETDESC_H
+ 
++#include <cstdint>
+ #include <memory>
+ #include <string>
+ 
diff --git a/thirdparty/patches/rocksdb-gcc15-part1.patch b/thirdparty/patches/rocksdb-gcc15-part1.patch
new file mode 100644
index 000000000..95c12fab4
--- /dev/null
+++ b/thirdparty/patches/rocksdb-gcc15-part1.patch
@@ -0,0 +1,49 @@
+From 72c38871673f34ef22c66b8fcb9292812272a961 Mon Sep 17 00:00:00 2001
+From: Jay Huh <je...@meta.com>
+Date: Mon, 28 Apr 2025 13:35:48 -0700
+Subject: [PATCH] Fix build (#13579)
+
+Summary:
+- [Failed CI run](https://productionresultssa17.blob.core.windows.net/actions-results/fd083599-6c98-4aec-8732-fcb280c96021/workflow-job-run-2f73efd7-c93d-53ea-a18f-1c7e17604f7e/logs/job/job-logs.txt?rsct=text%2Fplain&se=2025-04-28T17%3A15%3A01Z&sig=YJevYF5xH4RClY3klBe6Z3tnCWuYZFLlBYRHwftW9lc%3D&ske=2025-04-29T01%3A55%3A36Z&skoid=ca7593d4-ee42-46cd-af88-8b886a2f84eb&sks=b&skt=2025-04-28T13%3A55%3A36Z&sktid=398a6654-997b-47e9-b12b-9515b896b4de&skv=2025-01-05&sp=r&spr=https&sr=b&st=2025-04- [...]
+
+```
+2025-04-28T16:56:00.5775476Z In file included from <stdin>:1:
+2025-04-28T16:56:00.5776056Z db/blob/blob_file_meta.h:28:7: error: 'uint64_t' has not been declared
+2025-04-28T16:56:00.5776715Z    28 |       uint64_t blob_file_number, uint64_t total_blob_count,
+2025-04-28T16:56:00.5777153Z       |       ^~~~~~~~
+2025-04-28T16:56:00.5778083Z db/blob/blob_file_meta.h:15:1: note: 'uint64_t' is defined in header '<cstdint>'; this is probably fixable by adding '#include <cstdint>'
+2025-04-28T16:56:00.5779293Z    14 | #include "rocksdb/rocksdb_namespace.h"
+2025-04-28T16:56:00.5782126Z   +++ |+#include <cstdint>
+2025-04-28T16:56:00.5782780Z    15 |
+2025-04-28T16:56:00.5783204Z db/blob/blob_file_meta.h:28:34: error: 'uint64_t' has not been declared
+2025-04-28T16:56:00.5783832Z    28 |       uint64_t blob_file_number, uint64_t total_blob_count,
+2025-04-28T16:56:00.5784301Z       |                                  ^~~~~~~~
+```
+
+Pull Request resolved: https://github.com/facebook/rocksdb/pull/13579
+
+Test Plan: [CI](https://github.com/facebook/rocksdb/actions/runs/14713618495/job/41291839382?pr=13579)
+
+Reviewed By: archang19, cbi42
+
+Differential Revision: D73799590
+
+Pulled By: jaykorean
+
+fbshipit-source-id: 7ead97914c05958bb7146f1934c48615599bc4f8
+---
+ db/blob/blob_file_meta.h | 1 +
+ 1 file changed, 1 insertion(+)
+
+diff --git a/db/blob/blob_file_meta.h b/db/blob/blob_file_meta.h
+index d7c8a124336d..2e47726f8d11 100644
+--- a/db/blob/blob_file_meta.h
++++ b/db/blob/blob_file_meta.h
+@@ -6,6 +6,7 @@
+ #pragma once
+ 
+ #include <cassert>
++#include <cstdint>
+ #include <iosfwd>
+ #include <memory>
+ #include <string>
diff --git a/thirdparty/patches/rocksdb-gcc15-part2.patch b/thirdparty/patches/rocksdb-gcc15-part2.patch
new file mode 100644
index 000000000..f93f4dc69
--- /dev/null
+++ b/thirdparty/patches/rocksdb-gcc15-part2.patch
@@ -0,0 +1,35 @@
+From 58420b7c60c4541ea5d91bb895cc7f9d535ebff9 Mon Sep 17 00:00:00 2001
+From: Jiffin Tony Thottan <th...@gmail.com>
+Date: Fri, 13 Jun 2025 09:47:52 -0700
+Subject: [PATCH] include cstdint to trace_record.h (#13651)
+
+Summary:
+There are compilation errors on gcc 15 in fedora 42 while compiling ceph.
+
+This is similar to PR https://github.com/facebook/rocksdb/issues/13573.
+
+Pull Request resolved: https://github.com/facebook/rocksdb/pull/13651
+
+Reviewed By: jaykorean
+
+Differential Revision: D76062855
+
+Pulled By: cbi42
+
+fbshipit-source-id: d213debbda39fdfac01641daa567687fc104d260
+---
+ include/rocksdb/trace_record.h | 1 +
+ 1 file changed, 1 insertion(+)
+
+diff --git a/include/rocksdb/trace_record.h b/include/rocksdb/trace_record.h
+index 8f9c3ee2f0f5..d321f538745d 100644
+--- a/include/rocksdb/trace_record.h
++++ b/include/rocksdb/trace_record.h
+@@ -5,6 +5,7 @@
+ 
+ #pragma once
+ 
++#include <cstdint>
+ #include <memory>
+ #include <string>
+ #include <vector>
diff --git a/thirdparty/patches/rocksdb-gcc15-part3.patch b/thirdparty/patches/rocksdb-gcc15-part3.patch
new file mode 100644
index 000000000..84a09f14f
--- /dev/null
+++ b/thirdparty/patches/rocksdb-gcc15-part3.patch
@@ -0,0 +1,25 @@
+From b0b89f2dfc4524858c3837a579fe895e671c32c9 Mon Sep 17 00:00:00 2001
+From: Joe McDonnell <jo...@cloudera.com>
+Date: Wed, 26 Aug 2026 23:02:49 -0700
+Subject: [PATCH] Fix GCC15 compilation by adding cstdint include to
+ checkpoint.h
+
+This is a manual patch, because there isn't an upstream patch
+containing only this fix. In upstream, the cstdint header came
+in via a patch containing other functionality.
+---
+ include/rocksdb/utilities/checkpoint.h | 1 +
+ 1 file changed, 1 insertion(+)
+
+diff --git a/include/rocksdb/utilities/checkpoint.h b/include/rocksdb/utilities/checkpoint.h
+index 7fb9d489a..f7b4e6f03 100644
+--- a/include/rocksdb/utilities/checkpoint.h
++++ b/include/rocksdb/utilities/checkpoint.h
+@@ -8,6 +8,7 @@
+ #pragma once
+ #ifndef ROCKSDB_LITE
+ 
++#include <cstdint>
+ #include <string>
+ #include <vector>
+ #include "rocksdb/status.h"
diff --git a/thirdparty/patches/yaml-fix-missing-cstdint-for-GCC15.patch b/thirdparty/patches/yaml-fix-missing-cstdint-for-GCC15.patch
new file mode 100644
index 000000000..137f5f0e4
--- /dev/null
+++ b/thirdparty/patches/yaml-fix-missing-cstdint-for-GCC15.patch
@@ -0,0 +1,38 @@
+From 7b469b4220f96fb3d036cf68cd7bd30bd39e61d2 Mon Sep 17 00:00:00 2001
+From: Christopher Fore <cs...@posteo.net>
+Date: Wed, 14 Aug 2024 21:02:32 -0400
+Subject: [PATCH] emitterutils: Explicitly include <cstdint>
+
+GCC 15 will no longer include it by default, resulting in build
+failures in projects that do not explicitly include it.
+
+Error:
+src/emitterutils.cpp:221:11: error: 'uint16_t' was not declared in this scope
+  221 | std::pair<uint16_t, uint16_t> EncodeUTF16SurrogatePair(int codePoint) {
+      |           ^~~~~~~~
+src/emitterutils.cpp:13:1: note: 'uint16_t' is defined in header '<cstdint>';
+this is probably fixable by adding '#include <cstdint>'
+   12 | #include "yaml-cpp/null.h"
+  +++ |+#include <cstdint>
+   13 | #include "yaml-cpp/ostream_wrapper.h"
+
+Tests pass.
+
+Closes: #1307
+See-also: https://gcc.gnu.org/pipermail/gcc-cvs/2024-August/407124.html
+See-also: https://bugs.gentoo.org/937412
+Signed-off-by: Christopher Fore <cs...@posteo.net>
+---
+ src/emitterutils.cpp | 1 +
+ 1 file changed, 1 insertion(+)
+
+diff --git a/src/emitterutils.cpp b/src/emitterutils.cpp
+index fc41011a5..f801b1d0c 100644
+--- a/src/emitterutils.cpp
++++ b/src/emitterutils.cpp
+@@ -1,4 +1,5 @@
+ #include <algorithm>
++#include <cstdint>
+ #include <iomanip>
+ #include <sstream>
+ 
diff --git a/thirdparty/vars.sh b/thirdparty/vars.sh
index 11f674a9d..63d46701a 100644
--- a/thirdparty/vars.sh
+++ b/thirdparty/vars.sh
@@ -386,7 +386,7 @@ LIBUNWIND_PATCHES=(
 # The include-what-you-use is built along with LLVM in its source tree.
 IWYU_VERSION=0.15
 LLVM_VERSION=11.0.0
-LLVM_PATCHLEVEL=11
+LLVM_PATCHLEVEL=12
 LLVM_NAME=llvm-$LLVM_VERSION.src
 LLVM_SOURCE=$TP_SOURCE_DIR/$LLVM_NAME
 LLVM_ARCHIVE=llvm-$LLVM_VERSION-iwyu-$IWYU_VERSION.src.tar.gz
@@ -408,6 +408,7 @@ LLVM_PATCHES=(
  "patch -p1 < $TP_DIR/patches/llvm-chrono-duration-01.patch"
  "patch -p1 < $TP_DIR/patches/llvm-section-mm-memory-mapper.patch"
  "patch -p1 < $TP_DIR/patches/llvm-section-mm-extra-methods.patch"
+ "patch -p2 < $TP_DIR/patches/llvm-gcc15-fix-missing-cstdint-include.patch"
 )
 
 # All libcxxabi's variables are pointing to LLVM's: libcxxabi is built
@@ -554,10 +555,13 @@ HADOOP_ANY_ARCH=1
 HADOOP_ANY_OS=1
 
 YAML_VERSION=0.8.0
-YAML_PATCHLEVEL=0
+YAML_PATCHLEVEL=1
 YAML_NAME=yaml-cpp-yaml-cpp-$YAML_VERSION
 YAML_SOURCE=$TP_SOURCE_DIR/$YAML_NAME
 YAML_ARCHIVE=$YAML_NAME.tar.gz
+YAML_PATCHES=(
+ "patch -p1 < $TP_DIR/patches/yaml-fix-missing-cstdint-for-GCC15.patch"
+)
 
 CHRONY_VERSION=4.6.1
 CHRONY_PATCHLEVEL=1
@@ -655,12 +659,15 @@ JWT_CPP_ANY_ARCH=1
 JWT_CPP_ANY_OS=1
 
 ROCKSDB_VERSION=7.7.3
-ROCKSDB_PATCHLEVEL=1
+ROCKSDB_PATCHLEVEL=2
 ROCKSDB_NAME=rocksdb-$ROCKSDB_VERSION
 ROCKSDB_SOURCE=$TP_SOURCE_DIR/$ROCKSDB_NAME
 ROCKSDB_ARCHIVE=$ROCKSDB_NAME.tar.gz
 ROCKSDB_PATCHES=(
  "patch -p1 < $TP_DIR/patches/rocksdb-gcc13.patch"
+ "patch -p1 < $TP_DIR/patches/rocksdb-gcc15-part1.patch"
+ "patch -p1 < $TP_DIR/patches/rocksdb-gcc15-part2.patch"
+ "patch -p1 < $TP_DIR/patches/rocksdb-gcc15-part3.patch"
 )
 
 # Prometheus is downloaded as a prebuilt binary from the S3 bucket.