You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by vi...@apache.org on 2026/07/13 03:56:49 UTC
(hudi) branch asf-site updated: docs: SEO/AEO foundation: structured data, llms.txt, vendors page, and top-post retrofits (#19268)
This is an automated email from the ASF dual-hosted git repository.
vinothchandar pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 2e9aad57e693 docs: SEO/AEO foundation: structured data, llms.txt, vendors page, and top-post retrofits (#19268)
2e9aad57e693 is described below
commit 2e9aad57e693db568874d3b8a98890f5ac661758
Author: vinoth chandar <vi...@users.noreply.github.com>
AuthorDate: Sun Jul 12 20:56:42 2026 -0700
docs: SEO/AEO foundation: structured data, llms.txt, vendors page, and top-post retrofits (#19268)
* docs(site): add robots.txt and llms.txt
* docs(site): AEO infrastructure: last-update dates, sitemap lastmod, FAQ structured data
* chore: add upsert to blog tag allowlist
* docs(site): add vendors page
* docs(learn): reposition general FAQ around Hudi differentiators
* docs: refresh migration guide with XTable
* docs(blog): AEO retrofit of top posts: answer-first intros, FAQ, last_update
* docs(blog): mirror excerpt into description front matter for SEO
* docs: address review feedback: stale defaults, deprecated configs, empty descriptions, XTable phrasing
---
.github/scripts/validate-blog.py | 2 +-
.github/workflows/asf-site.ci.yml | 8 ++-
.../blog/2019-05-14-registering-dataset-to-hive.md | 1 +
.../blog/2019-09-09-ingesting-database-changes.md | 1 +
website/blog/2020-01-15-delete-support-in-hudi.md | 1 +
.../blog/2020-01-20-change-capture-using-aws.md | 1 +
website/blog/2020-03-22-exporting-hudi-datasets.md | 1 +
.../blog/2020-04-27-apache-hudi-apache-zepplin.md | 1 +
...0-05-28-monitoring-hudi-metrics-with-datadog.md | 1 +
...18-hudi-incremental-processing-on-data-lakes.md | 20 +++++-
...-efficient-migration-of-large-parquet-tables.md | 3 +
...2020-08-21-async-compaction-deployment-model.md | 1 +
...2020-08-22-ingest-multiple-tables-using-hudi.md | 1 +
...020-10-06-cdc-solution-using-hudi-by-nclouds.md | 1 +
.../2020-10-15-apache-hudi-meets-apache-flink.md | 1 +
.../2020-10-19-hudi-meets-aws-emr-and-aws-dms.md | 1 +
.../blog/2020-11-11-hudi-indexing-mechanisms.md | 23 ++++++-
...gh-perf-data-lake-with-hudi-and-alluxio-t3go.md | 1 +
website/blog/2021-01-27-hudi-clustering-intro.md | 1 +
website/blog/2021-02-13-hudi-key-generators.md | 22 +++++-
website/blog/2021-03-01-hudi-file-sizing.md | 1 +
...loying-right-configurations-for-hudi-cleaner.md | 1 +
.../2021-07-21-streaming-data-lake-platform.md | 15 ++++-
.../blog/2021-08-16-kafka-custom-deserializer.md | 1 +
.../blog/2021-08-18-improving-marker-mechanism.md | 1 +
website/blog/2021-08-18-virtual-keys.md | 1 +
website/blog/2021-08-23-async-clustering.md | 22 ++++--
website/blog/2021-08-23-s3-events-source.md | 1 +
...g-eb-level-data-lake-using-hudi-at-bytedance.md | 1 +
...e-concurrency-control-are-we-too-optimistic.mdx | 19 +++++-
...hudi-zorder-and-hilbert-space-filling-curves.md | 1 +
...022-01-06-apache-hudi-2021-a-year-in-review.mdx | 1 +
...-data-capture-with-debezium-and-apache-hudi.mdx | 1 +
...ild-open-lakehouse-using-apache-hudi-and-dbt.md | 3 +-
...irst-Hudi-Lakehouse-with-AWS-Glue-and-AWS-S3.md | 1 +
...022-12-29-Apache-Hudi-2022-A-Year-In-Review.mdx | 1 +
...loring-various-storage-types-in-Apache-Hudi.mdx | 1 +
...25-Delta-Hudi-Iceberg-Which-is-most-popular.mdx | 1 +
.../2023-08-28-Apache-Hudi-From-Zero-To-One.mdx | 1 +
...-Delta-Hudi-Iceberg-A-Benchmark-Compilation.mdx | 1 +
...l-Queries-with-Apache-Hudi-and-Apache-Flink.mdx | 1 +
...3-09-06-Apache-Hudi-From-Zero-To-One-blog-2.mdx | 1 +
...23-09-06-Lakehouse-or-Warehouse-Part-1-of-2.mdx | 1 +
...udi-Understanding-Read-and-Write-Operations.mdx | 1 +
...23-09-12-Lakehouse-or-Warehouse-Part-2-of-2.mdx | 1 +
...n-data-lakes-using-AWS-Glue-and-Apache-Hudi.mdx | 1 +
...3-09-15-Apache-Hudi-From-Zero-To-One-blog-3.mdx | 1 +
...f-Apache-Iceberg-Delta-Lake-and-Apache-Hudi.mdx | 1 +
...3-09-27-Apache-Hudi-From-Zero-To-One-blog-4.mdx | 1 +
...3-10-06-Apache-Hudi-Copy-on-Write-CoW-Table.mdx | 1 +
...y-performance-with-apache-hudi-and-onehouse.mdx | 1 +
...-by-implementing-key-design-concepts-Part-1.mdx | 1 +
...3-10-18-Apache-Hudi-From-Zero-To-One-blog-5.mdx | 1 +
...transactional-data-lakes-to-data-warehouses.mdx | 1 +
...0-Its-Time-for-the-Universal-Data-Lakehouse.mdx | 1 +
...el-Index-Global-Bloom-Global-Simple-Indexes.mdx | 1 +
website/blog/2023-11-01-record-level-index.md | 22 ++++++
...3-11-13-Apache-Hudi-From-Zero-To-One-blog-6.mdx | 1 +
...n-Guide-Local-Ingestion-from-Parquet-Source.mdx | 1 +
...-Apache-Hudi-support-with-AWS-Glue-crawlers.mdx | 1 +
...istry-and-DeltaStreamer-Guide-for-Begineers.mdx | 1 +
...-Apache-Hudi-Part-1-History-Getting-Started.mdx | 1 +
...eep-Dive-into-MINIO-Hudi-and-Delta-Streamer.mdx | 1 +
...2023-12-01-Getting-started-with-Apache-Hudi.mdx | 1 +
...3-12-06-Apache-Hudi-From-Zero-To-One-blog-7.mdx | 1 +
...2023-12-09-Getting-started-with-Apache-Hudi.mdx | 1 +
website/blog/2023-12-13-what-is-apache-hudi.mdx | 1 +
...023-12-28-apache-hudi-2023-a-year-in-review.mdx | 1 +
...-Level-Index-with-FastAPI-and-Spark-Connect.mdx | 1 +
...h-Apache-Doris-Apache-Flink-and-Apache-Hudi.mdx | 1 +
...4-01-05-Apache-Hudi-From-Zero-To-One-blog-8.mdx | 1 +
.../2024-01-05-Small-Talk-about-Apache-Hudi.mdx | 1 +
.../2024-01-09-introduction-to-apache-hudi.mdx | 1 +
...e-Data-Lake-with-CDC-Processing-Hudi-Docker.mdx | 1 +
...azon-EMR-integrated-with-AWS-Lake-Formation.mdx | 1 +
...mer-in-UPSERT-Mode-with-Kafka-Avro-Messages.mdx | 1 +
...ng-Bootstrapping-Data-lake-with-Apache-Hudi.mdx | 1 +
...a-From-MongoDB-to-Apache-Hudi-Using-PySpark.mdx | 1 +
...our-open-source-transactional-table-formats.mdx | 1 +
...-Optimize-Data-Retrieval-Costs-on-the-cloud.mdx | 1 +
...anaging-Partition-on-a-petabyte-scale-table.mdx | 1 +
...Postgres-Hive-Metastore-Minio-and-StarRocks.mdx | 1 +
...-lakehouse-through-close-team-collaboration.mdx | 1 +
...-Debezium-Apicurio-Registry-and-Apache-Hudi.mdx | 1 +
...4-03-05-Apache-Hudi-From-Zero-To-One-blog-9.mdx | 1 +
...-04-13-Apache-Hudi-From-Zero-To-One-blog-10.mdx | 1 +
.../blog/2024-07-11-what-is-a-data-lakehouse.md | 15 ++++-
website/blog/2024-07-30-data-lake-cdc.md | 15 +++++
website/blog/2024-07-31-hudi-file-formats.md | 1 +
.../2024-11-19-automated-small-file-handling.md | 1 +
.../2024-12-06-non-blocking-concurrency-control.md | 21 +++++-
website/blog/2024-12-16-announcing-hudi-1-0-0.mdx | 1 +
...024-12-29-apache-hudi-2024-a-year-in-review.mdx | 1 +
.../2025-01-15-outofbox-key-generators-in-hudi.mdx | 1 +
website/blog/2025-01-28-concurrency-control.md | 21 +++++-
website/blog/2025-03-03-record-mergers-in-hudi.mdx | 1 +
.../2025-03-05-hudi-21-unique-differentiators.mdx | 15 ++++-
website/blog/2025-03-31-amazon-hudi.md | 1 +
website/blog/2025-04-02-secondary-index.md | 20 +++++-
website/blog/2025-05-29-lsm-timeline.md | 1 +
.../blog/2025-06-16-Apache-Hudi-does-XYZ-110.mdx | 1 +
website/blog/2025-06-30-uber-hudi.md | 1 +
...025-07-15-modernizing-datainfra-peloton-hudi.md | 1 +
website/blog/2025-07-21-mor-comparison.md | 1 +
website/blog/2025-09-17-hudi-auto-gen-keys.mdx | 1 +
...-Time-Cloud-Security-Graphs-Hudi+PuppyGraph.mdx | 1 +
...form-with-Apache-Hudi-DBT-and-EMR-Serverless.md | 1 +
...ion_Stats_Enhancing_Column_Stats_in_Hudi_1.0.md | 1 +
...ve-into-hudis-indexing-subsystem-part-1-of-2.md | 1 +
...ses-apache-hudi-to-power-its-data-lakehouse.mdx | 1 +
...ve-into-hudis-indexing-subsystem-part-2-of-2.md | 1 +
...5-11-25-apache-hudi-release-1-1-announcement.md | 1 +
...-12-01-apache-hudi-JD-meetup-asia-2025-recap.md | 1 +
...ve-optimizing-streaming-ingestion-with-flink.md | 15 +++++
.../blog/2025-12-16-maximizing-throughput-nbcc.md | 1 +
...how-zupee-cut-s3-costs-60-percent-with-hudi.mdx | 1 +
...025-12-29-apache-hudi-2025-a-year-in-review.mdx | 1 +
...09-hudi-11-deep-dive-flink-async-instant-gen.md | 1 +
website/blog/2026-01-15-funding-circle.mdx | 1 +
...2026-01-22-apache-hudi-at-applied-intuition.mdx | 1 +
...age-optimization-its-an-architectural-shift.mdx | 1 +
...05-why-metadata-has-to-be-mutation-friendly.mdx | 1 +
...-06-07-apache-hudi-release-1-2-announcement.mdx | 1 +
...ts-for-flink-streaming-in-apache-hudi-1-2-0.mdx | 1 +
website/blog/2026-06-15-apache-hudi-at-metica.mdx | 1 +
.../2026-06-15-apache-hudi-at-penn-interactive.mdx | 1 +
...026-06-15-apache-hudi-at-southwest-airlines.mdx | 1 +
...6-06-25-building-indexes-on-a-moving-target.mdx | 1 +
website/docs/migration_guide.md | 11 ++-
website/docusaurus.config.js | 20 ++++++
website/plugins/remark-faq-structured-data.js | 75 +++++++++++++++++++++
website/src/components/PostFAQ/index.js | 57 ++++++++++++++++
website/src/pages/faq/general.md | 20 +++++-
website/src/pages/vendors.md | 51 ++++++++++++++
website/src/theme/MDXComponents.js | 10 +++
website/static/llms.txt | 78 ++++++++++++++++++++++
website/static/robots.txt | 4 ++
137 files changed, 691 insertions(+), 26 deletions(-)
diff --git a/.github/scripts/validate-blog.py b/.github/scripts/validate-blog.py
index 63cee6129d2e..8f15cbb08d5b 100644
--- a/.github/scripts/validate-blog.py
+++ b/.github/scripts/validate-blog.py
@@ -49,7 +49,7 @@ ALLOWED_TAGS = {
'bytearray', 'best practices', 'backfilling', 'architecture',
'apicurio registry', 'apache zeppelin', 'apache orc', 'apache dolphinscheduler',
'apache avro', 'apache', 'access control', 'lakehouse', 'merge on read', 'record level index','rli', 'penn interactive', 'southwest airlines', 'metica',
- 'async indexing',
+ 'async indexing', 'upsert',
}
# Tags that should not be used
diff --git a/.github/workflows/asf-site.ci.yml b/.github/workflows/asf-site.ci.yml
index d880a1a24c2c..e617a998210c 100644
--- a/.github/workflows/asf-site.ci.yml
+++ b/.github/workflows/asf-site.ci.yml
@@ -21,9 +21,11 @@ jobs:
- uses: actions/checkout@v5
with:
token: ${{ secrets.GITHUB_TOKEN }}
- # For PR: need enough commits for diff (100 should cover most PRs)
- # For push: only need latest commit
- fetch-depth: ${{ github.event_name == 'pull_request' && 100 || 1 }}
+ # Full history (blobless) so git-derived last-update timestamps are
+ # real: a shallow clone makes Docusaurus stamp every page with the
+ # deploy date in sitemap lastmod and BlogPosting dateModified
+ fetch-depth: 0
+ filter: blob:none
# Skip tags - this saves significant time
fetch-tags: false
- name: Check for files in content/
diff --git a/website/blog/2019-05-14-registering-dataset-to-hive.md b/website/blog/2019-05-14-registering-dataset-to-hive.md
index 70f259f45188..8e63b3fdba6d 100644
--- a/website/blog/2019-05-14-registering-dataset-to-hive.md
+++ b/website/blog/2019-05-14-registering-dataset-to-hive.md
@@ -1,6 +1,7 @@
---
title: "Registering sample dataset to Hive via beeline"
excerpt: "How to manually register HUDI dataset into Hive using beeline"
+description: "How to manually register HUDI dataset into Hive using beeline"
authors: [vinoth-chandar]
category: how-to
tags:
diff --git a/website/blog/2019-09-09-ingesting-database-changes.md b/website/blog/2019-09-09-ingesting-database-changes.md
index 374c90d7c4ce..05389bda9c26 100644
--- a/website/blog/2019-09-09-ingesting-database-changes.md
+++ b/website/blog/2019-09-09-ingesting-database-changes.md
@@ -1,6 +1,7 @@
---
title: "Ingesting Database changes via Sqoop/Hudi"
excerpt: "Learn how to ingesting changes from a HUDI dataset using Sqoop/Hudi"
+description: "Learn how to ingest database changes into a Hudi dataset using Sqoop and Hudi"
authors: [vinoth-chandar]
category: how-to
tags:
diff --git a/website/blog/2020-01-15-delete-support-in-hudi.md b/website/blog/2020-01-15-delete-support-in-hudi.md
index c4226d369dba..00ef7546ba2d 100644
--- a/website/blog/2020-01-15-delete-support-in-hudi.md
+++ b/website/blog/2020-01-15-delete-support-in-hudi.md
@@ -1,6 +1,7 @@
---
title: "Delete support in Hudi"
excerpt: "Deletes are supported at a record level in Hudi with 0.5.1 release. This blog is a “how to” blog on how to delete records in hudi."
+description: "Deletes are supported at a record level in Hudi with 0.5.1 release. This blog is a “how to” blog on how to delete records in hudi."
authors: [sivabalan]
category: how-to
tags:
diff --git a/website/blog/2020-01-20-change-capture-using-aws.md b/website/blog/2020-01-20-change-capture-using-aws.md
index 7b6e47560f8d..485dc6d6e4a0 100644
--- a/website/blog/2020-01-20-change-capture-using-aws.md
+++ b/website/blog/2020-01-20-change-capture-using-aws.md
@@ -1,6 +1,7 @@
---
title: "Change Capture Using AWS Database Migration Service and Hudi"
excerpt: "In this blog, we will build an end-end solution for capturing changes from a MySQL instance running on AWS RDS to a Hudi table on S3, using capabilities in the Hudi 0.5.1 release."
+description: "In this blog, we will build an end-end solution for capturing changes from a MySQL instance running on AWS RDS to a Hudi table on S3, using capabilities in the Hudi 0.5.1 release."
authors: [vinoth-chandar]
category: how-to
image: /assets/images/blog/change-capture-architecture.png
diff --git a/website/blog/2020-03-22-exporting-hudi-datasets.md b/website/blog/2020-03-22-exporting-hudi-datasets.md
index 1aec96c911cc..e232d21a10fc 100644
--- a/website/blog/2020-03-22-exporting-hudi-datasets.md
+++ b/website/blog/2020-03-22-exporting-hudi-datasets.md
@@ -1,6 +1,7 @@
---
title: "Export Hudi datasets as a copy or as different formats"
excerpt: "Learn how to copy or export HUDI dataset in various formats."
+description: "Learn how to copy or export HUDI dataset in various formats."
authors: [xushiyan]
category: how-to
tags:
diff --git a/website/blog/2020-04-27-apache-hudi-apache-zepplin.md b/website/blog/2020-04-27-apache-hudi-apache-zepplin.md
index 78fa3f0fcb27..42d703680dd7 100644
--- a/website/blog/2020-04-27-apache-hudi-apache-zepplin.md
+++ b/website/blog/2020-04-27-apache-hudi-apache-zepplin.md
@@ -1,6 +1,7 @@
---
title: "Apache Hudi Support on Apache Zeppelin"
excerpt: "Integrating HUDI's real-time and read-optimized query capabilities into Apache Zeppelin’s notebook"
+description: "Integrating HUDI's real-time and read-optimized query capabilities into Apache Zeppelin’s notebook"
author: leesf
category: how-to
tags:
diff --git a/website/blog/2020-05-28-monitoring-hudi-metrics-with-datadog.md b/website/blog/2020-05-28-monitoring-hudi-metrics-with-datadog.md
index 4fb6e37a0f06..9a50be883fde 100644
--- a/website/blog/2020-05-28-monitoring-hudi-metrics-with-datadog.md
+++ b/website/blog/2020-05-28-monitoring-hudi-metrics-with-datadog.md
@@ -1,6 +1,7 @@
---
title: "Monitor Hudi metrics with Datadog"
excerpt: "Introducing the feature of reporting Hudi metrics via Datadog HTTP API"
+description: "Introducing the feature of reporting Hudi metrics via Datadog HTTP API"
authors: [xushiyan]
category: how-to
tags:
diff --git a/website/blog/2020-08-18-hudi-incremental-processing-on-data-lakes.md b/website/blog/2020-08-18-hudi-incremental-processing-on-data-lakes.md
index 3c146887682a..a39e3f612db1 100644
--- a/website/blog/2020-08-18-hudi-incremental-processing-on-data-lakes.md
+++ b/website/blog/2020-08-18-hudi-incremental-processing-on-data-lakes.md
@@ -1,20 +1,24 @@
---
title: "Incremental Processing on the Data Lake"
excerpt: "How Apache Hudi provides ability for incremental data processing."
+description: "How Apache Hudi provides ability for incremental data processing."
author: vinoyang
category: deep-dive
image: /assets/images/blog/incr-processing/image7.png
+last_update:
+ date: 2026-07-06
tags:
- data lakehouse
- incremental processing
---
-### NOTE: This article is a translation of the infoq.cn article, found [here](https://www.infoq.cn/article/CAgIDpfJBVcJHKJLSbhe), with minor edits
-
-Apache Hudi is a data lake framework which provides the ability to ingest, manage and query large analytical data sets on a distributed file system/cloud stores.
+Incremental processing on a data lake is the ability to apply changes to tables at the record level (upserts) and let downstream consumers query only the records that changed since a point in time, instead of rewriting and rescanning entire partitions on every run. Apache Hudi is a data lake framework which provides exactly this ability to ingest, manage and query large analytical data sets on a distributed file system/cloud stores.
Hudi joined the Apache incubator for incubation in January 2019, and was promoted to the top Apache project in May 2020. This article mainly discusses the importance
of Hudi to the data lake from the perspective of "incremental processing". More information about Apache Hudi's framework functions, features, usage scenarios, and
latest developments can be found at [QCon Global Software Development Conference (Shanghai Station) 2020](https://qconplus.infoq.cn/2020/shanghai/presentation/2646).
+
+### NOTE: This article is a translation of the infoq.cn article, found [here](https://www.infoq.cn/article/CAgIDpfJBVcJHKJLSbhe), with minor edits
+
<!--truncate-->
Throughout the development of big data technology, Hadoop has steadily seized the opportunities of this era and has become the de-facto standard for enterprises to build big data infrastructure.
Among them, the distributed file system HDFS that supports the Hadoop ecosystem almost naturally has become the standard interface for big data storage systems. In recent years, with the rise of
@@ -277,3 +281,13 @@ In the data warehouse, some "pain points" caused by the lack of incremental proc
aspects: incremental processing of semantics of natural fit flow, the need for analytical scenarios, quasi-real-time scene resource/efficiency trade-offs, and unified lake architecture.
Finally, we introduced the open source data lake storage framework Apache Hudi's support for incremental processing and simple cases.
+
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {question: 'What is incremental processing on a data lake?', answer: 'Incremental processing is the ability to incorporate changes into existing data lake tables at the record level and to give downstream consumers a way to obtain only the data that changed since a point in time. It replaces the traditional pattern of rewriting entire immutable partitions and re-triggering full recomputation of every cascading task that consumed them.'},
+ {question: 'What are the two primitives needed for incremental processing?', answer: 'The first is upsert, the ability to update or insert records instead of rewriting whole partitions. The second is incremental consumption, a mechanism for downstream consumers to efficiently fetch just the records that changed since the last time they consumed the table. Together they enable end-to-end incremental pipelines on DFS or cloud storage.'},
+ {question: 'How does incremental processing improve efficiency over pure stream processing?', answer: 'In quasi-real-time scenarios, incremental batches let you use short-lived containers and merge accumulated changes in memory before updating the result store once, instead of keeping containers running continuously and issuing millions of fine-grained updates. This yields several times better CPU efficiency and orders of magnitude fewer writes to the result storage, because resources [...]
+ {question: 'How does Apache Hudi support incremental processing?', answer: 'Hudi provides update and delete support using fine-grained file and record level indexes with transactional guarantees on writes, and it provides a first-class change stream so you can query all records inserted, updated or deleted in a table from a given point in time. This incremental view can be queried from Spark to build incremental data pipelines.'},
+ {question: 'How does incremental processing simplify Lambda and Kappa architectures?', answer: 'With incremental processing, the same SQL or Spark logic can run incrementally on new data for fast quasi-real-time views and periodically on the full data to correct results, all on one infrastructure, unifying the Lambda architecture\'s speed and batch layers. In a Kappa-style setup, fast upserts on the lake can make a separate speed serving layer unnecessary, reducing overall system compl [...]
+]} />
diff --git a/website/blog/2020-08-20-efficient-migration-of-large-parquet-tables.md b/website/blog/2020-08-20-efficient-migration-of-large-parquet-tables.md
index d07fb2f89d72..99fa067e1176 100644
--- a/website/blog/2020-08-20-efficient-migration-of-large-parquet-tables.md
+++ b/website/blog/2020-08-20-efficient-migration-of-large-parquet-tables.md
@@ -1,12 +1,15 @@
---
title: "Efficient Migration of Large Parquet Tables to Apache Hudi"
excerpt: "Migrating a large parquet table to Apache Hudi without having to rewrite the entire dataset."
+description: "Migrating a large parquet table to Apache Hudi without having to rewrite the entire dataset."
author: vbalaji
category: how-to
image: /assets/images/blog/2020-08-20-skeleton.png
tags:
- migration
- bootstrap
+last_update:
+ date: 2026-07-06
---
We will look at how to migrate a large parquet table to Hudi without having to rewrite the entire dataset.
diff --git a/website/blog/2020-08-21-async-compaction-deployment-model.md b/website/blog/2020-08-21-async-compaction-deployment-model.md
index 9eeecd7703ac..6e93bef9058d 100644
--- a/website/blog/2020-08-21-async-compaction-deployment-model.md
+++ b/website/blog/2020-08-21-async-compaction-deployment-model.md
@@ -1,6 +1,7 @@
---
title: "Async Compaction Deployment Models"
excerpt: "Mechanisms for executing compaction jobs in Hudi asynchronously"
+description: "Mechanisms for executing compaction jobs in Hudi asynchronously"
author: vbalaji
category: how-to
tags:
diff --git a/website/blog/2020-08-22-ingest-multiple-tables-using-hudi.md b/website/blog/2020-08-22-ingest-multiple-tables-using-hudi.md
index 5ae3914c6a64..899159799683 100644
--- a/website/blog/2020-08-22-ingest-multiple-tables-using-hudi.md
+++ b/website/blog/2020-08-22-ingest-multiple-tables-using-hudi.md
@@ -1,6 +1,7 @@
---
title: "Ingest multiple tables using Hudi"
excerpt: "Ingesting multiple tables using Hudi at a single go is now possible. This blog gives a detailed explanation of how to achieve the same using `HoodieMultiTableDeltaStreamer.java`"
+description: "Ingesting multiple tables using Hudi at a single go is now possible. This blog gives a detailed explanation of how to achieve the same using `HoodieMultiTableDeltaStreamer.java`"
author: pratyakshsharma
category: how-to
tags:
diff --git a/website/blog/2020-10-06-cdc-solution-using-hudi-by-nclouds.md b/website/blog/2020-10-06-cdc-solution-using-hudi-by-nclouds.md
index c741218d2135..82b856014ca9 100644
--- a/website/blog/2020-10-06-cdc-solution-using-hudi-by-nclouds.md
+++ b/website/blog/2020-10-06-cdc-solution-using-hudi-by-nclouds.md
@@ -1,6 +1,7 @@
---
title: "How nClouds Helps Accelerate Data Delivery with Apache Hudi on Amazon EMR"
excerpt: "Solution to set up a new data and analytics platform using Apache Hudi on Amazon EMR and other managed services, including Amazon QuickSight for data visualization."
+description: "Solution to set up a new data and analytics platform using Apache Hudi on Amazon EMR and other managed services, including Amazon QuickSight for data visualization."
author: nclouds
category: case-study
image: /assets/images/blog/2020-10-06-cdc-solution-using-hudi-by-nclouds.jpg
diff --git a/website/blog/2020-10-15-apache-hudi-meets-apache-flink.md b/website/blog/2020-10-15-apache-hudi-meets-apache-flink.md
index 303b559f9e9f..5c10843c6cba 100644
--- a/website/blog/2020-10-15-apache-hudi-meets-apache-flink.md
+++ b/website/blog/2020-10-15-apache-hudi-meets-apache-flink.md
@@ -1,6 +1,7 @@
---
title: "Apache Hudi meets Apache Flink"
excerpt: "The design and latest progress of the integration of Apache Hudi and Apache Flink."
+description: "The design and latest progress of the integration of Apache Hudi and Apache Flink."
author: wangxianghu
category: how-to
image: /assets/images/blog/2020-10-15-apache-hudi-meets-apache-flink.png
diff --git a/website/blog/2020-10-19-hudi-meets-aws-emr-and-aws-dms.md b/website/blog/2020-10-19-hudi-meets-aws-emr-and-aws-dms.md
index 820583719602..56abdf758d26 100644
--- a/website/blog/2020-10-19-hudi-meets-aws-emr-and-aws-dms.md
+++ b/website/blog/2020-10-19-hudi-meets-aws-emr-and-aws-dms.md
@@ -1,6 +1,7 @@
---
title: "Apply record level changes from relational databases to Amazon S3 data lake using Apache Hudi on Amazon EMR and AWS Database Migration Service"
excerpt: "AWS blog showing how to build a CDC pipeline that captures data from an Amazon RDS for MySQL database using AWS DMS and applies those changes to an Amazon S3 dataset using Apache Hudi on Amazon EMR."
+description: "AWS blog showing how to build a CDC pipeline that captures data from an Amazon RDS for MySQL database using AWS DMS and applies those changes to an Amazon S3 dataset using Apache Hudi on Amazon EMR."
author: aws
category: how-to
image: /assets/images/blog/2020-10-19-hudi-meets-aws-emr-and-aws-dms.jpeg
diff --git a/website/blog/2020-11-11-hudi-indexing-mechanisms.md b/website/blog/2020-11-11-hudi-indexing-mechanisms.md
index 198a023cc566..0c61292e48ce 100644
--- a/website/blog/2020-11-11-hudi-indexing-mechanisms.md
+++ b/website/blog/2020-11-11-hudi-indexing-mechanisms.md
@@ -1,22 +1,25 @@
---
title: "Employing the right indexes for fast updates, deletes in Apache Hudi"
excerpt: "Detailing different indexing mechanisms in Hudi and when to use each of them"
+description: "Detailing different indexing mechanisms in Hudi and when to use each of them"
authors: [vinoth-chandar]
category: how-to
image: /assets/images/blog/hudi-indexes/with-and-without-index.png
+last_update:
+ date: 2026-07-06
tags:
- indexing
---
-Apache Hudi employs an index to locate the file group, that an update/delete belongs to. For Copy-On-Write tables, this enables
+An index in Apache Hudi maps a record key + an optional partition path to the file group ID on storage that the record belongs to, letting writers
+route updates and deletes directly to the right files instead of joining against the entire dataset. For Copy-On-Write tables, this enables
fast upsert/delete operations, by avoiding the need to join against the entire dataset to determine which files to rewrite.
For Merge-On-Read tables, this design allows Hudi to bound the amount of records any given base file needs to be merged against.
Specifically, a given base file needs to merged only against updates for records that are part of that base file. In contrast,
designs without an indexing component (e.g: [Apache Hive ACID](https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions)),
could end up having to merge all the base files against all incoming updates/delete records.
<!--truncate-->
-At a high level, an index maps a record key + an optional partition path to a file group ID on storage (explained
-more in detail [here](/docs/concepts)) and during write operations, we lookup this mapping to route an incoming update/delete
+During write operations, we lookup this key-to-file-group mapping (explained more in detail [here](/docs/concepts)) to route an incoming update/delete
to a log file attached to the base file (MOR) or to the latest base file that now needs to be merged against (COW). The index also enables
Hudi to enforce unique constraints based on the record keys.
@@ -27,6 +30,10 @@ Given that Hudi already supports few different indexing techniques and is also c
attempts to explain different categories of workloads, from our experience and suggests what index types to use for each. We will also interlace
commentary on existing limitations, upcoming work and optimizations/tradeoffs along the way.
+:::info Newer content available
+This 2020 post predates Hudi 1.x. For the modern (1.x) indexing subsystem, see [Deep Dive Into Hudi's Indexing Subsystem Part 1](/blog/2025/10/29/deep-dive-into-hudis-indexing-subsystem-part-1-of-2) and [Part 2](/blog/2025/11/12/deep-dive-into-hudis-indexing-subsystem-part-2-of-2).
+:::
+
## Index Types in Hudi
Currently, Hudi supports the following indexing options.
@@ -122,6 +129,16 @@ Some interesting work underway in this area:
Going forward, this will remain an area of active investment for the project. we are always looking for contributors who can drive these roadmap items forward.
Please [engage](/community/get-involved) with our community if you want to get involved.
+
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {question: 'What is an index in Apache Hudi?', answer: 'An index in Hudi maps a record key, plus an optional partition path, to the file group on storage that the record belongs to. During writes, Hudi looks up this mapping to route each incoming update or delete to the right base file or log file, avoiding a join against the entire dataset. The index also lets Hudi enforce unique constraints on record keys.'},
+ {question: 'What index types does Apache Hudi support?', answer: 'At the time of this post, Hudi supports the Bloom index (the default), which uses bloom filters and optional key-range pruning; the Simple index, which performs a lean join of incoming records against keys extracted from storage; and the HBase index, which stores the mapping in an external Apache HBase table. A custom index implementation can also be plugged in via the hoodie.index.class config.'},
+ {question: 'What is the difference between global and non-global indexes in Hudi?', answer: 'A global index enforces key uniqueness across all partitions of the table, but its update and delete cost grows with the size of the table. A non-global index enforces uniqueness only within a partition and relies on the writer to supply a consistent partition path, but the lookup cost scales with the number of records updated or deleted, so it performs much better at high write volumes.'},
+ {question: 'Which Hudi index works best for late-arriving updates to fact tables?', answer: 'The Bloom index performs well for fact-table workloads, since the lookup prunes a lot of data files using well-sized bloom filters. If keys are constructed with an ordering, such as a timestamp prefix, range pruning cuts the number of files compared even further. Dynamic bloom filters can be enabled to keep the false positive ratio in check as file sizes vary.'},
+ {question: 'Which Hudi index should I use for random updates to dimension tables?', answer: 'For random writes that touch most files in the table, bloom filters return true positives almost everywhere, so the Simple index is a better fit because it directly joins incoming records against the interested fields from every data file. The HBase index gives even faster lookups if its operational overhead is acceptable. With a global index, also consider enabling the update.partition.path co [...]
+]} />
diff --git a/website/blog/2020-12-01-high-perf-data-lake-with-hudi-and-alluxio-t3go.md b/website/blog/2020-12-01-high-perf-data-lake-with-hudi-and-alluxio-t3go.md
index 86cbd5401238..14d91c6daaec 100644
--- a/website/blog/2020-12-01-high-perf-data-lake-with-hudi-and-alluxio-t3go.md
+++ b/website/blog/2020-12-01-high-perf-data-lake-with-hudi-and-alluxio-t3go.md
@@ -1,6 +1,7 @@
---
title: "Building High-Performance Data Lake Using Apache Hudi and Alluxio at T3Go"
excerpt: "How T3Go’s high-performance data lake using Apache Hudi and Alluxio shortened the time for data ingestion into the lake by up to a factor of 2. Data analysts using Presto, Hudi, and Alluxio in conjunction to query data on the lake saw queries speed up by 10 times faster."
+description: "How T3Go’s high-performance data lake using Apache Hudi and Alluxio shortened the time for data ingestion into the lake by up to a factor of 2. Data analysts using Presto, Hudi, and Alluxio in conjunction to query data on the lake saw queries speed up by 10 times faster."
author: t3go
category: case-study
image: /assets/images/blog/2020-12-01-t3go-architecture.png
diff --git a/website/blog/2021-01-27-hudi-clustering-intro.md b/website/blog/2021-01-27-hudi-clustering-intro.md
index 45dfbd0a78fc..98aeac59114e 100644
--- a/website/blog/2021-01-27-hudi-clustering-intro.md
+++ b/website/blog/2021-01-27-hudi-clustering-intro.md
@@ -1,6 +1,7 @@
---
title: "Optimize Data lake layout using Clustering in Apache Hudi"
excerpt: "Introduce clustering feature to change data layout"
+description: "Introduce clustering feature to change data layout"
author: satish.kotha
category: deep-dive
image: /assets/images/blog/2021-01-27-hudi-clustering-intro.png
diff --git a/website/blog/2021-02-13-hudi-key-generators.md b/website/blog/2021-02-13-hudi-key-generators.md
index 431f36f08e00..f4777ad0cc81 100644
--- a/website/blog/2021-02-13-hudi-key-generators.md
+++ b/website/blog/2021-02-13-hudi-key-generators.md
@@ -1,14 +1,18 @@
---
title: "Apache Hudi Key Generators"
excerpt: "Different key generators available with Apache Hudi"
+description: "Different key generators available with Apache Hudi"
authors: [sivabalan]
category: how-to
+last_update:
+ date: 2026-07-06
tags:
- key generation
---
-Every record in Hudi is uniquely identified by a primary key, which is a pair of record key and partition path where
-the record belongs to. Using primary keys, Hudi can impose a) partition level uniqueness integrity constraint
+Key generators in Apache Hudi are pluggable classes that derive each record's primary key - a pair of record key and
+partition path - from the fields of the incoming record. Every record in Hudi is uniquely identified by such a primary key.
+Using primary keys, Hudi can impose a) partition level uniqueness integrity constraint
b) enable fast updates and deletes on records. One should choose the partitioning scheme wisely as it could be a
determining factor for your ingestion and query latency.
<!--truncate-->
@@ -17,6 +21,10 @@ commonly used), each record is uniquely identified by a pair of record key and p
global index, each record is uniquely identified by just the record key. There won't be any duplicate record keys across
partitions.
+:::info Newer content available
+For an updated walkthrough of the key generators shipped with recent Hudi releases, see [Out-of-box key generators in Hudi](/blog/2025/01/15/outofbox-key-generators-in-hudi).
+:::
+
## Key Generators
Hudi provides several key generators out of the box that users can use based on their need, while having a pluggable
@@ -197,3 +205,13 @@ partition for all records. In other words, all records go to the same partition
Hope this blog gave you a good understanding of different types of Key Generators available in Apache Hudi. Thanks for your continued support for Hudi's community.
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {question: 'What is a key generator in Apache Hudi?', answer: 'A key generator is a class that produces the primary key for every incoming record, namely the pair of record key and partition path. Hudi ships several key generators out of the box and also exposes a pluggable KeyGenerator interface so users can implement their own.'},
+ {question: 'Which key generators does Hudi provide out of the box?', answer: 'Hudi provides SimpleKeyGenerator for single-field keys and partition paths, ComplexKeyGenerator for keys or partition paths built from multiple fields, TimestampBasedKeyGenerator for timestamp-based partition paths, CustomKeyGenerator for mixing simple and timestamp partition fields, GlobalDeleteKeyGenerator for deletes with a global index, and NonpartitionedKeyGenerator for unpartitioned datasets.'},
+ {question: 'What configs are required to set up a key generator?', answer: 'Three configs are mandatory: hoodie.datasource.write.recordkey.field for the record key field, hoodie.datasource.write.partitionpath.field for the partition path field, and hoodie.datasource.write.keygenerator.class for the key generator class. Optional configs control URL encoding of partition paths and hive style partitioning, and TimestampBasedKeyGenerator needs a few more configs for timestamp type, formats [...]
+ {question: 'When should I use TimestampBasedKeyGenerator?', answer: 'Use it when your partition field is a timestamp. The field values are interpreted as timestamps rather than converted to strings as is, with configs specifying the timestamp type, input and output date formats, and timezone. The record key is still chosen by field name as usual.'},
+ {question: 'What if my Hudi dataset is not partitioned?', answer: 'Use NonpartitionedKeyGenerator, which returns an empty partition path for all records. In other words, all records go to the same, empty partition.'},
+]} />
+
diff --git a/website/blog/2021-03-01-hudi-file-sizing.md b/website/blog/2021-03-01-hudi-file-sizing.md
index 916d8fcbba3a..4117ac8838f2 100644
--- a/website/blog/2021-03-01-hudi-file-sizing.md
+++ b/website/blog/2021-03-01-hudi-file-sizing.md
@@ -1,6 +1,7 @@
---
title: "Streaming Responsibly - How Apache Hudi maintains optimum sized files"
excerpt: "Maintaining well-sized files can improve query performance significantly"
+description: "Maintaining well-sized files can improve query performance significantly"
authors: [sivabalan]
category: deep-dive
image: /assets/images/blog/2021-03-01-hudi-file-sizing.png
diff --git a/website/blog/2021-06-10-employing-right-configurations-for-hudi-cleaner.md b/website/blog/2021-06-10-employing-right-configurations-for-hudi-cleaner.md
index a70b738499a6..14aadece4534 100644
--- a/website/blog/2021-06-10-employing-right-configurations-for-hudi-cleaner.md
+++ b/website/blog/2021-06-10-employing-right-configurations-for-hudi-cleaner.md
@@ -1,6 +1,7 @@
---
title: "Employing correct configurations for Hudi's cleaner table service"
excerpt: "Ensuring isolation between Hudi writers and readers using `HoodieCleaner.java`"
+description: "Ensuring isolation between Hudi writers and readers using `HoodieCleaner.java`"
author: pratyakshsharma
category: how-to
image: /assets/images/blog/hoodie-cleaner/Initial_timeline.png
diff --git a/website/blog/2021-07-21-streaming-data-lake-platform.md b/website/blog/2021-07-21-streaming-data-lake-platform.md
index 27e4b7b87b59..9db7ac21fb03 100644
--- a/website/blog/2021-07-21-streaming-data-lake-platform.md
+++ b/website/blog/2021-07-21-streaming-data-lake-platform.md
@@ -1,14 +1,17 @@
---
title: "Apache Hudi - The Data Lake Platform"
excerpt: "It's been called many things. But, we have always been building a data lake platform"
+description: "It's been called many things. But, we have always been building a data lake platform"
authors: [vinoth-chandar]
category: deep-dive
image: /assets/images/blog/hudi_streaming.png
+last_update:
+ date: 2026-07-06
tags:
- data lakehouse
---
-As early as 2016, we set out a [bold, new vision](https://www.oreilly.com/content/ubers-case-for-incremental-processing-on-hadoop/) reimagining batch data processing through a new “**incremental**” data processing stack - alongside the existing batch and streaming stacks.
+Apache Hudi is a streaming data lake platform built around a database kernel—not merely a table format or a transactional layer, but a complete software stack with built-in table services, data services, and incremental processing capabilities. As early as 2016, we set out a [bold, new vision](https://www.oreilly.com/content/ubers-case-for-incremental-processing-on-hadoop/) reimagining batch data processing through a new “**incremental**” data processing stack - alongside the existing ba [...]
While a stream processing pipeline does row-oriented processing, delivering a few seconds of processing latency, an incremental pipeline would apply the same principles to *columnar* data in the data lake,
delivering orders of magnitude improvements in processing efficiency within few minutes, on extremely scalable batch storage/compute infrastructure. This new stack would be able to effortlessly support regular batch processing for bulk reprocessing/backfilling as well.
Hudi was built as the manifestation of this vision, rooted in real, hard problems faced at [Uber](https://eng.uber.com/uber-big-data-platform/) and later took a life of its own in the open source community. Together, we have been able to
@@ -152,3 +155,13 @@ There is a fundamental tradeoff today in data lakes between faster writing and g
## Onwards
We hope that this blog painted a complete picture of Apache Hudi, staying true to its founding principles. Interested users and readers can expect blogs delving into each layer of the stack and an overhaul of our docs along these lines in the coming weeks/months. We view the current efforts around table formats as merely removing decade-old bottlenecks in data lake storage/query planes, problems which have been already solved very well in cloud warehouses like Big Query/Snowflake. We wou [...]
+
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {question: 'What is Apache Hudi?', answer: 'Apache Hudi is a streaming data lake platform built around a database kernel. By optimizing for fast upserts and change streams, it provides primitives to data lake workloads comparable to what Apache Kafka does for event streaming, while also providing a self-managing data plane for large-scale batch processing, ad hoc queries, and ML pipelines on the lake.'},
+ {question: 'Is Apache Hudi just a table format or a transactional layer?', answer: 'No. The functionality provided by a table format is merely one layer in the Hudi software stack. Hudi implements a table format internally, but adds a transactional layer designed around an event log, pluggable indexes, built-in table services, and data services like the DeltaStreamer ingestion utility—together forming a complete platform.'},
+ {question: 'What is incremental processing?', answer: 'Incremental processing applies stream processing principles to columnar data in the data lake, delivering orders of magnitude improvements in processing efficiency within a few minutes of latency, on scalable batch storage and compute infrastructure. It sits alongside the existing batch and streaming stacks and can also support regular batch processing for bulk reprocessing or backfills.'},
+ {question: 'How does Hudi handle concurrency between writers and table services?', answer: 'Hudi provides snapshot isolation between writers, table services, and readers. It uses optimistic concurrency control between writers, while providing lock-free, non-blocking MVCC-based concurrency control between writers and table services and between different table services. This allows, for example, compaction to run in the background without blocking ingestion.'},
+ {question: 'What table services are built into Hudi?', answer: 'Hudi ships with built-in services that keep table storage and metadata performant: archival to manage timeline history, cleaning to remove file slices past the retention period, compaction to merge base files with delta logs, clustering to reorganize records for query efficiency, and bootstrap for zero-copy migration of plain Parquet tables. They can run synchronously after each write or asynchronously as background jobs.'},
+]} />
diff --git a/website/blog/2021-08-16-kafka-custom-deserializer.md b/website/blog/2021-08-16-kafka-custom-deserializer.md
index c446152f9e80..2dba037af35e 100644
--- a/website/blog/2021-08-16-kafka-custom-deserializer.md
+++ b/website/blog/2021-08-16-kafka-custom-deserializer.md
@@ -1,6 +1,7 @@
---
title: "Schema evolution with DeltaStreamer using KafkaSource"
excerpt: "Evolve schema used in Kafkasource of DeltaStreamer to keep data up to date with business"
+description: "Evolve schema used in Kafkasource of DeltaStreamer to keep data up to date with business"
author: sbernauer
category: how-to
image: /assets/images/blog/hudi_schemaevolution.png
diff --git a/website/blog/2021-08-18-improving-marker-mechanism.md b/website/blog/2021-08-18-improving-marker-mechanism.md
index 2a9df1c61d00..386b8da5f3d0 100644
--- a/website/blog/2021-08-18-improving-marker-mechanism.md
+++ b/website/blog/2021-08-18-improving-marker-mechanism.md
@@ -1,6 +1,7 @@
---
title: "Improving Marker Mechanism in Apache Hudi"
excerpt: "We introduce a new marker mechanism leveraging the timeline server to address performance bottlenecks due to rate-limiting on cloud storage like AWS S3."
+description: "We introduce a new marker mechanism leveraging the timeline server to address performance bottlenecks due to rate-limiting on cloud storage like AWS S3."
authors: [ethan-guo]
category: deep-dive
image: /assets/images/blog/marker-mechanism/timeline-server-based-marker-mechanism.png
diff --git a/website/blog/2021-08-18-virtual-keys.md b/website/blog/2021-08-18-virtual-keys.md
index 86dcd078dc18..bee35ba0e67d 100644
--- a/website/blog/2021-08-18-virtual-keys.md
+++ b/website/blog/2021-08-18-virtual-keys.md
@@ -1,6 +1,7 @@
---
title: "Adding support for Virtual Keys in Hudi"
excerpt: "Supporting Virtual keys in Hudi for reducing storage overhead"
+description: "Supporting Virtual keys in Hudi for reducing storage overhead"
authors: [sivabalan]
category: deep-dive
tags:
diff --git a/website/blog/2021-08-23-async-clustering.md b/website/blog/2021-08-23-async-clustering.md
index 55e064e7af4c..fc86fe823810 100644
--- a/website/blog/2021-08-23-async-clustering.md
+++ b/website/blog/2021-08-23-async-clustering.md
@@ -1,17 +1,21 @@
---
title: "Asynchronous Clustering using Hudi"
excerpt: "How to setup Hudi for asynchronous clustering"
+description: "How to setup Hudi for asynchronous clustering"
authors: [sagar-sumit]
category: deep-dive
image: /assets/images/blog/clustering/example_perf_improvement.png
+last_update:
+ date: 2026-07-06
tags:
- clustering
---
-In one of the [previous blog](/blog/2021/01/27/hudi-clustering-intro) posts, we introduced a new
-kind of table service called clustering to reorganize data for improved query performance without compromising on
-ingestion speed. We learnt how to setup inline clustering. In this post, we will discuss what has changed since then and
-see how asynchronous clustering can be setup using HoodieClusteringJob as well as DeltaStreamer utility.
+Asynchronous clustering is a way to run Hudi's clustering table service in the background, reorganizing data for improved query performance while writers continue ingesting into the table without being blocked. In one of the [previous blog](/blog/2021/01/27/hudi-clustering-intro) posts, we introduced clustering and learnt how to setup inline clustering. In this post, we will discuss what has changed since then and see how asynchronous clustering can be setup using HoodieClusteringJob as [...]
+
+:::info
+For a newer, format-agnostic take on this topic, see our 2025 [clustering post](/blog/2025/03/26/clustering) on what clustering means in an open data lakehouse.
+:::
<!--truncate-->
@@ -219,3 +223,13 @@ over yet and future work entails:
Please follow this [JIRA](https://issues.apache.org/jira/browse/HUDI-1042) to learn more about active development on
this issue. We look forward to contributions from the community. Hope you enjoyed this post. Put your Hudi on and keep
streaming!
+
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {question: 'What is asynchronous clustering in Apache Hudi?', answer: 'Asynchronous clustering runs Hudi\'s clustering table service in the background while regular writers keep ingesting into the table. Hudi\'s multi-writer support provides snapshot isolation between table services, so data can be reorganized for better query performance without compromising ingestion speed.'},
+ {question: 'What clustering plan strategies does Hudi provide?', answer: 'Hudi ships three pluggable plan strategies. SparkSizeBasedClusteringPlanStrategy groups small file slices up to a max size per group, SparkRecentDaysClusteringPlanStrategy (the default when this post was written; current releases default to SparkSizeBasedClusteringPlanStrategy) clusters small files in the previous N days of partitions, and SparkSelectedPartitionsClusteringPlanStrategy clusters only partitions wit [...]
+ {question: 'What are the modes of HoodieClusteringJob?', answer: 'Since Hudi 0.9.0, HoodieClusteringJob supports three modes passed via the -m option. The schedule mode makes a clustering plan and returns an instant, the execute mode runs a plan at a given instant time, and scheduleAndExecute does both in one step.'},
+ {question: 'How do I enable asynchronous clustering with DeltaStreamer?', answer: 'Set hoodie.clustering.async.enabled to true and put the other clustering configs in the properties file passed via the props option, then run DeltaStreamer in continuous mode. The same config also enables async clustering with a Spark structured streaming sink.'},
+ {question: 'Can clustering run on a table that is receiving updates?', answer: 'By default no, clustering can only be scheduled for tables or partitions not receiving concurrent updates, and the default SparkRejectUpdateStrategy throws an exception if a file group gets updates during clustering. If updates are sparse, you can set the update strategy to SparkAllowUpdateStrategy, and running HoodieClusteringJob alongside a writer requires enabling optimistic concurrency control with a lo [...]
+]} />
diff --git a/website/blog/2021-08-23-s3-events-source.md b/website/blog/2021-08-23-s3-events-source.md
index 2db99fdb7d22..556ccad48497 100644
--- a/website/blog/2021-08-23-s3-events-source.md
+++ b/website/blog/2021-08-23-s3-events-source.md
@@ -1,6 +1,7 @@
---
title: "Reliable ingestion from AWS S3 using Hudi"
excerpt: "From listing to log-based approach, a reliable way of ingesting data from AWS S3 into Hudi."
+description: "From listing to log-based approach, a reliable way of ingesting data from AWS S3 into Hudi."
authors: [sagar-sumit]
category: how-to
image: /assets/images/blog/s3_events_source_design.png
diff --git a/website/blog/2021-09-01-building-eb-level-data-lake-using-hudi-at-bytedance.md b/website/blog/2021-09-01-building-eb-level-data-lake-using-hudi-at-bytedance.md
index 80f725b13d9c..a267a8e32915 100644
--- a/website/blog/2021-09-01-building-eb-level-data-lake-using-hudi-at-bytedance.md
+++ b/website/blog/2021-09-01-building-eb-level-data-lake-using-hudi-at-bytedance.md
@@ -1,6 +1,7 @@
---
title: "Building an ExaByte-level Data Lake Using Apache Hudi at ByteDance"
excerpt: "Ziyue Guan from Bytedance shares the production experience of building an ExaByte-level data lake using Apache Hudi and how it is used in the recommendation system at Bytedance."
+description: "Ziyue Guan from Bytedance shares the production experience of building an ExaByte-level data lake using Apache Hudi and how it is used in the recommendation system at Bytedance."
author: Ziyue Guan, translated to English by yihua
category: case-study
image: /assets/images/blog/bytedance_hudi.png
diff --git a/website/blog/2021-12-16-lakehouse-concurrency-control-are-we-too-optimistic.mdx b/website/blog/2021-12-16-lakehouse-concurrency-control-are-we-too-optimistic.mdx
index c3e0bdc30f25..389cf9e2f3c7 100644
--- a/website/blog/2021-12-16-lakehouse-concurrency-control-are-we-too-optimistic.mdx
+++ b/website/blog/2021-12-16-lakehouse-concurrency-control-are-we-too-optimistic.mdx
@@ -1,19 +1,26 @@
---
title: "Lakehouse Concurrency Control: Are we too optimistic?"
excerpt: "Vinoth Chandar, original creator of Apache Hudi, dives into concurrency control mechanisms"
+description: "Vinoth Chandar, original creator of Apache Hudi, dives into concurrency control mechanisms"
authors: [vinoth-chandar]
category: deep-dive
image: /assets/images/blog/concurrency/MultiWriter.gif
+last_update:
+ date: 2026-07-12
tags:
- concurrency control
---
import SlackCommunity from '@site/src/components/SlackCommunity';
-Transactions on data lakes are now considered a key characteristic of a [Lakehouse](https://hudi.apache.org/blog/2024/07/11/what-is-a-data-lakehouse/) these days. But what has actually been accomplished so far? What are the current approaches? How do they fare in real-world scenarios? These questions are the focus of this blog.
+Optimistic concurrency control (OCC) on a lakehouse lets concurrent jobs write without locks and simply checks for overlapping files at commit time, aborting the entire job if a conflict is found - an approach that struggles under real contention, because long-running jobs that touch overlapping files keep starving each other and burning hours of compute on failed retries. Transactions on data lakes are now considered a key characteristic of a [Lakehouse](https://hudi.apache.org/blog/202 [...]
<!--truncate-->
+:::info Related reading
+For where these ideas landed in Hudi, see the [concurrency control docs](/docs/concurrency_control), a broader [survey of concurrency control across open table formats](/blog/2025/01/28/concurrency-control), and Hudi 1.0's [non-blocking concurrency control](/blog/2024/12/06/non-blocking-concurrency-control) that realizes the lock-free vision sketched at the end of this post.
+:::
+
Having had the good fortune of working on diverse database projects - an RDBMS ([Oracle](https://www.oracle.com/database/)), a NoSQL key-value store ([Voldemort](https://www.slideshare.net/vinothchandar/voldemort-prototype-to-production-nectar-edits)), a streaming database ([ksqlDB](https://www.confluent.io/blog/ksqldb-pull-queries-high-availability/)), a closed-source real-time datastore and of course, Apache Hudi, I can safely say that the nature of workloads deeply influence the concu [...]
First, let's set the record straight. RDBMS databases offer the richest set of transactional capabilities and the widest array of concurrency control [mechanisms](https://dev.mysql.com/doc/refman/5.7/en/innodb-locking-transaction-model.html). Different isolation levels, fine grained locking, deadlock detection/avoidance, and more are possible because they have to support row-level mutations and reads across many tables while enforcing [key constraints](https://dev.mysql.com/doc/refman/8. [...]
@@ -55,3 +62,13 @@ All this said, there are still many ways we can improve upon this foundation.
* Touching upon key constraints, Hudi is the only lake transactional layer that ensures unique [key](https://hudi.apache.org/docs/key_generation) constraints today, but limited to the record key of the table. We will be looking to expand this capability in a more general form to non-primary key fields, with the said newer concurrency models.
Finally, for data lakes to transform successfully into lakehouses, we must learn from the failing of the "hadoop warehouse" vision, which shared similar goals with the new "[lakehouse](https://hudi.apache.org/blog/2024/07/11/what-is-a-data-lakehouse/)" vision. Designers did not pay closer attention to the missing technology gaps against warehouses and created unrealistic expectations from the actual software. As transactions and database functionality finally goes mainstream on data lake [...]
+
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {question: 'What is optimistic concurrency control (OCC) on a data lakehouse?', answer: 'OCC is a lock-free approach where each writer proceeds assuming no conflicts, then takes a table-level lock at commit time to check whether it touched files that overlap with another writer. If a conflict is detected, the job aborts its operation completely and must retry from scratch.'},
+ {question: 'Why does OCC struggle under contention?', answer: 'OCC assumes real contention never happens. When transactions are long-running - like a multi-hour GDPR deletion job overlapping a half-hourly ingest job - the probability of overlap rises with transaction length, so the longer job is almost guaranteed to starve and fail to commit each time, wasting hours of compute on every failed attempt.'},
+ {question: 'How does Apache Hudi approach concurrency control differently?', answer: 'Hudi implements a file-level, log-based concurrency control protocol on its timeline, borrowing from stream processing where events are serialized into a single ordered log instead of coordinated through locks. This supports deployment models with greater concurrency than pure OCC approaches that only track table snapshots.'},
+ {question: 'Does Hudi need locks to run table services like compaction and clustering?', answer: 'No. With a single writer, Hudi runs table services either inline after each write or asynchronously in the same process, with no locking at all. Execution plans are idempotent, persisted to the timeline, and auto-recover from failures.'},
+ {question: 'Does Hudi support multiple concurrent writers?', answer: 'Yes. With multiple distributed writers, Hudi offers optimistic concurrency control between the actual writers, while table services still execute lock-free and async. Because compaction does the heavy lifting of rewriting columnar data, writer transactions stay short and the wastage from any conflict is drastically lower.'},
+]} />
diff --git a/website/blog/2021-12-29-hudi-zorder-and-hilbert-space-filling-curves.md b/website/blog/2021-12-29-hudi-zorder-and-hilbert-space-filling-curves.md
index 1827761ece1a..f54279138b24 100644
--- a/website/blog/2021-12-29-hudi-zorder-and-hilbert-space-filling-curves.md
+++ b/website/blog/2021-12-29-hudi-zorder-and-hilbert-space-filling-curves.md
@@ -1,6 +1,7 @@
---
title: "Hudi Z-Order and Hilbert Space Filling Curves"
excerpt: "Explore the benefits of new Apache Hudi Z-Order and Hilbert Curves"
+description: "Explore the benefits of new Apache Hudi Z-Order and Hilbert Curves"
author: Alexey Kudinkin and Tao Meng
category: deep-dive
image: /assets/images/zordercurve.png
diff --git a/website/blog/2022-01-06-apache-hudi-2021-a-year-in-review.mdx b/website/blog/2022-01-06-apache-hudi-2021-a-year-in-review.mdx
index dad3df087766..139a87982bc1 100644
--- a/website/blog/2022-01-06-apache-hudi-2021-a-year-in-review.mdx
+++ b/website/blog/2022-01-06-apache-hudi-2021-a-year-in-review.mdx
@@ -1,6 +1,7 @@
---
title: "Apache Hudi - 2021 a Year in Review"
excerpt: "A reflection on the growth and momentum of Apache Hudi in 2021"
+description: "A reflection on the growth and momentum of Apache Hudi in 2021"
authors: [vinoth-chandar]
category: community
image: /assets/images/Hudi_community.png
diff --git a/website/blog/2022-01-14-change-data-capture-with-debezium-and-apache-hudi.mdx b/website/blog/2022-01-14-change-data-capture-with-debezium-and-apache-hudi.mdx
index abcfad4d1e4c..bd1994c7fcde 100644
--- a/website/blog/2022-01-14-change-data-capture-with-debezium-and-apache-hudi.mdx
+++ b/website/blog/2022-01-14-change-data-capture-with-debezium-and-apache-hudi.mdx
@@ -1,6 +1,7 @@
---
title: "Change Data Capture with Debezium and Apache Hudi"
excerpt: "A review of new Debezium source connector for Apache Hudi"
+description: "A review of new Debezium source connector for Apache Hudi"
authors: [rajesh-mahindra]
category: how-to
image: /assets/images/blog/debezium.png
diff --git a/website/blog/2022-07-11-build-open-lakehouse-using-apache-hudi-and-dbt.md b/website/blog/2022-07-11-build-open-lakehouse-using-apache-hudi-and-dbt.md
index c25139e42914..a84bb22e1466 100644
--- a/website/blog/2022-07-11-build-open-lakehouse-using-apache-hudi-and-dbt.md
+++ b/website/blog/2022-07-11-build-open-lakehouse-using-apache-hudi-and-dbt.md
@@ -1,6 +1,7 @@
---
title: "Build Open Lakehouse using Apache Hudi & dbt"
-excerpt: "How to style blog focused projects on teaching how to build an open Lakehouse using Apache Hudi & dbt"
+excerpt: "A hands-on tutorial for building an open lakehouse using Apache Hudi and dbt."
+description: "A hands-on tutorial for building an open lakehouse using Apache Hudi and dbt."
author: Vinoth Govindarajan
category: how-to
image: /assets/images/blog/hudi_dbt_lakehouse.png
diff --git a/website/blog/2022-12-19-Build-Your-First-Hudi-Lakehouse-with-AWS-Glue-and-AWS-S3.md b/website/blog/2022-12-19-Build-Your-First-Hudi-Lakehouse-with-AWS-Glue-and-AWS-S3.md
index d02339871bd3..81da47458ceb 100644
--- a/website/blog/2022-12-19-Build-Your-First-Hudi-Lakehouse-with-AWS-Glue-and-AWS-S3.md
+++ b/website/blog/2022-12-19-Build-Your-First-Hudi-Lakehouse-with-AWS-Glue-and-AWS-S3.md
@@ -1,6 +1,7 @@
---
title: "Build Your First Hudi Lakehouse with AWS S3 and AWS Glue"
excerpt: "Follow this tutorial on building your first hudi lakehouse with AWS S3 & AWS Glue"
+description: "Follow this tutorial on building your first hudi lakehouse with AWS S3 & AWS Glue"
authors: [nadine-farah]
category: how-to
image: /assets/images/blog/DataCouncil.jpg
diff --git a/website/blog/2022-12-29-Apache-Hudi-2022-A-Year-In-Review.mdx b/website/blog/2022-12-29-Apache-Hudi-2022-A-Year-In-Review.mdx
index 149aa6e67435..8851e8e858ce 100644
--- a/website/blog/2022-12-29-Apache-Hudi-2022-A-Year-In-Review.mdx
+++ b/website/blog/2022-12-29-Apache-Hudi-2022-A-Year-In-Review.mdx
@@ -1,6 +1,7 @@
---
title: "Apache Hudi 2022 - A year in Review"
excerpt: "2022 was the best year for Apache Hudi yet! Huge thank you to everyone who contributed!"
+description: "2022 was the best year for Apache Hudi yet! Huge thank you to everyone who contributed!"
authors: [sivabalan]
category: community
image: /assets/images/blog/Apache-Hudi-2022-Review.png
diff --git a/website/blog/2023-08-22-Exploring-various-storage-types-in-Apache-Hudi.mdx b/website/blog/2023-08-22-Exploring-various-storage-types-in-Apache-Hudi.mdx
index 56949e1c50af..258e32afd11d 100644
--- a/website/blog/2023-08-22-Exploring-various-storage-types-in-Apache-Hudi.mdx
+++ b/website/blog/2023-08-22-Exploring-various-storage-types-in-Apache-Hudi.mdx
@@ -1,6 +1,7 @@
---
title: "Exploring various storage types in Apache Hudi"
excerpt: "Hudi Storage Format Overview"
+description: "Hudi Storage Format Overview"
author: Arun Kumar Nagaraj
category: deep-dive
image: /assets/images/blog/2023-08-22-Exploring-various-storage-types-in-Apache-Hudi.png
diff --git a/website/blog/2023-08-25-Delta-Hudi-Iceberg-Which-is-most-popular.mdx b/website/blog/2023-08-25-Delta-Hudi-Iceberg-Which-is-most-popular.mdx
index 6965f6f67170..d7787be43790 100644
--- a/website/blog/2023-08-25-Delta-Hudi-Iceberg-Which-is-most-popular.mdx
+++ b/website/blog/2023-08-25-Delta-Hudi-Iceberg-Which-is-most-popular.mdx
@@ -1,6 +1,7 @@
---
title: "Delta, Hudi, Iceberg — Which is most popular?"
excerpt: "Popular Lakehoue Project"
+description: "Popular Lakehoue Project"
author: Kyle Weller
category: deep-dive
image: /assets/images/blog/2023-08-25-Delta-Hudi-Iceberg-Which-is-most-popular.png
diff --git a/website/blog/2023-08-28-Apache-Hudi-From-Zero-To-One.mdx b/website/blog/2023-08-28-Apache-Hudi-From-Zero-To-One.mdx
index 74d2a4af9e16..8100987fbb02 100644
--- a/website/blog/2023-08-28-Apache-Hudi-From-Zero-To-One.mdx
+++ b/website/blog/2023-08-28-Apache-Hudi-From-Zero-To-One.mdx
@@ -1,6 +1,7 @@
---
title: "Apache Hudi: From Zero To One (1/10)"
excerpt: "A first glance at Hudi's storage format"
+description: "A first glance at Hudi's storage format"
authors: [xushiyan]
category: deep-dive
image: /assets/images/blog/2023-08-28-Apache-Hudi-From-Zero-To-One-blog-1.png
diff --git a/website/blog/2023-08-28-Delta-Hudi-Iceberg-A-Benchmark-Compilation.mdx b/website/blog/2023-08-28-Delta-Hudi-Iceberg-A-Benchmark-Compilation.mdx
index 0424f6980370..7fff9adc5558 100644
--- a/website/blog/2023-08-28-Delta-Hudi-Iceberg-A-Benchmark-Compilation.mdx
+++ b/website/blog/2023-08-28-Delta-Hudi-Iceberg-A-Benchmark-Compilation.mdx
@@ -1,6 +1,7 @@
---
title: "Delta, Hudi, Iceberg — A Benchmark Compilation"
excerpt: "Benchmark Compilation"
+description: "Benchmark Compilation"
author: Kyle Weller
category: deep-dive
image: /assets/images/blog/2023-08-28-Delta-Hudi-Iceberg-A-Benchmark-Compilation.png
diff --git a/website/blog/2023-08-31-Incremental-Queries-with-Apache-Hudi-and-Apache-Flink.mdx b/website/blog/2023-08-31-Incremental-Queries-with-Apache-Hudi-and-Apache-Flink.mdx
index 20226edab1e1..e885b94850b9 100644
--- a/website/blog/2023-08-31-Incremental-Queries-with-Apache-Hudi-and-Apache-Flink.mdx
+++ b/website/blog/2023-08-31-Incremental-Queries-with-Apache-Hudi-and-Apache-Flink.mdx
@@ -1,6 +1,7 @@
---
title: "Incremental Queries with Apache Hudi and Apache Flink"
excerpt: "Incremental Queries with Apache Hudi and Apache Flink"
+description: "Incremental Queries with Apache Hudi and Apache Flink"
author: nello
category: how-to
image: /assets/images/blog/2023-08-31-Incremental-Queries-with-Apache-Hudi-and-Apache-Flink.png
diff --git a/website/blog/2023-09-06-Apache-Hudi-From-Zero-To-One-blog-2.mdx b/website/blog/2023-09-06-Apache-Hudi-From-Zero-To-One-blog-2.mdx
index d0a8853a1947..e8bb2318163a 100644
--- a/website/blog/2023-09-06-Apache-Hudi-From-Zero-To-One-blog-2.mdx
+++ b/website/blog/2023-09-06-Apache-Hudi-From-Zero-To-One-blog-2.mdx
@@ -1,6 +1,7 @@
---
title: "Apache Hudi: From Zero To One (2/10)"
excerpt: "Dive into read operation flow and query types"
+description: "Dive into read operation flow and query types"
authors: [xushiyan]
category: deep-dive
image: /assets/images/blog/2023-09-06-Apache-Hudi-From-Zero-To-One-blog-2.png
diff --git a/website/blog/2023-09-06-Lakehouse-or-Warehouse-Part-1-of-2.mdx b/website/blog/2023-09-06-Lakehouse-or-Warehouse-Part-1-of-2.mdx
index a6ad72ddadff..79ae9663b616 100644
--- a/website/blog/2023-09-06-Lakehouse-or-Warehouse-Part-1-of-2.mdx
+++ b/website/blog/2023-09-06-Lakehouse-or-Warehouse-Part-1-of-2.mdx
@@ -1,6 +1,7 @@
---
title: "Lakehouse or Warehouse? Part 1 of 2"
excerpt: "Lakehouse or Warehouse? Part 1 of 2"
+description: "Lakehouse or Warehouse? Part 1 of 2"
author: Floyd Smith
category: deep-dive
image: /assets/images/blog/2023-09-06-Lakehouse-or-Warehouse-Part-1-of-2.png
diff --git a/website/blog/2023-09-10-Demystifying-Copy-on-Write-in-Apache-Hudi-Understanding-Read-and-Write-Operations.mdx b/website/blog/2023-09-10-Demystifying-Copy-on-Write-in-Apache-Hudi-Understanding-Read-and-Write-Operations.mdx
index 2a8f4100154c..25ae521dac67 100644
--- a/website/blog/2023-09-10-Demystifying-Copy-on-Write-in-Apache-Hudi-Understanding-Read-and-Write-Operations.mdx
+++ b/website/blog/2023-09-10-Demystifying-Copy-on-Write-in-Apache-Hudi-Understanding-Read-and-Write-Operations.mdx
@@ -1,6 +1,7 @@
---
title: "Demystifying Copy-on-Write in Apache Hudi: Understanding Read and Write Operations"
excerpt: "COW Overview"
+description: "COW Overview"
author: Eswaramoorthy P
category: deep-dive
image: /assets/images/blog/2023-09-10-Demystifying-Copy-on-Write-in-Apache-Hudi-Understanding-Read-and-Write-Operations.png
diff --git a/website/blog/2023-09-12-Lakehouse-or-Warehouse-Part-2-of-2.mdx b/website/blog/2023-09-12-Lakehouse-or-Warehouse-Part-2-of-2.mdx
index b8348d425a79..afdc5bcafd46 100644
--- a/website/blog/2023-09-12-Lakehouse-or-Warehouse-Part-2-of-2.mdx
+++ b/website/blog/2023-09-12-Lakehouse-or-Warehouse-Part-2-of-2.mdx
@@ -1,6 +1,7 @@
---
title: "Lakehouse or Warehouse? Part 2 of 2"
excerpt: "Lakehouse or Warehouse? Part 2 of 2"
+description: "Lakehouse or Warehouse? Part 2 of 2"
author: Floyd Smith
category: deep-dive
image: /assets/images/blog/2023-09-12-Lakehouse-or-Warehouse-Part-2-of-2.png
diff --git a/website/blog/2023-09-13-Simplify-operational-data-processing-in-data-lakes-using-AWS-Glue-and-Apache-Hudi.mdx b/website/blog/2023-09-13-Simplify-operational-data-processing-in-data-lakes-using-AWS-Glue-and-Apache-Hudi.mdx
index c97cabf69762..0717315a7ce2 100644
--- a/website/blog/2023-09-13-Simplify-operational-data-processing-in-data-lakes-using-AWS-Glue-and-Apache-Hudi.mdx
+++ b/website/blog/2023-09-13-Simplify-operational-data-processing-in-data-lakes-using-AWS-Glue-and-Apache-Hudi.mdx
@@ -1,6 +1,7 @@
---
title: "Simplify operational data processing in data lakes using AWS Glue and Apache Hudi"
excerpt: "Use AWS Glue and Apache Hudi for data processing"
+description: "Use AWS Glue and Apache Hudi for data processing"
authors:
- name: Srinivas Kandi
- name: Ravi Itha
diff --git a/website/blog/2023-09-15-Apache-Hudi-From-Zero-To-One-blog-3.mdx b/website/blog/2023-09-15-Apache-Hudi-From-Zero-To-One-blog-3.mdx
index 3ed6abcbe52a..0011e0f03d76 100644
--- a/website/blog/2023-09-15-Apache-Hudi-From-Zero-To-One-blog-3.mdx
+++ b/website/blog/2023-09-15-Apache-Hudi-From-Zero-To-One-blog-3.mdx
@@ -1,6 +1,7 @@
---
title: "Apache Hudi: From Zero To One (3/10)"
excerpt: "Understand write flows and operations"
+description: "Understand write flows and operations"
authors: [xushiyan]
category: deep-dive
image: /assets/images/blog/2023-09-15-Apache-Hudi-From-Zero-To-One-blog-3.png
diff --git a/website/blog/2023-09-22-Exploring-the-Architecture-of-Apache-Iceberg-Delta-Lake-and-Apache-Hudi.mdx b/website/blog/2023-09-22-Exploring-the-Architecture-of-Apache-Iceberg-Delta-Lake-and-Apache-Hudi.mdx
index a95a9337c8cd..0d43938fc29a 100644
--- a/website/blog/2023-09-22-Exploring-the-Architecture-of-Apache-Iceberg-Delta-Lake-and-Apache-Hudi.mdx
+++ b/website/blog/2023-09-22-Exploring-the-Architecture-of-Apache-Iceberg-Delta-Lake-and-Apache-Hudi.mdx
@@ -1,6 +1,7 @@
---
title: "Exploring the Architecture of Apache Iceberg, Delta Lake, and Apache Hudi"
excerpt: "Exploring the Architecture of Apache Iceberg, Delta Lake, and Apache Hudi"
+description: "Exploring the Architecture of Apache Iceberg, Delta Lake, and Apache Hudi"
author: Alex Merced
category: deep-dive
image: /assets/images/blog/2023-09-22-Exploring-the-Architecture-of-Apache-Iceberg-Delta-Lake-and-Apache-Hudi.png
diff --git a/website/blog/2023-09-27-Apache-Hudi-From-Zero-To-One-blog-4.mdx b/website/blog/2023-09-27-Apache-Hudi-From-Zero-To-One-blog-4.mdx
index e69571929f56..bc008a2d3509 100644
--- a/website/blog/2023-09-27-Apache-Hudi-From-Zero-To-One-blog-4.mdx
+++ b/website/blog/2023-09-27-Apache-Hudi-From-Zero-To-One-blog-4.mdx
@@ -1,6 +1,7 @@
---
title: "Apache Hudi: From Zero To One (4/10)"
excerpt: "All about writer indexes"
+description: "All about writer indexes"
authors: [xushiyan]
category: deep-dive
image: /assets/images/blog/2023-09-27-Apache-Hudi-From-Zero-To-One-blog-4.png
diff --git a/website/blog/2023-10-06-Apache-Hudi-Copy-on-Write-CoW-Table.mdx b/website/blog/2023-10-06-Apache-Hudi-Copy-on-Write-CoW-Table.mdx
index c842281f2700..c39ad5161028 100644
--- a/website/blog/2023-10-06-Apache-Hudi-Copy-on-Write-CoW-Table.mdx
+++ b/website/blog/2023-10-06-Apache-Hudi-Copy-on-Write-CoW-Table.mdx
@@ -1,6 +1,7 @@
---
title: "Apache Hudi: Copy on Write(CoW) Table"
excerpt: "Apache Hudi: Copy on Write(CoW) Table"
+description: "Apache Hudi: Copy on Write(CoW) Table"
authors:
- name: Ankur Ranjan
category: deep-dive
diff --git a/website/blog/2023-10-11-starrocks-query-performance-with-apache-hudi-and-onehouse.mdx b/website/blog/2023-10-11-starrocks-query-performance-with-apache-hudi-and-onehouse.mdx
index 34bad702a3bb..55eeb3730266 100644
--- a/website/blog/2023-10-11-starrocks-query-performance-with-apache-hudi-and-onehouse.mdx
+++ b/website/blog/2023-10-11-starrocks-query-performance-with-apache-hudi-and-onehouse.mdx
@@ -1,6 +1,7 @@
---
title: "StarRocks query performance with Apache Hudi and Onehouse"
excerpt: "StarRocks Query Performance with Apache Hudi"
+description: "StarRocks Query Performance with Apache Hudi"
authors:
- name: Albert Wong
category: case-study
diff --git a/website/blog/2023-10-17-Get-started-with-Apache-Hudi-using-AWS-Glue-by-implementing-key-design-concepts-Part-1.mdx b/website/blog/2023-10-17-Get-started-with-Apache-Hudi-using-AWS-Glue-by-implementing-key-design-concepts-Part-1.mdx
index 1bd08db8a38e..66bb0ead7497 100644
--- a/website/blog/2023-10-17-Get-started-with-Apache-Hudi-using-AWS-Glue-by-implementing-key-design-concepts-Part-1.mdx
+++ b/website/blog/2023-10-17-Get-started-with-Apache-Hudi-using-AWS-Glue-by-implementing-key-design-concepts-Part-1.mdx
@@ -1,6 +1,7 @@
---
title: "Get started with Apache Hudi using AWS Glue by implementing key design concepts – Part 1"
excerpt: "Get started with Apache Hudi using AWS Glue by implementing key design concepts – Part 1"
+description: "Get started with Apache Hudi using AWS Glue by implementing key design concepts – Part 1"
authors:
- name: Srinivas Kandi
- name: Ravi Itha
diff --git a/website/blog/2023-10-18-Apache-Hudi-From-Zero-To-One-blog-5.mdx b/website/blog/2023-10-18-Apache-Hudi-From-Zero-To-One-blog-5.mdx
index 11aafad4c7a9..936cfa7328be 100644
--- a/website/blog/2023-10-18-Apache-Hudi-From-Zero-To-One-blog-5.mdx
+++ b/website/blog/2023-10-18-Apache-Hudi-From-Zero-To-One-blog-5.mdx
@@ -1,6 +1,7 @@
---
title: "Apache Hudi: From Zero To One (5/10)"
excerpt: "Introduce table services: compaction, cleaning, and indexing"
+description: "Introduce table services: compaction, cleaning, and indexing"
authors: [xushiyan]
category: deep-dive
image: /assets/images/blog/2023-10-18-Apache-Hudi-From-Zero-To-One-blog-5.png
diff --git a/website/blog/2023-10-19-load-data-incrementally-from-transactional-data-lakes-to-data-warehouses.mdx b/website/blog/2023-10-19-load-data-incrementally-from-transactional-data-lakes-to-data-warehouses.mdx
index 2e0b068fd12c..ebc0853e5786 100644
--- a/website/blog/2023-10-19-load-data-incrementally-from-transactional-data-lakes-to-data-warehouses.mdx
+++ b/website/blog/2023-10-19-load-data-incrementally-from-transactional-data-lakes-to-data-warehouses.mdx
@@ -1,6 +1,7 @@
---
title: "Load data incrementally from transactional data lakes to data warehouses"
excerpt: "Load data incrementally from Apache Hudi table to Amazon Redshift using a Hudi incremental query"
+description: "Load data incrementally from Apache Hudi table to Amazon Redshift using a Hudi incremental query"
author: Noritaka Sekiyama
category: how-to
image: /assets/images/blog/2023-10-19-load-data-incrementally-from-transactional-data-lakes-to-data-warehouses.png
diff --git a/website/blog/2023-10-20-Its-Time-for-the-Universal-Data-Lakehouse.mdx b/website/blog/2023-10-20-Its-Time-for-the-Universal-Data-Lakehouse.mdx
index 59cebcda7bce..259daaf469eb 100644
--- a/website/blog/2023-10-20-Its-Time-for-the-Universal-Data-Lakehouse.mdx
+++ b/website/blog/2023-10-20-Its-Time-for-the-Universal-Data-Lakehouse.mdx
@@ -1,6 +1,7 @@
---
title: "It's Time for the Universal Data Lakehouse"
excerpt: "Universal Lakehouse"
+description: "Universal Lakehouse"
authors: [vinoth-chandar]
category: deep-dive
image: /assets/images/blog/2023-10-20-Its-Time-for-the-Universal-Data-Lakehouse.png
diff --git a/website/blog/2023-10-29-UPSERT-Performance-Evaluation-of-Hudi-0-14-and-Spark-3-4-1-Record-Level-Index-Global-Bloom-Global-Simple-Indexes.mdx b/website/blog/2023-10-29-UPSERT-Performance-Evaluation-of-Hudi-0-14-and-Spark-3-4-1-Record-Level-Index-Global-Bloom-Global-Simple-Indexes.mdx
index f135c775b420..7bf713c502ec 100644
--- a/website/blog/2023-10-29-UPSERT-Performance-Evaluation-of-Hudi-0-14-and-Spark-3-4-1-Record-Level-Index-Global-Bloom-Global-Simple-Indexes.mdx
+++ b/website/blog/2023-10-29-UPSERT-Performance-Evaluation-of-Hudi-0-14-and-Spark-3-4-1-Record-Level-Index-Global-Bloom-Global-Simple-Indexes.mdx
@@ -1,6 +1,7 @@
---
title: "UPSERT Performance Evaluation of Hudi 0.14 and Spark 3.4.1: Record Level Index vs. Global Bloom & Global Simple Indexes"
excerpt: "Record Level Index Performance"
+description: "Record Level Index Performance"
author: Soumil Shah
category: how-to
image: /assets/images/blog/2023-10-29-UPSERT-Performance-Evaluation-of-Hudi-0-14-and-Spark-3-4-1-Record-Level-Index-Global-Bloom-Global-Simple-Indexes.png
diff --git a/website/blog/2023-11-01-record-level-index.md b/website/blog/2023-11-01-record-level-index.md
index a24f373b4ced..6e50af75faec 100644
--- a/website/blog/2023-11-01-record-level-index.md
+++ b/website/blog/2023-11-01-record-level-index.md
@@ -1,14 +1,26 @@
---
title: "Record Level Index: Hudi's blazing fast indexing for large-scale datasets"
excerpt: "Announcing the Record Level Index in Apache Hudi"
+description: "Announcing the Record Level Index in Apache Hudi"
authors: [xushiyan, sivabalan]
category: deep-dive
image: /assets/images/blog/record-level-index/03.RLI_bulkinsert.png
+last_update:
+ date: 2026-07-06
tags:
- indexing
- metadata
---
+The Record Level Index (RLI) is a global index in Apache Hudi, introduced in [Hudi 0.14.0](https://hudi.apache.org/releases/release-0.14#release-0140), that stores the mapping
+from each record key to its file group inside Hudi's metadata table, dramatically speeding up writes as well as key-matching queries without
+any external system to operate. This post walks through RLI's design and workflows, benchmarks it against other Hudi index types, and closes
+with future work.
+
+:::info Newer content available
+RLI has evolved since this 2023 post. For the current (1.x) picture, see [Deep Dive Into Hudi's Indexing Subsystem Part 2](/blog/2025/11/12/deep-dive-into-hudis-indexing-subsystem-part-2-of-2), and [Introducing Secondary Index in Apache Hudi](/blog/2025/04/02/secondary-index) for indexing beyond the record key.
+:::
+
## Introduction
Index is a critical component that facilitates quick updates and deletes for Hudi writers, and it plays a pivotal
@@ -229,3 +241,13 @@ by Hudi metadata tables.
[^3] As of now, query engine integration is only available for Spark, with plans to support additional engines in the future.
[^4] The query improvement is specific to record-key-matching queries and does not reflect a general reduction in latency by enabling RLI. In the case of the single record-key query, 99.995% of file groups (19999 out of 20000) were pruned during query execution.
+
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {question: 'What is the Record Level Index in Apache Hudi?', answer: 'The Record Level Index (RLI) is a global index introduced in Hudi 0.14.0 that stores one-to-one mappings between record keys and their file groups in a dedicated partition of Hudi\'s metadata table. It lets writers and readers locate the exact file group for a record key, drastically reducing the number of files that need to be scanned.'},
+ {question: 'How do I enable the Record Level Index in Hudi?', answer: 'On current releases, set hoodie.metadata.global.record.level.index.enable=true and hoodie.index.type=GLOBAL_RECORD_LEVEL_INDEX, with the metadata table enabled via hoodie.metadata.enable=true (hoodie.metadata.record.index.enable and RECORD_INDEX are the deprecated pre-1.1 names). Since the number of file groups in the RLI partition is fixed at initialization, it is recommended to configure the file group count and s [...]
+ {question: 'How much faster is RLI compared to other Hudi index types?', answer: 'In benchmarks on a 1TB dataset of 2 billion records, RLI improved average write latency and index look-up latency by 72% over the Global Simple Index, while shuffling 92% less data. On the read side, a query on a single record key dropped from 977 seconds to 12 seconds, a 98% latency reduction.'},
+ {question: 'How much storage does the Record Level Index consume?', answer: 'With Gzip compression and a 4MB block size, an individual RLI record averages only 48 bytes. For a 100TB table with about 1 billion records, the RLI partition needs roughly 48GB, which is less than 0.05% of the total data size.'},
+ {question: 'When should I use the Record Level Index, and what are its limitations?', answer: 'RLI shines for write-heavy workloads such as GDPR deletes and for key-matching queries like tracing lookups. As a global index, it requires record keys to be unique across all partitions, its initialization can take time on large tables, and the current design fixes the number of file groups once the partition is created, so extremely skewed workloads may not achieve the desired performance.'},
+]} />
diff --git a/website/blog/2023-11-13-Apache-Hudi-From-Zero-To-One-blog-6.mdx b/website/blog/2023-11-13-Apache-Hudi-From-Zero-To-One-blog-6.mdx
index 7072284fecb7..991808888837 100644
--- a/website/blog/2023-11-13-Apache-Hudi-From-Zero-To-One-blog-6.mdx
+++ b/website/blog/2023-11-13-Apache-Hudi-From-Zero-To-One-blog-6.mdx
@@ -1,6 +1,7 @@
---
title: "Apache Hudi: From Zero To One (6/10)"
excerpt: "Demystify clustering and space-filling curves"
+description: "Demystify clustering and space-filling curves"
authors: [xushiyan]
category: deep-dive
image: /assets/images/blog/2023-11-13-Apache-Hudi-From-Zero-To-One-blog-6.png
diff --git a/website/blog/2023-11-19-Hudi-Streamer-DeltaStreamer-Hands-On-Guide-Local-Ingestion-from-Parquet-Source.mdx b/website/blog/2023-11-19-Hudi-Streamer-DeltaStreamer-Hands-On-Guide-Local-Ingestion-from-Parquet-Source.mdx
index 28619d72621b..422a62861f0c 100644
--- a/website/blog/2023-11-19-Hudi-Streamer-DeltaStreamer-Hands-On-Guide-Local-Ingestion-from-Parquet-Source.mdx
+++ b/website/blog/2023-11-19-Hudi-Streamer-DeltaStreamer-Hands-On-Guide-Local-Ingestion-from-Parquet-Source.mdx
@@ -1,6 +1,7 @@
---
title: "Hudi Streamer (Delta Streamer) Hands-On Guide: Local Ingestion from Parquet Source"
excerpt: "Hudi Streamer (Delta Streamer) Hands-On Guide: Local Ingestion from Parquet Source"
+description: "Hudi Streamer (Delta Streamer) Hands-On Guide: Local Ingestion from Parquet Source"
author: Soumil Shah
category: how-to
image: /assets/images/blog/2023-11-19-Hudi-Streamer-DeltaStreamer-Hands-On-Guide-Local-Ingestion-from-Parquet-Source.png
diff --git a/website/blog/2023-11-22-Introducing-Apache-Hudi-support-with-AWS-Glue-crawlers.mdx b/website/blog/2023-11-22-Introducing-Apache-Hudi-support-with-AWS-Glue-crawlers.mdx
index 1594e4a90e32..871122e92e5f 100644
--- a/website/blog/2023-11-22-Introducing-Apache-Hudi-support-with-AWS-Glue-crawlers.mdx
+++ b/website/blog/2023-11-22-Introducing-Apache-Hudi-support-with-AWS-Glue-crawlers.mdx
@@ -1,6 +1,7 @@
---
title: "Introducing Apache Hudi support with AWS Glue crawlers"
excerpt: "Introducing Apache Hudi support with AWS Glue crawlers"
+description: "Introducing Apache Hudi support with AWS Glue crawlers"
author: Noritaka Sekiyama, Kyle Duong, Sandeep Adwankar
category: how-to
image: /assets/images/blog/2023-11-22-Introducing-Apache-Hudi-support-with-AWS-Glue-crawlers.png
diff --git a/website/blog/2023-11-26-Real-Time-Data-Processing-with-Postgres-Debezium-Kafka-Schema-Registry-and-DeltaStreamer-Guide-for-Begineers.mdx b/website/blog/2023-11-26-Real-Time-Data-Processing-with-Postgres-Debezium-Kafka-Schema-Registry-and-DeltaStreamer-Guide-for-Begineers.mdx
index 7ff325be0798..cccd2fc1689e 100644
--- a/website/blog/2023-11-26-Real-Time-Data-Processing-with-Postgres-Debezium-Kafka-Schema-Registry-and-DeltaStreamer-Guide-for-Begineers.mdx
+++ b/website/blog/2023-11-26-Real-Time-Data-Processing-with-Postgres-Debezium-Kafka-Schema-Registry-and-DeltaStreamer-Guide-for-Begineers.mdx
@@ -1,6 +1,7 @@
---
title: "Real-Time Data Processing with Postgres, Debezium, Kafka, Schema Registry, and Delta Streamer Guide for Begineers"
excerpt: "Real-Time Data Processing with Postgres, Debezium, Kafka, Schema Registry, and Delta Streamer Guide for Begineers"
+description: "Real-Time Data Processing with Postgres, Debezium, Kafka, Schema Registry, and Delta Streamer Guide for Begineers"
author: Soumil Shah
category: how-to
image: /assets/images/blog/2023-11-26-Real-Time-Data-Processing-with-Postgres-Debezium-Kafka-Schema-Registry-and-DeltaStreamer-Guide-for-Begineers.png
diff --git a/website/blog/2023-11-28-Apache-Hudi-Part-1-History-Getting-Started.mdx b/website/blog/2023-11-28-Apache-Hudi-Part-1-History-Getting-Started.mdx
index 2b5fb32a08b2..2cd7a48698ac 100644
--- a/website/blog/2023-11-28-Apache-Hudi-Part-1-History-Getting-Started.mdx
+++ b/website/blog/2023-11-28-Apache-Hudi-Part-1-History-Getting-Started.mdx
@@ -1,6 +1,7 @@
---
title: "Apache Hudi (Part 1): History, Getting Started"
excerpt: "Apache Hudi (Part 1): History, Getting Started"
+description: "Apache Hudi (Part 1): History, Getting Started"
authors: [dipankar-mazumdar]
category: how-to
image: /assets/images/blog/2023-11-28-Apache-Hudi-Part-1-History-Getting-Started.png
diff --git a/website/blog/2023-11-30-Mastering-Data-Lakes-A-Deep-Dive-into-MINIO-Hudi-and-Delta-Streamer.mdx b/website/blog/2023-11-30-Mastering-Data-Lakes-A-Deep-Dive-into-MINIO-Hudi-and-Delta-Streamer.mdx
index c5c3d4d59338..f993d11aec3c 100644
--- a/website/blog/2023-11-30-Mastering-Data-Lakes-A-Deep-Dive-into-MINIO-Hudi-and-Delta-Streamer.mdx
+++ b/website/blog/2023-11-30-Mastering-Data-Lakes-A-Deep-Dive-into-MINIO-Hudi-and-Delta-Streamer.mdx
@@ -1,6 +1,7 @@
---
title: "Mastering Data Lakes: A Deep Dive into MINIO, Hudi, and Delta Streamer"
excerpt: "A Deep Dive into MINIO, Hudi, and Delta Streamer"
+description: "A Deep Dive into MINIO, Hudi, and Delta Streamer"
author: Soumil Shah
category: how-to
image: /assets/images/blog/2023-11-30-Mastering-Data-Lakes-A-Deep-Dive-into-MINIO-Hudi-and-Delta-Streamer.png
diff --git a/website/blog/2023-12-01-Getting-started-with-Apache-Hudi.mdx b/website/blog/2023-12-01-Getting-started-with-Apache-Hudi.mdx
index ea8f7364d443..7193f51842db 100644
--- a/website/blog/2023-12-01-Getting-started-with-Apache-Hudi.mdx
+++ b/website/blog/2023-12-01-Getting-started-with-Apache-Hudi.mdx
@@ -1,6 +1,7 @@
---
title: "Getting started with Apache Hudi"
excerpt: "Getting started with Apache Hudi"
+description: "Getting started with Apache Hudi"
author: DataCouch
category: how-to
image: /assets/images/blog/2023-12-01-Getting-started-with-Apache-Hudi.png
diff --git a/website/blog/2023-12-06-Apache-Hudi-From-Zero-To-One-blog-7.mdx b/website/blog/2023-12-06-Apache-Hudi-From-Zero-To-One-blog-7.mdx
index 066cfed64050..5dbb987bb77f 100644
--- a/website/blog/2023-12-06-Apache-Hudi-From-Zero-To-One-blog-7.mdx
+++ b/website/blog/2023-12-06-Apache-Hudi-From-Zero-To-One-blog-7.mdx
@@ -1,6 +1,7 @@
---
title: "Apache Hudi: From Zero To One (7/10)"
excerpt: "Concurrently run writers and table services"
+description: "Concurrently run writers and table services"
authors: [xushiyan]
category: deep-dive
image: /assets/images/blog/2023-12-06-Apache-Hudi-From-Zero-To-One-blog-7.png
diff --git a/website/blog/2023-12-09-Getting-started-with-Apache-Hudi.mdx b/website/blog/2023-12-09-Getting-started-with-Apache-Hudi.mdx
index 39f6381a7c90..dc9d7a1da27f 100644
--- a/website/blog/2023-12-09-Getting-started-with-Apache-Hudi.mdx
+++ b/website/blog/2023-12-09-Getting-started-with-Apache-Hudi.mdx
@@ -1,6 +1,7 @@
---
title: "Getting started with Apache Hudi"
excerpt: "Getting started with Apache Hudi"
+description: "Getting started with Apache Hudi"
author: DataCouch
category: how-to
image: /assets/images/blog/2023-12-09-Getting-started-with-Apache-Hudi.png
diff --git a/website/blog/2023-12-13-what-is-apache-hudi.mdx b/website/blog/2023-12-13-what-is-apache-hudi.mdx
index 1168c98942d8..fd32361b2c58 100644
--- a/website/blog/2023-12-13-what-is-apache-hudi.mdx
+++ b/website/blog/2023-12-13-what-is-apache-hudi.mdx
@@ -1,6 +1,7 @@
---
title: "What is Apache Hudi"
excerpt: "What is Apache Hudi"
+description: "What is Apache Hudi"
author: Karim Faiz
category: how-to
image: /assets/images/blog/2023-12-13-what-is-apache-hudi.png
diff --git a/website/blog/2023-12-28-apache-hudi-2023-a-year-in-review.mdx b/website/blog/2023-12-28-apache-hudi-2023-a-year-in-review.mdx
index 314f2c7703ab..0969cfc3bb0e 100644
--- a/website/blog/2023-12-28-apache-hudi-2023-a-year-in-review.mdx
+++ b/website/blog/2023-12-28-apache-hudi-2023-a-year-in-review.mdx
@@ -1,6 +1,7 @@
---
title: "Apache Hudi 2023: A Year In Review"
excerpt: "Reflect on and celebrate the myriad of exciting developments and accomplishments that have defined the year 2023 for the Hudi community."
+description: "Reflect on and celebrate the myriad of exciting developments and accomplishments that have defined the year 2023 for the Hudi community."
authors: [xushiyan]
category: community
image: /assets/images/blog/2023-12-28-a-year-in-review-2023/00.cover.png
diff --git a/website/blog/2024-01-01-From-Data-lake-to-Microservices-Unleashing-the-Power-of-Apache-Hudi-Record-Level-Index-with-FastAPI-and-Spark-Connect.mdx b/website/blog/2024-01-01-From-Data-lake-to-Microservices-Unleashing-the-Power-of-Apache-Hudi-Record-Level-Index-with-FastAPI-and-Spark-Connect.mdx
index 01da5bcb529a..da20d29cc4a9 100644
--- a/website/blog/2024-01-01-From-Data-lake-to-Microservices-Unleashing-the-Power-of-Apache-Hudi-Record-Level-Index-with-FastAPI-and-Spark-Connect.mdx
+++ b/website/blog/2024-01-01-From-Data-lake-to-Microservices-Unleashing-the-Power-of-Apache-Hudi-Record-Level-Index-with-FastAPI-and-Spark-Connect.mdx
@@ -1,6 +1,7 @@
---
title: "From Data lake to Microservices: Unleashing the Power of Apache Hudi's Record Level Index with FastAPI and Spark Connect"
excerpt: "From Data lake to Microservices: Unleashing the Power of Apache Hudi's Record Level Index with FastAPI and Spark Connect"
+description: "From Data lake to Microservices: Unleashing the Power of Apache Hudi's Record Level Index with FastAPI and Spark Connect"
author: Soumil Shah
category: how-to
image: /assets/images/blog/2024-01-01-From-Data-lake-to-Microservices-Unleashing-the-Power-of-Apache-Hudi-Record-Level-Index-with-FastAPI-and-Spark-Connect.png
diff --git a/website/blog/2024-01-02-Build-a-federated-query-solution-with-Apache-Doris-Apache-Flink-and-Apache-Hudi.mdx b/website/blog/2024-01-02-Build-a-federated-query-solution-with-Apache-Doris-Apache-Flink-and-Apache-Hudi.mdx
index d842f8ba67d3..d0accbd1b3fb 100644
--- a/website/blog/2024-01-02-Build-a-federated-query-solution-with-Apache-Doris-Apache-Flink-and-Apache-Hudi.mdx
+++ b/website/blog/2024-01-02-Build-a-federated-query-solution-with-Apache-Doris-Apache-Flink-and-Apache-Hudi.mdx
@@ -1,6 +1,7 @@
---
title: "Build a federated query solution with Apache Doris, Apache Flink, and Apache Hudi"
excerpt: "Build a federated query solution with Apache Doris, Apache Flink, and Apache Hudi"
+description: "Build a federated query solution with Apache Doris, Apache Flink, and Apache Hudi"
author: Apache Doris
category: how-to
image: /assets/images/blog/2024-01-02-Build-a-federated-query-solution-with-Apache-Doris-Apache-Flink-and-Apache-Hudi.png
diff --git a/website/blog/2024-01-05-Apache-Hudi-From-Zero-To-One-blog-8.mdx b/website/blog/2024-01-05-Apache-Hudi-From-Zero-To-One-blog-8.mdx
index 30db6acbdb9a..18ce347899ce 100644
--- a/website/blog/2024-01-05-Apache-Hudi-From-Zero-To-One-blog-8.mdx
+++ b/website/blog/2024-01-05-Apache-Hudi-From-Zero-To-One-blog-8.mdx
@@ -1,6 +1,7 @@
---
title: "Apache Hudi: From Zero To One (8/10)"
excerpt: "Read and process incrementally"
+description: "Read and process incrementally"
authors: [xushiyan]
category: deep-dive
image: /assets/images/blog/2024-01-05-Apache-Hudi-From-Zero-To-One-blog-8.png
diff --git a/website/blog/2024-01-05-Small-Talk-about-Apache-Hudi.mdx b/website/blog/2024-01-05-Small-Talk-about-Apache-Hudi.mdx
index e5857a0a2124..b574f216c6e2 100644
--- a/website/blog/2024-01-05-Small-Talk-about-Apache-Hudi.mdx
+++ b/website/blog/2024-01-05-Small-Talk-about-Apache-Hudi.mdx
@@ -1,6 +1,7 @@
---
title: "Small Talk about Apache Hudi"
excerpt: "Small Talk about Apache Hudi"
+description: "Small Talk about Apache Hudi"
author: Ashok Kumar Kunkala
category: how-to
image: /assets/images/blog/2024-01-05-Small-Talk-about-Apache-Hudi.png
diff --git a/website/blog/2024-01-09-introduction-to-apache-hudi.mdx b/website/blog/2024-01-09-introduction-to-apache-hudi.mdx
index cdb0291069ef..5f645dd99162 100644
--- a/website/blog/2024-01-09-introduction-to-apache-hudi.mdx
+++ b/website/blog/2024-01-09-introduction-to-apache-hudi.mdx
@@ -1,6 +1,7 @@
---
title: "Introduction to Apache Hudi"
excerpt: "Introduction to Apache Hudi"
+description: "Introduction to Apache Hudi"
author: Andrew Savchyns
category: how-to
image: /assets/images/blog/2024-01-09-introduction-to-apache-hudi.png
diff --git a/website/blog/2024-01-11-In-House-Data-Lake-with-CDC-Processing-Hudi-Docker.mdx b/website/blog/2024-01-11-In-House-Data-Lake-with-CDC-Processing-Hudi-Docker.mdx
index 322008ccb265..b607837a7ea3 100644
--- a/website/blog/2024-01-11-In-House-Data-Lake-with-CDC-Processing-Hudi-Docker.mdx
+++ b/website/blog/2024-01-11-In-House-Data-Lake-with-CDC-Processing-Hudi-Docker.mdx
@@ -1,6 +1,7 @@
---
title: "In-House Data Lake with CDC Processing, Hudi, Docker"
excerpt: "In-House Data Lake with CDC Processing, Hudi, Docker"
+description: "In-House Data Lake with CDC Processing, Hudi, Docker"
author: Rahul
category: how-to
image: /assets/images/blog/2024-01-11-In-House-Data-Lake-with-CDC-Processing-Hudi-Docker.png
diff --git a/website/blog/2024-01-17-Enforce-fine-grained-access-control-on-Open-Table-Formats-via-Amazon-EMR-integrated-with-AWS-Lake-Formation.mdx b/website/blog/2024-01-17-Enforce-fine-grained-access-control-on-Open-Table-Formats-via-Amazon-EMR-integrated-with-AWS-Lake-Formation.mdx
index f4bfdc769485..e8474e0b5a68 100644
--- a/website/blog/2024-01-17-Enforce-fine-grained-access-control-on-Open-Table-Formats-via-Amazon-EMR-integrated-with-AWS-Lake-Formation.mdx
+++ b/website/blog/2024-01-17-Enforce-fine-grained-access-control-on-Open-Table-Formats-via-Amazon-EMR-integrated-with-AWS-Lake-Formation.mdx
@@ -1,6 +1,7 @@
---
title: "Enforce fine-grained access control on Open Table Formats via Amazon EMR integrated with AWS Lake Formation"
excerpt: "Enforce fine-grained access control on Open Table Formats via Amazon EMR integrated with AWS Lake Formation"
+description: "Enforce fine-grained access control on Open Table Formats via Amazon EMR integrated with AWS Lake Formation"
author: Raymond Lai, Aditya Shah, Bin Wang, and Melody Yang
category: how-to
image: /assets/images/blog/2024-01-17-Enforce-fine-grained-access-control-on-Open-Table-Formats-via-Amazon-EMR-integrated-with-AWS-Lake-Formation.png
diff --git a/website/blog/2024-01-18-Deleting-Items-from-Apache-Hudi-using-Delta-Streamer-in-UPSERT-Mode-with-Kafka-Avro-Messages.mdx b/website/blog/2024-01-18-Deleting-Items-from-Apache-Hudi-using-Delta-Streamer-in-UPSERT-Mode-with-Kafka-Avro-Messages.mdx
index 7c30b523387a..af27ebc161d7 100644
--- a/website/blog/2024-01-18-Deleting-Items-from-Apache-Hudi-using-Delta-Streamer-in-UPSERT-Mode-with-Kafka-Avro-Messages.mdx
+++ b/website/blog/2024-01-18-Deleting-Items-from-Apache-Hudi-using-Delta-Streamer-in-UPSERT-Mode-with-Kafka-Avro-Messages.mdx
@@ -1,6 +1,7 @@
---
title: "Deleting Items from Apache Hudi using Delta Streamer in UPSERT Mode with Kafka Avro Messages"
excerpt: "Deleting Items from Apache Hudi using Delta Streamer in UPSERT Mode with Kafka Avro Messages"
+description: "Deleting Items from Apache Hudi using Delta Streamer in UPSERT Mode with Kafka Avro Messages"
author: Soumil Shah
category: how-to
image: /assets/images/blog/2024-01-18-Deleting-Items-from-Apache-Hudi-using-Delta-Streamer-in-UPSERT-Mode-with-Kafka-Avro-Messages.png
diff --git a/website/blog/2024-01-20-Data-Engineering-Bootstrapping-Data-lake-with-Apache-Hudi.mdx b/website/blog/2024-01-20-Data-Engineering-Bootstrapping-Data-lake-with-Apache-Hudi.mdx
index e3646f55b504..8b4f5eb2297e 100644
--- a/website/blog/2024-01-20-Data-Engineering-Bootstrapping-Data-lake-with-Apache-Hudi.mdx
+++ b/website/blog/2024-01-20-Data-Engineering-Bootstrapping-Data-lake-with-Apache-Hudi.mdx
@@ -1,6 +1,7 @@
---
title: "Data Engineering: Bootstrapping Data lake with Apache Hudi"
excerpt: "Data Engineering: Bootstrapping Data lake with Apache Hudi"
+description: "Data Engineering: Bootstrapping Data lake with Apache Hudi"
author: Krishna Prasad
category: how-to
image: /assets/images/blog/2024-01-20-Data-Engineering-Bootstrapping-Data-lake-with-Apache-Hudi.png
diff --git a/website/blog/2024-01-20-Learn-How-to-Move-Data-From-MongoDB-to-Apache-Hudi-Using-PySpark.mdx b/website/blog/2024-01-20-Learn-How-to-Move-Data-From-MongoDB-to-Apache-Hudi-Using-PySpark.mdx
index 8d49cfab41bf..4a0bd640f81e 100644
--- a/website/blog/2024-01-20-Learn-How-to-Move-Data-From-MongoDB-to-Apache-Hudi-Using-PySpark.mdx
+++ b/website/blog/2024-01-20-Learn-How-to-Move-Data-From-MongoDB-to-Apache-Hudi-Using-PySpark.mdx
@@ -1,6 +1,7 @@
---
title: "Learn How to Move Data From MongoDB to Apache Hudi Using PySpark"
excerpt: "Learn How to Move Data From MongoDB to Apache Hudi Using PySpark"
+description: "Learn How to Move Data From MongoDB to Apache Hudi Using PySpark"
author: Soumil Shah
category: how-to
image: /assets/images/blog/2024-01-20-Learn-How-to-Move-Data-From-MongoDB-to-Apache-Hudi-Using-PySpark.png
diff --git a/website/blog/2024-01-24-Use-Amazon-Athena-with-Spark-SQL-for-your-open-source-transactional-table-formats.mdx b/website/blog/2024-01-24-Use-Amazon-Athena-with-Spark-SQL-for-your-open-source-transactional-table-formats.mdx
index cc0b1bc5546e..6f56b4e440b3 100644
--- a/website/blog/2024-01-24-Use-Amazon-Athena-with-Spark-SQL-for-your-open-source-transactional-table-formats.mdx
+++ b/website/blog/2024-01-24-Use-Amazon-Athena-with-Spark-SQL-for-your-open-source-transactional-table-formats.mdx
@@ -1,6 +1,7 @@
---
title: "Use Amazon Athena with Spark SQL for your open-source transactional table formats"
excerpt: "Use Amazon Athena with Spark SQL for your open-source transactional table formats"
+description: "Use Amazon Athena with Spark SQL for your open-source transactional table formats"
author: Pathik Shah, Raj Devnath
category: how-to
image: /assets/images/blog/2024-01-24-Use-Amazon-Athena-with-Spark-SQL-for-your-open-source-transactional-table-formats.png
diff --git a/website/blog/2024-01-30-Leverage-Partition-Paths-of-your-data-lake-tables-to-Optimize-Data-Retrieval-Costs-on-the-cloud.mdx b/website/blog/2024-01-30-Leverage-Partition-Paths-of-your-data-lake-tables-to-Optimize-Data-Retrieval-Costs-on-the-cloud.mdx
index e3ebcc7a57b6..2e3a2f081041 100644
--- a/website/blog/2024-01-30-Leverage-Partition-Paths-of-your-data-lake-tables-to-Optimize-Data-Retrieval-Costs-on-the-cloud.mdx
+++ b/website/blog/2024-01-30-Leverage-Partition-Paths-of-your-data-lake-tables-to-Optimize-Data-Retrieval-Costs-on-the-cloud.mdx
@@ -1,6 +1,7 @@
---
title: "Leverage Partition Paths of your data lake tables to Optimize Data Retrieval Costs on the cloud"
excerpt: "Leverage Partition Paths of your data lake tables to Optimize Data Retrieval Costs on the cloud"
+description: "Leverage Partition Paths of your data lake tables to Optimize Data Retrieval Costs on the cloud"
author: Krishna Prasad
category: how-to
image: /assets/images/blog/2024-01-30-Leverage-Partition-Paths-of-your-data-lake-tables-to-Optimize-Data-Retrieval-Costs-on-the-cloud.png
diff --git a/website/blog/2024-02-04-Apache-Hudi-Managing-Partition-on-a-petabyte-scale-table.mdx b/website/blog/2024-02-04-Apache-Hudi-Managing-Partition-on-a-petabyte-scale-table.mdx
index ab33a49d548a..bc3bb81a487a 100644
--- a/website/blog/2024-02-04-Apache-Hudi-Managing-Partition-on-a-petabyte-scale-table.mdx
+++ b/website/blog/2024-02-04-Apache-Hudi-Managing-Partition-on-a-petabyte-scale-table.mdx
@@ -1,6 +1,7 @@
---
title: "Apache Hudi: Managing Partition on a petabyte-scale table"
excerpt: "Apache Hudi: Managing Partition on a petabyte-scale table"
+description: "Apache Hudi: Managing Partition on a petabyte-scale table"
author: Krishna Prasad
category: how-to
image: /assets/images/blog/2024-02-04-Apache-Hudi-Managing-Partition-on-a-petabyte-scale-table.png
diff --git a/website/blog/2024-02-06-Building-an-Open-Source-Data-Lake-House-with-Hudi-Postgres-Hive-Metastore-Minio-and-StarRocks.mdx b/website/blog/2024-02-06-Building-an-Open-Source-Data-Lake-House-with-Hudi-Postgres-Hive-Metastore-Minio-and-StarRocks.mdx
index 6eccca708efe..eaad2ce44dd3 100644
--- a/website/blog/2024-02-06-Building-an-Open-Source-Data-Lake-House-with-Hudi-Postgres-Hive-Metastore-Minio-and-StarRocks.mdx
+++ b/website/blog/2024-02-06-Building-an-Open-Source-Data-Lake-House-with-Hudi-Postgres-Hive-Metastore-Minio-and-StarRocks.mdx
@@ -1,6 +1,7 @@
---
title: "Building an Open Source Data Lake House with Hudi, Postgres Hive Metastore, Minio, and StarRocks"
excerpt: "Building an Open Source Data Lake House with Hudi, Postgres Hive Metastore, Minio, and StarRocks"
+description: "Building an Open Source Data Lake House with Hudi, Postgres Hive Metastore, Minio, and StarRocks"
author: Soumil Shah
category: how-to
image: /assets/images/blog/2024-02-06-Building-an-Open-Source-Data-Lake-House-with-Hudi-Postgres-Hive-Metastore-Minio-and-StarRocks.png
diff --git a/website/blog/2024-02-12-How-a-POC-became-a-production-ready-Hudi-data-lakehouse-through-close-team-collaboration.mdx b/website/blog/2024-02-12-How-a-POC-became-a-production-ready-Hudi-data-lakehouse-through-close-team-collaboration.mdx
index 1341341dbb70..e95764c51bca 100644
--- a/website/blog/2024-02-12-How-a-POC-became-a-production-ready-Hudi-data-lakehouse-through-close-team-collaboration.mdx
+++ b/website/blog/2024-02-12-How-a-POC-became-a-production-ready-Hudi-data-lakehouse-through-close-team-collaboration.mdx
@@ -1,6 +1,7 @@
---
title: "How a POC became a production-ready Hudi data lakehouse through close team collaboration"
excerpt: "How a POC became a production-ready Hudi data lakehouse through close team collaboration"
+description: "How a POC became a production-ready Hudi data lakehouse through close team collaboration"
author: Xiaoxiao Rey and Hussein Awala
category: how-to
image: /assets/images/blog/2024-02-12-How-a-POC-became-a-production-ready-Hudi-data-lakehouse-through-close-team-collaboration.png
diff --git a/website/blog/2024-02-27-Building-Data-Lakes-on-AWS-with-Kafka-Connect-Debezium-Apicurio-Registry-and-Apache-Hudi.mdx b/website/blog/2024-02-27-Building-Data-Lakes-on-AWS-with-Kafka-Connect-Debezium-Apicurio-Registry-and-Apache-Hudi.mdx
index df98f62b95d3..2b5799939372 100644
--- a/website/blog/2024-02-27-Building-Data-Lakes-on-AWS-with-Kafka-Connect-Debezium-Apicurio-Registry-and-Apache-Hudi.mdx
+++ b/website/blog/2024-02-27-Building-Data-Lakes-on-AWS-with-Kafka-Connect-Debezium-Apicurio-Registry-and-Apache-Hudi.mdx
@@ -1,6 +1,7 @@
---
title: "Building Data Lakes on AWS with Kafka Connect, Debezium, Apicurio Registry, and Apache Hudi"
excerpt: "Building Data Lakes on AWS with Kafka Connect, Debezium, Apicurio Registry, and Apache Hudi"
+description: "Building Data Lakes on AWS with Kafka Connect, Debezium, Apicurio Registry, and Apache Hudi"
author: Gary A. Stafford
category: how-to
image: /assets/images/blog/2024-02-27-Building-Data-Lakes-on-AWS-with-Kafka-Connect-Debezium-Apicurio-Registry-and-Apache-Hudi.png
diff --git a/website/blog/2024-03-05-Apache-Hudi-From-Zero-To-One-blog-9.mdx b/website/blog/2024-03-05-Apache-Hudi-From-Zero-To-One-blog-9.mdx
index 9efb6b8c7cb4..be5acb1e806d 100644
--- a/website/blog/2024-03-05-Apache-Hudi-From-Zero-To-One-blog-9.mdx
+++ b/website/blog/2024-03-05-Apache-Hudi-From-Zero-To-One-blog-9.mdx
@@ -1,6 +1,7 @@
---
title: "Apache Hudi: From Zero To One (9/10)"
excerpt: "Hudi Streamer - a Swiss Army knife for ingestion"
+description: "Hudi Streamer - a Swiss Army knife for ingestion"
authors: [xushiyan]
category: deep-dive
image: /assets/images/blog/2024-03-05-Apache-Hudi-From-Zero-To-One-blog-9.png
diff --git a/website/blog/2024-04-13-Apache-Hudi-From-Zero-To-One-blog-10.mdx b/website/blog/2024-04-13-Apache-Hudi-From-Zero-To-One-blog-10.mdx
index 8d2332f4c22c..3da839c0bffe 100644
--- a/website/blog/2024-04-13-Apache-Hudi-From-Zero-To-One-blog-10.mdx
+++ b/website/blog/2024-04-13-Apache-Hudi-From-Zero-To-One-blog-10.mdx
@@ -1,6 +1,7 @@
---
title: "Apache Hudi: From Zero To One (10/10)"
excerpt: "Becoming 'One' - the upcoming 1.0 highlights"
+description: "Becoming 'One' - the upcoming 1.0 highlights"
authors: [xushiyan]
category: deep-dive
image: /assets/images/blog/2024-04-13-Apache-Hudi-From-Zero-To-One-blog-10.jpg
diff --git a/website/blog/2024-07-11-what-is-a-data-lakehouse.md b/website/blog/2024-07-11-what-is-a-data-lakehouse.md
index 3f3133440e30..dd97b28c2454 100644
--- a/website/blog/2024-07-11-what-is-a-data-lakehouse.md
+++ b/website/blog/2024-07-11-what-is-a-data-lakehouse.md
@@ -1,9 +1,12 @@
---
title: "What is a Data Lakehouse & How does it Work?"
excerpt: "Explains the concept of the lakehouse architecture"
+description: "Explains the concept of the lakehouse architecture"
authors: [dipankar-mazumdar]
category: how-to
image: /assets/images/blog/dlh_1200.png
+last_update:
+ date: 2026-07-06
tags:
- data lakehouse
- apache iceberg
@@ -11,7 +14,7 @@ tags:
- open architecture
---
-A data lakehouse is a hybrid data architecture that combines the best attributes of data warehouses and data lakes to address their respective limitations. This innovative approach to data management brings the transactional capabilities of data warehouses to cloud-based data lakes, offering scalability at lower costs.
+A data lakehouse is a hybrid data architecture that combines the best attributes of data warehouses and data lakes to address their respective limitations. It works by layering an open table format—such as Apache Hudi, Apache Iceberg, or Delta Lake—over low-cost cloud object storage, bringing the transactional capabilities of data warehouses to cloud-based data lakes while offering scalability at lower costs.
<p align = "center">Figure: Data Lakehouse Architecture</p>
@@ -151,3 +154,13 @@ The future of data lakehouses is shaped by their truly open data architecture, w
## Conclusion
The data lakehouse architecture utilizes an open data foundation to blend the best features of data lakes and warehouses, establishing a versatile platform that effectively handles a range of analytical workloads. This architecture marries cost-effective data management with robust performance, offering a cohesive system for both batch and streaming data processes. By enabling organizations to work on a single data store, this approach not only simplifies management but also equips busin [...]
+
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {question: 'What is a data lakehouse?', answer: 'A data lakehouse is a hybrid data architecture that combines the transactional capabilities of data warehouses with the scalability and low cost of cloud data lakes. It stores data in open formats on cloud object storage and adds a table format layer that provides ACID transactions, schema management, and time travel. This lets organizations run BI, machine learning, and streaming workloads on a single data store.'},
+ {question: 'How is a data lakehouse different from a data warehouse or a data lake?', answer: 'Data warehouses offer strong transactional guarantees but rely on proprietary storage formats that risk lock-in and handle unstructured data poorly. Data lakes store any data type cheaply in open file formats but lack ACID transactions and often struggle with data quality and governance. A lakehouse merges the two, bringing warehouse-like reliability and performance to openly accessible data [...]
+ {question: 'What are the main components of a data lakehouse architecture?', answer: 'A lakehouse consists of six modular components: lake storage (cloud object stores like Amazon S3), open file formats (Apache Parquet or ORC), a table format (Apache Hudi, Apache Iceberg, or Delta Lake), a storage engine that handles clustering, compaction, cleaning, and indexing, a catalog that tracks tables and their metadata, and compute engines such as Spark, Flink, Presto, or Trino that read and w [...]
+ {question: 'What role does the table format play in a lakehouse?', answer: 'The table format is an open metadata layer above file formats like Apache Parquet that defines a schema on top of immutable data files. It allows different engines to concurrently read and write the same dataset with ACID transactions, and brings features such as schema evolution, partitioning, and time travel.'},
+ {question: 'Why is Apache Hudi considered more than a table format?', answer: 'Apache Hudi is a transactional data lakehouse platform built around a database kernel, not just a metadata specification. Beyond table-level abstractions, it includes tightly integrated table services for clustering, compaction, cleaning, and indexing, plus platform services like ingestion tools, catalog sync, and an admin CLI. It also supports fast updates and deletes through a pluggable indexing mechanism [...]
+]} />
diff --git a/website/blog/2024-07-30-data-lake-cdc.md b/website/blog/2024-07-30-data-lake-cdc.md
index 1540a5805cf6..9954f120611f 100644
--- a/website/blog/2024-07-30-data-lake-cdc.md
+++ b/website/blog/2024-07-30-data-lake-cdc.md
@@ -1,14 +1,19 @@
---
title: "Understanding Data Lake Change Data Capture"
excerpt: "Explains the concept of CDC in data lakes"
+description: "Explains the concept of CDC in data lakes"
authors: [sagar-lakshmipathy]
category: deep-dive
image: /assets/images/blog/data-lake-cdc/hudi-cdc.jpg
+last_update:
+ date: 2026-07-06
tags:
- data lakehouse
- cdc
---
+Data lake change data capture (CDC) is a technique that continuously identifies and captures changes—inserts, updates, and deletes—from source databases and applies them to a data lake, keeping lake tables fresh and consistent with operational systems. Combining CDC with data lakes addresses several challenges commonly faced by ETL pipelines delivering data from transactional databases to analytical systems: maintaining data freshness, ensuring consistency, and improving efficiency in da [...]
+
## Introduction
In data management, two concepts have garnered significant attention: data lakes and change data capture (CDC).
### Data Lake
@@ -101,5 +106,15 @@ Several methods exist for implementing CDC, including timestamp-based, trigger-b
Using tools like Apache Hudi can significantly enhance the CDC process by streamlining incremental data processing and data pipeline development. Hudi provides efficient storage management, supports record-level operations for privacy regulations, and offers near real-time access to data. It also simplifies the management of streaming data and ingestion pipelines by automatically tracking changes and optimizing file sizes, thereby reducing the need for custom solutions.
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {question: 'What is change data capture (CDC)?', answer: 'Change data capture is a technique used to identify and capture data changes—inserts, updates, and deletes—from a source system so they can be applied to downstream systems. It ensures data remains fresh and consistent across various systems without repeatedly transferring entire datasets.'},
+ {question: 'What are the main methods of implementing CDC?', answer: 'The three common approaches are timestamp-based (querying a LAST_UPDATED column), trigger-based (database triggers writing changes to shadow tables), and log-based (reading the database transaction log). Each has trade-offs: timestamp-based is simple but cannot track deletes, trigger-based adds maintenance and performance overhead, and log-based requires handling vendor-specific log formats.'},
+ {question: 'Why is log-based CDC generally preferred?', answer: 'Log-based CDC reads changes directly from the database transaction log, so it has minimal performance impact on the source database. It supports all transaction types including inserts, updates, and deletes, and requires no schema changes in the source system. Its main drawback is the lack of standardization in transaction log formats across database vendors.'},
+ {question: 'Why combine CDC with a data lake?', answer: 'Data lakes offer flexibility and lower storage costs than warehouses, supporting structured, semi-structured, and unstructured data queryable from multiple engines. CDC streamlines ETL by continuously capturing and applying only the changed data, reducing complexity, processing overhead, and load on source systems.'},
+ {question: 'How does Apache Hudi help with CDC on a data lake?', answer: 'Apache Hudi streamlines incremental data processing and supports record-level insert, update, upsert, and delete operations, which simplifies compliance with privacy regulations like GDPR and CCPA. Starting with release 0.13.0, Hudi natively supports CDC by logging before and after images of changed records along with the write operation type. It also automatically manages file sizes and integrates with engines l [...]
+]} />
+
diff --git a/website/blog/2024-07-31-hudi-file-formats.md b/website/blog/2024-07-31-hudi-file-formats.md
index 597783a95548..2a81fb0b3564 100644
--- a/website/blog/2024-07-31-hudi-file-formats.md
+++ b/website/blog/2024-07-31-hudi-file-formats.md
@@ -1,6 +1,7 @@
---
title: "Column File Formats: How Hudi Leverages Parquet and ORC "
excerpt: "Explains how Hudi uses Parquet and ORC"
+description: "Explains how Hudi uses Parquet and ORC"
author: Albert Wong
category: deep-dive
image: /assets/images/blog/hudi-parquet-orc.png
diff --git a/website/blog/2024-11-19-automated-small-file-handling.md b/website/blog/2024-11-19-automated-small-file-handling.md
index 40745c045355..b81c0fc6bbb3 100644
--- a/website/blog/2024-11-19-automated-small-file-handling.md
+++ b/website/blog/2024-11-19-automated-small-file-handling.md
@@ -1,6 +1,7 @@
---
title: "Hudi’s Automatic File Sizing Delivers Unmatched Performance"
excerpt: "Explains how Hudi handles small files during ingestion and its benefits"
+description: "Explains how Hudi handles small files during ingestion and its benefits"
authors: [aditya-goenka]
category: deep-dive
image: /assets/images/blog/2024-06-07-apache-hudi-a-deep-dive-with-python-code-examples.png
diff --git a/website/blog/2024-12-06-non-blocking-concurrency-control.md b/website/blog/2024-12-06-non-blocking-concurrency-control.md
index f51218dc8319..6f5d0eb99823 100644
--- a/website/blog/2024-12-06-non-blocking-concurrency-control.md
+++ b/website/blog/2024-12-06-non-blocking-concurrency-control.md
@@ -1,14 +1,23 @@
---
title: "Introducing Hudi's Non-blocking Concurrency Control for streaming, high-frequency writes"
excerpt: "Announcing the Non-blocking Concurrency Control in Apache Hudi"
+description: "Announcing the Non-blocking Concurrency Control in Apache Hudi"
authors: [danny-chan]
category: deep-dive
image: /assets/images/blog/non-blocking-concurrency-control/lsm_archive_timeline.png
+last_update:
+ date: 2026-07-12
tags:
- streaming
- concurrency control
---
+Non-blocking Concurrency Control (NBCC) is a multi-writer concurrency model introduced in Apache Hudi 1.0 that lets multiple streaming writers ingest into the same table simultaneously, without any conflict-resolution overhead at write time - conflicts are instead resolved later by the compaction service and query readers, while preserving event-time ordering semantics. This post explains the storage layout and timeline changes that make NBCC possible, walks through its design, and shows [...]
+
+:::info Related reading
+See the [concurrency control docs](/docs/concurrency_control) for configuration details, the [survey of concurrency control across open table formats](/blog/2025/01/28/concurrency-control) for how NBCC fits alongside OCC and MVCC, and [Lakehouse Concurrency Control: Are we too optimistic?](/blog/2021/12/16/lakehouse-concurrency-control-are-we-too-optimistic/) for the motivation behind moving past OCC.
+:::
+
## Introduction
In streaming ingestion scenarios, there are plenty of use cases that require concurrent ingestion from multiple streaming sources.
@@ -161,4 +170,14 @@ here are some plans that improve the Hudi core features:
---
-[^1] [RFC-66](https://github.com/apache/hudi/blob/master/rfc/rfc-66/rfc-66.md) well-explained the completion time based file slicing with a pseudocode.
\ No newline at end of file
+[^1] [RFC-66](https://github.com/apache/hudi/blob/master/rfc/rfc-66/rfc-66.md) well-explained the completion time based file slicing with a pseudocode.
+
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {question: 'What is non-blocking concurrency control (NBCC) in Apache Hudi?', answer: 'NBCC is a general-purpose concurrency model, introduced in Hudi 1.0.0, aimed at stream processing and high-contention or frequent-writing scenarios. It allows multiple streaming writers to write to the same Hudi table without any conflict-resolution overhead, while keeping event-time ordering semantics and supporting asynchronous table services like compaction, archiving, and cleaning.'},
+ {question: 'How is NBCC different from optimistic concurrency control (OCC)?', answer: 'Under OCC, writers abort the transaction if there is a hint of contention. Under NBCC, concurrent writers all proceed: the log writers append records without coordinating, and the compaction table service takes care of conflict resolution. Readers can merge records in either natural (processing time) order or event-time order.'},
+ {question: 'What storage changes in Hudi 1.0 made NBCC possible?', answer: 'Hudi 1.0 introduced a new file layout based on both requested and completion times of actions, treating them as an interval. Log file names now carry the write\'s own requested instant time instead of the base instant time, so writers no longer query the file layout before flushing, and compaction can be scheduled at any time without blocking writers. An LSM-tree-based archived timeline supports the fast instan [...]
+ {question: 'How do I enable NBCC?', answer: 'Set hoodie.write.concurrency.mode to NON_BLOCKING_CONCURRENCY_CONTROL. The initial implementation works with the simple bucket index on a merge-on-read table for Flink. When running multiple jobs against the same table, enable table services such as compaction and cleaning on one job and disable them on the others.'},
+ {question: 'What are the current limitations and future plans for NBCC?', answer: 'NBCC currently relies on the bucket index on MOR tables. The roadmap includes NBCC support for the metadata table, for clustering, and for other index types.'},
+]} />
\ No newline at end of file
diff --git a/website/blog/2024-12-16-announcing-hudi-1-0-0.mdx b/website/blog/2024-12-16-announcing-hudi-1-0-0.mdx
index c3b269e24b40..ffb0d5b2bf8b 100644
--- a/website/blog/2024-12-16-announcing-hudi-1-0-0.mdx
+++ b/website/blog/2024-12-16-announcing-hudi-1-0-0.mdx
@@ -1,6 +1,7 @@
---
title: "Announcing Apache Hudi 1.0 and the Next Generation of Data Lakehouses"
excerpt: "game-changing major release, that reimagines Hudi and Data Lakehouses."
+description: "game-changing major release, that reimagines Hudi and Data Lakehouses."
authors: [vinoth-chandar]
category: community
image: /assets/images/blog/dlms-hierarchy.png
diff --git a/website/blog/2024-12-29-apache-hudi-2024-a-year-in-review.mdx b/website/blog/2024-12-29-apache-hudi-2024-a-year-in-review.mdx
index a1f9ef1bee7f..2ccc8809b1da 100644
--- a/website/blog/2024-12-29-apache-hudi-2024-a-year-in-review.mdx
+++ b/website/blog/2024-12-29-apache-hudi-2024-a-year-in-review.mdx
@@ -1,6 +1,7 @@
---
title: "Apache Hudi 2024: A Year In Review"
excerpt: "Reflect on and celebrate the myriad of exciting developments and accomplishments that have defined the year 2024 for the Hudi community."
+description: "Reflect on and celebrate the myriad of exciting developments and accomplishments that have defined the year 2024 for the Hudi community."
authors: [xushiyan]
category: community
image: /assets/images/blog/2024-12-29-a-year-in-review-2024/cover.jpg
diff --git a/website/blog/2025-01-15-outofbox-key-generators-in-hudi.mdx b/website/blog/2025-01-15-outofbox-key-generators-in-hudi.mdx
index 6d7de88de280..53e83a3369f5 100644
--- a/website/blog/2025-01-15-outofbox-key-generators-in-hudi.mdx
+++ b/website/blog/2025-01-15-outofbox-key-generators-in-hudi.mdx
@@ -1,6 +1,7 @@
---
title: "Out of the box Key Generators in Apache Hudi"
excerpt: "Explain need for key gerators and out of box key generators in Apache Hudi"
+description: "Explain need for key gerators and out of box key generators in Apache Hudi"
authors: [aditya-goenka]
category: how-to
image: /assets/images/blog/2024-06-07-apache-hudi-a-deep-dive-with-python-code-examples.png
diff --git a/website/blog/2025-01-28-concurrency-control.md b/website/blog/2025-01-28-concurrency-control.md
index a699e46dacc6..77b91d75b649 100644
--- a/website/blog/2025-01-28-concurrency-control.md
+++ b/website/blog/2025-01-28-concurrency-control.md
@@ -1,15 +1,24 @@
---
title: "Concurrency Control in Open Data Lakehouse"
excerpt: "How various concurrency control techniques works in Apache Hudi, Apache Iceberg & Delta Lake"
+description: "How various concurrency control techniques works in Apache Hudi, Apache Iceberg & Delta Lake"
authors: [dipankar-mazumdar]
category: deep-dive
image: /assets/images/blog/concurrency_control/concurrency_blog_thumb.jpg
+last_update:
+ date: 2026-07-12
tags:
- concurrency control
- apache iceberg
- delta lake
---
+Concurrency control in an open data lakehouse determines how multiple writers, table services, and readers can safely operate on the same table at once, and open table formats implement it using methods adapted from database systems: optimistic concurrency control (OCC), multi-version concurrency control (MVCC), and - in Apache Hudi - non-blocking concurrency control (NBCC). This post covers the database foundations of these methods, then examines how Apache Hudi, Apache Iceberg, and Del [...]
+
+:::info Related reading
+See the [Hudi concurrency control docs](/docs/concurrency_control) for configuration reference, [Lakehouse Concurrency Control: Are we too optimistic?](/blog/2021/12/16/lakehouse-concurrency-control-are-we-too-optimistic/) for a critique of OCC under contention, and the [NBCC deep dive](/blog/2024/12/06/non-blocking-concurrency-control) for how non-blocking multi-writer support works.
+:::
+
## Introduction
Concurrency control is critical in database management systems to ensure consistent and safe access to shared data by multiple users. Relational databases (RDBMS) such as [MySQL (InnoDB)](https://dev.mysql.com/doc/refman/5.7/en/innodb-locking-transaction-model.html) and analytical databases (such as data warehouses) have been offering robust concurrency control mechanisms to effectively deal with this. As data grows in scale and complexity, managing concurrent access becomes more challen [...]
@@ -208,4 +217,14 @@ Additionally, Delta Lake employs Multi-Version Concurrency Control (MVCC) within
Concurrency control is critical for Open lakehouse architectures, especially when your architecture has multiple concurrent pipelines interacting with the same table. Open table formats such as Apache Hudi bring well-established concurrency control methods from traditional database systems into the Lakehouse architecture to handle these operations while maintaining data consistency and scalability. Apache Hudi’s unique design to distinguish between writers, table services, and readers en [...]
----
\ No newline at end of file
+---
+
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {question: 'What concurrency control methods do open table formats use?', answer: 'The main methods, adapted from database systems, are pessimistic locking (2PL), optimistic concurrency control (OCC), and multi-version concurrency control (MVCC). Apache Iceberg and Delta Lake rely on OCC for concurrent writes, with Delta Lake also using MVCC to separate reads from writes. Apache Hudi supports OCC between writers, MVCC between writers and table services, and additionally non-blocking co [...]
+ {question: 'When does optimistic concurrency control become a problem?', answer: 'OCC assumes conflicts are rare, so it suits simple, append-only jobs. With frequent updates or deletes, a detected conflict aborts the job entirely. For example, an ingest job writing every 30 minutes and a deletion job running every two hours may often overlap, causing the deletion job to fail repeatedly - and with long-running transactions the chance of conflict grows over time.'},
+ {question: 'How does Apache Hudi reduce contention between writers and table services?', answer: 'Hudi distinguishes between writers, table services, and readers, providing snapshot isolation across all three. It uses MVCC between writers and table services, so operations like compaction and clustering run without blocking active writes. Locks are acquired only at critical points, such as during a commit, rather than across the whole transaction, and OCC conflict detection works at the [...]
+ {question: 'What is non-blocking concurrency control (NBCC)?', answer: 'NBCC is a concurrency mode introduced in Hudi 1.0 where multiple writers operate on the same table simultaneously, with conflicts resolved automatically by query readers and the compactor rather than by aborting writers. The only lock needed is for writing commit metadata to the timeline, and the final serialization of writes is determined by completion times using Hudi\'s TrueTime semantics.'},
+ {question: 'Do I need an external lock provider to run multiple Hudi writers?', answer: 'For multi-writer setups using OCC, yes - an external lock provider such as Amazon DynamoDB, Zookeeper, or Hive Metastore coordinates concurrent access, together with hoodie.write.concurrency.mode=optimistic_concurrency_control and a LAZY failed-writes cleaning policy. Table services can still run lock-free and asynchronously when they operate in the same process as the writer.'},
+]} />
\ No newline at end of file
diff --git a/website/blog/2025-03-03-record-mergers-in-hudi.mdx b/website/blog/2025-03-03-record-mergers-in-hudi.mdx
index a26cff927823..d39ec83f1d3c 100644
--- a/website/blog/2025-03-03-record-mergers-in-hudi.mdx
+++ b/website/blog/2025-03-03-record-mergers-in-hudi.mdx
@@ -1,6 +1,7 @@
---
title: "Record Mergers in Apache Hudi"
excerpt: "Explain need for record mergers in Apache Hudi and implemenation details"
+description: "Explain need for record mergers in Apache Hudi and implemenation details"
authors: [aditya-goenka]
category: deep-dive
image: /assets/images/blog/2025-03-03-record-mergers-in-apache-hudi.png
diff --git a/website/blog/2025-03-05-hudi-21-unique-differentiators.mdx b/website/blog/2025-03-05-hudi-21-unique-differentiators.mdx
index 6b32a660ac66..ba4c21f914cd 100644
--- a/website/blog/2025-03-05-hudi-21-unique-differentiators.mdx
+++ b/website/blog/2025-03-05-hudi-21-unique-differentiators.mdx
@@ -1,9 +1,12 @@
---
title: "21 Unique Reasons Why Apache Hudi Should Be Your Next Data Lakehouse"
excerpt: "Unique Differentiators of Apache Hudi, that stand out from other projects"
+description: "Unique Differentiators of Apache Hudi, that stand out from other projects"
authors: [vinoth-chandar]
category: deep-dive
image: /assets/images/blog/2025-03-05-21-reasons-why.png
+last_update:
+ date: 2026-07-12
tags:
- data lakehouse
- apache iceberg
@@ -11,7 +14,7 @@ tags:
- table format
---
-Apache Hudi is continuously [redefining](https://hudi.apache.org/blog/2024/12/16/announcing-hudi-1-0-0) the data lakehouse, pushing the technical boundaries and offering cutting-edge features to handle data quickly and efficiently. If you have ever wondered how Apache Hudi has sustained its position over the years as the most comprehensive, open, high-performance data lakehouse project, this blog aims to give you some concise answers. Below, we shine a light on some unique capabilities i [...]
+What actually differentiates Apache Hudi from other data lakehouse projects is that it is a complete database-like platform for the lake - pairing an open storage format with [database-style indexes](/docs/indexes), [concurrency control purpose-built for lake workloads](/docs/concurrency_control), built-in table services and streaming-first ingestion - rather than a table format plus metadata alone. Apache Hudi is continuously [redefining](https://hudi.apache.org/blog/2024/12/16/announci [...]
**1\. Well-Balanced Storage Format**
@@ -101,3 +104,13 @@ In summary, there is no secret sauce. The answer to the original question is sim
Apache Hudi is the **best-in-class open-source data lakehouse platform** —powerful, efficient, and future-proof. Start exploring it today\! 🚀
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {question: 'What differentiates Apache Hudi from Apache Iceberg and Delta Lake?', answer: 'Hudi is a database-like platform rather than a table format alone. Its storage format balances record-level write speed with query performance, and it adds secondary indexes, an efficient merge-on-read design, built-in table services like compaction and clustering, concurrency control designed for lake workloads including non-blocking concurrency control, and out-of-box CDC and streaming ingestio [...]
+ {question: 'Does Apache Hudi support secondary indexes?', answer: 'Yes. Hudi supports multi-modal indexing including record-level indexes, bloom filters, and even indexes on expressions over columns - features inspired by relational databases like Postgres. It also ships about half a dozen writer-side indexes, such as range indexes built on interval trees and consistent-hashed bucket indexes, to scale writes.'},
+ {question: 'Does Hudi require external orchestration for table maintenance?', answer: 'No. Hudi comes with automated table services - compaction, clustering, indexing, de-duplication, archiving, TTL enforcement, and cleaning - that are scheduled, executed, and retried automatically with every write, without external orchestration or manual SQL maintenance commands.'},
+ {question: 'How does Hudi handle multiple concurrent writers?', answer: 'Hudi provides MVCC-based approaches and non-blocking concurrency control so data pipelines and table services don\'t fail or livelock each other. Even in the optimistic concurrency control model, Hudi adds early conflict detection to pre-emptively abort writes that would eventually fail, saving compute hours.'},
+ {question: 'Can Hudi tables interoperate with Iceberg and Delta Lake?', answer: 'Yes. Through Apache XTable (Incubating), Hudi enables interoperability with the Iceberg and Delta Lake table formats without duplicating data storage or processing, and Hudi\'s catalog sync mechanism supports about six different data catalogs to keep table definitions in sync.'},
+]} />
+
diff --git a/website/blog/2025-03-31-amazon-hudi.md b/website/blog/2025-03-31-amazon-hudi.md
index 5f85c6eb2453..99238d55dc89 100644
--- a/website/blog/2025-03-31-amazon-hudi.md
+++ b/website/blog/2025-03-31-amazon-hudi.md
@@ -1,6 +1,7 @@
---
title: "Powering Amazon Unit Economics at Scale Using Apache Hudi"
excerpt: "How Amazon's Profit Intelligence team uses Apache Hudi to power hundreds of pipelines"
+description: "How Amazon's Profit Intelligence team uses Apache Hudi to power hundreds of pipelines"
author: Jason, Abhishek, Sethu in collaboration with Dipankar
category: case-study
image: /assets/images/blog/amz-1200x600.jpg
diff --git a/website/blog/2025-04-02-secondary-index.md b/website/blog/2025-04-02-secondary-index.md
index 5938117d0fa5..7c52511fe0e9 100644
--- a/website/blog/2025-04-02-secondary-index.md
+++ b/website/blog/2025-04-02-secondary-index.md
@@ -1,14 +1,19 @@
---
title: "Introducing Secondary Index in Apache Hudi Lakehouse Platform"
excerpt: "What's & How's of Secondary indexes in Hudi 1.0"
+description: "What's & How's of Secondary indexes in Hudi 1.0"
authors: [dipankar-mazumdar, aditya-goenka]
category: deep-dive
image: /assets/images/blog/sec-thumb.jpg
+last_update:
+ date: 2026-07-06
tags:
- indexing
- performance
---
+A secondary index in Apache Hudi is an index on any column other than the record key (primary key), introduced in Hudi 1.0, that maps secondary key values to record keys so queries filtering on non-primary-key fields can skip irrelevant data files instead of scanning the whole table.
+
:::tip TL;DR
Apache Hudi 1.0 introduces Secondary Indexes, enabling faster queries on non-primary key fields. This improves data retrieval in Lakehouse architectures by reducing data scans. Hudi also offers asynchronous indexing for scalability and efficient index maintenance without disrupting data ingestion. By the end of this blog, you'll understand how these features enhance Hudi's capabilities as a high-performance lakehouse platform.
@@ -29,6 +34,9 @@ Hudi's indexing mechanism is designed to efficiently manage record lookups and u
This strategy allows Hudi to determine whether a record exists and pinpoint its exact location, enabling faster upserts and deletes.
+:::info Newer content available
+For a comprehensive tour of how secondary indexes fit into Hudi 1.x's broader indexing subsystem, see [Deep Dive Into Hudi's Indexing Subsystem Part 2](/blog/2025/11/12/deep-dive-into-hudis-indexing-subsystem-part-2-of-2).
+:::
## Apache Hudi's Multi-Modal Indexing System
@@ -193,4 +201,14 @@ Indexing has been a core component of Apache Hudi since its inception, enabling
Additionally, to ensure that index maintenance does not introduce bottlenecks, Hudi’s *asynchronous indexing* service decouples index updates from ingestion, enabling seamless scaling while keeping indexes timeline-consistent and ACID-compliant. These advancements further solidify Hudi’s role as a high-performance lakehouse platform, making data structures such as secondary indexes more accessible.
----
\ No newline at end of file
+---
+
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {question: 'What is a secondary index in Apache Hudi?', answer: 'A secondary index, introduced in Hudi 1.0, lets users index columns that are not part of the record key. Hudi stores mappings between secondary key values and record keys in its metadata table, so queries filtering on non-primary-key fields can prune files via data skipping instead of scanning the full table.'},
+ {question: 'How do I create a secondary index in Hudi?', answer: 'On a table with the record index enabled, run a SQL statement such as CREATE INDEX idx_city ON hudi_table(city). Secondary indexes can also be configured through the Spark DataSource API using hoodie.metadata.index.secondary.enable and hoodie.datasource.write.secondarykey.column.'},
+ {question: 'How much does a secondary index improve query performance?', answer: 'In a TPCDS 1TB benchmark with an index on the web_sales table, the same join query ran about 33% faster on the first run and 58% faster on the second, while the data scanned dropped by roughly 90%, from 67GB across 5000 files to 7GB across 521 files.'},
+ {question: 'Which query engines support Hudi secondary indexes?', answer: 'Secondary-index-based pruning is available through Apache Spark today. Other engines such as Trino can query Hudi tables but do not currently apply secondary-index pruning, since Trino dropped Hudi metadata table reads in Trino 419. Reduced data scans particularly benefit cloud query engines like AWS Athena that price by data scanned.'},
+ {question: 'Does maintaining a secondary index slow down ingestion?', answer: 'It does not have to. Hudi\'s asynchronous indexing service decouples index building from ingestion by scheduling an indexing action on the transactional timeline, so indexes are built in the background while remaining timeline-consistent and ACID-compliant. Indexes can be added or dropped without downtime.'},
+]} />
\ No newline at end of file
diff --git a/website/blog/2025-05-29-lsm-timeline.md b/website/blog/2025-05-29-lsm-timeline.md
index f102f4e61e13..40fb6e9ab84f 100644
--- a/website/blog/2025-05-29-lsm-timeline.md
+++ b/website/blog/2025-05-29-lsm-timeline.md
@@ -1,6 +1,7 @@
---
title: "Exploring Apache Hudi’s New Log-Structured Merge (LSM) Timeline"
excerpt: "What is the new LSM timeline in Hudi & how is it implemented"
+description: "What is the new LSM timeline in Hudi & how is it implemented"
authors: [dipankar-mazumdar]
category: deep-dive
image: /assets/images/blog/lsm-1200x600.jpg
diff --git a/website/blog/2025-06-16-Apache-Hudi-does-XYZ-110.mdx b/website/blog/2025-06-16-Apache-Hudi-does-XYZ-110.mdx
index ed8c72117f57..4feb7d223fff 100644
--- a/website/blog/2025-06-16-Apache-Hudi-does-XYZ-110.mdx
+++ b/website/blog/2025-06-16-Apache-Hudi-does-XYZ-110.mdx
@@ -1,6 +1,7 @@
---
title: "Apache Hudi does XYZ (1/10): File pruning with multi-modal index"
excerpt: "File pruning with multi-modal index"
+description: "File pruning with multi-modal index"
authors: [xushiyan]
category: how-to
image: /assets/images/blog/2025-06-16-Apache-Hudi-does-XYZ-110-cover.jpg
diff --git a/website/blog/2025-06-30-uber-hudi.md b/website/blog/2025-06-30-uber-hudi.md
index 2a3203f3949f..271d4e3b7a09 100644
--- a/website/blog/2025-06-30-uber-hudi.md
+++ b/website/blog/2025-06-30-uber-hudi.md
@@ -1,6 +1,7 @@
---
title: "Scaling Complex Data Workflows at Uber Using Apache Hudi"
excerpt: "How Uber's Core Services Data Engineering team supports a wide range of use cases with Apache Hudi"
+description: "How Uber's Core Services Data Engineering team supports a wide range of use cases with Apache Hudi"
author: Ankit Shrivastava in collaboration with Dipankar
category: case-study
image: /assets/images/blog/uber1200x600.jpg
diff --git a/website/blog/2025-07-15-modernizing-datainfra-peloton-hudi.md b/website/blog/2025-07-15-modernizing-datainfra-peloton-hudi.md
index 8332dc79c932..d998fb9c19a1 100644
--- a/website/blog/2025-07-15-modernizing-datainfra-peloton-hudi.md
+++ b/website/blog/2025-07-15-modernizing-datainfra-peloton-hudi.md
@@ -1,6 +1,7 @@
---
title: "Modernizing Data Infrastructure at Peloton Using Apache Hudi"
excerpt: "How Peloton's Data Platform team scaled their data infrastructure using Hudi"
+description: "How Peloton's Data Platform team scaled their data infrastructure using Hudi"
author: Amaresh Bingumalla, Thinh Kenny Vu, Gabriel Wang, Arun Vasudevan in collaboration with Dipankar Mazumdar
category: case-study
image: /assets/images/blog/2025-07-15-modernizing-datainfra-peloton-hudi/peloton-1200x600.jpg
diff --git a/website/blog/2025-07-21-mor-comparison.md b/website/blog/2025-07-21-mor-comparison.md
index 882999a85320..78649097bf63 100644
--- a/website/blog/2025-07-21-mor-comparison.md
+++ b/website/blog/2025-07-21-mor-comparison.md
@@ -1,6 +1,7 @@
---
title: "A Deep Dive on Merge-on-Read (MoR) in Lakehouse Table Formats"
excerpt: "How is MoR implemented in Hudi, Iceberg, Delta and how it impacts workloads"
+description: "How is MoR implemented in Hudi, Iceberg, Delta and how it impacts workloads"
authors: [dipankar-mazumdar]
category: deep-dive
image: /assets/images/blog/2025-07-21-mor-comparison/mor-1200x600.jpg
diff --git a/website/blog/2025-09-17-hudi-auto-gen-keys.mdx b/website/blog/2025-09-17-hudi-auto-gen-keys.mdx
index c27157ae141b..f7540e62f9a3 100644
--- a/website/blog/2025-09-17-hudi-auto-gen-keys.mdx
+++ b/website/blog/2025-09-17-hudi-auto-gen-keys.mdx
@@ -1,6 +1,7 @@
---
title: "Automatic Record Key Generation in Apache Hudi"
excerpt: ""
+description: "How Apache Hudi automatically generates record keys, and when to rely on auto-generated keys versus defining your own."
authors: [xushiyan]
category: deep-dive
image: /assets/images/blog/2025-09-17-hudi-auto-gen-keys/2025-09-17-hudi-auto-gen-keys.fig2.jpg
diff --git a/website/blog/2025-10-02-Real-Time-Cloud-Security-Graphs-Hudi+PuppyGraph.mdx b/website/blog/2025-10-02-Real-Time-Cloud-Security-Graphs-Hudi+PuppyGraph.mdx
index 5063304f6375..8fb2f37d9d0f 100644
--- a/website/blog/2025-10-02-Real-Time-Cloud-Security-Graphs-Hudi+PuppyGraph.mdx
+++ b/website/blog/2025-10-02-Real-Time-Cloud-Security-Graphs-Hudi+PuppyGraph.mdx
@@ -1,6 +1,7 @@
---
title: "Real-Time Cloud Security Graphs with Apache Hudi and PuppyGraph"
excerpt: "Hudi tables support fast upserts and incremental processing. PuppyGraph queries relationships in place using openCypher or Gremlin. In this blog, we explore how to get started with real-time security graph analytics at scale using the data already stored in your Hudi lakehouse tables."
+description: "Hudi tables support fast upserts and incremental processing. PuppyGraph queries relationships in place using openCypher or Gremlin. In this blog, we explore how to get started with real-time security graph analytics at scale using the data already stored in your Hudi lakehouse tables."
author: Jaz Samantha Ku, in collaboration with Shiyan Xu
category: how-to
image: /assets/images/blog/2025-10-02-Real-Time-Cloud-Security-Graphs-Hudi+PuppyGraph/fig-4-Sample-Architecture-of-PuppyGraph-Hudi.png
diff --git a/website/blog/2025-10-16-Modernizing-Upstox-Data-Platform-with-Apache-Hudi-DBT-and-EMR-Serverless.md b/website/blog/2025-10-16-Modernizing-Upstox-Data-Platform-with-Apache-Hudi-DBT-and-EMR-Serverless.md
index a39255f50319..233f7d74e87b 100644
--- a/website/blog/2025-10-16-Modernizing-Upstox-Data-Platform-with-Apache-Hudi-DBT-and-EMR-Serverless.md
+++ b/website/blog/2025-10-16-Modernizing-Upstox-Data-Platform-with-Apache-Hudi-DBT-and-EMR-Serverless.md
@@ -1,6 +1,7 @@
---
title: "Modernizing Upstox's Data Platform with Apache Hudi, dbt, and EMR Serverless"
excerpt: ""
+description: "How Upstox modernized its data platform with Apache Hudi, dbt and EMR Serverless."
author: The Hudi Community
category: case-study
image: /assets/images/blog/2025-10-16-Modernizing-Upstox-Data-Platform-with-Apache-Hudi-DBT-and-EMR-Serverless/fig1.png
diff --git a/website/blog/2025-10-22-Partition_Stats_Enhancing_Column_Stats_in_Hudi_1.0.md b/website/blog/2025-10-22-Partition_Stats_Enhancing_Column_Stats_in_Hudi_1.0.md
index f7e4ccda8534..85f6f81f9590 100644
--- a/website/blog/2025-10-22-Partition_Stats_Enhancing_Column_Stats_in_Hudi_1.0.md
+++ b/website/blog/2025-10-22-Partition_Stats_Enhancing_Column_Stats_in_Hudi_1.0.md
@@ -1,6 +1,7 @@
---
title: "Partition Stats: Enhancing Column Stats in Hudi 1.0"
excerpt: ""
+description: "How partition stats in Hudi 1.0 extend column stats to prune entire partitions at query planning time, speeding up selective queries."
authors: [aditya-goenka, xushiyan]
category: deep-dive
image: /assets/images/blog/2025-10-22-Partition_Stats_Enhancing_Column_Stats_in_Hudi_1.0/fig1.jpg
diff --git a/website/blog/2025-10-29-deep-dive-into-hudis-indexing-subsystem-part-1-of-2.md b/website/blog/2025-10-29-deep-dive-into-hudis-indexing-subsystem-part-1-of-2.md
index bafeaa293af0..cc3d595c7694 100644
--- a/website/blog/2025-10-29-deep-dive-into-hudis-indexing-subsystem-part-1-of-2.md
+++ b/website/blog/2025-10-29-deep-dive-into-hudis-indexing-subsystem-part-1-of-2.md
@@ -1,6 +1,7 @@
---
title: "Deep Dive Into Hudi’s Indexing Subsystem (Part 1 of 2)"
excerpt: ""
+description: "Part 1 of a deep dive into Hudi's indexing subsystem: files, column stats and bloom filter indexes in the metadata table."
authors: [xushiyan]
category: deep-dive
image: /assets/images/blog/2025-10-29-deep-dive-into-hudis-indexing-subsystem-part-1-of-2/fig1.png
diff --git a/website/blog/2025-11-07-how-freewheel-uses-apache-hudi-to-power-its-data-lakehouse.mdx b/website/blog/2025-11-07-how-freewheel-uses-apache-hudi-to-power-its-data-lakehouse.mdx
index 12e6403a2a0c..1e81f67525f6 100644
--- a/website/blog/2025-11-07-how-freewheel-uses-apache-hudi-to-power-its-data-lakehouse.mdx
+++ b/website/blog/2025-11-07-how-freewheel-uses-apache-hudi-to-power-its-data-lakehouse.mdx
@@ -1,6 +1,7 @@
---
title: "How FreeWheel Uses Apache Hudi to Power Its Data Lakehouse"
excerpt: "How FreeWheel unified batch and streaming with an Apache Hudi–powered lakehouse to improve freshness, simplify operations, and scale analytics."
+description: "How FreeWheel unified batch and streaming with an Apache Hudi–powered lakehouse to improve freshness, simplify operations, and scale analytics."
author: The Hudi Community
category: case-study
image: /assets/images/blog/2025-11-07-how-freewheel-uses-apache-hudi-to-power-its-data-lakehouse/image1.png
diff --git a/website/blog/2025-11-12-deep-dive-into-hudis-indexing-subsystem-part-2-of-2.md b/website/blog/2025-11-12-deep-dive-into-hudis-indexing-subsystem-part-2-of-2.md
index 2c715b29a270..3a8b254bfe87 100644
--- a/website/blog/2025-11-12-deep-dive-into-hudis-indexing-subsystem-part-2-of-2.md
+++ b/website/blog/2025-11-12-deep-dive-into-hudis-indexing-subsystem-part-2-of-2.md
@@ -1,6 +1,7 @@
---
title: Deep Dive Into Hudi's Indexing Subsystem (Part 2 of 2)
excerpt: 'Explore advanced indexing in Apache Hudi: record and secondary indexes for fast point lookups, expression indexes for transformed predicates, and async indexing for building indexes without blocking writes.'
+description: 'Explore advanced indexing in Apache Hudi: record and secondary indexes for fast point lookups, expression indexes for transformed predicates, and async indexing for building indexes without blocking writes.'
authors: [xushiyan]
category: deep-dive
image: /assets/images/blog/2025-11-12-deep-dive-into-hudis-indexing-subsystem-part-2-of-2/fig1.png
diff --git a/website/blog/2025-11-25-apache-hudi-release-1-1-announcement.md b/website/blog/2025-11-25-apache-hudi-release-1-1-announcement.md
index 6ea6213f2181..f08b5b72d4ea 100644
--- a/website/blog/2025-11-25-apache-hudi-release-1-1-announcement.md
+++ b/website/blog/2025-11-25-apache-hudi-release-1-1-announcement.md
@@ -1,6 +1,7 @@
---
title: Apache Hudi 1.1 is Here—Building the Foundation for the Next Generation of Lakehouse
excerpt: ''
+description: "Apache Hudi 1.1 release highlights: partitioned record-level index, Flink writer overhaul, and foundations for the next generation of the lakehouse."
authors: [xushiyan]
category: community
image: /assets/images/blog/2025-11-25-apache-hudi-release-1-1-announcement/1-pluggable-TF.png
diff --git a/website/blog/2025-12-01-apache-hudi-JD-meetup-asia-2025-recap.md b/website/blog/2025-12-01-apache-hudi-JD-meetup-asia-2025-recap.md
index 3326a169e85c..11cbdb7ae567 100644
--- a/website/blog/2025-12-01-apache-hudi-JD-meetup-asia-2025-recap.md
+++ b/website/blog/2025-12-01-apache-hudi-JD-meetup-asia-2025-recap.md
@@ -1,6 +1,7 @@
---
title: "Next Generation Lakehouse: New Engine for the Intelligent Future | Apache Hudi Meetup Asia Recap"
excerpt: "A comprehensive recap of the Apache Hudi Meetup Asia held at JD.com headquarters, featuring insights from Onehouse, JD.com, Kuaishou, and Huawei on Hudi 1.1, AI-native architectures, and production optimizations."
+description: "A comprehensive recap of the Apache Hudi Meetup Asia held at JD.com headquarters, featuring insights from Onehouse, JD.com, Kuaishou, and Huawei on Hudi 1.1, AI-native architectures, and production optimizations."
author: "Team at JD.com"
category: community
image: /assets/images/blog/2025-12-01-apache-hudi-JD-meetup-asia-2025-recap/jdpost-image7.jpg
diff --git a/website/blog/2025-12-10-apache-hudi-11-deep-dive-optimizing-streaming-ingestion-with-flink.md b/website/blog/2025-12-10-apache-hudi-11-deep-dive-optimizing-streaming-ingestion-with-flink.md
index f35867323a6c..8cb6d17c1987 100644
--- a/website/blog/2025-12-10-apache-hudi-11-deep-dive-optimizing-streaming-ingestion-with-flink.md
+++ b/website/blog/2025-12-10-apache-hudi-11-deep-dive-optimizing-streaming-ingestion-with-flink.md
@@ -1,15 +1,20 @@
---
title: "Apache Hudi 1.1 Deep Dive: Optimizing Streaming Ingestion with Apache Flink"
excerpt: ''
+description: "How Apache Hudi 1.1 optimizes streaming ingestion with Apache Flink: Flink-native writers, faster operator SerDe, lower GC pressure, and benchmarks vs Hudi 1.0 and Apache Paimon."
authors: [shuo-cheng]
category: deep-dive
image: /assets/images/blog/2025-12-10-apache-hudi-11-deep-dive-optimizing-streaming-ingestion-with-flink/benchmark-string-schemas.png
+last_update:
+ date: 2026-07-12
tags:
- apache flink
- apache paimon
- performance
---
+Apache Hudi 1.1 optimizes streaming ingestion with Apache Flink through three writer-side changes - a Flink-native record representation that avoids Kryo serialization during operator shuffles, a new write path that operates directly on Flink's `RowData` instead of converting records to Avro, and the elimination of record-level bytes copy when writing MOR log files - delivering about 3.5x the ingestion throughput of Hudi 1.0 in benchmarks. This post walks through each optimization and th [...]
+
---
_This blog was translated from the [original blog in Chinese](https://mp.weixin.qq.com/s/ek80Fzw30FOawk1qeWxtCw)._
@@ -129,3 +134,13 @@ Additionally, the schema of a table also has a significant impact on write perfo
## Summary
The optimizations in Hudi 1.1 around writer performance for Flink have brought significant, multi-fold improvements to streaming ingestion throughput. These enhancements are transparent and backward-compatible, allowing users to seamlessly upgrade their jobs from earlier Hudi versions to the latest version and enjoy the substantial performance gains without any additional operational overhead.
+
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {question: 'How much faster is Flink streaming ingestion in Hudi 1.1 compared to Hudi 1.0?', answer: 'In benchmarks on a merge-on-read table with UPSERT operation and bucket index, Hudi 1.1 achieved about 3.5 times the streaming ingestion throughput of Hudi 1.0 on a numeric-heavy schema. The optimized shuffle SerDe alone (RFC-84) boosts average throughput by about 25%.'},
+ {question: 'Why was Avro a bottleneck in the older Flink write path?', answer: 'Before Hudi 1.1, Flink RowData was converted to Avro records and serialized to Avro bytes, then deserialized again during log writing - redundant SerDe overhead at every step. The in-memory buffer of intermediate Avro objects also increased heap usage and GC pressure, and shuffles of the generic HoodieRecord type fell back to Flink\'s Kryo serializer, which is slow.'},
+ {question: 'What is the Flink-native write path introduced in Hudi 1.1?', answer: 'RFC-87 makes the Flink writer wrap RowData directly inside a specialized HoodieFlinkRecord instead of converting it to an Avro record, so the entire write path operates on RowData. It adds a self-managed, reusable binary buffer that reduces GC pressure under high-throughput workloads, and supports both Avro (default, row-oriented) and Parquet (columnar) data blocks for MOR log files.'},
+ {question: 'How does Hudi 1.1 compare with Apache Paimon for streaming ingestion?', answer: 'On the numeric-heavy Schema1 from the Paimon cluster benchmark, Paimon\'s throughput was slightly higher than Hudi 1.1, mainly because each HoodieRecord carries 5 extra String-type metadata fields by default. On string-heavy schemas, which are more common in production, Hudi 1.1 achieved the best throughput, since writing row-based Avro log files avoids the Parquet compression overhead Paimon i [...]
+ {question: 'Do I need to change my Flink jobs to get these improvements?', answer: 'No. The optimizations are transparent and backward-compatible, so existing jobs can upgrade from earlier Hudi versions and get the performance gains without additional operational overhead.'},
+]} />
diff --git a/website/blog/2025-12-16-maximizing-throughput-nbcc.md b/website/blog/2025-12-16-maximizing-throughput-nbcc.md
index f2bbe5d30836..9c9e13dd2638 100644
--- a/website/blog/2025-12-16-maximizing-throughput-nbcc.md
+++ b/website/blog/2025-12-16-maximizing-throughput-nbcc.md
@@ -1,6 +1,7 @@
---
title: "Maximizing Throughput with Apache Hudi NBCC: Stop Retrying, Start Scaling"
excerpt: "Learn how Hudi's Non-Blocking Concurrency Control eliminates retry storms for concurrent writers, maximizing throughput in streaming and mixed workloads."
+description: "Learn how Hudi's Non-Blocking Concurrency Control eliminates retry storms for concurrent writers, maximizing throughput in streaming and mixed workloads."
authors: [xushiyan]
category: deep-dive
image: /assets/images/blog/2025-12-16-maximizing-throughput-nbcc/p6-nbcc-compaction.png
diff --git a/website/blog/2025-12-22-how-zupee-cut-s3-costs-60-percent-with-hudi.mdx b/website/blog/2025-12-22-how-zupee-cut-s3-costs-60-percent-with-hudi.mdx
index f1bbe3dda289..61a35da4e759 100644
--- a/website/blog/2025-12-22-how-zupee-cut-s3-costs-60-percent-with-hudi.mdx
+++ b/website/blog/2025-12-22-how-zupee-cut-s3-costs-60-percent-with-hudi.mdx
@@ -1,6 +1,7 @@
---
title: "How Zupee Cut S3 Costs by 60% with Apache Hudi"
excerpt: "How Zupee, India's largest skill-based Ludo platform, reduced S3 costs by 60% and achieved 15-minute ingestion SLAs using Apache Hudi's Metadata Table and Hudi Streamer."
+description: "How Zupee, India's largest skill-based Ludo platform, reduced S3 costs by 60% and achieved 15-minute ingestion SLAs using Apache Hudi's Metadata Table and Hudi Streamer."
author: The Hudi Community
category: case-study
image: /assets/images/blog/2025-12-22-how-zupee-cut-s3-costs-60-percent-with-hudi/og.png
diff --git a/website/blog/2025-12-29-apache-hudi-2025-a-year-in-review.mdx b/website/blog/2025-12-29-apache-hudi-2025-a-year-in-review.mdx
index 8fd15de12af8..e73e5187b184 100644
--- a/website/blog/2025-12-29-apache-hudi-2025-a-year-in-review.mdx
+++ b/website/blog/2025-12-29-apache-hudi-2025-a-year-in-review.mdx
@@ -1,6 +1,7 @@
---
title: "Apache Hudi 2025: A Year In Review"
excerpt: ""
+description: "A look back at Apache Hudi in 2025: releases, community growth and ecosystem milestones."
authors: [xushiyan]
category: community
image: /assets/images/blog/2025-12-29-apache-hudi-2025-a-year-in-review/00-og.png
diff --git a/website/blog/2026-01-09-hudi-11-deep-dive-flink-async-instant-gen.md b/website/blog/2026-01-09-hudi-11-deep-dive-flink-async-instant-gen.md
index 9247b6f2b0d7..42c94195a86f 100644
--- a/website/blog/2026-01-09-hudi-11-deep-dive-flink-async-instant-gen.md
+++ b/website/blog/2026-01-09-hudi-11-deep-dive-flink-async-instant-gen.md
@@ -1,6 +1,7 @@
---
title: "Apache Hudi 1.1 Deep Dive: Async Instant Time Generation for Flink Writers"
excerpt: "Explore how Hudi 1.1 introduces asynchronous instant generation for Flink writers to eliminate throughput fluctuations in streaming ingestion."
+description: "Explore how Hudi 1.1 introduces asynchronous instant generation for Flink writers to eliminate throughput fluctuations in streaming ingestion."
authors: [shuo-cheng]
category: deep-dive
image: /assets/images/blog/2026-01-09-hudi-11-deep-dive-flink-async-instant-gen/async-instant-gen.png
diff --git a/website/blog/2026-01-15-funding-circle.mdx b/website/blog/2026-01-15-funding-circle.mdx
index 265269353aaa..b20e481fbcb7 100644
--- a/website/blog/2026-01-15-funding-circle.mdx
+++ b/website/blog/2026-01-15-funding-circle.mdx
@@ -1,6 +1,7 @@
---
title: "From Legacy to Leading: Funding Circle's Journey with Apache Hudi"
excerpt: ""
+description: "How Funding Circle modernized its legacy data platform into an Apache Hudi lakehouse."
author: The Hudi Community
category: case-study
image: /assets/images/blog/2026-01-15-funding-circle/01-hudi-sync-funding-circle.png
diff --git a/website/blog/2026-01-22-apache-hudi-at-applied-intuition.mdx b/website/blog/2026-01-22-apache-hudi-at-applied-intuition.mdx
index 1d6431f6e816..361db2873dac 100644
--- a/website/blog/2026-01-22-apache-hudi-at-applied-intuition.mdx
+++ b/website/blog/2026-01-22-apache-hudi-at-applied-intuition.mdx
@@ -1,6 +1,7 @@
---
title: "Scaling Autonomous Vehicle Data Infrastructure with Apache Hudi at Applied Intuition"
excerpt: "How Applied Intuition reduced query times from 10 minutes to under 25 seconds and achieved 20x storage compression by migrating to an Apache Hudi-powered data lakehouse."
+description: "How Applied Intuition reduced query times from 10 minutes to under 25 seconds and achieved 20x storage compression by migrating to an Apache Hudi-powered data lakehouse."
author: The Hudi Community
category: case-study
image: /assets/images/blog/2026-01-22-apache-hudi-at-applied-intuition/img1.png
diff --git a/website/blog/2026-05-14-mor-isnt-a-storage-optimization-its-an-architectural-shift.mdx b/website/blog/2026-05-14-mor-isnt-a-storage-optimization-its-an-architectural-shift.mdx
index 08f8b9baa581..4b11aa1842d6 100644
--- a/website/blog/2026-05-14-mor-isnt-a-storage-optimization-its-an-architectural-shift.mdx
+++ b/website/blog/2026-05-14-mor-isnt-a-storage-optimization-its-an-architectural-shift.mdx
@@ -1,6 +1,7 @@
---
title: "MOR Isn't a Storage Optimization. It's an Architectural Shift"
excerpt: "Why Merge-on-Read is a foundational design decision, not a storage tuning knob — and how it reshapes everything from concurrency to metadata."
+description: "Why Merge-on-Read is a foundational design decision, not a storage tuning knob — and how it reshapes everything from concurrency to metadata."
authors: [sivabalan]
category: deep-dive
image: /assets/images/blog/2026-05-14-mor-isnt-a-storage-optimization-its-an-architectural-shift.png
diff --git a/website/blog/2026-06-05-why-metadata-has-to-be-mutation-friendly.mdx b/website/blog/2026-06-05-why-metadata-has-to-be-mutation-friendly.mdx
index 2a9bb780a28b..7e25cbc0875f 100644
--- a/website/blog/2026-06-05-why-metadata-has-to-be-mutation-friendly.mdx
+++ b/website/blog/2026-06-05-why-metadata-has-to-be-mutation-friendly.mdx
@@ -1,6 +1,7 @@
---
title: "Why Metadata Has to Be Mutation-Friendly"
excerpt: "The Hudi metadata table is itself a Merge-on-Read table — here's why MOR's append-first mutation model is what makes a continuously evolving, indexing-rich metadata layer scale."
+description: "The Hudi metadata table is itself a Merge-on-Read table — here's why MOR's append-first mutation model is what makes a continuously evolving, indexing-rich metadata layer scale."
authors: [sivabalan]
category: deep-dive
image: /assets/images/blog/2026-06-05-why-metadata-has-to-be-mutation-friendly/mdt-stops-catalog-state.png
diff --git a/website/blog/2026-06-07-apache-hudi-release-1-2-announcement.mdx b/website/blog/2026-06-07-apache-hudi-release-1-2-announcement.mdx
index 1769a509d001..613783446dfc 100644
--- a/website/blog/2026-06-07-apache-hudi-release-1-2-announcement.mdx
+++ b/website/blog/2026-06-07-apache-hudi-release-1-2-announcement.mdx
@@ -1,6 +1,7 @@
---
title: "Apache Hudi 1.2: Expanding the Open Lakehouse for AI and Multimodal Data"
excerpt: "Apache Hudi 1.2 makes the open lakehouse a unified foundation for analytics and AI, with first-class support for vectors, binary objects, and semi-structured data alongside the records you already manage."
+description: "Apache Hudi 1.2 makes the open lakehouse a unified foundation for analytics and AI, with first-class support for vectors, binary objects, and semi-structured data alongside the records you already manage."
authors: [rahil-chertara, sivabalan, ethan-guo]
category: community
image: /assets/images/blog/2026-06-07-apache-hudi-release-1-2-announcement/3-hudi-foundation-multimodal-data.png
diff --git a/website/blog/2026-06-10-stateless-global-upserts-for-flink-streaming-in-apache-hudi-1-2-0.mdx b/website/blog/2026-06-10-stateless-global-upserts-for-flink-streaming-in-apache-hudi-1-2-0.mdx
index c0535397b3b8..223566ed589c 100644
--- a/website/blog/2026-06-10-stateless-global-upserts-for-flink-streaming-in-apache-hudi-1-2-0.mdx
+++ b/website/blog/2026-06-10-stateless-global-upserts-for-flink-streaming-in-apache-hudi-1-2-0.mdx
@@ -1,6 +1,7 @@
---
title: "Stateless Global Upserts for Flink Streaming in Apache Hudi 1.2.0"
excerpt: "How Hudi 1.2.0's Record Level Index brings stateless global upserts to Flink — replacing multi-hundred-GB keyed state with a table-backed, engine-agnostic index."
+description: "How Hudi 1.2.0's Record Level Index brings stateless global upserts to Flink — replacing multi-hundred-GB keyed state with a table-backed, engine-agnostic index."
authors: [danny-chan, shuo-cheng]
category: deep-dive
image: /assets/images/blog/stateless-global-upserts-for-flink-streaming-in-apache-hudi-1-2-0/img3.png
diff --git a/website/blog/2026-06-15-apache-hudi-at-metica.mdx b/website/blog/2026-06-15-apache-hudi-at-metica.mdx
index de2eca4e9cc6..ffb02387a901 100644
--- a/website/blog/2026-06-15-apache-hudi-at-metica.mdx
+++ b/website/blog/2026-06-15-apache-hudi-at-metica.mdx
@@ -1,6 +1,7 @@
---
title: "Accelerating Data Operations: Metica's Journey with Apache Hudi"
excerpt: "Metica built its first lakehouse from scratch on Apache Hudi and Amazon EMR — scaling from ~16 GB to billions of events without re-architecting, where every growth milestone became a config change, not a migration."
+description: "Metica built its first lakehouse from scratch on Apache Hudi and Amazon EMR — scaling from ~16 GB to billions of events without re-architecting, where every growth milestone became a config change, not a migration."
author: The Hudi Community
category: case-study
image: /assets/images/blog/2026-06-15-metica/img1.png
diff --git a/website/blog/2026-06-15-apache-hudi-at-penn-interactive.mdx b/website/blog/2026-06-15-apache-hudi-at-penn-interactive.mdx
index 4decd02f111d..4e061fa57112 100644
--- a/website/blog/2026-06-15-apache-hudi-at-penn-interactive.mdx
+++ b/website/blog/2026-06-15-apache-hudi-at-penn-interactive.mdx
@@ -1,6 +1,7 @@
---
title: "From Concept to Reality: Apache Hudi at the Foundation of Penn Entertainment's Data Platform"
excerpt: "How Penn Interactive, the online sports betting arm of Penn Entertainment, modernized its data platform using Apache Hudi HudiStreamer to ingest CDC from thousands of Kafka topics, orchestrate 100+ concurrent jobs with Argo Workflows, and deliver near real-time data freshness across thousands of tables."
+description: "How Penn Interactive, the online sports betting arm of Penn Entertainment, modernized its data platform using Apache Hudi HudiStreamer to ingest CDC from thousands of Kafka topics, orchestrate 100+ concurrent jobs with Argo Workflows, and deliver near real-time data freshness across thousands of tables."
author: The Hudi Community
category: case-study
image: /assets/images/blog/2026-06-15-penn-interactive/img1.png
diff --git a/website/blog/2026-06-15-apache-hudi-at-southwest-airlines.mdx b/website/blog/2026-06-15-apache-hudi-at-southwest-airlines.mdx
index 8c70a2109a37..55203130a314 100644
--- a/website/blog/2026-06-15-apache-hudi-at-southwest-airlines.mdx
+++ b/website/blog/2026-06-15-apache-hudi-at-southwest-airlines.mdx
@@ -1,6 +1,7 @@
---
title: "Modernizing Data Infrastructure at Southwest with Apache Hudi"
excerpt: "Southwest Airlines migrated its reservation data pipeline from an on-premise Ab Initio warehouse to an AWS lakehouse on Apache Hudi and EMR — implementing SCD Type 2 at scale and cutting job runtime from ~5 hours to ~1.5."
+description: "Southwest Airlines migrated its reservation data pipeline from an on-premise Ab Initio warehouse to an AWS lakehouse on Apache Hudi and EMR — implementing SCD Type 2 at scale and cutting job runtime from ~5 hours to ~1.5."
author: The Hudi Community
category: case-study
image: /assets/images/blog/2026-06-15-southwest-airlines/img1.png
diff --git a/website/blog/2026-06-25-building-indexes-on-a-moving-target.mdx b/website/blog/2026-06-25-building-indexes-on-a-moving-target.mdx
index 58cdfc6101f6..1f02abf864cc 100644
--- a/website/blog/2026-06-25-building-indexes-on-a-moving-target.mdx
+++ b/website/blog/2026-06-25-building-indexes-on-a-moving-target.mdx
@@ -1,6 +1,7 @@
---
title: "Building Indexes on a Moving Target"
excerpt: "Async indexing in Hudi lets you add a new index to a multi-petabyte table without stopping ingestion — and the design only works because MOR put the right primitives in place years earlier."
+description: "Async indexing in Hudi lets you add a new index to a multi-petabyte table without stopping ingestion — and the design only works because MOR put the right primitives in place years earlier."
authors: [sivabalan]
category: deep-dive
image: /assets/images/blog/2026-06-25-building-indexes-on-a-moving-target/async-indexing-architectural-consequence.png
diff --git a/website/docs/migration_guide.md b/website/docs/migration_guide.md
index f9aaffa72769..513a201e6672 100644
--- a/website/docs/migration_guide.md
+++ b/website/docs/migration_guide.md
@@ -2,7 +2,7 @@
title: Bootstrapping
keywords: [ hudi, migration, use case]
summary: In this page, we will discuss some available tools for migrating your existing table into a Hudi table
-last_modified_at: 2019-12-30T15:59:57-04:00
+last_modified_at: 2026-07-06T12:00:00-07:00
toc: true
toc_min_heading_level: 2
toc_max_heading_level: 4
@@ -97,6 +97,15 @@ hudi->bootstrap run --srcPath /tmp/source_table --targetPath /tmp/hoodie/bootstr
```
Unlike Hudi Streamer, FULL_RECORD or METADATA_ONLY is set with --selectorClass, see details with help "bootstrap run".
+### Migrating from Delta Lake or Apache Iceberg
+
+Tables already managed by Delta Lake or Apache Iceberg store their data as Parquet files, so they can be migrated to
+Hudi without rewriting data by translating their metadata with [Apache XTable](https://xtable.apache.org). XTable reads
+the source table's metadata and generates Hudi metadata alongside the existing data files, and it can also run in the
+reverse direction — keeping an Iceberg or Delta view of a Hudi table in sync so existing readers continue to work
+during and after the migration. See [Syncing to XTable](syncing_xtable.md) for the Hudi-side setup. For a full rewrite
+instead, read the source table with Spark and write it to Hudi using the approaches above.
+
## Configs
diff --git a/website/docusaurus.config.js b/website/docusaurus.config.js
index ff86770cbf67..034eb38667db 100644
--- a/website/docusaurus.config.js
+++ b/website/docusaurus.config.js
@@ -342,6 +342,10 @@ module.exports = {
label: "Integrations",
to: "/ecosystem",
},
+ {
+ label: "Vendors",
+ to: "/vendors",
+ },
],
},
{ to: "/releases/download", label: "Download", position: "left" },
@@ -448,6 +452,10 @@ module.exports = {
label: "Get Involved",
to: "/community/get-involved",
},
+ {
+ label: "Vendors",
+ to: "/vendors",
+ },
{
label: "Slack",
href: originalSlackUrl,
@@ -584,9 +592,21 @@ module.exports = {
title: "Apache Hudi: User-Facing Analytics",
},
showReadingTime: true,
+ // emits dateModified in the BlogPosting structured data (sourced
+ // from front-matter last_update or git history)
+ showLastUpdateTime: true,
onUntruncatedBlogPosts: "ignore",
onInlineAuthors: "ignore",
},
+ sitemap: {
+ lastmod: "date",
+ changefreq: null,
+ priority: null,
+ },
+ pages: {
+ // emits FAQPage JSON-LD on the src/pages/faq/* pages
+ remarkPlugins: [require("./plugins/remark-faq-structured-data")],
+ },
theme: {
customCss: require.resolve("./src/css/custom.css"),
},
diff --git a/website/plugins/remark-faq-structured-data.js b/website/plugins/remark-faq-structured-data.js
new file mode 100644
index 000000000000..ce280cb2365d
--- /dev/null
+++ b/website/plugins/remark-faq-structured-data.js
@@ -0,0 +1,75 @@
+const path = require('path');
+
+// Plain-text rendering of an mdast subtree (questions/answers for JSON-LD).
+function textOf(node) {
+ if (!node) {
+ return '';
+ }
+ if (node.type === 'text' || node.type === 'inlineCode') {
+ return node.value;
+ }
+ if (node.type === 'code') {
+ return node.value;
+ }
+ if (Array.isArray(node.children)) {
+ return node.children.map(textOf).join(node.type === 'list' ? ' ' : '');
+ }
+ return '';
+}
+
+/**
+ * Remark plugin that turns the `### Question` + prose structure of the
+ * src/pages/faq/*.md pages into schema.org FAQPage JSON-LD, by appending a
+ * <FAQStructuredData json="..."/> element (resolved through the global
+ * MDX components map). Content pages outside src/pages/faq are untouched.
+ */
+module.exports = function remarkFaqStructuredData() {
+ const faqDir = path.join('src', 'pages', 'faq') + path.sep;
+ return (tree, file) => {
+ const filePath = file.path || (file.history && file.history[0]) || '';
+ if (!filePath.includes(faqDir)) {
+ return;
+ }
+
+ const faqs = [];
+ let current = null;
+ for (const node of tree.children) {
+ if (node.type === 'heading' && node.depth === 3) {
+ current = {question: textOf(node).trim(), answer: []};
+ faqs.push(current);
+ } else if (node.type === 'heading') {
+ current = null;
+ } else if (current) {
+ const text = textOf(node).trim();
+ if (text) {
+ current.answer.push(text);
+ }
+ }
+ }
+
+ const items = faqs.filter((faq) => faq.question && faq.answer.length);
+ if (!items.length) {
+ return;
+ }
+
+ const json = JSON.stringify({
+ '@context': 'https://schema.org',
+ '@type': 'FAQPage',
+ mainEntity: items.map(({question, answer}) => ({
+ '@type': 'Question',
+ name: question,
+ acceptedAnswer: {
+ '@type': 'Answer',
+ text: answer.join('\n\n'),
+ },
+ })),
+ });
+
+ tree.children.push({
+ type: 'mdxJsxFlowElement',
+ name: 'FAQStructuredData',
+ attributes: [{type: 'mdxJsxAttribute', name: 'json', value: json}],
+ children: [],
+ });
+ };
+};
diff --git a/website/src/components/PostFAQ/index.js b/website/src/components/PostFAQ/index.js
new file mode 100644
index 000000000000..4ad8f602b34a
--- /dev/null
+++ b/website/src/components/PostFAQ/index.js
@@ -0,0 +1,57 @@
+import React from 'react';
+import Head from '@docusaurus/Head';
+
+/**
+ * Emits schema.org FAQPage JSON-LD. Accepts either `items`
+ * ([{question, answer}]) or a pre-serialized `json` string (used by the
+ * remark-faq-structured-data plugin on the /faq pages).
+ */
+export function FAQStructuredData({items, json}) {
+ const data = json
+ ? JSON.parse(json)
+ : {
+ '@context': 'https://schema.org',
+ '@type': 'FAQPage',
+ mainEntity: (items ?? []).map(({question, answer}) => ({
+ '@type': 'Question',
+ name: question,
+ acceptedAnswer: {
+ '@type': 'Answer',
+ text: answer,
+ },
+ })),
+ };
+ return (
+ <Head>
+ <script type="application/ld+json">{JSON.stringify(data)}</script>
+ </Head>
+ );
+}
+
+/**
+ * Visible FAQ section for blog posts, backed by FAQPage structured data.
+ * Answers are plain strings so the on-page text and the JSON-LD stay in sync.
+ *
+ * Usage in a post (no import needed, registered via src/theme/MDXComponents):
+ * ## FAQ
+ * <PostFAQ heading={null} items={[{question: '...', answer: '...'}]} />
+ */
+export default function PostFAQ({items, heading = 'Frequently Asked Questions'}) {
+ if (!items?.length) {
+ return null;
+ }
+ return (
+ <section>
+ <FAQStructuredData items={items} />
+ {heading && <h2>{heading}</h2>}
+ {items.map(({question, answer}) => (
+ <details key={question} style={{marginBottom: '0.75rem'}}>
+ <summary>
+ <strong>{question}</strong>
+ </summary>
+ <p style={{marginTop: '0.5rem'}}>{answer}</p>
+ </details>
+ ))}
+ </section>
+ );
+}
diff --git a/website/src/pages/faq/general.md b/website/src/pages/faq/general.md
index c55addbad68f..fe19916b825d 100644
--- a/website/src/pages/faq/general.md
+++ b/website/src/pages/faq/general.md
@@ -7,10 +7,28 @@ keywords: [hudi, writing, reading]
### When is Hudi useful for me or my organization?
-If you are looking to quickly ingest data onto HDFS or cloud storage, Hudi provides you [tools](/docs/hoodie_streaming_ingestion). Also, if you have ETL/hive/spark jobs which are slow/taking up a lot of resources, Hudi can potentially help by providing an incremental approach to reading and writing data.
+If you are looking to quickly ingest data onto HDFS or cloud storage, Hudi provides you [tools](/docs/hoodie_streaming_ingestion). Also, if you have ETL/hive/spark jobs which are slow/taking up a lot of resources, Hudi can potentially help by providing an incremental approach to reading and writing data. Hudi remains the de facto lakehouse format for fast incremental writes and reads, and it ships with automated table maintenance built in, so tables stay optimized without external orches [...]
As an organization, Hudi can help you build an [efficient data lake](https://docs.google.com/presentation/d/1FHhsvh70ZP6xXlHdVsAI0g__B_6Mpto5KQFlZ0b8-mM/edit#slide=id.p), solving some of the most complex, low-level storage management problems, while putting data into hands of your data analysts, engineers and scientists much quicker.
+### What makes Hudi different from other lakehouse formats?
+
+Hudi offers a set of core capabilities today that other lakehouse formats do not. The [21 unique differentiators](/blog/2025/03/05/hudi-21-unique-differentiators) post covers the technical crux in depth; the highlights are:
+
+* **_Multi-modal indexing:_** Hudi maintains a range of [indexes](/docs/indexes) — record-level indexes, bloom filters, bucket indexes and more — that speed up upserts and deletes on the write side, plus read-side secondary indexes (including expression indexes on columns) that prune queries, much like a relational database.
+* **_Non-blocking concurrency control:_** Hudi's MVCC-based [concurrency control](/docs/concurrency_control#non-blocking-concurrency-control) lets multiple writers and table services modify a table concurrently without failing or blocking each other, avoiding wasted compute from retries and livelocks.
+* **_Async compaction and built-in table services:_** compaction, clustering, cleaning, file sizing, indexing and archival are scheduled and executed automatically alongside writes — no external orchestration or manual maintenance commands. Hudi is the only lakehouse project that can rapidly ingest data while handling small-file compaction without blocking those writes. This kind of table maintenance is something you typically pay a vendor for; in Hudi it is open source and built in.
+* **_Ingestion utilities:_** production-ready [ingestion tools](/docs/hoodie_streaming_ingestion) like Hudi Streamer and the Flink writer build lakehouse tables from Kafka, Pulsar, S3/GCS and popular CDC formats (Debezium, AWS DMS, Mongo) with a single command.
+* **_Blob/unstructured data support:_** starting with Hudi 1.2, tables can manage blob and unstructured data alongside structured records, extending the lakehouse beyond tabular workloads.
+
+Combined with a storage format that balances write speed and query performance, these capabilities make Hudi the leader in incremental write performance and the de facto format for fast incremental writes and reads.
+
+### How does Hudi relate to Apache Iceberg? Are Hudi tables compatible with Iceberg?
+
+The two projects were engineered around different workloads. Iceberg's design centers on the traditional batch, scan-oriented workloads that Apache Hive served — large periodic rewrites and full-table scans. Hudi was engineered for fast-moving, mutable data: streaming ingestion, CDC, record-level upserts and deletes, and incremental pipelines that process only what changed. Choosing between them is a question of workload fit, not either/or on data access.
+
+That is because both formats store data as Parquet files, and [Apache XTable](/docs/syncing_xtable) (incubating) translates Hudi table metadata into Iceberg metadata in place — no data is copied or rewritten — making Hudi copy-on-write tables fully format-compatible with Iceberg readers: a single copy of data on cloud storage is readable as both Hudi and Iceberg. (Merge-on-read tables expose their compacted read-optimized view to Iceberg readers.) You can ingest and manage tables with Hu [...]
+
### What are some non-goals for Hudi?
Hudi is not designed for any OLTP use-cases, where typically you are using existing NoSQL/RDBMS data stores. Hudi cannot replace your in-memory analytical database (at-least not yet!). Hudi support near-real time ingestion in the order of few minutes, trading off latency for efficient batching. If you truly desirable sub-minute processing delays, then stick with your favorite stream processing solution.
diff --git a/website/src/pages/vendors.md b/website/src/pages/vendors.md
new file mode 100644
index 000000000000..dbabc0694f22
--- /dev/null
+++ b/website/src/pages/vendors.md
@@ -0,0 +1,51 @@
+---
+title: Vendors
+description: Vendors that provide managed services or commercial support for Apache Hudi, listed alphabetically.
+---
+# Vendors
+
+The following vendors provide managed services, cloud platform integrations, or commercial support for Apache Hudi. Vendors are listed alphabetically; inclusion on this page is for the convenience of the community and does not imply endorsement by the Apache Software Foundation or the Apache Hudi project.
+
+To add or update a listing, please [open a pull request](https://github.com/apache/hudi/tree/asf-site/website/src/pages/vendors.md) or reach out on the [dev mailing list](mailto:dev@hudi.apache.org).
+
+## Alibaba Cloud
+
+Alibaba Cloud E-MapReduce ships with Apache Hudi support, and Hudi tables can be stored on [Alibaba Cloud OSS](/docs/oss_hoodie). Alibaba Cloud engineers are long-time contributors to the project.
+
+[alibabacloud.com](https://www.alibabacloud.com/help/en/emr/emr-on-ecs/user-guide/hudi/)
+
+## Amazon Web Services
+
+Amazon EMR includes Apache Hudi out of the box, and Hudi tables on [Amazon S3](/docs/s3_hoodie) can be queried from Amazon Athena and Amazon Redshift Spectrum, with catalog integration via AWS Glue.
+
+[aws.amazon.com](https://aws.amazon.com/emr/features/hudi/)
+
+## ByteDance
+
+ByteDance runs one of the largest Apache Hudi deployments in the world and offers Hudi through Volcano Engine's LakeHouse Analytics Service (LAS) and E-MapReduce products.
+
+[volcengine.com](https://www.volcengine.com/docs/6491/198105)
+
+## Google Cloud
+
+Google Cloud Dataproc includes Apache Hudi, Hudi tables can be stored on [Google Cloud Storage](/docs/gcs_hoodie), and BigQuery integrates with Hudi tables through the [BigQuery integration](/docs/gcp_bigquery).
+
+[cloud.google.com](https://cloud.google.com/dataproc/docs/concepts/components/hudi)
+
+## Huawei Cloud
+
+Huawei Cloud MRS (MapReduce Service) provides Apache Hudi as part of its data lake offering, with Hudi used as the storage layer for its FusionInsight lakehouse solutions.
+
+[support.huaweicloud.com](https://support.huaweicloud.com/intl/en-us/productdesc-mrs/mrs_08_0083.html)
+
+## Onehouse
+
+Onehouse provides a fully managed universal data lakehouse service built on Apache Hudi, founded by the original creators of the project. Onehouse offers commercial support for Apache Hudi and contributes heavily to its development.
+
+[onehouse.ai](https://www.onehouse.ai/solutions/managed-hudi)
+
+## Tencent Cloud
+
+Tencent Cloud EMR supports Apache Hudi, and Hudi tables can be stored on [Tencent Cloud Object Storage](/docs/cos_hoodie).
+
+[tencentcloud.com](https://www.tencentcloud.com/document/product/1026/35587)
diff --git a/website/src/theme/MDXComponents.js b/website/src/theme/MDXComponents.js
new file mode 100644
index 000000000000..6499666f9ca2
--- /dev/null
+++ b/website/src/theme/MDXComponents.js
@@ -0,0 +1,10 @@
+import MDXComponents from '@theme-original/MDXComponents';
+import PostFAQ, {FAQStructuredData} from '@site/src/components/PostFAQ';
+
+// Components available in all MDX content (blog posts, docs, pages)
+// without an explicit import.
+export default {
+ ...MDXComponents,
+ PostFAQ,
+ FAQStructuredData,
+};
diff --git a/website/static/llms.txt b/website/static/llms.txt
new file mode 100644
index 000000000000..138ed0fcc51f
--- /dev/null
+++ b/website/static/llms.txt
@@ -0,0 +1,78 @@
+# Apache Hudi
+
+> Apache Hudi is an open source data lakehouse platform that brings database-like capabilities — ACID transactions, record-level upserts and deletes, indexing, change data capture and incremental processing — to data stored on cloud object storage or HDFS. Hudi tables can be written and queried from Apache Spark, Apache Flink, Presto, Trino, Hive and more, and can interoperate with Apache Iceberg and Delta Lake metadata via Apache XTable.
+
+Hudi (pronounced "hoodie") pioneered transactional capabilities on top of data lakes. Hudi lakehouses were in production at Uber and other places in 2017, years before the term was coined in the industry.
+It manages tables as a timeline of actions over open file formats (Apache Parquet, Apache ORC, Lance file format, Apache Avro), supports Copy-on-Write and Merge-on-Read storage, ships a multi-modal indexing subsystem
+(bloom filters, record-level index, secondary indexes, expression indexes, vector search), and includes built-in table services such as compaction, clustering and cleaning.
+It can also synchronize your data to half dozen data catalogs to keep table consistent with changes. This site hosts the project documentation, an extensive engineering blog, FAQs and community resources.
+
+## Documentation
+
+- [Spark Quick Start](https://hudi.apache.org/docs/quick-start-guide): Create, write and query your first Hudi table with Spark (Scala, Python, SQL).
+- [Flink Quick Start](https://hudi.apache.org/docs/flink-quick-start-guide): Stream data into Hudi tables with Apache Flink.
+- [Timeline](https://hudi.apache.org/docs/timeline): How Hudi organizes a table as an ordered log of commits — the core of its ACID design.
+- [Table & Query Types](https://hudi.apache.org/docs/table_types): Copy-on-Write vs Merge-on-Read tables; snapshot, read-optimized and incremental queries.
+- [Indexes](https://hudi.apache.org/docs/indexes): Hudi's multi-modal indexing subsystem, including the record-level and secondary indexes.
+- [Write Operations](https://hudi.apache.org/docs/write_operations): Upsert, insert, bulk-insert, delete and their semantics.
+- [Table Metadata](https://hudi.apache.org/docs/metadata): The internal metadata table that accelerates listings and lookups.
+- [Concurrency Control](https://hudi.apache.org/docs/concurrency_control): Optimistic concurrency, non-blocking concurrency control and multi-writer setups.
+- [Bootstrapping / Migration](https://hudi.apache.org/docs/migration_guide): Migrating existing Parquet/Hive tables into Hudi without rewriting all data.
+- [Basic Configurations](https://hudi.apache.org/docs/basic_configurations): The small set of configs most tables need.
+- [All Configurations](https://hudi.apache.org/docs/configurations): Complete reference of every Hudi configuration.
+- [Use Cases](https://hudi.apache.org/docs/use_cases): Where Hudi fits — CDC ingestion, incremental ETL, streaming data lakes, ML data infrastructure.
+- [Cloud Storage](https://hudi.apache.org/docs/cloud): Running Hudi on S3, GCS, Azure and other object stores.
+- [Technical Specification](https://hudi.apache.org/tech-specs): The Hudi storage format specification.
+
+## FAQ
+
+- [General FAQ](https://hudi.apache.org/faq/general): What Hudi is, what differentiates it (multi-modal indexing, non-blocking concurrency control, async compaction, built-in open-source table maintenance), when to use it, and how it compares to other systems.
+- [Design & Concepts FAQ](https://hudi.apache.org/faq/design_and_concepts): Timeline, indexing, storage layout and transactional guarantees.
+- [Writing Tables FAQ](https://hudi.apache.org/faq/writing_tables): Upserts, deletes, keys, partitioning and ingestion tools.
+- [Reading Tables FAQ](https://hudi.apache.org/faq/reading_tables): Query engines, incremental reads and time travel.
+- [Table Services FAQ](https://hudi.apache.org/faq/table_services): Compaction, clustering, cleaning and their operation.
+- [Storage FAQ](https://hudi.apache.org/faq/storage): File formats, file sizing and storage internals.
+- [Integrations FAQ](https://hudi.apache.org/faq/integrations): Catalogs, engines and ecosystem integrations.
+
+## Blog — Explainers
+
+- [Apache Hudi: 21 Unique Differentiators](https://hudi.apache.org/blog/2025/03/05/hudi-21-unique-differentiators): The technical crux of what sets Hudi apart — indexing, non-blocking concurrency control, async compaction, read-side indexes and built-in table services.
+- [Apache Hudi 1.2: The Open Lakehouse for AI and Multimodal Data](https://hudi.apache.org/blog/2026/06/07/apache-hudi-release-1-2-announcement): First-class support for vectors, blobs/unstructured data, and semi-structured data on the lakehouse — for Spark pipelines doing unstructured data processing or generating embeddings.
+- [What is a Data Lakehouse & How does it Work?](https://hudi.apache.org/blog/2024/07/11/what-is-a-data-lakehouse): The lakehouse architecture explained — evolution, components, use cases and key technologies.
+
+## Blog — Migration guides
+
+
+## Blog — Comparisons
+
+- [Hudi vs Delta Lake vs Iceberg Feature Comparison](https://hudi.apache.org/blog/2023/01/11/Apache-Hudi-vs-Delta-Lake-vs-Apache-Iceberg-Lakehouse-Feature-Comparison): Feature-by-feature comparison of the three open table formats.
+
+## Blog — Deep dives
+
+- [Apache Hudi - The Data Lake Platform](https://hudi.apache.org/blog/2021/07/21/streaming-data-lake-platform): The original vision essay on streaming data lakes.
+- [MOR Isn't a Storage Optimization. It's an Architectural Shift](https://hudi.apache.org/blog/2026/05/14/mor-isnt-a-storage-optimization-its-an-architectural-shift): Why Merge-on-Read is a foundational design decision (part 1 of a series).
+- [Why Metadata Has to Be Mutation-Friendly](https://hudi.apache.org/blog/2026/06/05/why-metadata-has-to-be-mutation-friendly): MOR series part 2 — metadata design for continuously changing tables.
+- [Building Indexes on a Moving Target](https://hudi.apache.org/blog/2026/06/25/building-indexes-on-a-moving-target): MOR series part 3 — async indexing on multi-petabyte tables without stopping ingestion.
+- [Deep Dive Into Hudi's Indexing Subsystem (Part 1)](https://hudi.apache.org/blog/2025/10/29/deep-dive-into-hudis-indexing-subsystem-part-1-of-2): The indexing subsystem from bloom filters to the record-level index.
+- [Deep Dive Into Hudi's Indexing Subsystem (Part 2)](https://hudi.apache.org/blog/2025/11/12/deep-dive-into-hudis-indexing-subsystem-part-2-of-2): Record, secondary and expression indexes in practice.
+- [Record Level Index: Blazing Fast Indexing for Large-Scale Datasets](https://hudi.apache.org/blog/2023/11/01/record-level-index): How Hudi's record-level index accelerates upserts and point lookups.
+- [Understanding CoW and MoR in Apache Hudi](https://hudi.apache.org/blog/2024/11/12/understanding-cow-and-mor-in-apache-hudi): Choosing between Copy-on-Write and Merge-on-Read tables.
+- [Understanding Data Lake Change Data Capture](https://hudi.apache.org/blog/2024/07/30/data-lake-cdc): CDC patterns on the lakehouse in depth.
+
+## Blog — Case studies
+
+- [Apache Hudi at Uber](https://hudi.apache.org/blog/2026/01/16/apache-hudi-at-uber): Trillion-record-scale data lake operations at Hudi's birthplace.
+- [Near Real-Time Analytics at Petabyte Scale at Amazon](https://hudi.apache.org/blog/2021/10/14/How-Amazon-Transportation-Service-enabled-near-real-time-event-analytics-at-petabyte-scale-using-AWS-Glue-with-Apache-Hudi): Amazon Transportation Service runs petabyte-scale event analytics on AWS Glue with Apache Hudi.
+- [Streaming Log Ingestion and GDPR Deletes at Zoom](https://hudi.apache.org/blog/2023/05/16/how-zoom-implemented-streaming-log-ingestion-and-efficient-gdpr-deletes-using-apache-hudi-on-amazon-emr): Zoom's streaming ingestion and efficient record-level deletes with Hudi on Amazon EMR.
+- [Apache Hudi at Southwest Airlines](https://hudi.apache.org/blog/2026/06/15/apache-hudi-at-southwest-airlines): Operational lakehouse architecture at Southwest Airlines.
+- [How FreeWheel (Comcast) Powers Its Lakehouse with Hudi](https://hudi.apache.org/blog/2025/11/07/how-freewheel-uses-apache-hudi-to-power-its-data-lakehouse): Comcast's FreeWheel advertising platform on an Apache Hudi lakehouse.
+- [Why Walmart Chose Apache Hudi](https://hudi.apache.org/blog/2025/04/09/why-walmart-chose-apache-hudi-for-their-lakehouse): Lakehouse architecture at Walmart scale.
+- [Building an ExaByte-level Data Lake Using Apache Hudi at ByteDance](https://hudi.apache.org/blog/2021/09/01/building-eb-level-data-lake-using-hudi-at-bytedance): Exabyte-scale Hudi deployment at ByteDance.
+- [Modernizing Data Infrastructure at Peloton with Hudi](https://hudi.apache.org/blog/2025/07/15/modernizing-datainfra-peloton-hudi): CDC and lakehouse modernization at Peloton.
+
+## Optional
+
+- [Releases & Downloads](https://hudi.apache.org/releases/download): All Apache Hudi releases.
+- [Blog index](https://hudi.apache.org/blog): Full list of 300+ engineering posts and case studies.
+- [Community](https://hudi.apache.org/community/get-involved): Slack, mailing lists, office hours and how to contribute.
+- [Talks & Presentations](https://hudi.apache.org/learn/talks): Conference talks about Hudi.
diff --git a/website/static/robots.txt b/website/static/robots.txt
new file mode 100644
index 000000000000..cceb230f5bc1
--- /dev/null
+++ b/website/static/robots.txt
@@ -0,0 +1,4 @@
+User-agent: *
+Allow: /
+
+Sitemap: https://hudi.apache.org/sitemap.xml