You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@xtable.apache.org by "vinishjail97 (via GitHub)" <gi...@apache.org> on 2026/07/21 21:45:53 UTC

[I] XTable 4th Release [incubator-xtable]

vinishjail97 opened a new issue, #855:
URL: https://github.com/apache/incubator-xtable/issues/855

   XTable 4th release updates (0.4.0-incubating).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] XTable 4th Release [incubator-xtable]

Posted by "vinishjail97 (via GitHub)" <gi...@apache.org>.

vinishjail97 commented on issue #855:
URL: https://github.com/apache/incubator-xtable/issues/855#issuecomment-5223005730

   # Release Notes for release-0.4.0-incubating-rc2
   
   Apache XTable™ (Incubating) is a cross-table converter for table formats that facilitates omni-directional interoperability across data processing systems and query engines. Currently, Apache XTable™ supports widely adopted open-source table formats such as Apache Hudi, Apache Iceberg, Delta Lake, and (as a source) Apache Paimon.
   
   ## 🔁 Changes since rc1
   
   rc1 was cancelled. This candidate contains the following additional changes.
   
   - **Make the source release build with `mvn package`** (#864, #868). `quarkus-maven-plugin` bootstraps the application to generate code, so it resolves `xtable-service`'s dependencies with Quarkus' own bootstrap resolver instead of through the reactor. That resolver discovered the workspace from the raw poms without interpolating properties, so the module declaring `xtable-core_${scala.binary.version}` was never matched against the `xtable-core_2.12` dependency and resolution fell back to Maven Central, where the artifact does not exist until the release is published. `.mvn/maven.config` now sets `-Dquarkus.bootstrap.effective-model-builder=true` so workspace discovery uses the effective model. A new **Source Build Check** workflow builds `xtable-service` with the local repository cleared of XTable artifacts and no caching, so this cannot regress silently.
   - **Cover 2026 in the NOTICE copyright range** (#872, #874), and check the range at release time as part of `validate_source_copyright.sh`.
   - Exclude the `xtable-service/examples` screenshots from the source release tarball, so the source release contains no binary files.
   
   ### Known issues
   
   The license documentation issues raised during the rc1 review remain open and are targeted for the next release: incomplete license information for bundled jars (#700), multiple LICENSE files in some jars (#701), and orphaned license files in bundled jars (#865).
   
   ## ✨ What's Changed
   ### 🔄 Core Functionality Enhancements
   - **New formats**:
     - Added a **Delta Kernel** conversion target (#801, #729).
     - Added **Apache Paimon source** support, with incremental sync and column stats (#742, #780, #767).
     - Added **Apache Parquet source** support, with partition extraction and incremental sync (#728, #669, #768, #805).
   - **Sync improvements**:
     - Delta source: extract **nested field statistics** (#769).
     - **Nested-column partition** support in `HudiPartitionValuesExtractor` with hive-style partitioning (#738).
     - Fall back to **parquet footers** when metadata-table column stats are missing (#818).
     - Enable **Delta checkpoint CRC** exception throwing by default (#817).
     - Add additional properties when converting for **Hudi** (#714).
   - **Bug fixes**:
     - Delta: null partition value for composite generated-column partitions (#828), map-key path handling (#826), NPE for binary in map/array schemas (#795), column-mapping-enabled source tables (#766).
     - Hudi: batch `INSERT_OVERWRITE` replacecommits dropping adds (#816), empty `EarliestCommitToRetain` (#732).
     - Iceberg: nested comments with qualified name (#797), schema sync (#749), empty source table (#756).
     - Parquet: snapshot sync on multiple commits with partitions (#806), schema-conversion bugs (#805).
   
   ### 📦 Packaging & Distribution
   - Added **xtable-spark-runtime**, a thin drop-in Spark bundle for Spark 3.4 and 3.5 (#843).
   - Made the bundled **xtable-utilities** jar runnable again (#841, #736).
   - Fixed the scope of spark-catalyst in the utilities bundle (#790).
   
   ### 🧪 Testing & Quality
   - Added an initial integration test for the **XTable REST service** (#716).
   - Improved `TestHudiInstantUtils` with parameterization and more cases (#703).
   - Removed wildcard imports and enforced with **Spotless** (#809).
   - Removed redundant `getSnapshotAt` calls per commit (#791).
   
   ### 📘 Documentation & Community
   - Added a **0.3.0 release page** (#725) and updated the **community sync** page (#734).
   
   ### 🧰 Infrastructure & Maintenance
   - Upgraded **Apache Iceberg** to 1.9.2 and **Avro** to 1.12.0 (#784).
   - Added a **Maven wrapper** for consistent builds (#788).
   - Automated **license file generation and validation** (#822); removed a duplicate `javassist` LICENSE entry (#745).
   - Restored the **delombok** execution so release builds generate javadoc jars (#853).
   - Added a **Source Build Check** workflow that builds the source release without any installed XTable artifacts (#868).
   - Added `.sdkmanrc` to `.gitignore` (#793); bumped GitHub Actions (`checkout`, `setup-node`, `setup-java`).
   
   ## Detailed Change Log
   * [553] Parquet Schema and Column Stats Converters by @sapienza88 in https://github.com/apache/incubator-xtable/pull/669
   * Add additional properties when doing conversion for hudi by @rahil-c in https://github.com/apache/incubator-xtable/pull/714
   * Improved all tests in TestHudiInstantUtils by parameterising and adding more test cases by @Monilnarang in https://github.com/apache/incubator-xtable/pull/703
   * Fix typo in InternalTable by @vinishjail97 in https://github.com/apache/incubator-xtable/pull/720
   * [DOCS] Add a new 0.3.0 release page by @sagarlakshmipathy in https://github.com/apache/incubator-xtable/pull/725
   * Add intial IT for XTable REST Service by @rahil-c in https://github.com/apache/incubator-xtable/pull/716
   * Update community sync page by @pratyakshsharma in https://github.com/apache/incubator-xtable/pull/734
   * Bump actions/setup-java from 4 to 5 by @dependabot[bot] in https://github.com/apache/incubator-xtable/pull/740
   * [739] Nested column partition support in HudiPartitionValuesExtractor when hive style partitioning is enabled by @kroushan-nit in https://github.com/apache/incubator-xtable/pull/738
   * Bump actions/setup-node from 4 to 5 by @dependabot[bot] in https://github.com/apache/incubator-xtable/pull/741
   * Bump actions/checkout from 4 to 5 by @dependabot[bot] in https://github.com/apache/incubator-xtable/pull/737
   * [553] parquet source with partition extraction by @sapienza88 in https://github.com/apache/incubator-xtable/pull/728
   * [LICENSE] Remove duplicate entry for org.javassist:javassist:3.25.0-G… by @piyushdubey in https://github.com/apache/incubator-xtable/pull/745
   * [750] Fix schema sync for Iceberg tables by @the-other-tim-brown in https://github.com/apache/incubator-xtable/pull/749
   * Paimon Source Support by @mikedias in https://github.com/apache/incubator-xtable/pull/742
   * Handle empty Iceberg table source in conversion by @brishi19791 in https://github.com/apache/incubator-xtable/pull/756
   * Bump actions/checkout from 5 to 6 by @dependabot[bot] in https://github.com/apache/incubator-xtable/pull/761
   * Delta Kernel Draft PR by @vaibhavk1992 in https://github.com/apache/incubator-xtable/pull/729
   * Delta source: extract nested field's stats by @emilie-wang in https://github.com/apache/incubator-xtable/pull/769
   * Fix Delta to Iceberg not working on column mapping enabled Delta source table by @xr-chen in https://github.com/apache/incubator-xtable/pull/766
   * Implement Paimon Source Incremental Sync by @mikedias in https://github.com/apache/incubator-xtable/pull/780
   * [787] Add maven wrapper for consistent builds by @the-other-tim-brown in https://github.com/apache/incubator-xtable/pull/788
   * [755] Support column stats for paimon by @mao-liu in https://github.com/apache/incubator-xtable/pull/767
   * [789] Fix scope of spark catalyst in utilities bundle by @the-other-tim-brown in https://github.com/apache/incubator-xtable/pull/790
   * Upgrade Iceberg to 1.9.2 and Avro to 1.12.0 by @kevinjqliu in https://github.com/apache/incubator-xtable/pull/784
   * fix(delta): avoid NPE for binary in map/array schemas by @parisni in https://github.com/apache/incubator-xtable/pull/795
   * Remove redundant getSnapshotAt calls per commit by @brishi19791 in https://github.com/apache/incubator-xtable/pull/791
   * fix(iceberg): nested comments with qualified name by @parisni in https://github.com/apache/incubator-xtable/pull/797
   * add .sdkmanrc to .gitignore by @kevinjqliu in https://github.com/apache/incubator-xtable/pull/793
   * Remove wildcard imports and enforce with spotless by @the-other-tim-brown in https://github.com/apache/incubator-xtable/pull/809
   * Parquet source: Expand column stats support, fix bugs in schema conversion by @the-other-tim-brown in https://github.com/apache/incubator-xtable/pull/805
   * [bug] Parquet Source: snapshot sync fails on multiple commits with partitions by @the-other-tim-brown in https://github.com/apache/incubator-xtable/pull/806
   * Enable Delta checkpoint CRC exception throwing by default by @vinishjail97 in https://github.com/apache/incubator-xtable/pull/817
   * Bump actions/setup-node from 5 to 6 by @dependabot[bot] in https://github.com/apache/incubator-xtable/pull/753
   * Handle empty EarliestCommitToRetain by @alexr17 in https://github.com/apache/incubator-xtable/pull/732
   * [bug] Fix batch INSERT_OVERWRITE replacecommits dropping adds in HudiDataFileExtractor by @vinishjail97 in https://github.com/apache/incubator-xtable/pull/816
   * Fallback to parquet footers when metadata table col stats are missing by @vinishjail97 in https://github.com/apache/incubator-xtable/pull/818
   * Automate license file generation and validation by @the-other-tim-brown in https://github.com/apache/incubator-xtable/pull/822
   * Delta kernel conversion target by @vaibhavk1992 in https://github.com/apache/incubator-xtable/pull/801
   * Fix map key path handling in Delta schema extractors by @brishi19791 in https://github.com/apache/incubator-xtable/pull/826
   * Parquet Incremental Sync by @sapienza88 in https://github.com/apache/incubator-xtable/pull/768
   * [MINOR] Preserve original cause in ReadException on close failure by @alealandreev in https://github.com/apache/incubator-xtable/pull/827
   * fix(delta): null partition value for composite generated-column partitions by @alealandreev in https://github.com/apache/incubator-xtable/pull/828
   * [736] Make the bundled xtable-utilities jar runnable again (Hudi 0.x) + CI on branch-0.4 by @vinishjail97 in https://github.com/apache/incubator-xtable/pull/841
   * [836] xtable-spark-runtime: thin drop-in Spark bundle for 0.4.x (Spark 3.4 + 3.5) by @vinishjail97 in https://github.com/apache/incubator-xtable/pull/843
   * fix(build): restore delombok execution so release builds generate javadoc jars by @vinishjail97 in https://github.com/apache/incubator-xtable/pull/853
   * [864] Make the source release build with mvn package by @vinishjail97 in https://github.com/apache/incubator-xtable/pull/868
   * [872] Cover 2026 in the NOTICE copyright range and check it at release time by @vinishjail97 in https://github.com/apache/incubator-xtable/pull/874
   
   ## New Contributors
   * @sapienza88 made their first contribution in https://github.com/apache/incubator-xtable/pull/669
   * @Monilnarang made their first contribution in https://github.com/apache/incubator-xtable/pull/703
   * @pratyakshsharma made their first contribution in https://github.com/apache/incubator-xtable/pull/734
   * @mikedias made their first contribution in https://github.com/apache/incubator-xtable/pull/742
   * @xr-chen made their first contribution in https://github.com/apache/incubator-xtable/pull/766
   * @mao-liu made their first contribution in https://github.com/apache/incubator-xtable/pull/767
   * @kevinjqliu made their first contribution in https://github.com/apache/incubator-xtable/pull/784
   * @parisni made their first contribution in https://github.com/apache/incubator-xtable/pull/795
   * @alexr17 made their first contribution in https://github.com/apache/incubator-xtable/pull/732
   * @alealandreev made their first contribution in https://github.com/apache/incubator-xtable/pull/827
   
   **Full Changelog**: https://github.com/apache/incubator-xtable/compare/0.3.0-incubating...0.4.0-incubating-rc2


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] XTable 4th Release [incubator-xtable]

Posted by "vinishjail97 (via GitHub)" <gi...@apache.org>.

vinishjail97 commented on issue #855:
URL: https://github.com/apache/incubator-xtable/issues/855#issuecomment-5040820566

   # Apache XTable (Incubating) 0.4.0 — Release Notes
   
   Changes since 0.3.0-incubating.
   
   ## New Features
   
   - **Delta Kernel conversion target** — a new Delta target built on Delta Kernel (#801, #729)
   - **Apache Paimon source support** — read Paimon tables as a conversion source (#742), with incremental sync (#780) and column stats (#767)
   - **Apache Parquet source support** — snapshot sync with partition extraction (#728, #669) and incremental sync (#768)
   - **xtable-spark-runtime** — a thin drop-in Spark bundle for Spark 3.4 and 3.5 (#843)
   
   ## Improvements
   
   - Make the bundled `xtable-utilities` jar runnable again (#841, #736)
   - Upgrade Apache Iceberg to 1.9.2 and Avro to 1.12.0 (#784)
   - Add a Maven wrapper for consistent builds (#788)
   - Parquet source: expand column stats support (#805)
   - Delta source: extract nested field statistics (#769)
   - Nested-column partition support in `HudiPartitionValuesExtractor` with hive-style partitioning (#738)
   - Fall back to parquet footers when metadata-table column stats are missing (#818)
   - Enable Delta checkpoint CRC exception throwing by default (#817)
   - Remove redundant `getSnapshotAt` calls per commit (#791)
   - Add additional properties when converting for Hudi (#714)
   - Preserve the original cause in `ReadException` on close failure (#827)
   - Automate license file generation and validation (#822)
   - Remove wildcard imports and enforce with Spotless (#809)
   - Add initial integration test for the XTable REST service (#716)
   
   ## Bug Fixes
   
   - Delta: null partition value for composite generated-column partitions (#828)
   - Delta: fix map key path handling in schema extractors (#826)
   - Delta: avoid NPE for binary in map/array schemas (#795)
   - Delta: fix Delta-to-Iceberg on column-mapping-enabled source tables (#766)
   - Hudi: fix batch `INSERT_OVERWRITE` replacecommits dropping adds in `HudiDataFileExtractor` (#816)
   - Hudi: handle empty `EarliestCommitToRetain` (#732)
   - Iceberg: fix nested comments with qualified name (#797)
   - Iceberg: fix schema sync for Iceberg tables (#749)
   - Iceberg: handle empty Iceberg table source in conversion (#756)
   - Parquet source: fix snapshot sync failing on multiple commits with partitions (#806)
   - Parquet: fix bugs in schema conversion (#805)
   - Restore `lombok-maven-plugin` delombok execution so javadoc jars are generated (#853)
   - Fix typo in `InternalTable` (#720)
   
   ## Build, Infra, Docs & Dependencies
   
   - Enable CI on the `branch-0.4` release branch (#841)
   - Remove duplicate LICENSE entry for `org.javassist:javassist:3.25.0-GA` (#745)
   - Improve `TestHudiInstantUtils` with parameterization and more cases (#703)
   - Add `.sdkmanrc` to `.gitignore` (#793)
   - Docs: add a 0.3.0 release page (#725); update `sync.md` (#734)
   - Bump `actions/checkout` 4→5 (#737) and 5→6 (#761)
   - Bump `actions/setup-node` 4→5 (#741) and 5→6 (#753)
   - Bump `actions/setup-java` 4→5 (#740)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] XTable 4th Release [incubator-xtable]

Posted by "vinishjail97 (via GitHub)" <gi...@apache.org>.

vinishjail97 closed issue #855: XTable 4th Release
URL: https://github.com/apache/incubator-xtable/issues/855


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org