You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tsfile.apache.org by "shuolin_l@163.com" <sh...@163.com> on 2026/01/19 06:18:45 UTC
Proposal on C++ Standard Strategy for Optional Arrow-based Features in TsFile
Hi all,
During the recent work on enabling batch read/write support for TsFile based on Apache Arrow, I noticed a divergence between Arrow and our current project in terms of supported C++ standards. This raises an important question regarding the technical strategy for TsFile as a common component across both embedded and cloud environments.
According to the official Arrow documentation, starting from version 10.0.0, C++11 is no longer supported and C++17 is required (see issue [1]). Therefore, introducing Arrow-related capabilities inevitably requires a build environment that supports a newer C++ standard.
In embedded scenarios, although we already provide C-based interfaces to minimize dependency on C++ versions, in practice some embedded or legacy edge devices still only support relatively old C++ standards. In contrast, cloud environments or platforms used for Python-based training and data processing usually have more modern toolchains, where C++17 support is generally not an issue.
Given these differences in usage scenarios, I propose adopting a tiered strategy for C++ standard support within the project:
* The core and basic functionality of the project will continue to use C++11 as the minimum required standard, to ensure compatibility with resource-constrained or legacy platforms.
* Features that depend on newer capabilities or third-party libraries such as Arrow will be provided as optional extension modules, and implemented and built using C++17.
* Users can decide whether to enable these optional features based on their runtime environment and functional requirements, and accordingly choose the appropriate C++ standard for compilation.
With this approach, we aim to preserve overall compatibility while still enabling the introduction of high-performance data processing features, and to better accommodate the differences between embedded and cloud environments in terms of toolchains and functional needs.
[1] https://github.com/apache/arrow/issues/32415
Best regards,
Colin.
Re: Proposal on tiered C++ standard support for TsFile
Posted by Hongzhi Gao <ho...@foxmail.com>.
Hi all,
I generally support the tiered C++ standard approach.
Since Arrow requires C++17 starting from 10.0, Arrow-related functionality should be treated as an optional extension rather than part of the TsFile core. The core library should explicitly remain at C++11 to preserve compatibility with embedded and legacy environments.
From an implementation perspective, Arrow support can live in a separate module (e.g. tsfile-arrow), built with C++17 and guarded by a dedicated build option (such as TSFILE_ENABLE_ARROW). The interface between the core and this module should be kept narrow and avoid exposing Arrow types or C++17 features in public headers.
This keeps the core stable while still allowing Arrow-based capabilities where the toolchain and deployment environment permit.
Best regards,
Hingzhi Gao
Hongzhi Gao
hongzhigao@foxmail.com
原始邮件
发件人：shuolin_l <shuolin_l@163.com>
发件时间：2026年1月19日 14:18
收件人：dev <dev@tsfile.apache.org>
主题：Proposal on C++ Standard Strategy for Optional Arrow-based Featuresin TsFile
Hi all,
During the recent work on enabling batch read/write support for TsFile based on Apache Arrow, I noticed a divergence between Arrow and our current project in terms of supported C++ standards. This raises an important question regarding the technical strategy for TsFile as a common component across both embedded and cloud environments.
According to the official Arrow documentation, starting from version 10.0.0, C++11 is no longer supported and C++17 is required (see issue [1]). Therefore, introducing Arrow-related capabilities inevitably requires a build environment that supports a newer C++ standard.
In embedded scenarios, although we already provide C-based interfaces to minimize dependency on C++ versions, in practice some embedded or legacy edge devices still only support relatively old C++ standards. In contrast, cloud environments or platforms used for Python-based training and data processing usually have more modern toolchains, where C++17 support is generally not an issue.
Given these differences in usage scenarios, I propose adopting a tiered strategy for C++ standard support within the project:
* The core and basic functionality of the project will continue to use C++11 as the minimum required standard, to ensure compatibility with resource-constrained or legacy platforms.
* Features that depend on newer capabilities or third-party libraries such as Arrow will be provided as optional extension modules, and implemented and built using C++17.
* Users can decide whether to enable these optional features based on their runtime environment and functional requirements, and accordingly choose the appropriate C++ standard for compilation.
With this approach, we aim to preserve overall compatibility while still enabling the introduction of high-performance data processing features, and to better accommodate the differences between embedded and cloud environments in terms of toolchains and functional needs.
[1] https://github.com/apache/arrow/issues/32415
Best regards,
Colin.
Re: Proposal on C++ Standard Strategy for Optional Arrow-based Features in TsFile
Posted by Tian Jiang <jt...@163.com>.
It is an inspiring proposal and also a useful one.
In other Cpp libraries, there will be a ./configure script running a configuration check,
finding the supported highest standard and automatically adding some options to the modified cmake file.
I think it is high time that we introduce a similar solution.
Best,
Tian Jiang
---- Replied Message ----
| From | shuolin_l@163.com<sh...@163.com> |
| Date | 1/19/2026 14:19 |
| To | dev<de...@tsfile.apache.org> |
| Subject | Proposal on C++ Standard Strategy for Optional Arrow-based Features in TsFile |
Hi all,
During the recent work on enabling batch read/write support for TsFile based on Apache Arrow, I noticed a divergence between Arrow and our current project in terms of supported C++ standards. This raises an important question regarding the technical strategy for TsFile as a common component across both embedded and cloud environments.
According to the official Arrow documentation, starting from version 10.0.0, C++11 is no longer supported and C++17 is required (see issue [1]). Therefore, introducing Arrow-related capabilities inevitably requires a build environment that supports a newer C++ standard.
In embedded scenarios, although we already provide C-based interfaces to minimize dependency on C++ versions, in practice some embedded or legacy edge devices still only support relatively old C++ standards. In contrast, cloud environments or platforms used for Python-based training and data processing usually have more modern toolchains, where C++17 support is generally not an issue.
Given these differences in usage scenarios, I propose adopting a tiered strategy for C++ standard support within the project:
* The core and basic functionality of the project will continue to use C++11 as the minimum required standard, to ensure compatibility with resource-constrained or legacy platforms.
* Features that depend on newer capabilities or third-party libraries such as Arrow will be provided as optional extension modules, and implemented and built using C++17.
* Users can decide whether to enable these optional features based on their runtime environment and functional requirements, and accordingly choose the appropriate C++ standard for compilation.
With this approach, we aim to preserve overall compatibility while still enabling the introduction of high-performance data processing features, and to better accommodate the differences between embedded and cloud environments in terms of toolchains and functional needs.
[1] https://github.com/apache/arrow/issues/32415
Best regards,
Colin.