You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tsfile.apache.org by Jialin Qiao <qi...@apache.org> on 2024/04/15 10:03:37 UTC
Start of TsFile C
Hi,
To support more scenarios on the edge, we plan to start the C language
TsFile development, The code will be in the same repo of
apache/tsfile.
Welcome to participate in this project :)
Jialin Qiao
AW: Start of TsFile C
Posted by Christofer Dutz <ch...@c-ware.de>.
Well in C you have linked lists, consisting of structs with 2 or 3 pointers:
* Pointer to Data
* Pointer to next element
* Pointer to previous element (optional)
Actually, quite trivial.
But it all depends on what’s really needed. But yes … explicitly allocating and freeing memory is of upmost importance.
Chris
Von: Tian Jiang <jt...@163.com>
Datum: Dienstag, 16. April 2024 um 10:36
An: dev@tsfile.apache.org <de...@tsfile.apache.org>
Betreff: Re: Start of TsFile C
Totally agreed. The ultimate goal is to provide only a few high-level interfaces that can do the same as the Java version.
The implementation details are not as concerned since I assume enforcing object-oriented thinking in C can be very painful.
Luckily, TsFile does not need very complicated classes like CompletableFuture.
Most important classes (like headers and metadata) can be translated into C structs with little effort.
Since TsFile mainly involves in-memory data organization and IO operations, my primary concern lies in containers and memory management.
C++ provides convenient (and also efficient) containers like vectors and maps, but I am not familiar with alternatives in C.
Also, preventing memory leaking can be tricky and vital.
We may need advice from someone more experienced like you.
Tian Jiang
At 2024-04-16 15:42:36, "Christofer Dutz" <ch...@c-ware.de> wrote:
>Hi Tian,
>
>I think the library should be close to the Java version semantically but should not focus on mimicking Java functionality.
>
>Conceptually you should be doing the same high-level operations as in Java but do them in a way that a C/C++ developer feels comfortable with.
>
>So, not implementing some CompletableFuture in C, but to use programming paradigms that you use in C.
>
>Also did I build PLC4C aimed at embedded devices too and therefore implement a lot less bells & whistles in drivers, because each bell & whistle needs memory and CPU time.
>I guess TsFile is a bit different with that respect. But yeah … initially I was planning on doing PLC4CPP, but decided to go for PLC4C as the number of devices able to run simple low-level C was a lot higher.
>But they were also totally different requirements.
>
>Sure, I guess I’d be able to help with C … after all that was the first real programming Language I learnt some time around 1995, while I was still at school ;-)
>
>I would however lie, if I said I wouldn’t prefer high-level-languages such as Java, but happy to help ;-)
>
>Chris
>
>
>Von: Tian Jiang <jt...@163.com>
>Datum: Dienstag, 16. April 2024 um 03:31
>An: dev@tsfile.apache.org <de...@tsfile.apache.org>
>Betreff: Re: Start of TsFile C
>Hi Chris,
>
>
>Thanks for your concerns and advice.
>I think it is good enough to keep our current method for the C++ client, namely using CMake for the build and using Maven to call CMake and do the packaging.
>However, things are going to be tough this time since we are not building clients that are simply wrappers of the generated Thrift code.
>We are going to provide a fully functional TsFile library, which should have write, read, and schema interfaces logically identical to those of the Java version.
>The library is especially (and maybe only) meant for embedded systems, where advanced features like class, intelligent point, or generic types in C++ may not apply.
>This also limits the use of third-party libraries greatly, as you have suggested.
>
>
>Fortunately, the TsFile has been well-designed and implemented in Java, so the whole thing is possibly merely a code translation (a massive one, though).
>Unfortunately, we have very limited experience in primitive C programming.
>I assume we will probably struggle to make an available version for months.
>I think we might need your help to check whether the code can work well in an embedded system.
>
>
>Best regards,
>Tian Jiang
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>At 2024-04-15 21:50:36, "Jialin Qiao" <qi...@apache.org> wrote:
>>Hi, I support one large maven build, and manage each in their own.
>>
>>Jialin Qiao
>>
>>Christofer Dutz <ch...@c-ware.de> 于2024年4月15日周一 20:51写道：
>>>
>>> Also, should we probably decide on how to build the project and how to structure the sub-modules.
>>>
>>> Here, I would also like to propose using what we have in PLC4X, as getting such a build to work is quite challenging and we’ve already managed to integrate building the following languages in our build:
>>>
>>>
>>> * Java
>>> * Go
>>> * Python
>>> * C
>>> * C#
>>> * Rust
>>>
>>> The core principles we tried here, is to have one large maven build, that is able to build all parts in one big reactor, but that each of the sub-projects feel natural for developers in their particular languages.
>>>
>>> So, we use IntelliJ for developing the java parts, GoLand for the go part, CLion for c, Raider for c#, PyCharm for Python, … also some people use VSCode this way.
>>>
>>> As there are multiple ways of building C stuff, we decided to use CMake as this seems very portable for Win, Mac and Linux. We should also try to stay clear of requiring boost for the C-related parts as this complicates the build a lot and having something that relies on core C is a lot more portable, especially with respect to porting it to run on PLCs directly.
>>>
>>> For Rust we use the build in cargo stuff, for Go also the built-in stuff. For python I think they use venv in combination with pip3.
>>>
>>> Chris
>>>
>>> Von: Jialin Qiao <qi...@apache.org>
>>> Datum: Montag, 15. April 2024 um 12:03
>>> An: dev@tsfile.apache.org <de...@tsfile.apache.org>
>>> Betreff: Start of TsFile C
>>> Hi,
>>>
>>> To support more scenarios on the edge, we plan to start the C language
>>> TsFile development, The code will be in the same repo of
>>> apache/tsfile.
>>> Welcome to participate in this project :)
>>>
>>> Jialin Qiao
Re: Start of TsFile C
Posted by Tian Jiang <jt...@163.com>.
Totally agreed. The ultimate goal is to provide only a few high-level interfaces that can do the same as the Java version.
The implementation details are not as concerned since I assume enforcing object-oriented thinking in C can be very painful.
Luckily, TsFile does not need very complicated classes like CompletableFuture.
Most important classes (like headers and metadata) can be translated into C structs with little effort.
Since TsFile mainly involves in-memory data organization and IO operations, my primary concern lies in containers and memory management.
C++ provides convenient (and also efficient) containers like vectors and maps, but I am not familiar with alternatives in C.
Also, preventing memory leaking can be tricky and vital.
We may need advice from someone more experienced like you.
Tian Jiang
At 2024-04-16 15:42:36, "Christofer Dutz" <ch...@c-ware.de> wrote:
>Hi Tian,
>
>I think the library should be close to the Java version semantically but should not focus on mimicking Java functionality.
>
>Conceptually you should be doing the same high-level operations as in Java but do them in a way that a C/C++ developer feels comfortable with.
>
>So, not implementing some CompletableFuture in C, but to use programming paradigms that you use in C.
>
>Also did I build PLC4C aimed at embedded devices too and therefore implement a lot less bells & whistles in drivers, because each bell & whistle needs memory and CPU time.
>I guess TsFile is a bit different with that respect. But yeah … initially I was planning on doing PLC4CPP, but decided to go for PLC4C as the number of devices able to run simple low-level C was a lot higher.
>But they were also totally different requirements.
>
>Sure, I guess I’d be able to help with C … after all that was the first real programming Language I learnt some time around 1995, while I was still at school ;-)
>
>I would however lie, if I said I wouldn’t prefer high-level-languages such as Java, but happy to help ;-)
>
>Chris
>
>
>Von: Tian Jiang <jt...@163.com>
>Datum: Dienstag, 16. April 2024 um 03:31
>An: dev@tsfile.apache.org <de...@tsfile.apache.org>
>Betreff: Re: Start of TsFile C
>Hi Chris,
>
>
>Thanks for your concerns and advice.
>I think it is good enough to keep our current method for the C++ client, namely using CMake for the build and using Maven to call CMake and do the packaging.
>However, things are going to be tough this time since we are not building clients that are simply wrappers of the generated Thrift code.
>We are going to provide a fully functional TsFile library, which should have write, read, and schema interfaces logically identical to those of the Java version.
>The library is especially (and maybe only) meant for embedded systems, where advanced features like class, intelligent point, or generic types in C++ may not apply.
>This also limits the use of third-party libraries greatly, as you have suggested.
>
>
>Fortunately, the TsFile has been well-designed and implemented in Java, so the whole thing is possibly merely a code translation (a massive one, though).
>Unfortunately, we have very limited experience in primitive C programming.
>I assume we will probably struggle to make an available version for months.
>I think we might need your help to check whether the code can work well in an embedded system.
>
>
>Best regards,
>Tian Jiang
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>At 2024-04-15 21:50:36, "Jialin Qiao" <qi...@apache.org> wrote:
>>Hi, I support one large maven build, and manage each in their own.
>>
>>Jialin Qiao
>>
>>Christofer Dutz <ch...@c-ware.de> 于2024年4月15日周一 20:51写道：
>>>
>>> Also, should we probably decide on how to build the project and how to structure the sub-modules.
>>>
>>> Here, I would also like to propose using what we have in PLC4X, as getting such a build to work is quite challenging and we’ve already managed to integrate building the following languages in our build:
>>>
>>>
>>> * Java
>>> * Go
>>> * Python
>>> * C
>>> * C#
>>> * Rust
>>>
>>> The core principles we tried here, is to have one large maven build, that is able to build all parts in one big reactor, but that each of the sub-projects feel natural for developers in their particular languages.
>>>
>>> So, we use IntelliJ for developing the java parts, GoLand for the go part, CLion for c, Raider for c#, PyCharm for Python, … also some people use VSCode this way.
>>>
>>> As there are multiple ways of building C stuff, we decided to use CMake as this seems very portable for Win, Mac and Linux. We should also try to stay clear of requiring boost for the C-related parts as this complicates the build a lot and having something that relies on core C is a lot more portable, especially with respect to porting it to run on PLCs directly.
>>>
>>> For Rust we use the build in cargo stuff, for Go also the built-in stuff. For python I think they use venv in combination with pip3.
>>>
>>> Chris
>>>
>>> Von: Jialin Qiao <qi...@apache.org>
>>> Datum: Montag, 15. April 2024 um 12:03
>>> An: dev@tsfile.apache.org <de...@tsfile.apache.org>
>>> Betreff: Start of TsFile C
>>> Hi,
>>>
>>> To support more scenarios on the edge, we plan to start the C language
>>> TsFile development, The code will be in the same repo of
>>> apache/tsfile.
>>> Welcome to participate in this project :)
>>>
>>> Jialin Qiao
AW: Start of TsFile C
Posted by Christofer Dutz <ch...@c-ware.de>.
Hi Tian,
I think the library should be close to the Java version semantically but should not focus on mimicking Java functionality.
Conceptually you should be doing the same high-level operations as in Java but do them in a way that a C/C++ developer feels comfortable with.
So, not implementing some CompletableFuture in C, but to use programming paradigms that you use in C.
Also did I build PLC4C aimed at embedded devices too and therefore implement a lot less bells & whistles in drivers, because each bell & whistle needs memory and CPU time.
I guess TsFile is a bit different with that respect. But yeah … initially I was planning on doing PLC4CPP, but decided to go for PLC4C as the number of devices able to run simple low-level C was a lot higher.
But they were also totally different requirements.
Sure, I guess I’d be able to help with C … after all that was the first real programming Language I learnt some time around 1995, while I was still at school ;-)
I would however lie, if I said I wouldn’t prefer high-level-languages such as Java, but happy to help ;-)
Chris
Von: Tian Jiang <jt...@163.com>
Datum: Dienstag, 16. April 2024 um 03:31
An: dev@tsfile.apache.org <de...@tsfile.apache.org>
Betreff: Re: Start of TsFile C
Hi Chris,
Thanks for your concerns and advice.
I think it is good enough to keep our current method for the C++ client, namely using CMake for the build and using Maven to call CMake and do the packaging.
However, things are going to be tough this time since we are not building clients that are simply wrappers of the generated Thrift code.
We are going to provide a fully functional TsFile library, which should have write, read, and schema interfaces logically identical to those of the Java version.
The library is especially (and maybe only) meant for embedded systems, where advanced features like class, intelligent point, or generic types in C++ may not apply.
This also limits the use of third-party libraries greatly, as you have suggested.
Fortunately, the TsFile has been well-designed and implemented in Java, so the whole thing is possibly merely a code translation (a massive one, though).
Unfortunately, we have very limited experience in primitive C programming.
I assume we will probably struggle to make an available version for months.
I think we might need your help to check whether the code can work well in an embedded system.
Best regards,
Tian Jiang
At 2024-04-15 21:50:36, "Jialin Qiao" <qi...@apache.org> wrote:
>Hi, I support one large maven build, and manage each in their own.
>
>Jialin Qiao
>
>Christofer Dutz <ch...@c-ware.de> 于2024年4月15日周一 20:51写道：
>>
>> Also, should we probably decide on how to build the project and how to structure the sub-modules.
>>
>> Here, I would also like to propose using what we have in PLC4X, as getting such a build to work is quite challenging and we’ve already managed to integrate building the following languages in our build:
>>
>>
>> * Java
>> * Go
>> * Python
>> * C
>> * C#
>> * Rust
>>
>> The core principles we tried here, is to have one large maven build, that is able to build all parts in one big reactor, but that each of the sub-projects feel natural for developers in their particular languages.
>>
>> So, we use IntelliJ for developing the java parts, GoLand for the go part, CLion for c, Raider for c#, PyCharm for Python, … also some people use VSCode this way.
>>
>> As there are multiple ways of building C stuff, we decided to use CMake as this seems very portable for Win, Mac and Linux. We should also try to stay clear of requiring boost for the C-related parts as this complicates the build a lot and having something that relies on core C is a lot more portable, especially with respect to porting it to run on PLCs directly.
>>
>> For Rust we use the build in cargo stuff, for Go also the built-in stuff. For python I think they use venv in combination with pip3.
>>
>> Chris
>>
>> Von: Jialin Qiao <qi...@apache.org>
>> Datum: Montag, 15. April 2024 um 12:03
>> An: dev@tsfile.apache.org <de...@tsfile.apache.org>
>> Betreff: Start of TsFile C
>> Hi,
>>
>> To support more scenarios on the edge, we plan to start the C language
>> TsFile development, The code will be in the same repo of
>> apache/tsfile.
>> Welcome to participate in this project :)
>>
>> Jialin Qiao
Re: Start of TsFile C
Posted by Tian Jiang <jt...@163.com>.
Hi Chris,
Thanks for your concerns and advice.
I think it is good enough to keep our current method for the C++ client, namely using CMake for the build and using Maven to call CMake and do the packaging.
However, things are going to be tough this time since we are not building clients that are simply wrappers of the generated Thrift code.
We are going to provide a fully functional TsFile library, which should have write, read, and schema interfaces logically identical to those of the Java version.
The library is especially (and maybe only) meant for embedded systems, where advanced features like class, intelligent point, or generic types in C++ may not apply.
This also limits the use of third-party libraries greatly, as you have suggested.
Fortunately, the TsFile has been well-designed and implemented in Java, so the whole thing is possibly merely a code translation (a massive one, though).
Unfortunately, we have very limited experience in primitive C programming.
I assume we will probably struggle to make an available version for months.
I think we might need your help to check whether the code can work well in an embedded system.
Best regards,
Tian Jiang
At 2024-04-15 21:50:36, "Jialin Qiao" <qi...@apache.org> wrote:
>Hi, I support one large maven build, and manage each in their own.
>
>Jialin Qiao
>
>Christofer Dutz <ch...@c-ware.de> 于2024年4月15日周一 20:51写道：
>>
>> Also, should we probably decide on how to build the project and how to structure the sub-modules.
>>
>> Here, I would also like to propose using what we have in PLC4X, as getting such a build to work is quite challenging and we’ve already managed to integrate building the following languages in our build:
>>
>>
>> * Java
>> * Go
>> * Python
>> * C
>> * C#
>> * Rust
>>
>> The core principles we tried here, is to have one large maven build, that is able to build all parts in one big reactor, but that each of the sub-projects feel natural for developers in their particular languages.
>>
>> So, we use IntelliJ for developing the java parts, GoLand for the go part, CLion for c, Raider for c#, PyCharm for Python, … also some people use VSCode this way.
>>
>> As there are multiple ways of building C stuff, we decided to use CMake as this seems very portable for Win, Mac and Linux. We should also try to stay clear of requiring boost for the C-related parts as this complicates the build a lot and having something that relies on core C is a lot more portable, especially with respect to porting it to run on PLCs directly.
>>
>> For Rust we use the build in cargo stuff, for Go also the built-in stuff. For python I think they use venv in combination with pip3.
>>
>> Chris
>>
>> Von: Jialin Qiao <qi...@apache.org>
>> Datum: Montag, 15. April 2024 um 12:03
>> An: dev@tsfile.apache.org <de...@tsfile.apache.org>
>> Betreff: Start of TsFile C
>> Hi,
>>
>> To support more scenarios on the edge, we plan to start the C language
>> TsFile development, The code will be in the same repo of
>> apache/tsfile.
>> Welcome to participate in this project :)
>>
>> Jialin Qiao
Re: Start of TsFile C
Posted by Jialin Qiao <qi...@apache.org>.
Hi, I support one large maven build, and manage each in their own.
Jialin Qiao
Christofer Dutz <ch...@c-ware.de> 于2024年4月15日周一 20:51写道：
>
> Also, should we probably decide on how to build the project and how to structure the sub-modules.
>
> Here, I would also like to propose using what we have in PLC4X, as getting such a build to work is quite challenging and we’ve already managed to integrate building the following languages in our build:
>
>
> * Java
> * Go
> * Python
> * C
> * C#
> * Rust
>
> The core principles we tried here, is to have one large maven build, that is able to build all parts in one big reactor, but that each of the sub-projects feel natural for developers in their particular languages.
>
> So, we use IntelliJ for developing the java parts, GoLand for the go part, CLion for c, Raider for c#, PyCharm for Python, … also some people use VSCode this way.
>
> As there are multiple ways of building C stuff, we decided to use CMake as this seems very portable for Win, Mac and Linux. We should also try to stay clear of requiring boost for the C-related parts as this complicates the build a lot and having something that relies on core C is a lot more portable, especially with respect to porting it to run on PLCs directly.
>
> For Rust we use the build in cargo stuff, for Go also the built-in stuff. For python I think they use venv in combination with pip3.
>
> Chris
>
> Von: Jialin Qiao <qi...@apache.org>
> Datum: Montag, 15. April 2024 um 12:03
> An: dev@tsfile.apache.org <de...@tsfile.apache.org>
> Betreff: Start of TsFile C
> Hi,
>
> To support more scenarios on the edge, we plan to start the C language
> TsFile development, The code will be in the same repo of
> apache/tsfile.
> Welcome to participate in this project :)
>
> Jialin Qiao
AW: Start of TsFile C
Posted by Christofer Dutz <ch...@c-ware.de>.
Also, should we probably decide on how to build the project and how to structure the sub-modules.
Here, I would also like to propose using what we have in PLC4X, as getting such a build to work is quite challenging and we’ve already managed to integrate building the following languages in our build:
* Java
* Go
* Python
* C
* C#
* Rust
The core principles we tried here, is to have one large maven build, that is able to build all parts in one big reactor, but that each of the sub-projects feel natural for developers in their particular languages.
So, we use IntelliJ for developing the java parts, GoLand for the go part, CLion for c, Raider for c#, PyCharm for Python, … also some people use VSCode this way.
As there are multiple ways of building C stuff, we decided to use CMake as this seems very portable for Win, Mac and Linux. We should also try to stay clear of requiring boost for the C-related parts as this complicates the build a lot and having something that relies on core C is a lot more portable, especially with respect to porting it to run on PLCs directly.
For Rust we use the build in cargo stuff, for Go also the built-in stuff. For python I think they use venv in combination with pip3.
Chris
Von: Jialin Qiao <qi...@apache.org>
Datum: Montag, 15. April 2024 um 12:03
An: dev@tsfile.apache.org <de...@tsfile.apache.org>
Betreff: Start of TsFile C
Hi,
To support more scenarios on the edge, we plan to start the C language
TsFile development, The code will be in the same repo of
apache/tsfile.
Welcome to participate in this project :)
Jialin Qiao
AW: Start of TsFile C
Posted by Christofer Dutz <ch...@c-ware.de>.
I would like to pitch an idea:
As in PLC4X we also deal with the problem of generating code for reading and writing binary messages in various languages, it might be an idea to evaluate PLC4Xs way of generating code.
In this case we’d create a tsfile.mspec (which julian had already started working on) and then we could auto-generate the biggest part of reading and writing TsFiles for Java, Go, C, Python and possibly even the less supported languages C# and Rust.
Chris
Von: Jialin Qiao <qi...@apache.org>
Datum: Montag, 15. April 2024 um 12:03
An: dev@tsfile.apache.org <de...@tsfile.apache.org>
Betreff: Start of TsFile C
Hi,
To support more scenarios on the edge, we plan to start the C language
TsFile development, The code will be in the same repo of
apache/tsfile.
Welcome to participate in this project :)
Jialin Qiao