Introducing Apache IoTDB use in COVESA Central Data Service Playground

[Stephen Lawrence <st...@renesas.com>]

Hi,

As suggested in the wiki I am introducing myself and a new OSS project using Apache IoTDB.

My name is Stephen Lawrence from Renesas Electronics and I work in the COVESA[1] open automotive alliance. COVESA is focused on developing common approaches and technologies for connected vehicles, such as the VSS data model [2]. I lead the COVESA Data Architecture and Infrastructure group and am a Board member.

We recently released the first version of the OSS COVESA Central Data Service Playground (CDSP)[3] which uses Apache IoTDB as a data store/server. The Playground is a neutral, open playground for investigating data services in-vehicle and off-board in the context of data-centric architectures. Importantly in community terms it provides a means to publish and collaborate on such work in the open.

Constructed as a "project of projects" this first release was focused on creating the first version of the "toolkit". Now we pivot to improving it, creating examples for its use and investigating industry issues. Currently we are focused on Docker deployment for ease of development and connections to other technology. Deployment on Automotive h/w is part of the future roadmap. The contributors include various large vehicle OEMs.

At a recent COVESA Conference I demonstrated this first release. The online documentation is a work in progress but contains a summary of the What/Why/How of the project. If you are interested in more details I led a track of sessions about the Playground at the conference, the presentation for which you can find here [4].

I know I am going to have an increasing number of questions in the coming months, so I wanted to first introduce myself and the project. I hope to be able to contribute back to the IoTDB project as well.

Apologies for the corporately expanded reference URLs in plain text:
[1] http://www.covesa.global/
[2] https://covesa.global/project/vehicle-signal-specification/
[3] https://github.com/COVESA/cdsp
[4] https://wiki.covesa.global/download/attachments/98271360/Central_Data_Service_Playground_track_v2.pptx?version=1&modificationDate=1714486822982&api=v2

Best Wishes,

Stephen Lawrence
COVESA Data Architecture Lead and Board Member
(Renesas Electronics)

RE: Introducing Apache IoTDB use in COVESA Central Data Service Playground

[Stephen Lawrence <st...@renesas.com>]

Hi,

Thank you both, for the encouragement and the offers of support.

> -----Original Message-----
> From: Jialin Qiao <qi...@apache.org>
> Sent: Tuesday, June 18, 2024 9:50 AM
> To: dev@iotdb.apache.org
> Subject: Re: Introducing Apache IoTDB use in COVESA Central Data Service
> Playground
> 
> Hi,
> 
> Glad to see the integration!
> Welcome and feel free to ask :D
> 
> Jialin Qiao
> 
> Pengcheng Zheng <ma...@gmail.com> 于2024年6月15日周六 04:38写道：
> >
> > Hi Stephen,
> >
> > Very exciting project!
> >
> > Thanks for sharing the details of COVESA/CDSP. I remember that you asked
> > some questions in Slack several months ago, and seems that you have made
> > great progress ;-)

Yes, originally I had some questions about running IoTDB on embedded h/w and 
the answers helped me commit to using IoTDB in the Playground.

BTW as part of the project I have done some work integrating IoTDB to other
technology:

1) COVESA and the W3C co-developed a REST-like get/set/sub API for accessing 
vehicle data in-vehicle called VISS. VISS has a reference implementation 
maintained in the VISSR project that supported SQLite and Redis as data
store backends. I upstreamed support for VISSR to also support IoTDB.

2) The Playground includes a bridge for streaming real car data from the 
RemotiveLabs virtual signal platform in the cloud into IoTDB. It has a
very simple implementation currently for the IoTDB connection but it has
no requirements on any other part of the Playground and could be a useful
example to anyone wanting to using RemotiveLabs with IoTDB. 

> >
> > IoTDB has already proved its excellence in the scenario of connected
> > vehicles before, and the key questions you've listed in the
> > presentation happen to be exactly what IoTDB & TsFile are addressing,
> such
> > as data syncing between all touchpoints/ bidirectional sync/ permissions
> &
> > privacy/ model updating/ subscription, etc.
> >
> > Also your ideas of the deployment in Docker and edge-cloud sync (if my
> > memory serves me right..) shall benefit a lot from IoTDB.

Yes, IoTDB has many features that made it attractive for me to use it in
the Playground. I am sure I will have questions as the project progresses😊
Especially, I would like to leverage the greater IoTDB experience here to
check we are using it effectively. So please excuse the odd basic question.

Over the last year I have looked through the presentations on the IoTDB website
and occasionally checked back for updates. Including the Chinese language 
ones where I have been able to translate. I remember seeing some mentions 
of IoTDB implementation in automotive but not many details. If I remember
correctly there was a major instance for a telematics use-case. I also saw 
that Table 4 of the IoTDB paper from SIGMOD 2023 listed three automotive 
deployments.

If you have details of IoTDB automotive deployments I would be very interested.

> >
> > Hopefully IoTDB could support the CDSP project well and help transform
> the
> > "Playground" to POC - and finally from theory to practice.

I think it will. Put simply the Playground is a toolset of Lego pieces.
These pieces are intended to be connected in a range of ways to create
PoCs or examples of logical concepts large and small and prove them out
in implementation. Hence "Playground". This is not a project to make a 
specific middleware 'product' or address a specific singular use-case.
At least in the current project setup.

The open source community aspect of COVESA means for effective collaboration
we need to also support diversity in architecture. In-vehicle architecture is
very complex and OEM A will make different choices to OEM B.

So, production will be a second step for individual companies to take
with their partners suited to their architectural and purchasing decisions.
That said, of course the advantage of using proven technology like IoTDB 
is that the path to production is there.

> >
> > Looking forward to your updates!

Thanks.

Best Wishes,

Stephen Lawrence

> >
> >
> > Best Wishes,
> > Pengcheng Zheng
> > Timecho Europe

[snip]

Re: Introducing Apache IoTDB use in COVESA Central Data Service Playground

[Jialin Qiao <qi...@apache.org>]

Hi,

Glad to see the integration!
Welcome and feel free to ask :D

Jialin Qiao

Pengcheng Zheng <ma...@gmail.com> 于2024年6月15日周六 04:38写道：
>
> Hi Stephen,
>
> Very exciting project!
>
> Thanks for sharing the details of COVESA/CDSP. I remember that you asked
> some questions in Slack several months ago, and seems that you have made
> great progress ;-)
>
> IoTDB has already proved its excellence in the scenario of connected
> vehicles before, and the key questions you've listed in the
> presentation happen to be exactly what IoTDB & TsFile are addressing, such
> as data syncing between all touchpoints/ bidirectional sync/ permissions &
> privacy/ model updating/ subscription, etc.
>
> Also your ideas of the deployment in Docker and edge-cloud sync (if my
> memory serves me right..) shall benefit a lot from IoTDB.
>
> Hopefully IoTDB could support the CDSP project well and help transform the
> "Playground" to POC - and finally from theory to practice.
>
> Looking forward to your updates!
>
>
> Best Wishes,
> Pengcheng Zheng
> Timecho Europe
>
> Am Fr., 14. Juni 2024 um 21:12 Uhr schrieb Stephen Lawrence <
> stephen.lawrence@renesas.com>:
>
> > Hi,
> >
> > As suggested in the wiki I am introducing myself and a new OSS project
> > using Apache IoTDB.
> >
> > My name is Stephen Lawrence from Renesas Electronics and I work in the
> > COVESA[1] open automotive alliance. COVESA is focused on developing common
> > approaches and technologies for connected vehicles, such as the VSS data
> > model [2]. I lead the COVESA Data Architecture and Infrastructure group and
> > am a Board member.
> >
> > We recently released the first version of the OSS COVESA Central Data
> > Service Playground (CDSP)[3] which uses Apache IoTDB as a data
> > store/server. The Playground is a neutral, open playground for
> > investigating data services in-vehicle and off-board in the context of
> > data-centric architectures. Importantly in community terms it provides a
> > means to publish and collaborate on such work in the open.
> >
> > Constructed as a "project of projects" this first release was focused on
> > creating the first version of the "toolkit". Now we pivot to improving it,
> > creating examples for its use and investigating industry issues. Currently
> > we are focused on Docker deployment for ease of development and connections
> > to other technology. Deployment on Automotive h/w is part of the future
> > roadmap. The contributors include various large vehicle OEMs.
> >
> > At a recent COVESA Conference I demonstrated this first release. The
> > online documentation is a work in progress but contains a summary of the
> > What/Why/How of the project. If you are interested in more details I led a
> > track of sessions about the Playground at the conference, the presentation
> > for which you can find here [4].
> >
> > I know I am going to have an increasing number of questions in the coming
> > months, so I wanted to first introduce myself and the project. I hope to be
> > able to contribute back to the IoTDB project as well.
> >
> > Apologies for the corporately expanded reference URLs in plain text:
> > [1] http://www.covesa.global/
> > [2] https://covesa.global/project/vehicle-signal-specification/
> > [3] https://github.com/COVESA/cdsp
> > [4]
> > https://wiki.covesa.global/download/attachments/98271360/Central_Data_Service_Playground_track_v2.pptx?version=1&modificationDate=1714486822982&api=v2
> >
> > Best Wishes,
> >
> > Stephen Lawrence
> > COVESA Data Architecture Lead and Board Member
> > (Renesas Electronics)
> >

Re: Introducing Apache IoTDB use in COVESA Central Data Service Playground

[Pengcheng Zheng <ma...@gmail.com>]

Hi Stephen,

Very exciting project!

Thanks for sharing the details of COVESA/CDSP. I remember that you asked
some questions in Slack several months ago, and seems that you have made
great progress ;-)

IoTDB has already proved its excellence in the scenario of connected
vehicles before, and the key questions you've listed in the
presentation happen to be exactly what IoTDB & TsFile are addressing, such
as data syncing between all touchpoints/ bidirectional sync/ permissions &
privacy/ model updating/ subscription, etc.

Also your ideas of the deployment in Docker and edge-cloud sync (if my
memory serves me right..) shall benefit a lot from IoTDB.

Hopefully IoTDB could support the CDSP project well and help transform the
"Playground" to POC - and finally from theory to practice.

Looking forward to your updates!


Best Wishes,
Pengcheng Zheng
Timecho Europe

Am Fr., 14. Juni 2024 um 21:12 Uhr schrieb Stephen Lawrence <
stephen.lawrence@renesas.com>:

> Hi,
>
> As suggested in the wiki I am introducing myself and a new OSS project
> using Apache IoTDB.
>
> My name is Stephen Lawrence from Renesas Electronics and I work in the
> COVESA[1] open automotive alliance. COVESA is focused on developing common
> approaches and technologies for connected vehicles, such as the VSS data
> model [2]. I lead the COVESA Data Architecture and Infrastructure group and
> am a Board member.
>
> We recently released the first version of the OSS COVESA Central Data
> Service Playground (CDSP)[3] which uses Apache IoTDB as a data
> store/server. The Playground is a neutral, open playground for
> investigating data services in-vehicle and off-board in the context of
> data-centric architectures. Importantly in community terms it provides a
> means to publish and collaborate on such work in the open.
>
> Constructed as a "project of projects" this first release was focused on
> creating the first version of the "toolkit". Now we pivot to improving it,
> creating examples for its use and investigating industry issues. Currently
> we are focused on Docker deployment for ease of development and connections
> to other technology. Deployment on Automotive h/w is part of the future
> roadmap. The contributors include various large vehicle OEMs.
>
> At a recent COVESA Conference I demonstrated this first release. The
> online documentation is a work in progress but contains a summary of the
> What/Why/How of the project. If you are interested in more details I led a
> track of sessions about the Playground at the conference, the presentation
> for which you can find here [4].
>
> I know I am going to have an increasing number of questions in the coming
> months, so I wanted to first introduce myself and the project. I hope to be
> able to contribute back to the IoTDB project as well.
>
> Apologies for the corporately expanded reference URLs in plain text:
> [1] http://www.covesa.global/
> [2] https://covesa.global/project/vehicle-signal-specification/
> [3] https://github.com/COVESA/cdsp
> [4]
> https://wiki.covesa.global/download/attachments/98271360/Central_Data_Service_Playground_track_v2.pptx?version=1&modificationDate=1714486822982&api=v2
>
> Best Wishes,
>
> Stephen Lawrence
> COVESA Data Architecture Lead and Board Member
> (Renesas Electronics)
>