Read all data present in a Data Sets

["Iorio, Michele" <mi...@gft.com>]

Hi,
I would like implemented a component that take in input the values from a Data Sets and put in to array and then call a mathematic function that elaborate them all together.
How do I know when the Data Sets have been read all in order to start the mathematical function?

Thanks
Michele

Michele Iorio
_________________________________________________________________

GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy

T +39 010 583158
michele.iorio@gft.com
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it>

[cid:ActoForTheFuture_GFT@1,2x_68f73108-d76e-4497-aa5c-ff8a5693ce57.png]<https://www.gft.com/it/it/about-us/Sustainability>

Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in

Company subject to the direction and coordination activity of GFT Technologies SE

Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Societ? sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.

In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________

Re: Read all data present in a Data Sets

[Philipp Zehnder <ze...@apache.org>]

Hi Michele,

 

unfortunately, this is not so easy to do.

You could for example add a timer that triggers the calculation once events do not arrive for a certain period.

Or you define an end of file “value” that triggers the calculation.

 

Both options are not ideal, the reason is that processing elements are usually designed for event streams.

The best option would be to use a time window for the calculation based on the timestamp of the events. 

In this case, the processing element would be generic and would also work with event streams.

 

 

Cheers, 

Philipp

 

 

Von: "Iorio, Michele" <mi...@gft.com>
Antworten an: "users@streampipes.apache.org" <us...@streampipes.apache.org>
Datum: Dienstag, 10. Mai 2022 um 16:07
An: "users@streampipes.apache.org" <us...@streampipes.apache.org>
Betreff: Read all data present in a Data Sets

 

Hi,

I would like implemented a component that take in input the values from a Data Sets and put in to array and then call a mathematic function that elaborate them all together.

How do I know when the Data Sets have been read all in order to start the mathematical function?

 

Thanks

Michele

Michele Iorio
_________________________________________________________________

GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy

T +39 010 583158
michele.iorio@gft.com
www.gft.com/it
https://blog.gft.com
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it 



Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in

Company subject to the direction and coordination activity of GFT Technologies SE

Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all’azienda. 

In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________