You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@streampipes.apache.org by "Rullo, Federico" <Fe...@gft.com> on 2023/04/19 14:34:37 UTC
Compose Issue
Hello,
When we use the "compose" component to aggregate data, it does not output all data from the stream where the timestamp was not selected but returns all data from the stream with the selected timestamp.
We also tried to aggregate our data with the "synchronize two streams" component,
but it does not output any our data even though it consumes the data.
Moreover, when we use these components with two streams ad hoc from the machine data simulator, they work well.
Do you know if this is a know issue? Can you give us more information about it?
We are currently running on the version 0.70.0, because of an issue with version 0.90.0.
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Societ? sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:signature-cloud_e31f4244-bb3f-4ff4-be4f-552f04b8a5ac.png]<https://www.gft.com/it/it/solutions/SOLUTIONS/it/IT-Metaverso>
RE: Compose Issue
Posted by "Rullo, Federico" <Fe...@gft.com>.
Hello Philipp,
Yes, you have got right, but it seems it is not the reason for the problems with the compose component. By directly connecting our adapters to the compose component we experienced the same problem.
Our HTTP stream-based data source adapter retrieves a set of data at each polling and passes it to a processor. So, on further testing, we noticed that this could be related to the type of data source (set or stream) we are using.
It seems that the compose component does not like the sets of data we give to it, because by using a file stream adapter and not our adapter (or a file set adapter) with the same data, it responds better.
Best,
Fede
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:signature-cloud_e31f4244-bb3f-4ff4-be4f-552f04b8a5ac.png]<https://www.gft.com/it/it/solutions/SOLUTIONS/it/IT-Metaverso>
From: Philipp Zehnder <ph...@bytefabrik.ai>
Sent: venerdì 5 maggio 2023 16:34
To: users@streampipes.apache.org
Subject: AW: Compose Issue
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Fede,
so I assume you are using the adapter with the schema (date_unix_ts, value) as input to the signal monitoring processor, right?
It looks like you used a fixed output strategy and named the timestamp ('timestamp').
In the original stream the timestamp is named ('date_unix_ts'), did you convert this value accordingly in your onEvent method, or could this be the reason for the problems you have with the compose component?
Cheers,
Philipp
.........................................................
Philipp Zehnder
Bytefabrik.AI GmbH
Haid-und-Neu-Str. 10-14
76131 Karlsruhe
Tel.: +49 721 9654-805
philipp.zehnder@bytefabrik.ai<ma...@bytefabrik.ai>
www.bytefabrik.ai <https://www.bytefabrik.ai/> | www.twitter.com/bytefabrik<https://www.bytefabrik.ai/>
.........................................................
Bytefabrik.AI GmbH
Sitz der Gesellschaft: Karlsruhe
Registergericht: Amtsgericht Mannheim, HRB 741226
Geschäftsführer: Dr.-Ing. Dominik Riemer, Dr.-Ing. Philipp Zehnder
.........................................................
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Dienstag, 2. Mai 2023 um 14:23
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: RE: Compose Issue
Hi Philipp,
Please find attached some screenshots of our schema.
Best,
Fede.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com<https://blog.gft.com/>
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:image001.png@01D98256.375C6260]
From: Philipp Zehnder <ph...@bytefabrik.ai>>
Sent: martedì 2 maggio 2023 11:10
To: users@streampipes.apache.org<ma...@streampipes.apache.org>
Subject: AW: Compose Issue
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Federico,
what is the output strategy of your processors?
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Mittwoch, 26. April 2023 um 09:46
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: RE: Compose Issue
Hi Philipp,
We also tried with a vanilla version of the 0.91.0 stable and had no luck.
We are creating our own, Data Sources, yes, but the problem we have is only with the Compose and the Synchronize Streams, which shows only the stream of data of which we selected the timestamp but seems to be working if we create a datasource using the machine simulator adapter.
On the other hand, most of the default components work with our "custom" data sources.
To give you more context our data is composed of a timestamp field, and a value field.
The custom adapter that creates this data sources is based on the Http Stream adapter.
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com<https://blog.gft.com/>
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:image001.png@01D98256.375C6260]
From: Philipp Zehnder <ph...@bytefabrik.ai>>
Sent: martedì 25 aprile 2023 16:01
To: users@streampipes.apache.org<ma...@streampipes.apache.org>
Subject: AW: Compose Issue
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Federico,
sorry as far as I know the problem this is not a known issue.
Maybe you can try it with the newest version 0.91.0.
If I understand it correctly you implemented your own components that produce the input for the compose processor, right?
Maybe you can check if there is a problem with the output events from your component.
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Mittwoch, 19. April 2023 um 16:35
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: Compose Issue
Hello,
When we use the "compose" component to aggregate data, it does not output all data from the stream where the timestamp was not selected but returns all data from the stream with the selected timestamp.
We also tried to aggregate our data with the "synchronize two streams" component,
but it does not output any our data even though it consumes the data.
Moreover, when we use these components with two streams ad hoc from the machine data simulator, they work well.
Do you know if this is a know issue? Can you give us more information about it?
We are currently running on the version 0.70.0, because of an issue with version 0.90.0.
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com<https://blog.gft.com/>
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:image001.png@01D98256.375C6260]
AW: Compose Issue
Posted by Philipp Zehnder <ph...@bytefabrik.ai>.
Hi Fede,
ok I think I understand the problem now.
The compose component expects two data streams.
Data Sets are replayed when the pipeline is started. Thereby the data is played "as fast as possible".
That means when the data from the other data stream comes, all the data from the data set has already been played.
So, they cannot be processed in the Compose component.
Since we have several problems with datasets, we had a discussion [1] about it.
We decided to remove them in the future releases and concentrate on streams.
Cheers,
Philipp
[1] https://github.com/apache/streampipes/discussions/1115
Von: Rullo, Federico <Fe...@gft.com>
Datum: Dienstag, 9. Mai 2023 um 09:18
An: users@streampipes.apache.org <us...@streampipes.apache.org>
Betreff: RE: Compose Issue
Hello Philipp,
Thank you for your reply and please ignore the previous email as it was not complete.
Yes, we are using the adapter with the schema (date_unix_ts, value) as input to the signal monitoring processor and we have used a fixed output strategy to name the timestamp ‘date_unix_ts’ as 'timestamp'. Moreover, the conversion in our onEvent method was well done. Therefore, it is not the reason for the problems with the compose component; also, because when we connect directly our adapters to the compose component we experienced the same issue.
On further testing, we noticed that this could be related to the type of data source (set or stream) we are using: By using data source from a file set adapter, we get the same problem. But by using a file stream adapter with the same data, the compose component responds better.
So, since our HTTP stream-based data source adapter retrieves a set of data at each polling and passes it to a processor like a file set, it seems that the compose component is not able to handle the sets of data we give to it.
Thank you in advance,
Best,
Fede
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all’azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:image001.png@01D98257.087DFB60]
From: Philipp Zehnder <ph...@bytefabrik.ai>
Sent: venerdì 5 maggio 2023 16:34
To: users@streampipes.apache.org
Subject: AW: Compose Issue
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Fede,
so I assume you are using the adapter with the schema (date_unix_ts, value) as input to the signal monitoring processor, right?
It looks like you used a fixed output strategy and named the timestamp ('timestamp').
In the original stream the timestamp is named ('date_unix_ts'), did you convert this value accordingly in your onEvent method, or could this be the reason for the problems you have with the compose component?
Cheers,
Philipp
…………………………………………………
Philipp Zehnder
Bytefabrik.AI GmbH
Haid-und-Neu-Str. 10–14
76131 Karlsruhe
Tel.: +49 721 9654-805
philipp.zehnder@bytefabrik.ai<ma...@bytefabrik.ai>
www.bytefabrik.ai <https://www.bytefabrik.ai/> | www.twitter.com/bytefabrik<https://www.bytefabrik.ai/>
…………………………………………………
Bytefabrik.AI GmbH
Sitz der Gesellschaft: Karlsruhe
Registergericht: Amtsgericht Mannheim, HRB 741226
Geschäftsführer: Dr.-Ing. Dominik Riemer, Dr.-Ing. Philipp Zehnder
…………………………………………………
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Dienstag, 2. Mai 2023 um 14:23
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: RE: Compose Issue
Hi Philipp,
Please find attached some screenshots of our schema.
Best,
Fede.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com<https://blog.gft.com/>
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all’azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:image001.png@01D98257.087DFB60]
From: Philipp Zehnder <ph...@bytefabrik.ai>>
Sent: martedì 2 maggio 2023 11:10
To: users@streampipes.apache.org<ma...@streampipes.apache.org>
Subject: AW: Compose Issue
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Federico,
what is the output strategy of your processors?
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Mittwoch, 26. April 2023 um 09:46
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: RE: Compose Issue
Hi Philipp,
We also tried with a vanilla version of the 0.91.0 stable and had no luck.
We are creating our own, Data Sources, yes, but the problem we have is only with the Compose and the Synchronize Streams, which shows only the stream of data of which we selected the timestamp but seems to be working if we create a datasource using the machine simulator adapter.
On the other hand, most of the default components work with our “custom” data sources.
To give you more context our data is composed of a timestamp field, and a value field.
The custom adapter that creates this data sources is based on the Http Stream adapter.
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com<https://blog.gft.com/>
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all’azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:image001.png@01D98257.087DFB60]
From: Philipp Zehnder <ph...@bytefabrik.ai>>
Sent: martedì 25 aprile 2023 16:01
To: users@streampipes.apache.org<ma...@streampipes.apache.org>
Subject: AW: Compose Issue
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Federico,
sorry as far as I know the problem this is not a known issue.
Maybe you can try it with the newest version 0.91.0.
If I understand it correctly you implemented your own components that produce the input for the compose processor, right?
Maybe you can check if there is a problem with the output events from your component.
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Mittwoch, 19. April 2023 um 16:35
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: Compose Issue
Hello,
When we use the "compose" component to aggregate data, it does not output all data from the stream where the timestamp was not selected but returns all data from the stream with the selected timestamp.
We also tried to aggregate our data with the "synchronize two streams" component,
but it does not output any our data even though it consumes the data.
Moreover, when we use these components with two streams ad hoc from the machine data simulator, they work well.
Do you know if this is a know issue? Can you give us more information about it?
We are currently running on the version 0.70.0, because of an issue with version 0.90.0.
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com<https://blog.gft.com/>
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all’azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:image001.png@01D98257.087DFB60]
RE: Compose Issue
Posted by "Rullo, Federico" <Fe...@gft.com>.
Hello Philipp,
Thank you for your reply and please ignore the previous email as it was not complete.
Yes, we are using the adapter with the schema (date_unix_ts, value) as input to the signal monitoring processor and we have used a fixed output strategy to name the timestamp 'date_unix_ts' as 'timestamp'. Moreover, the conversion in our onEvent method was well done. Therefore, it is not the reason for the problems with the compose component; also, because when we connect directly our adapters to the compose component we experienced the same issue.
On further testing, we noticed that this could be related to the type of data source (set or stream) we are using: By using data source from a file set adapter, we get the same problem. But by using a file stream adapter with the same data, the compose component responds better.
So, since our HTTP stream-based data source adapter retrieves a set of data at each polling and passes it to a processor like a file set, it seems that the compose component is not able to handle the sets of data we give to it.
Thank you in advance,
Best,
Fede
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:signature-cloud_e31f4244-bb3f-4ff4-be4f-552f04b8a5ac.png]<https://www.gft.com/it/it/solutions/SOLUTIONS/it/IT-Metaverso>
From: Philipp Zehnder <ph...@bytefabrik.ai>
Sent: venerdì 5 maggio 2023 16:34
To: users@streampipes.apache.org
Subject: AW: Compose Issue
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Fede,
so I assume you are using the adapter with the schema (date_unix_ts, value) as input to the signal monitoring processor, right?
It looks like you used a fixed output strategy and named the timestamp ('timestamp').
In the original stream the timestamp is named ('date_unix_ts'), did you convert this value accordingly in your onEvent method, or could this be the reason for the problems you have with the compose component?
Cheers,
Philipp
.........................................................
Philipp Zehnder
Bytefabrik.AI GmbH
Haid-und-Neu-Str. 10-14
76131 Karlsruhe
Tel.: +49 721 9654-805
philipp.zehnder@bytefabrik.ai<ma...@bytefabrik.ai>
www.bytefabrik.ai <https://www.bytefabrik.ai/> | www.twitter.com/bytefabrik<https://www.bytefabrik.ai/>
.........................................................
Bytefabrik.AI GmbH
Sitz der Gesellschaft: Karlsruhe
Registergericht: Amtsgericht Mannheim, HRB 741226
Geschäftsführer: Dr.-Ing. Dominik Riemer, Dr.-Ing. Philipp Zehnder
.........................................................
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Dienstag, 2. Mai 2023 um 14:23
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: RE: Compose Issue
Hi Philipp,
Please find attached some screenshots of our schema.
Best,
Fede.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com<https://blog.gft.com/>
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:image001.png@01D98257.087DFB60]
From: Philipp Zehnder <ph...@bytefabrik.ai>>
Sent: martedì 2 maggio 2023 11:10
To: users@streampipes.apache.org<ma...@streampipes.apache.org>
Subject: AW: Compose Issue
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Federico,
what is the output strategy of your processors?
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Mittwoch, 26. April 2023 um 09:46
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: RE: Compose Issue
Hi Philipp,
We also tried with a vanilla version of the 0.91.0 stable and had no luck.
We are creating our own, Data Sources, yes, but the problem we have is only with the Compose and the Synchronize Streams, which shows only the stream of data of which we selected the timestamp but seems to be working if we create a datasource using the machine simulator adapter.
On the other hand, most of the default components work with our "custom" data sources.
To give you more context our data is composed of a timestamp field, and a value field.
The custom adapter that creates this data sources is based on the Http Stream adapter.
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com<https://blog.gft.com/>
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:image001.png@01D98257.087DFB60]
From: Philipp Zehnder <ph...@bytefabrik.ai>>
Sent: martedì 25 aprile 2023 16:01
To: users@streampipes.apache.org<ma...@streampipes.apache.org>
Subject: AW: Compose Issue
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Federico,
sorry as far as I know the problem this is not a known issue.
Maybe you can try it with the newest version 0.91.0.
If I understand it correctly you implemented your own components that produce the input for the compose processor, right?
Maybe you can check if there is a problem with the output events from your component.
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Mittwoch, 19. April 2023 um 16:35
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: Compose Issue
Hello,
When we use the "compose" component to aggregate data, it does not output all data from the stream where the timestamp was not selected but returns all data from the stream with the selected timestamp.
We also tried to aggregate our data with the "synchronize two streams" component,
but it does not output any our data even though it consumes the data.
Moreover, when we use these components with two streams ad hoc from the machine data simulator, they work well.
Do you know if this is a know issue? Can you give us more information about it?
We are currently running on the version 0.70.0, because of an issue with version 0.90.0.
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com<https://blog.gft.com/>
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:image001.png@01D98257.087DFB60]
AW: Compose Issue
Posted by Philipp Zehnder <ph...@bytefabrik.ai>.
Hi Fede,
so I assume you are using the adapter with the schema (date_unix_ts, value) as input to the signal monitoring processor, right?
It looks like you used a fixed output strategy and named the timestamp ('timestamp').
In the original stream the timestamp is named ('date_unix_ts'), did you convert this value accordingly in your onEvent method, or could this be the reason for the problems you have with the compose component?
Cheers,
Philipp
…………………………………………………
Philipp Zehnder
Bytefabrik.AI GmbH
Haid-und-Neu-Str. 10–14
76131 Karlsruhe
Tel.: +49 721 9654-805
philipp.zehnder@bytefabrik.ai
www.bytefabrik.ai <https://www.bytefabrik.ai/> | www.twitter.com/bytefabrik<https://www.bytefabrik.ai/>
…………………………………………………
Bytefabrik.AI GmbH
Sitz der Gesellschaft: Karlsruhe
Registergericht: Amtsgericht Mannheim, HRB 741226
Geschäftsführer: Dr.-Ing. Dominik Riemer, Dr.-Ing. Philipp Zehnder
…………………………………………………
Von: Rullo, Federico <Fe...@gft.com>
Datum: Dienstag, 2. Mai 2023 um 14:23
An: users@streampipes.apache.org <us...@streampipes.apache.org>
Betreff: RE: Compose Issue
Hi Philipp,
Please find attached some screenshots of our schema.
Best,
Fede.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all’azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:image001.png@01D97D01.AA994130]
From: Philipp Zehnder <ph...@bytefabrik.ai>
Sent: martedì 2 maggio 2023 11:10
To: users@streampipes.apache.org
Subject: AW: Compose Issue
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Federico,
what is the output strategy of your processors?
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Mittwoch, 26. April 2023 um 09:46
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: RE: Compose Issue
Hi Philipp,
We also tried with a vanilla version of the 0.91.0 stable and had no luck.
We are creating our own, Data Sources, yes, but the problem we have is only with the Compose and the Synchronize Streams, which shows only the stream of data of which we selected the timestamp but seems to be working if we create a datasource using the machine simulator adapter.
On the other hand, most of the default components work with our “custom” data sources.
To give you more context our data is composed of a timestamp field, and a value field.
The custom adapter that creates this data sources is based on the Http Stream adapter.
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com<https://blog.gft.com/>
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all’azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:image001.png@01D97D01.AA994130]
From: Philipp Zehnder <ph...@bytefabrik.ai>>
Sent: martedì 25 aprile 2023 16:01
To: users@streampipes.apache.org<ma...@streampipes.apache.org>
Subject: AW: Compose Issue
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Federico,
sorry as far as I know the problem this is not a known issue.
Maybe you can try it with the newest version 0.91.0.
If I understand it correctly you implemented your own components that produce the input for the compose processor, right?
Maybe you can check if there is a problem with the output events from your component.
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Mittwoch, 19. April 2023 um 16:35
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: Compose Issue
Hello,
When we use the "compose" component to aggregate data, it does not output all data from the stream where the timestamp was not selected but returns all data from the stream with the selected timestamp.
We also tried to aggregate our data with the "synchronize two streams" component,
but it does not output any our data even though it consumes the data.
Moreover, when we use these components with two streams ad hoc from the machine data simulator, they work well.
Do you know if this is a know issue? Can you give us more information about it?
We are currently running on the version 0.70.0, because of an issue with version 0.90.0.
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com<https://blog.gft.com/>
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all’azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:image001.png@01D97D01.AA994130]
RE: Compose Issue
Posted by "Rullo, Federico" <Fe...@gft.com>.
Hi Philipp,
Please find attached some screenshots of our schema.
Best,
Fede.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:signature-cloud_e31f4244-bb3f-4ff4-be4f-552f04b8a5ac.png]<https://www.gft.com/it/it/solutions/SOLUTIONS/it/IT-Metaverso>
From: Philipp Zehnder <ph...@bytefabrik.ai>
Sent: martedì 2 maggio 2023 11:10
To: users@streampipes.apache.org
Subject: AW: Compose Issue
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Federico,
what is the output strategy of your processors?
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Mittwoch, 26. April 2023 um 09:46
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: RE: Compose Issue
Hi Philipp,
We also tried with a vanilla version of the 0.91.0 stable and had no luck.
We are creating our own, Data Sources, yes, but the problem we have is only with the Compose and the Synchronize Streams, which shows only the stream of data of which we selected the timestamp but seems to be working if we create a datasource using the machine simulator adapter.
On the other hand, most of the default components work with our "custom" data sources.
To give you more context our data is composed of a timestamp field, and a value field.
The custom adapter that creates this data sources is based on the Http Stream adapter.
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com<https://blog.gft.com/>
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:image001.png@01D97D01.AA994130]
From: Philipp Zehnder <ph...@bytefabrik.ai>>
Sent: martedì 25 aprile 2023 16:01
To: users@streampipes.apache.org<ma...@streampipes.apache.org>
Subject: AW: Compose Issue
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Federico,
sorry as far as I know the problem this is not a known issue.
Maybe you can try it with the newest version 0.91.0.
If I understand it correctly you implemented your own components that produce the input for the compose processor, right?
Maybe you can check if there is a problem with the output events from your component.
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Mittwoch, 19. April 2023 um 16:35
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: Compose Issue
Hello,
When we use the "compose" component to aggregate data, it does not output all data from the stream where the timestamp was not selected but returns all data from the stream with the selected timestamp.
We also tried to aggregate our data with the "synchronize two streams" component,
but it does not output any our data even though it consumes the data.
Moreover, when we use these components with two streams ad hoc from the machine data simulator, they work well.
Do you know if this is a know issue? Can you give us more information about it?
We are currently running on the version 0.70.0, because of an issue with version 0.90.0.
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com<https://blog.gft.com/>
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:image001.png@01D97D01.AA994130]
AW: Compose Issue
Posted by Philipp Zehnder <ph...@bytefabrik.ai>.
Hi Federico,
what is the output strategy of your processors?
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>
Datum: Mittwoch, 26. April 2023 um 09:46
An: users@streampipes.apache.org <us...@streampipes.apache.org>
Betreff: RE: Compose Issue
Hi Philipp,
We also tried with a vanilla version of the 0.91.0 stable and had no luck.
We are creating our own, Data Sources, yes, but the problem we have is only with the Compose and the Synchronize Streams, which shows only the stream of data of which we selected the timestamp but seems to be working if we create a datasource using the machine simulator adapter.
On the other hand, most of the default components work with our “custom” data sources.
To give you more context our data is composed of a timestamp field, and a value field.
The custom adapter that creates this data sources is based on the Http Stream adapter.
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all’azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:image001.png@01D97822.996FEA50]
From: Philipp Zehnder <ph...@bytefabrik.ai>
Sent: martedì 25 aprile 2023 16:01
To: users@streampipes.apache.org
Subject: AW: Compose Issue
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Federico,
sorry as far as I know the problem this is not a known issue.
Maybe you can try it with the newest version 0.91.0.
If I understand it correctly you implemented your own components that produce the input for the compose processor, right?
Maybe you can check if there is a problem with the output events from your component.
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Mittwoch, 19. April 2023 um 16:35
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: Compose Issue
Hello,
When we use the "compose" component to aggregate data, it does not output all data from the stream where the timestamp was not selected but returns all data from the stream with the selected timestamp.
We also tried to aggregate our data with the "synchronize two streams" component,
but it does not output any our data even though it consumes the data.
Moreover, when we use these components with two streams ad hoc from the machine data simulator, they work well.
Do you know if this is a know issue? Can you give us more information about it?
We are currently running on the version 0.70.0, because of an issue with version 0.90.0.
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com<https://blog.gft.com/>
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all’azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:image001.png@01D97822.996FEA50]
RE: Compose Issue
Posted by "Rullo, Federico" <Fe...@gft.com>.
Hi Philipp,
We also tried with a vanilla version of the 0.91.0 stable and had no luck.
We are creating our own, Data Sources, yes, but the problem we have is only with the Compose and the Synchronize Streams, which shows only the stream of data of which we selected the timestamp but seems to be working if we create a datasource using the machine simulator adapter.
On the other hand, most of the default components work with our "custom" data sources.
To give you more context our data is composed of a timestamp field, and a value field.
The custom adapter that creates this data sources is based on the Http Stream adapter.
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:signature-cloud_e31f4244-bb3f-4ff4-be4f-552f04b8a5ac.png]<https://www.gft.com/it/it/solutions/SOLUTIONS/it/IT-Metaverso>
From: Philipp Zehnder <ph...@bytefabrik.ai>
Sent: martedì 25 aprile 2023 16:01
To: users@streampipes.apache.org
Subject: AW: Compose Issue
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Federico,
sorry as far as I know the problem this is not a known issue.
Maybe you can try it with the newest version 0.91.0.
If I understand it correctly you implemented your own components that produce the input for the compose processor, right?
Maybe you can check if there is a problem with the output events from your component.
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Mittwoch, 19. April 2023 um 16:35
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: Compose Issue
Hello,
When we use the "compose" component to aggregate data, it does not output all data from the stream where the timestamp was not selected but returns all data from the stream with the selected timestamp.
We also tried to aggregate our data with the "synchronize two streams" component,
but it does not output any our data even though it consumes the data.
Moreover, when we use these components with two streams ad hoc from the machine data simulator, they work well.
Do you know if this is a know issue? Can you give us more information about it?
We are currently running on the version 0.70.0, because of an issue with version 0.90.0.
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com<https://blog.gft.com/>
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:image001.png@01D97822.996FEA50]
AW: Compose Issue
Posted by Philipp Zehnder <ph...@bytefabrik.ai>.
Hi Federico,
sorry as far as I know the problem this is not a known issue.
Maybe you can try it with the newest version 0.91.0.
If I understand it correctly you implemented your own components that produce the input for the compose processor, right?
Maybe you can check if there is a problem with the output events from your component.
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>
Datum: Mittwoch, 19. April 2023 um 16:35
An: users@streampipes.apache.org <us...@streampipes.apache.org>
Betreff: Compose Issue
Hello,
When we use the "compose" component to aggregate data, it does not output all data from the stream where the timestamp was not selected but returns all data from the stream with the selected timestamp.
We also tried to aggregate our data with the "synchronize two streams" component,
but it does not output any our data even though it consumes the data.
Moreover, when we use these components with two streams ad hoc from the machine data simulator, they work well.
Do you know if this is a know issue? Can you give us more information about it?
We are currently running on the version 0.70.0, because of an issue with version 0.90.0.
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all’azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
[cid:signature-cloud_e31f4244-bb3f-4ff4-be4f-552f04b8a5ac.png]