You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@streampipes.apache.org by "Rullo, Federico" <Fe...@gft.com> on 2022/06/23 15:44:15 UTC
StreamPipes File Stream Creation Problem
Hello,
I am trying to upload a Json file of 9.2MB of size, containing 50000 entries, to create a datastream.
I previously uploaded a smaller version of the file, and everything went fine, then I had to reinstall SP, and uploading the same file, gave me the same error I am having right now.
The error is that it creates the DataStream, but no data is being produced.
To create the DataStream, I am using the Data Stream creator, and a JSON Array key for the file format.
What can be the problem in your opinion?
Also, I saw that using the rel/0.69.0 branch yields 0.69.0 as the version and not 0.69.1-SNAPSHOT anymore, the previous installation of SP I had was using the latter, do you think this can be the problem?
Thanks in advance for your reply,
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it>
[cid:ActoForTheFuture_GFT@1,2x_68f73108-d76e-4497-aa5c-ff8a5693ce57.png]<https://www.gft.com/it/it/about-us/Sustainability>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Societ? sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
AW: StreamPipes File Stream Creation Problem
Posted by Philipp Zehnder <ph...@bytefabrik.ai>.
Hi Michele,
I am happy that we could help.
Please write us if there are any other questions.
Cheers,
Philipp
Von: Canteri, Michele <Mi...@gft.com>
Datum: Donnerstag, 30. Juni 2022 um 12:04
An: users@streampipes.apache.org <us...@streampipes.apache.org>
Betreff: RE: StreamPipes File Stream Creation Problem
Hi Philipp,
Thank you for your answer, me and the other Michele tried the File Stream and not File Set but I think that isn’t a problem because I’ve tried on our server that is running via docker-compose on the code tree and had problems, not starting the adapter. Today I’ve started SP using the installer and so it downloaded all the images not using those compiled on the code, and it actually works, so our problem is on compiling and maybe retrieving the right branch/tag of the code, I guess. I think the right tag or branch to checkout is something that confuses me, sorry, my lack.
I retrieved the code cloning it from GIT and another time downloading it as a zip file, but I would prefer the first way in order to be able to patch it in case of bugs.
Thank you again for your help guys!
Regards,
Michele
Michele Canteri
_________________________________________________________________
GFT Italia S.r.l.
VIA SILE 18 / floor 4
20139 Milano, Italy
T +39 02 58158503
F +39 02 58158621
Michele.Canteri@gft.com
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
[cid:ActoForTheFuture_GFT@1,2x_68f73108-d76e-4497-aa5c-ff8a5693ce57.png]<https://www.gft.com/it/it/about-us/Sustainability>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all’azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
From: Philipp Zehnder <ph...@bytefabrik.ai>
Sent: mercoledì 29 giugno 2022 18:13
To: users@streampipes.apache.org
Subject: AW: StreamPipes File Stream Creation Problem
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Michele,
thanks for providing sample data.
I was able to create an adapter with it on my machine.
I used the “File Set” adapter with the format “Json” -> “Array Field” -> “Key: data”.
For the timestamp I selected the field “date_unix_ts”. I could start the adapter, stored the results in the data lake and visualized them in the data explorer.
Does the error occur on your systems with the same configurations?
One minor comment: I saw that your values in the data stream have different units. Depending on your use case, my suggestion would be to either create one data streams for each property, or if it is possible to create a bigger event with all the values at one point in time. This often eases the analytics part of the event streams.
Cheers,
Philipp
Von: Iorio, Michele <mi...@gft.com>>
Datum: Montag, 27. Juni 2022 um 18:26
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: RE: StreamPipes File Stream Creation Problem
Hi Philipp,
in attachment you will find a sample of data to try to recreate the problem.
Regarding version 0.69.1-SNAPSHOT, do you have any other information?
Thanks
Michele
Michele Iorio
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 010 583158
michele.iorio@gft.com<ma...@gft.com>
www.gft.com/it<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.gft.com%2Fit&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003491799%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iCCrTklKM4OzTKZBgFpvUryOn0Y%2FYlegBUTpxIPGRXI%3D&reserved=0>
https://blog.gft.com<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.gft.com%2F&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003491799%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=vqGlP4r1%2BX4DbSadWDmVWyURz77MhxJ5a6UG2onRau0%3D&reserved=0>
https://www.linkedin.com/company/gft-group/<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fgft-group%2F&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003491799%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=l7gOWRftGrZCtq8pU9ZGO8la4X0l9cscLPcN3uNQPwg%3D&reserved=0>
www.twitter.com/gft_it <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.twitter.com%2Fgft_it%2520&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003491799%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YFGDXOHDDpGEuVnlUCoAVliWZLgvbKryMBD6P0DA4%2Fg%3D&reserved=0>
[cid:image002.png@01D88C77.D7DC8560]<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.gft.com%2Fit%2Fit%2Fabout-us%2FSustainability&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003491799%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3w7LrTeJocuKFb83g%2FN0LJcG%2FERHfomlccuSW3635Bg%3D&reserved=0>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all’azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
From: Iorio, Michele
Sent: venerdì 24 giugno 2022 16:24
To: users@streampipes.apache.org<ma...@streampipes.apache.org>
Subject: RE: StreamPipes File Stream Creation Problem
Hi Philipp,
the problem occurs even if I upload a file with 100 entries.
The situation is strange, because sometimes it works with the small file, but other times it gives the same error.
At the moment I'm checking if I can send you a small sample of data to reproduce the problem.
Regarding the last part of Federico's email, why is version 0.69.1-SNAPSHOT no longer installed?
Could this be the cause of the problem?
Thanks
Michele
From: Philipp Zehnder <ph...@bytefabrik.ai>>
Sent: giovedì 23 giugno 2022 19:53
To: users@streampipes.apache.org<ma...@streampipes.apache.org>
Subject: AW: StreamPipes File Stream Creation Problem
Non si ricevono spesso messaggi di posta elettronica da philipp.zehnder@bytefabrik.ai<ma...@bytefabrik.ai>. Informazioni sul perché è importante<https://aka.ms/LearnAboutSenderIdentification>
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Frederico,
is there anything logged in the docker container running the adapter?
Can you try it with a very small file containing only a few events?
If you want you can also send me a small sample, so I can try to recreate the problem.
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Donnerstag, 23. Juni 2022 um 17:44
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: StreamPipes File Stream Creation Problem
Hello,
I am trying to upload a Json file of 9.2MB of size, containing 50000 entries, to create a datastream.
I previously uploaded a smaller version of the file, and everything went fine, then I had to reinstall SP, and uploading the same file, gave me the same error I am having right now.
The error is that it creates the DataStream, but no data is being produced.
To create the DataStream, I am using the Data Stream creator, and a JSON Array key for the file format.
What can be the problem in your opinion?
Also, I saw that using the rel/0.69.0 branch yields 0.69.0 as the version and not 0.69.1-SNAPSHOT anymore, the previous installation of SP I had was using the latter, do you think this can be the problem?
Thanks in advance for your reply,
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.gft.com%2Fit&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003647251%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=nziW761WhSbBY1yc9gvjs4U0GssKnsaHsTbxuPQanHs%3D&reserved=0>
https://blog.gft.com<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.gft.com%2F&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003647251%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=AYUzWAd582X6otG0k7kqiOm4Dr0pL73o%2BS4ilhKp2aU%3D&reserved=0>
https://www.linkedin.com/company/gft-group/<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fgft-group%2F&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003647251%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2BbnQ2jQgASKCc864tR7GwW4gMukkODo4Xq0edEDX5gA%3D&reserved=0>
www.twitter.com/gft_it <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.twitter.com%2Fgft_it%2520&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003647251%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=BkLAfnbDjcbP7kAHalTpTlQgaMI%2BLHt%2Bgo0N%2B3%2FDAhs%3D&reserved=0>
[cid:image003.png@01D88C77.D7DC8560]<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.gft.com%2Fit%2Fit%2Fabout-us%2FSustainability&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003647251%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2BAKCBcMct1JE0DW8ty%2FeQXUswiQ6bpJnNFseeshxigY%3D&reserved=0>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all’azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
RE: StreamPipes File Stream Creation Problem
Posted by "Canteri, Michele" <Mi...@gft.com>.
Hi Philipp,
Thank you for your answer, me and the other Michele tried the File Stream and not File Set but I think that isn't a problem because I've tried on our server that is running via docker-compose on the code tree and had problems, not starting the adapter. Today I've started SP using the installer and so it downloaded all the images not using those compiled on the code, and it actually works, so our problem is on compiling and maybe retrieving the right branch/tag of the code, I guess. I think the right tag or branch to checkout is something that confuses me, sorry, my lack.
I retrieved the code cloning it from GIT and another time downloading it as a zip file, but I would prefer the first way in order to be able to patch it in case of bugs.
Thank you again for your help guys!
Regards,
Michele
Michele Canteri
_________________________________________________________________
GFT Italia S.r.l.
VIA SILE 18 / floor 4
20139 Milano, Italy
T +39 02 58158503
F +39 02 58158621
Michele.Canteri@gft.com
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it>
[cid:ActoForTheFuture_GFT@1,2x_68f73108-d76e-4497-aa5c-ff8a5693ce57.png]<https://www.gft.com/it/it/about-us/Sustainability>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
From: Philipp Zehnder <ph...@bytefabrik.ai>
Sent: mercoledì 29 giugno 2022 18:13
To: users@streampipes.apache.org
Subject: AW: StreamPipes File Stream Creation Problem
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Michele,
thanks for providing sample data.
I was able to create an adapter with it on my machine.
I used the "File Set" adapter with the format "Json" -> "Array Field" -> "Key: data".
For the timestamp I selected the field "date_unix_ts". I could start the adapter, stored the results in the data lake and visualized them in the data explorer.
Does the error occur on your systems with the same configurations?
One minor comment: I saw that your values in the data stream have different units. Depending on your use case, my suggestion would be to either create one data streams for each property, or if it is possible to create a bigger event with all the values at one point in time. This often eases the analytics part of the event streams.
Cheers,
Philipp
Von: Iorio, Michele <mi...@gft.com>>
Datum: Montag, 27. Juni 2022 um 18:26
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: RE: StreamPipes File Stream Creation Problem
Hi Philipp,
in attachment you will find a sample of data to try to recreate the problem.
Regarding version 0.69.1-SNAPSHOT, do you have any other information?
Thanks
Michele
Michele Iorio
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 010 583158
michele.iorio@gft.com<ma...@gft.com>
www.gft.com/it<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.gft.com%2Fit&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003491799%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iCCrTklKM4OzTKZBgFpvUryOn0Y%2FYlegBUTpxIPGRXI%3D&reserved=0>
https://blog.gft.com<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.gft.com%2F&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003491799%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=vqGlP4r1%2BX4DbSadWDmVWyURz77MhxJ5a6UG2onRau0%3D&reserved=0>
https://www.linkedin.com/company/gft-group/<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fgft-group%2F&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003491799%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=l7gOWRftGrZCtq8pU9ZGO8la4X0l9cscLPcN3uNQPwg%3D&reserved=0>
www.twitter.com/gft_it <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.twitter.com%2Fgft_it%2520&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003491799%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YFGDXOHDDpGEuVnlUCoAVliWZLgvbKryMBD6P0DA4%2Fg%3D&reserved=0>
[cid:image002.png@01D88C77.D7DC8560]<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.gft.com%2Fit%2Fit%2Fabout-us%2FSustainability&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003491799%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3w7LrTeJocuKFb83g%2FN0LJcG%2FERHfomlccuSW3635Bg%3D&reserved=0>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
From: Iorio, Michele
Sent: venerdì 24 giugno 2022 16:24
To: users@streampipes.apache.org<ma...@streampipes.apache.org>
Subject: RE: StreamPipes File Stream Creation Problem
Hi Philipp,
the problem occurs even if I upload a file with 100 entries.
The situation is strange, because sometimes it works with the small file, but other times it gives the same error.
At the moment I'm checking if I can send you a small sample of data to reproduce the problem.
Regarding the last part of Federico's email, why is version 0.69.1-SNAPSHOT no longer installed?
Could this be the cause of the problem?
Thanks
Michele
From: Philipp Zehnder <ph...@bytefabrik.ai>>
Sent: giovedì 23 giugno 2022 19:53
To: users@streampipes.apache.org<ma...@streampipes.apache.org>
Subject: AW: StreamPipes File Stream Creation Problem
Non si ricevono spesso messaggi di posta elettronica da philipp.zehnder@bytefabrik.ai<ma...@bytefabrik.ai>. Informazioni sul perché è importante<https://aka.ms/LearnAboutSenderIdentification>
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Frederico,
is there anything logged in the docker container running the adapter?
Can you try it with a very small file containing only a few events?
If you want you can also send me a small sample, so I can try to recreate the problem.
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Donnerstag, 23. Juni 2022 um 17:44
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: StreamPipes File Stream Creation Problem
Hello,
I am trying to upload a Json file of 9.2MB of size, containing 50000 entries, to create a datastream.
I previously uploaded a smaller version of the file, and everything went fine, then I had to reinstall SP, and uploading the same file, gave me the same error I am having right now.
The error is that it creates the DataStream, but no data is being produced.
To create the DataStream, I am using the Data Stream creator, and a JSON Array key for the file format.
What can be the problem in your opinion?
Also, I saw that using the rel/0.69.0 branch yields 0.69.0 as the version and not 0.69.1-SNAPSHOT anymore, the previous installation of SP I had was using the latter, do you think this can be the problem?
Thanks in advance for your reply,
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.gft.com%2Fit&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003647251%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=nziW761WhSbBY1yc9gvjs4U0GssKnsaHsTbxuPQanHs%3D&reserved=0>
https://blog.gft.com<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.gft.com%2F&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003647251%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=AYUzWAd582X6otG0k7kqiOm4Dr0pL73o%2BS4ilhKp2aU%3D&reserved=0>
https://www.linkedin.com/company/gft-group/<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fgft-group%2F&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003647251%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2BbnQ2jQgASKCc864tR7GwW4gMukkODo4Xq0edEDX5gA%3D&reserved=0>
www.twitter.com/gft_it <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.twitter.com%2Fgft_it%2520&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003647251%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=BkLAfnbDjcbP7kAHalTpTlQgaMI%2BLHt%2Bgo0N%2B3%2FDAhs%3D&reserved=0>
[cid:image003.png@01D88C77.D7DC8560]<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.gft.com%2Fit%2Fit%2Fabout-us%2FSustainability&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003647251%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2BAKCBcMct1JE0DW8ty%2FeQXUswiQ6bpJnNFseeshxigY%3D&reserved=0>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
AW: StreamPipes File Stream Creation Problem
Posted by Philipp Zehnder <ph...@bytefabrik.ai>.
Hi Michele,
thanks for providing sample data.
I was able to create an adapter with it on my machine.
I used the “File Set” adapter with the format “Json” -> “Array Field” -> “Key: data”.
For the timestamp I selected the field “date_unix_ts”. I could start the adapter, stored the results in the data lake and visualized them in the data explorer.
Does the error occur on your systems with the same configurations?
One minor comment: I saw that your values in the data stream have different units. Depending on your use case, my suggestion would be to either create one data streams for each property, or if it is possible to create a bigger event with all the values at one point in time. This often eases the analytics part of the event streams.
Cheers,
Philipp
Von: Iorio, Michele <mi...@gft.com>
Datum: Montag, 27. Juni 2022 um 18:26
An: users@streampipes.apache.org <us...@streampipes.apache.org>
Betreff: RE: StreamPipes File Stream Creation Problem
Hi Philipp,
in attachment you will find a sample of data to try to recreate the problem.
Regarding version 0.69.1-SNAPSHOT, do you have any other information?
Thanks
Michele
Michele Iorio
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 010 583158
michele.iorio@gft.com
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
[cid:ActoForTheFuture_GFT@1,2x_68f73108-d76e-4497-aa5c-ff8a5693ce57.png]<https://www.gft.com/it/it/about-us/Sustainability>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all’azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
From: Iorio, Michele
Sent: venerdì 24 giugno 2022 16:24
To: users@streampipes.apache.org
Subject: RE: StreamPipes File Stream Creation Problem
Hi Philipp,
the problem occurs even if I upload a file with 100 entries.
The situation is strange, because sometimes it works with the small file, but other times it gives the same error.
At the moment I'm checking if I can send you a small sample of data to reproduce the problem.
Regarding the last part of Federico's email, why is version 0.69.1-SNAPSHOT no longer installed?
Could this be the cause of the problem?
Thanks
Michele
From: Philipp Zehnder <ph...@bytefabrik.ai>>
Sent: giovedì 23 giugno 2022 19:53
To: users@streampipes.apache.org<ma...@streampipes.apache.org>
Subject: AW: StreamPipes File Stream Creation Problem
Non si ricevono spesso messaggi di posta elettronica da philipp.zehnder@bytefabrik.ai<ma...@bytefabrik.ai>. Informazioni sul perché è importante<https://aka.ms/LearnAboutSenderIdentification>
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Frederico,
is there anything logged in the docker container running the adapter?
Can you try it with a very small file containing only a few events?
If you want you can also send me a small sample, so I can try to recreate the problem.
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Donnerstag, 23. Juni 2022 um 17:44
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: StreamPipes File Stream Creation Problem
Hello,
I am trying to upload a Json file of 9.2MB of size, containing 50000 entries, to create a datastream.
I previously uploaded a smaller version of the file, and everything went fine, then I had to reinstall SP, and uploading the same file, gave me the same error I am having right now.
The error is that it creates the DataStream, but no data is being produced.
To create the DataStream, I am using the Data Stream creator, and a JSON Array key for the file format.
What can be the problem in your opinion?
Also, I saw that using the rel/0.69.0 branch yields 0.69.0 as the version and not 0.69.1-SNAPSHOT anymore, the previous installation of SP I had was using the latter, do you think this can be the problem?
Thanks in advance for your reply,
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.gft.com%2Fit&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047673634%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=TZWw2LbLjpyL4DAAx9RjIyUP40qVRMoG2TSCrSqmuL4%3D&reserved=0>
https://blog.gft.com<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.gft.com%2F&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047829869%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Ovy0cewqOANT1CmDo7%2F8bcyIBkPgAdLl0qVLK2G71hM%3D&reserved=0>
https://www.linkedin.com/company/gft-group/<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fgft-group%2F&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047829869%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YQQ91LFlSziIhm9VV3LSLm7qDCAncsdeD48u6sf6CPc%3D&reserved=0>
www.twitter.com/gft_it <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.twitter.com%2Fgft_it%2520&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047829869%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=1QSBILyL9q%2FEDSppEU8RVyuAXDGOvhzVRp%2BEemtMlf4%3D&reserved=0>
[cid:image001.png@01D88A53.4732CD60]<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.gft.com%2Fit%2Fit%2Fabout-us%2FSustainability&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047829869%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=StW2olnL9VvMqBLg1MU47v0UrYDrTGD81I07hiVh05A%3D&reserved=0>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all’azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
RE: StreamPipes File Stream Creation Problem
Posted by Dominik Riemer <do...@bytefabrik.ai>.
Sorry, Michele, I mixed up the name 😉
From: Dominik Riemer
Sent: Tuesday, June 28, 2022 8:21 PM
To: users@streampipes.apache.org
Subject: RE: StreamPipes File Stream Creation Problem
Hi Frederico,
regarding version 0.69.1: This is not a release. We currently only release minor versions with 0.69.0 being the last one. In the release process, we create a patch version 0.69.1-SNAPSHOT in the release branch in case we decide to do any patch release. These branches are prefixed with “rel”. Releases are prefixed with “release”. I suggest that you use either a released version (0.69.0) or the latest development version from dev if you want to contribute to development.
Hope this clarifies things!
Cheers,
Dominik
From: Iorio, Michele <mi...@gft.com>>
Sent: Monday, June 27, 2022 6:26 PM
To: users@streampipes.apache.org<ma...@streampipes.apache.org>
Subject: RE: StreamPipes File Stream Creation Problem
Hi Philipp,
in attachment you will find a sample of data to try to recreate the problem.
Regarding version 0.69.1-SNAPSHOT, do you have any other information?
Thanks
Michele
Michele Iorio
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 010 583158
michele.iorio@gft.com<ma...@gft.com>
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
[cid:image001.png@01D88B2C.A8558B30]<https://www.gft.com/it/it/about-us/Sustainability>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all’azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
From: Iorio, Michele
Sent: venerdì 24 giugno 2022 16:24
To: users@streampipes.apache.org<ma...@streampipes.apache.org>
Subject: RE: StreamPipes File Stream Creation Problem
Hi Philipp,
the problem occurs even if I upload a file with 100 entries.
The situation is strange, because sometimes it works with the small file, but other times it gives the same error.
At the moment I'm checking if I can send you a small sample of data to reproduce the problem.
Regarding the last part of Federico's email, why is version 0.69.1-SNAPSHOT no longer installed?
Could this be the cause of the problem?
Thanks
Michele
From: Philipp Zehnder <ph...@bytefabrik.ai>>
Sent: giovedì 23 giugno 2022 19:53
To: users@streampipes.apache.org<ma...@streampipes.apache.org>
Subject: AW: StreamPipes File Stream Creation Problem
Non si ricevono spesso messaggi di posta elettronica da philipp.zehnder@bytefabrik.ai<ma...@bytefabrik.ai>. Informazioni sul perché è importante<https://aka.ms/LearnAboutSenderIdentification>
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Frederico,
is there anything logged in the docker container running the adapter?
Can you try it with a very small file containing only a few events?
If you want you can also send me a small sample, so I can try to recreate the problem.
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Donnerstag, 23. Juni 2022 um 17:44
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: StreamPipes File Stream Creation Problem
Hello,
I am trying to upload a Json file of 9.2MB of size, containing 50000 entries, to create a datastream.
I previously uploaded a smaller version of the file, and everything went fine, then I had to reinstall SP, and uploading the same file, gave me the same error I am having right now.
The error is that it creates the DataStream, but no data is being produced.
To create the DataStream, I am using the Data Stream creator, and a JSON Array key for the file format.
What can be the problem in your opinion?
Also, I saw that using the rel/0.69.0 branch yields 0.69.0 as the version and not 0.69.1-SNAPSHOT anymore, the previous installation of SP I had was using the latter, do you think this can be the problem?
Thanks in advance for your reply,
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.gft.com%2Fit&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047673634%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=TZWw2LbLjpyL4DAAx9RjIyUP40qVRMoG2TSCrSqmuL4%3D&reserved=0>
https://blog.gft.com<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.gft.com%2F&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047829869%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Ovy0cewqOANT1CmDo7%2F8bcyIBkPgAdLl0qVLK2G71hM%3D&reserved=0>
https://www.linkedin.com/company/gft-group/<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fgft-group%2F&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047829869%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YQQ91LFlSziIhm9VV3LSLm7qDCAncsdeD48u6sf6CPc%3D&reserved=0>
www.twitter.com/gft_it <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.twitter.com%2Fgft_it%2520&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047829869%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=1QSBILyL9q%2FEDSppEU8RVyuAXDGOvhzVRp%2BEemtMlf4%3D&reserved=0>
[cid:image002.png@01D88B2C.A8558B30]<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.gft.com%2Fit%2Fit%2Fabout-us%2FSustainability&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047829869%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=StW2olnL9VvMqBLg1MU47v0UrYDrTGD81I07hiVh05A%3D&reserved=0>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all’azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
RE: StreamPipes File Stream Creation Problem
Posted by Dominik Riemer <do...@bytefabrik.ai>.
Hi Frederico,
regarding version 0.69.1: This is not a release. We currently only release minor versions with 0.69.0 being the last one. In the release process, we create a patch version 0.69.1-SNAPSHOT in the release branch in case we decide to do any patch release. These branches are prefixed with "rel". Releases are prefixed with "release". I suggest that you use either a released version (0.69.0) or the latest development version from dev if you want to contribute to development.
Hope this clarifies things!
Cheers,
Dominik
From: Iorio, Michele <mi...@gft.com>
Sent: Monday, June 27, 2022 6:26 PM
To: users@streampipes.apache.org
Subject: RE: StreamPipes File Stream Creation Problem
Hi Philipp,
in attachment you will find a sample of data to try to recreate the problem.
Regarding version 0.69.1-SNAPSHOT, do you have any other information?
Thanks
Michele
Michele Iorio
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 010 583158
michele.iorio@gft.com<ma...@gft.com>
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
[cid:image002.png@01D88B2C.8E3C5490]<https://www.gft.com/it/it/about-us/Sustainability>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
From: Iorio, Michele
Sent: venerdì 24 giugno 2022 16:24
To: users@streampipes.apache.org<ma...@streampipes.apache.org>
Subject: RE: StreamPipes File Stream Creation Problem
Hi Philipp,
the problem occurs even if I upload a file with 100 entries.
The situation is strange, because sometimes it works with the small file, but other times it gives the same error.
At the moment I'm checking if I can send you a small sample of data to reproduce the problem.
Regarding the last part of Federico's email, why is version 0.69.1-SNAPSHOT no longer installed?
Could this be the cause of the problem?
Thanks
Michele
From: Philipp Zehnder <ph...@bytefabrik.ai>>
Sent: giovedì 23 giugno 2022 19:53
To: users@streampipes.apache.org<ma...@streampipes.apache.org>
Subject: AW: StreamPipes File Stream Creation Problem
Non si ricevono spesso messaggi di posta elettronica da philipp.zehnder@bytefabrik.ai<ma...@bytefabrik.ai>. Informazioni sul perché è importante<https://aka.ms/LearnAboutSenderIdentification>
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Frederico,
is there anything logged in the docker container running the adapter?
Can you try it with a very small file containing only a few events?
If you want you can also send me a small sample, so I can try to recreate the problem.
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Donnerstag, 23. Juni 2022 um 17:44
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: StreamPipes File Stream Creation Problem
Hello,
I am trying to upload a Json file of 9.2MB of size, containing 50000 entries, to create a datastream.
I previously uploaded a smaller version of the file, and everything went fine, then I had to reinstall SP, and uploading the same file, gave me the same error I am having right now.
The error is that it creates the DataStream, but no data is being produced.
To create the DataStream, I am using the Data Stream creator, and a JSON Array key for the file format.
What can be the problem in your opinion?
Also, I saw that using the rel/0.69.0 branch yields 0.69.0 as the version and not 0.69.1-SNAPSHOT anymore, the previous installation of SP I had was using the latter, do you think this can be the problem?
Thanks in advance for your reply,
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.gft.com%2Fit&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047673634%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=TZWw2LbLjpyL4DAAx9RjIyUP40qVRMoG2TSCrSqmuL4%3D&reserved=0>
https://blog.gft.com<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.gft.com%2F&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047829869%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Ovy0cewqOANT1CmDo7%2F8bcyIBkPgAdLl0qVLK2G71hM%3D&reserved=0>
https://www.linkedin.com/company/gft-group/<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fgft-group%2F&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047829869%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YQQ91LFlSziIhm9VV3LSLm7qDCAncsdeD48u6sf6CPc%3D&reserved=0>
www.twitter.com/gft_it <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.twitter.com%2Fgft_it%2520&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047829869%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=1QSBILyL9q%2FEDSppEU8RVyuAXDGOvhzVRp%2BEemtMlf4%3D&reserved=0>
[cid:image003.png@01D88B2C.8E3C5490]<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.gft.com%2Fit%2Fit%2Fabout-us%2FSustainability&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047829869%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=StW2olnL9VvMqBLg1MU47v0UrYDrTGD81I07hiVh05A%3D&reserved=0>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
RE: StreamPipes File Stream Creation Problem
Posted by "Iorio, Michele" <mi...@gft.com>.
Hi Philipp,
in attachment you will find a sample of data to try to recreate the problem.
Regarding version 0.69.1-SNAPSHOT, do you have any other information?
Thanks
Michele
Michele Iorio
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 010 583158
michele.iorio@gft.com
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it>
[cid:ActoForTheFuture_GFT@1,2x_68f73108-d76e-4497-aa5c-ff8a5693ce57.png]<https://www.gft.com/it/it/about-us/Sustainability>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
From: Iorio, Michele
Sent: venerdì 24 giugno 2022 16:24
To: users@streampipes.apache.org
Subject: RE: StreamPipes File Stream Creation Problem
Hi Philipp,
the problem occurs even if I upload a file with 100 entries.
The situation is strange, because sometimes it works with the small file, but other times it gives the same error.
At the moment I'm checking if I can send you a small sample of data to reproduce the problem.
Regarding the last part of Federico's email, why is version 0.69.1-SNAPSHOT no longer installed?
Could this be the cause of the problem?
Thanks
Michele
From: Philipp Zehnder <ph...@bytefabrik.ai>>
Sent: giovedì 23 giugno 2022 19:53
To: users@streampipes.apache.org<ma...@streampipes.apache.org>
Subject: AW: StreamPipes File Stream Creation Problem
Non si ricevono spesso messaggi di posta elettronica da philipp.zehnder@bytefabrik.ai<ma...@bytefabrik.ai>. Informazioni sul perché è importante<https://aka.ms/LearnAboutSenderIdentification>
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Frederico,
is there anything logged in the docker container running the adapter?
Can you try it with a very small file containing only a few events?
If you want you can also send me a small sample, so I can try to recreate the problem.
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Donnerstag, 23. Juni 2022 um 17:44
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: StreamPipes File Stream Creation Problem
Hello,
I am trying to upload a Json file of 9.2MB of size, containing 50000 entries, to create a datastream.
I previously uploaded a smaller version of the file, and everything went fine, then I had to reinstall SP, and uploading the same file, gave me the same error I am having right now.
The error is that it creates the DataStream, but no data is being produced.
To create the DataStream, I am using the Data Stream creator, and a JSON Array key for the file format.
What can be the problem in your opinion?
Also, I saw that using the rel/0.69.0 branch yields 0.69.0 as the version and not 0.69.1-SNAPSHOT anymore, the previous installation of SP I had was using the latter, do you think this can be the problem?
Thanks in advance for your reply,
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.gft.com%2Fit&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047673634%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=TZWw2LbLjpyL4DAAx9RjIyUP40qVRMoG2TSCrSqmuL4%3D&reserved=0>
https://blog.gft.com<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.gft.com%2F&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047829869%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Ovy0cewqOANT1CmDo7%2F8bcyIBkPgAdLl0qVLK2G71hM%3D&reserved=0>
https://www.linkedin.com/company/gft-group/<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fgft-group%2F&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047829869%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YQQ91LFlSziIhm9VV3LSLm7qDCAncsdeD48u6sf6CPc%3D&reserved=0>
www.twitter.com/gft_it <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.twitter.com%2Fgft_it%2520&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047829869%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=1QSBILyL9q%2FEDSppEU8RVyuAXDGOvhzVRp%2BEemtMlf4%3D&reserved=0>
[cid:image001.png@01D88A53.4732CD60]<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.gft.com%2Fit%2Fit%2Fabout-us%2FSustainability&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047829869%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=StW2olnL9VvMqBLg1MU47v0UrYDrTGD81I07hiVh05A%3D&reserved=0>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
RE: StreamPipes File Stream Creation Problem
Posted by "Iorio, Michele" <mi...@gft.com>.
Hi Philipp,
the problem occurs even if I upload a file with 100 entries.
The situation is strange, because sometimes it works with the small file, but other times it gives the same error.
At the moment I'm checking if I can send you a small sample of data to reproduce the problem.
Regarding the last part of Federico's email, why is version 0.69.1-SNAPSHOT no longer installed?
Could this be the cause of the problem?
Thanks
Michele
Michele Iorio
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 010 583158
michele.iorio@gft.com
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it>
[cid:ActoForTheFuture_GFT@1,2x_68f73108-d76e-4497-aa5c-ff8a5693ce57.png]<https://www.gft.com/it/it/about-us/Sustainability>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
From: Philipp Zehnder <ph...@bytefabrik.ai>
Sent: giovedì 23 giugno 2022 19:53
To: users@streampipes.apache.org
Subject: AW: StreamPipes File Stream Creation Problem
Non si ricevono spesso messaggi di posta elettronica da philipp.zehnder@bytefabrik.ai<ma...@bytefabrik.ai>. Informazioni sul perché è importante<https://aka.ms/LearnAboutSenderIdentification>
[EXTERNAL EMAIL]
This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe.
Hi Frederico,
is there anything logged in the docker container running the adapter?
Can you try it with a very small file containing only a few events?
If you want you can also send me a small sample, so I can try to recreate the problem.
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>>
Datum: Donnerstag, 23. Juni 2022 um 17:44
An: users@streampipes.apache.org<ma...@streampipes.apache.org> <us...@streampipes.apache.org>>
Betreff: StreamPipes File Stream Creation Problem
Hello,
I am trying to upload a Json file of 9.2MB of size, containing 50000 entries, to create a datastream.
I previously uploaded a smaller version of the file, and everything went fine, then I had to reinstall SP, and uploading the same file, gave me the same error I am having right now.
The error is that it creates the DataStream, but no data is being produced.
To create the DataStream, I am using the Data Stream creator, and a JSON Array key for the file format.
What can be the problem in your opinion?
Also, I saw that using the rel/0.69.0 branch yields 0.69.0 as the version and not 0.69.1-SNAPSHOT anymore, the previous installation of SP I had was using the latter, do you think this can be the problem?
Thanks in advance for your reply,
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com<ma...@gft.com>
www.gft.com/it<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.gft.com%2Fit&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047673634%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=TZWw2LbLjpyL4DAAx9RjIyUP40qVRMoG2TSCrSqmuL4%3D&reserved=0>
https://blog.gft.com<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.gft.com%2F&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047829869%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Ovy0cewqOANT1CmDo7%2F8bcyIBkPgAdLl0qVLK2G71hM%3D&reserved=0>
https://www.linkedin.com/company/gft-group/<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fgft-group%2F&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047829869%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YQQ91LFlSziIhm9VV3LSLm7qDCAncsdeD48u6sf6CPc%3D&reserved=0>
www.twitter.com/gft_it <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.twitter.com%2Fgft_it%2520&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047829869%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=1QSBILyL9q%2FEDSppEU8RVyuAXDGOvhzVRp%2BEemtMlf4%3D&reserved=0>
[cid:image001.png@01D887E6.DBE75B70]<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.gft.com%2Fit%2Fit%2Fabout-us%2FSustainability&data=05%7C01%7Cmichele.iorio%40gft.com%7C3b3abc618db94fba2dca08da55414353%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637916036047829869%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=StW2olnL9VvMqBLg1MU47v0UrYDrTGD81I07hiVh05A%3D&reserved=0>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________
AW: StreamPipes File Stream Creation Problem
Posted by Philipp Zehnder <ph...@bytefabrik.ai>.
Hi Frederico,
is there anything logged in the docker container running the adapter?
Can you try it with a very small file containing only a few events?
If you want you can also send me a small sample, so I can try to recreate the problem.
Cheers,
Philipp
Von: Rullo, Federico <Fe...@gft.com>
Datum: Donnerstag, 23. Juni 2022 um 17:44
An: users@streampipes.apache.org <us...@streampipes.apache.org>
Betreff: StreamPipes File Stream Creation Problem
Hello,
I am trying to upload a Json file of 9.2MB of size, containing 50000 entries, to create a datastream.
I previously uploaded a smaller version of the file, and everything went fine, then I had to reinstall SP, and uploading the same file, gave me the same error I am having right now.
The error is that it creates the DataStream, but no data is being produced.
To create the DataStream, I am using the Data Stream creator, and a JSON Array key for the file format.
What can be the problem in your opinion?
Also, I saw that using the rel/0.69.0 branch yields 0.69.0 as the version and not 0.69.1-SNAPSHOT anymore, the previous installation of SP I had was using the latter, do you think this can be the problem?
Thanks in advance for your reply,
Best,
Federico.
Federico Rullo
_________________________________________________________________
GFT Italia S.r.l.
Via A. Rimassa, 51/2
16129 Genova, Italy
T +39 025 8158169
Federico.Rullo@gft.com
www.gft.com/it<http://www.gft.com/it>
https://blog.gft.com
https://www.linkedin.com/company/gft-group/
www.twitter.com/gft_it <http://www.twitter.com/gft_it%20>
[cid:ActoForTheFuture_GFT@1,2x_68f73108-d76e-4497-aa5c-ff8a5693ce57.png]<https://www.gft.com/it/it/about-us/Sustainability>
Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy)
VAT Code and number of registration with the Companies' Register of Milan: 00819200478
Corporate capital: Euro 1,000,000 fully paid-in
Company subject to the direction and coordination activity of GFT Technologies SE
Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all’azienda.
In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company.
_________________________________________________________________