You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@streampipes.apache.org by Joel Oliveira <jo...@gmail.com> on 2022/04/17 10:11:00 UTC
[Stream Development]
Hi
I am using StreamPipes version 0.69.0, Windows 10 Pro.
I am trying to build a custom Data Stream. I started by following the
tutorial skeleton.
I have already developed the code for the data streaming.
When I run it, still in IntelIj environment, I can install it without any
problem.
However, when I try to use it to build a pipeline, the data published seems
to never reach the Streampipes application, as you can see in the example
attached, both the numerical filters are equally configured ("value", "<",
"10").
I changed the maven skeleton processor so that it makes no operation.
Output is the same as the input.
The stream I developed, shows no data, and cannot send it to any pipeline
elements. However, when I connect it with the skeleton processor, the data
can flow. Any hints on this?
The consul gives no errors (printscreen attached).
I have also attached my startup function and the thread executed as the
stream.
Also, after developing the elements, I am not sure how the streampipes add
command works, to add the service fully functional to the streampipes
environment. It would be very helpfull if you could clarify it.
Regards,
Joel Oliveira
Re: [Stream Development]
Posted by Joel Oliveira <jo...@gmail.com>.
I have found the problem, thanks a lot!
I looked into your examples, from the vehicle case, and the watertank. You
choose the kafka host/port dinamically.
I changed that according with your code, and, has you said, included the
image in the docker-compose file, and it worked perfectly.
Once again, many thanks.
Regards,
Joel
Joel Oliveira <jo...@gmail.com> escreveu no dia segunda,
18/04/2022 à(s) 17:16:
> The console logs only mention kafka once, at start up. I am not sure if
> that is what you are refering too. In any case I'll attach the output, it
> is easier.
> Regarding the SP_DEBUG, I am declaring it, in the env file, with the value
> "true".
>
> Also attaching prints about the monitoring view.
> The information that goes directly to the numerical filter, and the stream
> have negative values.
> The information that goes through the dummy processor seems to be ok.
>
> Dominik Riemer <ri...@fzi.de> escreveu no dia segunda, 18/04/2022 à(s)
> 13:20:
>
>> Hi Joel,
>>
>>
>>
>> happy to help! Do you see any output in your console when starting the
>> custom stream? It should give you a message that the connection to Kafka is
>> successful. Are you providing the SP_DEBUG variable for local development
>> to override the Kafka settings to a localhost/9094 combination? It would be
>> also interesting to know if there are any events displayed when you switch
>> to the pipelines view and navigate to the monitoring tab (the search icon
>> in the pipeline view).
>>
>>
>>
>> Regarding deployment, if you want to use Docker, you can build your
>> service with streams and processors with the provided Dockerfile and, e.g.,
>> attach it to the docker-compose of the installer. You can also run it
>> standalone, but make sure to attach your service to the spnet Docker
>> network.
>>
>>
>>
>> Cheers,
>> Dominik
>>
>>
>>
>>
>>
>> *From:* Joel Oliveira <jo...@gmail.com>
>> *Sent:* Sunday, April 17, 2022 12:11 PM
>> *To:* users@streampipes.apache.org
>> *Subject:* [Stream Development]
>>
>>
>>
>> Hi
>>
>> I am using StreamPipes version 0.69.0, Windows 10 Pro.
>>
>>
>>
>> I am trying to build a custom Data Stream. I started by following the
>> tutorial skeleton.
>>
>> I have already developed the code for the data streaming.
>>
>>
>>
>> When I run it, still in IntelIj environment, I can install it without any
>> problem.
>>
>> However, when I try to use it to build a pipeline, the data published
>> seems to never reach the Streampipes application, as you can see in the
>> example attached, both the numerical filters are equally configured
>> ("value", "<", "10").
>>
>> I changed the maven skeleton processor so that it makes no operation.
>> Output is the same as the input.
>>
>> The stream I developed, shows no data, and cannot send it to any pipeline
>> elements. However, when I connect it with the skeleton processor, the data
>> can flow. Any hints on this?
>>
>> The consul gives no errors (printscreen attached).
>>
>> I have also attached my startup function and the thread executed as the
>> stream.
>>
>>
>>
>>
>>
>> Also, after developing the elements, I am not sure how the streampipes
>> add command works, to add the service fully functional to the streampipes
>> environment. It would be very helpfull if you could clarify it.
>>
>>
>>
>> Regards,
>>
>> Joel Oliveira
>>
>>
>>
>
Re: [Stream Development]
Posted by Joel Oliveira <jo...@gmail.com>.
The console logs only mention kafka once, at start up. I am not sure if
that is what you are refering too. In any case I'll attach the output, it
is easier.
Regarding the SP_DEBUG, I am declaring it, in the env file, with the value
"true".
Also attaching prints about the monitoring view.
The information that goes directly to the numerical filter, and the stream
have negative values.
The information that goes through the dummy processor seems to be ok.
Dominik Riemer <ri...@fzi.de> escreveu no dia segunda, 18/04/2022 à(s)
13:20:
> Hi Joel,
>
>
>
> happy to help! Do you see any output in your console when starting the
> custom stream? It should give you a message that the connection to Kafka is
> successful. Are you providing the SP_DEBUG variable for local development
> to override the Kafka settings to a localhost/9094 combination? It would be
> also interesting to know if there are any events displayed when you switch
> to the pipelines view and navigate to the monitoring tab (the search icon
> in the pipeline view).
>
>
>
> Regarding deployment, if you want to use Docker, you can build your
> service with streams and processors with the provided Dockerfile and, e.g.,
> attach it to the docker-compose of the installer. You can also run it
> standalone, but make sure to attach your service to the spnet Docker
> network.
>
>
>
> Cheers,
> Dominik
>
>
>
>
>
> *From:* Joel Oliveira <jo...@gmail.com>
> *Sent:* Sunday, April 17, 2022 12:11 PM
> *To:* users@streampipes.apache.org
> *Subject:* [Stream Development]
>
>
>
> Hi
>
> I am using StreamPipes version 0.69.0, Windows 10 Pro.
>
>
>
> I am trying to build a custom Data Stream. I started by following the
> tutorial skeleton.
>
> I have already developed the code for the data streaming.
>
>
>
> When I run it, still in IntelIj environment, I can install it without any
> problem.
>
> However, when I try to use it to build a pipeline, the data published
> seems to never reach the Streampipes application, as you can see in the
> example attached, both the numerical filters are equally configured
> ("value", "<", "10").
>
> I changed the maven skeleton processor so that it makes no operation.
> Output is the same as the input.
>
> The stream I developed, shows no data, and cannot send it to any pipeline
> elements. However, when I connect it with the skeleton processor, the data
> can flow. Any hints on this?
>
> The consul gives no errors (printscreen attached).
>
> I have also attached my startup function and the thread executed as the
> stream.
>
>
>
>
>
> Also, after developing the elements, I am not sure how the streampipes add
> command works, to add the service fully functional to the streampipes
> environment. It would be very helpfull if you could clarify it.
>
>
>
> Regards,
>
> Joel Oliveira
>
>
>
RE: [Stream Development]
Posted by Dominik Riemer <ri...@fzi.de>.
Hi Joel,
happy to help! Do you see any output in your console when starting the custom stream? It should give you a message that the connection to Kafka is successful. Are you providing the SP_DEBUG variable for local development to override the Kafka settings to a localhost/9094 combination? It would be also interesting to know if there are any events displayed when you switch to the pipelines view and navigate to the monitoring tab (the search icon in the pipeline view).
Regarding deployment, if you want to use Docker, you can build your service with streams and processors with the provided Dockerfile and, e.g., attach it to the docker-compose of the installer. You can also run it standalone, but make sure to attach your service to the spnet Docker network.
Cheers,
Dominik
From: Joel Oliveira <jo...@gmail.com>
Sent: Sunday, April 17, 2022 12:11 PM
To: users@streampipes.apache.org
Subject: [Stream Development]
Hi
I am using StreamPipes version 0.69.0, Windows 10 Pro.
I am trying to build a custom Data Stream. I started by following the tutorial skeleton.
I have already developed the code for the data streaming.
When I run it, still in IntelIj environment, I can install it without any problem.
However, when I try to use it to build a pipeline, the data published seems to never reach the Streampipes application, as you can see in the example attached, both the numerical filters are equally configured ("value", "<", "10").
I changed the maven skeleton processor so that it makes no operation. Output is the same as the input.
The stream I developed, shows no data, and cannot send it to any pipeline elements. However, when I connect it with the skeleton processor, the data can flow. Any hints on this?
The consul gives no errors (printscreen attached).
I have also attached my startup function and the thread executed as the stream.
Also, after developing the elements, I am not sure how the streampipes add command works, to add the service fully functional to the streampipes environment. It would be very helpfull if you could clarify it.
Regards,
Joel Oliveira