You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@streampipes.apache.org by GitBox <gi...@apache.org> on 2023/04/19 15:16:10 UTC
[GitHub] [streampipes] bossenti created a discussion: Does switching docker-compose mode require to clear docker assets? 🤔
GitHub user bossenti created a discussion: Does switching docker-compose mode require to clear docker assets? 🤔
I just faced a surprising behavior of StreamPipes when using docker-compose as deployment mode:
First, I started StreamPipes with `docker-compose -f docker-compose.nats.yaml up -d` within `installer/compose`. After some time I shutdown StreamPipes with `docker-compose ... down`.
Eventually, I started StreamPipes again and switched the broker via using `docker-compose -f docker-compose.yaml`. All containers started as expected (Kafka was started, but not NATS). But after logging in, I couldn't really use StreamPipes, e.g. for creating adapters, because it still expected NATS as broker.
So it appears that StreamPipes did not recognize using the wrong configuration/volumes.
One workaround would be simply clear the docker assets when switching between the docker-compose options , but I think this is not ideal...
How are your thoughts about this?
GitHub link: https://github.com/apache/streampipes/discussions/1514
----
This is an automatically sent email for dev@streampipes.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@streampipes.apache.org
[GitHub] [streampipes] bossenti added a comment to the discussion: Does switching docker-compose mode require to clear docker assets? 🤔
Posted by GitBox <gi...@apache.org>.
GitHub user bossenti added a comment to the discussion: Does switching docker-compose mode require to clear docker assets? 🤔
AFAIK we also use this information in the Python library, but that shouldn't stop us from removing it.
We will surely find an alternative 🙂
GitHub link: https://github.com/apache/streampipes/discussions/1514#discussioncomment-5822383
----
This is an automatically sent email for dev@streampipes.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@streampipes.apache.org
[GitHub] [streampipes] bossenti added a comment to the discussion: Does switching docker-compose mode require to clear docker assets? 🤔
Posted by GitBox <gi...@apache.org>.
GitHub user bossenti added a comment to the discussion: Does switching docker-compose mode require to clear docker assets? 🤔
I think the behavior observed by me is only relevant for developing or exploration scenarios.
Usually, users do not frequently change their broker settings, I'd assume.
But if so, the current behavior is a bit confusing IMHO.
Is there a reason why the broker priority is stored in every adapter/pipeline?
The second scenario described by sounds like an ideal solution at the first sight.
GitHub link: https://github.com/apache/streampipes/discussions/1514#discussioncomment-5774453
----
This is an automatically sent email for dev@streampipes.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@streampipes.apache.org
[GitHub] [streampipes] dominikriemer added a comment to the discussion: Does switching docker-compose mode require to clear docker assets? 🤔
Posted by GitBox <gi...@apache.org>.
GitHub user dominikriemer added a comment to the discussion: Does switching docker-compose mode require to clear docker assets? 🤔
Hi @bossenti, yes, setting the messaging layer is currently a setup-time configuration option since the messaging settings for existing pipelines and adapters are not changed when using another broker. This would only account for newly created resources.
Is this something we would like to have? Then we could add this to the configuration view or as a startup config - I see the following options:
* Iterating over all pipelines and adapters after the broker priority was changed and changing all messaging settings in these resources
* We do not store the messaging info for in the pipeline and adapter descriptions (e.g., only the channel/topic) and dynamically assing them from the settings when starting adapters and pipelines.
GitHub link: https://github.com/apache/streampipes/discussions/1514#discussioncomment-5727943
----
This is an automatically sent email for dev@streampipes.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@streampipes.apache.org
[GitHub] [streampipes] tenthe added a comment to the discussion: Does switching docker-compose mode require to clear docker assets? 🤔
Posted by GitBox <gi...@apache.org>.
GitHub user tenthe added a comment to the discussion: Does switching docker-compose mode require to clear docker assets? 🤔
I think this is for some legacy reasons. I do not see a reason why we need to store this information in each resource.
I guess it should be possible to manage the messaging configuration by the backend service. Or are there any edge cases that I am currently not thinking of?
GitHub link: https://github.com/apache/streampipes/discussions/1514#discussioncomment-5777123
----
This is an automatically sent email for dev@streampipes.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@streampipes.apache.org