You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@streampipes.apache.org by Florian Micklich <mi...@apache.org> on 2022/11/22 17:03:22 UTC
Add functionality of host.docker.internal for linux users?
Hi,
unlike Windows or Mac, the host.docker.internal command is not working by default in Linux.
A solution would be to add following code snippet the the docker-compose files [1].
```
extra_hosts:
- "host.docker.internal:host-gateway"
```
But this requires at least docker version 20.10+.
I think on our website the required docker version is 1.17.
Should we take this step and increase the version requirements to add this functionality?
Greetings
Florian
[1] https://github.com/docker/for-linux/issues/264#issuecomment-784985736
RE: Add functionality of host.docker.internal for linux users?
Posted by Dominik Riemer <do...@bytefabrik.ai>.
What Philipp says is that you don't have to configure the .env file, but that you can provide the env variable directly as a startup argument (e.g., in the IDE settings). This doesn't change your source code.
Dominik
-----Original Message-----
From: Florian Micklich <mi...@apache.org>
Sent: Wednesday, November 30, 2022 1:01 PM
To: dev@streampipes.apache.org
Subject: Re: Add functionality of host.docker.internal for linux users?
Hi Philipp,
I guess we are talking at cross purposes :) Maybe for better understanding. How does auto discovery works with the CLI installer?
I thought the auto discovery is done with the host.docker.internal flag?
Everything is working after configure the SP_HOST in the environment list.
But this leads to a changed file in git in the ../incubator-streampipes/.idea/runConfigurations folder e.g. all_pipeline_elements_jvm.xml due this folder / files are not in the gitignore list.
So I have to handle this in git all the time and for better programmer experiences I though we can add this little feature for linux users as well, so that everything is working without extra editing.
Greetings
Florian
Am Dienstag, dem 29.11.2022 um 10:15 +0000 schrieb Philipp Zehnder:
> Hi Florian,<br>
>
> ah, sorry, I thought we don't have the IP problem anymore.<br> You can
> take a look at [1], the networking class implements auto-discovery of
> the IP address.<br> Maybe we can find a solution that will reliably
> determine the IP address for Linux as well.<br>
>
> In the meantime, my tip would be to set the environment variable in
> the IDE so you don't have any changes in the source files.<br>
>
> Cheers!<br>
> Philipp<br>
>
>
> Von: Florian Micklich
> <[micklich@apache.org](mailto:micklich@apache.org)><br>
> Datum: Samstag, 26. November 2022 um 20:17<br>
> An: [dev@streampipes.apache.org](mailto:dev@streampipes.apache.org)
> <[dev@streampipes.apache.org](mailto:dev@streampipes.apache.org)><br>
> Betreff: Re: Add functionality of host.docker.internal for linux
> users?<br> Hi Philipp,<br>
>
> are you using Linux with the CLI installation? How is here auto
> discovery possible?<br> So I need to set the SP_HOST variable in my
> environment or env file. This constantly results in a changed file in
> git that I have to pay attention every time I commit, push, ... so I
> don't accidentally commit it or clash into something else.<br>
>
>
> As it is written here [1] on Example walkthrough for
> pipeline-elements-jvm-all in IntelliJ:<br>
> -> For Linux user: You must change the environment variables stating
> -> host.docker.internal to you local machine's IP.<br>
>
> So I think this will increase compatibility for Linux users to use the
> default host.docker.internal.<br>
>
> ```<br>
> > extra_hosts:<br><br>
> > - "host.docker.internal:host-gateway"<br>
> ```<br>
>
> Greetings<br>
> Florian<br>
>
> P.S. A short poll, who uses which operating system, would be
> interesting :-)<br>
>
>
>
> [1]
> [https://cwiki.apache.org/confluence/display/STREAMPIPES/Extensions](h
> ttps://cwiki.apache.org/confluence/display/STREAMPIPES/Extensions)<br>
>
>
> This is a service that wraps all pipeline elements written with the
> standard Java wrapper, hence the the notion of "jvm-all".<br>
>
> You should automatically have a run configuration
> "all-pipeline-elements-jvm" as we ship in with the repository<br>
> (Optional) Modifiy environment variables depending on your OS
> (this is due to the dev setup of partly IDE and partly Docker, we need
> the local services to be reachable from within the Docker network)<br>
> For Windows/Mac user: you are good to go<br>
> For Linux user: You must change the environment variables
> stating host.docker.internal to you local machine's IP<br>
> Run the service<br>
> Navigate to "[http://localhost:7023/](http://localhost:7023/)" to
> see if the backend is running. You should see something similar to the
> following:<br>
>
> Am Mittwoch, dem 23.11.2022 um 16:50 +0000 schrieb Philipp
> Zehnder:<br>
> > Hi Florian,<br><br>
> >
> > what do you need “host.docker.internal” for?<br><br>
> >
> > I usually use the auto discovery of the IP address.<br><br>
> > Sometimes, when I change the network, I must restart the backend,
> > but then it will automatically find the correct IP address
> > again.<br><br>
> >
> > Cheers,<br><br>
> > Philipp<br><br>
> >
> > Von: Florian Micklich
> > <[[micklich@apache.org](mailto:micklich@apache.org)](mailto:[micklic
> > h@apache.org](mailto:micklich@apache.org))><br><br>
> > Datum: Dienstag, 22. November 2022 um 18:03<br><br>
> > An: StreanPipes Dev
> > <[[dev@streampipes.apache.org](mailto:dev@streampipes.apache.org)](m
> > ailto:[dev@streampipes.apache.org](mailto:dev@streampipes.apache.org
> > ))><br><br>
> > Betreff: Add functionality of host.docker.internal for linux
> > users?<br><br> Hi,<br><br>
> >
> > unlike Windows or Mac, the host.docker.internal command is not
> > working by default in Linux.<br><br> A solution would be to add
> > following code snippet the the docker-compose files [1].<br><br>
> >
> >
> > ```<br><br>
> > extra_hosts:<br><br>
> > - "host.docker.internal:host-gateway"<br><br>
> > ```<br><br>
> >
> > But this requires at least docker version 20.10+.<br><br> I think on
> > our website the required docker version is 1.17.<br><br> Should we
> > take this step and increase the version requirements to add this
> > functionality?<br><br>
> >
> >
> > Greetings<br><br>
> > Florian<br><br>
> >
> > [1]
> > [[https://github.com/docker/for-linux/issues/264#issuecomment-784985
> > 736](https://github.com/docker/for-linux/issues/264#issuecomment-784
> > 985736)](https://github.com/docker/for-linux/issues/264#issuecomment
> > -784985736](https://github.com/docker/for-linux/issues/264#issuecomm
> > ent-784985736))<br><br>
Re: Add functionality of host.docker.internal for linux users?
Posted by Florian Micklich <mi...@apache.org>.
Hi Philipp,
I guess we are talking at cross purposes :)
Maybe for better understanding. How does auto discovery works with the CLI installer?
I thought the auto discovery is done with the host.docker.internal flag?
Everything is working after configure the SP_HOST in the environment list.
But this leads to a changed file in git in the ../incubator-streampipes/.idea/runConfigurations folder
e.g. all_pipeline_elements_jvm.xml due this folder / files are not in the gitignore list.
So I have to handle this in git all the time and for better programmer experiences I though we can add this little feature for linux users as well, so that everything is working without extra editing.
Greetings
Florian
Am Dienstag, dem 29.11.2022 um 10:15 +0000 schrieb Philipp Zehnder:
> Hi Florian,<br>
>
> ah, sorry, I thought we don't have the IP problem anymore.<br>
> You can take a look at [1], the networking class implements auto-discovery of the IP address.<br>
> Maybe we can find a solution that will reliably determine the IP address for Linux as well.<br>
>
> In the meantime, my tip would be to set the environment variable in the IDE so you don't have any changes in the source files.<br>
>
> Cheers!<br>
> Philipp<br>
>
>
> Von: Florian Micklich <[micklich@apache.org](mailto:micklich@apache.org)><br>
> Datum: Samstag, 26. November 2022 um 20:17<br>
> An: [dev@streampipes.apache.org](mailto:dev@streampipes.apache.org) <[dev@streampipes.apache.org](mailto:dev@streampipes.apache.org)><br>
> Betreff: Re: Add functionality of host.docker.internal for linux users?<br>
> Hi Philipp,<br>
>
> are you using Linux with the CLI installation? How is here auto discovery possible?<br>
> So I need to set the SP_HOST variable in my environment or env file. This constantly results in a changed file in git that I have to pay attention every time I commit, push, ... so I don't accidentally commit it or clash into something else.<br>
>
>
> As it is written here [1] on Example walkthrough for pipeline-elements-jvm-all in IntelliJ:<br>
> -> For Linux user: You must change the environment variables stating host.docker.internal to you local machine's IP.<br>
>
> So I think this will increase compatibility for Linux users to use the default host.docker.internal.<br>
>
> ```<br>
> > extra_hosts:<br><br>
> > - "host.docker.internal:host-gateway"<br>
> ```<br>
>
> Greetings<br>
> Florian<br>
>
> P.S. A short poll, who uses which operating system, would be interesting :-)<br>
>
>
>
> [1] [https://cwiki.apache.org/confluence/display/STREAMPIPES/Extensions](https://cwiki.apache.org/confluence/display/STREAMPIPES/Extensions)<br>
>
>
> This is a service that wraps all pipeline elements written with the standard Java wrapper, hence the the notion of "jvm-all".<br>
>
> You should automatically have a run configuration "all-pipeline-elements-jvm" as we ship in with the repository<br>
> (Optional) Modifiy environment variables depending on your OS (this is due to the dev setup of partly IDE and partly Docker, we need the local services to be reachable from within the Docker network)<br>
> For Windows/Mac user: you are good to go<br>
> For Linux user: You must change the environment variables stating host.docker.internal to you local machine's IP<br>
> Run the service<br>
> Navigate to "[http://localhost:7023/](http://localhost:7023/)" to see if the backend is running. You should see something similar to the following:<br>
>
> Am Mittwoch, dem 23.11.2022 um 16:50 +0000 schrieb Philipp Zehnder:<br>
> > Hi Florian,<br><br>
> >
> > what do you need “host.docker.internal” for?<br><br>
> >
> > I usually use the auto discovery of the IP address.<br><br>
> > Sometimes, when I change the network, I must restart the backend, but then it will automatically find the correct IP address again.<br><br>
> >
> > Cheers,<br><br>
> > Philipp<br><br>
> >
> > Von: Florian Micklich <[[micklich@apache.org](mailto:micklich@apache.org)](mailto:[micklich@apache.org](mailto:micklich@apache.org))><br><br>
> > Datum: Dienstag, 22. November 2022 um 18:03<br><br>
> > An: StreanPipes Dev <[[dev@streampipes.apache.org](mailto:dev@streampipes.apache.org)](mailto:[dev@streampipes.apache.org](mailto:dev@streampipes.apache.org))><br><br>
> > Betreff: Add functionality of host.docker.internal for linux users?<br><br>
> > Hi,<br><br>
> >
> > unlike Windows or Mac, the host.docker.internal command is not working by default in Linux.<br><br>
> > A solution would be to add following code snippet the the docker-compose files [1].<br><br>
> >
> >
> > ```<br><br>
> > extra_hosts:<br><br>
> > - "host.docker.internal:host-gateway"<br><br>
> > ```<br><br>
> >
> > But this requires at least docker version 20.10+.<br><br>
> > I think on our website the required docker version is 1.17.<br><br>
> > Should we take this step and increase the version requirements to add this functionality?<br><br>
> >
> >
> > Greetings<br><br>
> > Florian<br><br>
> >
> > [1] [[https://github.com/docker/for-linux/issues/264#issuecomment-784985736](https://github.com/docker/for-linux/issues/264#issuecomment-784985736)](https://github.com/docker/for-linux/issues/264#issuecomment-784985736](https://github.com/docker/for-linux/issues/264#issuecomment-784985736))<br><br>
AW: Add functionality of host.docker.internal for linux users?
Posted by Philipp Zehnder <ph...@bytefabrik.ai>.
Hi Florian,
ah, sorry, I thought we don't have the IP problem anymore.
You can take a look at [1], the networking class implements auto-discovery of the IP address.
Maybe we can find a solution that will reliably determine the IP address for Linux as well.
In the meantime, my tip would be to set the environment variable in the IDE so you don't have any changes in the source files.
Cheers!
Philipp
Von: Florian Micklich <mi...@apache.org>
Datum: Samstag, 26. November 2022 um 20:17
An: dev@streampipes.apache.org <de...@streampipes.apache.org>
Betreff: Re: Add functionality of host.docker.internal for linux users?
Hi Philipp,
are you using Linux with the CLI installation? How is here auto discovery possible?
So I need to set the SP_HOST variable in my environment or env file. This constantly results in a changed file in git that I have to pay attention every time I commit, push, ... so I don't accidentally commit it or clash into something else.
As it is written here [1] on Example walkthrough for pipeline-elements-jvm-all in IntelliJ:
-> For Linux user: You must change the environment variables stating host.docker.internal to you local machine's IP.
So I think this will increase compatibility for Linux users to use the default host.docker.internal.
```
> extra_hosts:<br>
> - "host.docker.internal:host-gateway"
```
Greetings
Florian
P.S. A short poll, who uses which operating system, would be interesting :-)
[1] https://cwiki.apache.org/confluence/display/STREAMPIPES/Extensions
This is a service that wraps all pipeline elements written with the standard Java wrapper, hence the the notion of "jvm-all".
You should automatically have a run configuration "all-pipeline-elements-jvm" as we ship in with the repository
(Optional) Modifiy environment variables depending on your OS (this is due to the dev setup of partly IDE and partly Docker, we need the local services to be reachable from within the Docker network)
For Windows/Mac user: you are good to go
For Linux user: You must change the environment variables stating host.docker.internal to you local machine's IP
Run the service
Navigate to "http://localhost:7023/" to see if the backend is running. You should see something similar to the following:
Am Mittwoch, dem 23.11.2022 um 16:50 +0000 schrieb Philipp Zehnder:
> Hi Florian,<br>
>
> what do you need “host.docker.internal” for?<br>
>
> I usually use the auto discovery of the IP address.<br>
> Sometimes, when I change the network, I must restart the backend, but then it will automatically find the correct IP address again.<br>
>
> Cheers,<br>
> Philipp<br>
>
> Von: Florian Micklich <[micklich@apache.org](mailto:micklich@apache.org)><br>
> Datum: Dienstag, 22. November 2022 um 18:03<br>
> An: StreanPipes Dev <[dev@streampipes.apache.org](mailto:dev@streampipes.apache.org)><br>
> Betreff: Add functionality of host.docker.internal for linux users?<br>
> Hi,<br>
>
> unlike Windows or Mac, the host.docker.internal command is not working by default in Linux.<br>
> A solution would be to add following code snippet the the docker-compose files [1].<br>
>
>
> ```<br>
> extra_hosts:<br>
> - "host.docker.internal:host-gateway"<br>
> ```<br>
>
> But this requires at least docker version 20.10+.<br>
> I think on our website the required docker version is 1.17.<br>
> Should we take this step and increase the version requirements to add this functionality?<br>
>
>
> Greetings<br>
> Florian<br>
>
> [1] [https://github.com/docker/for-linux/issues/264#issuecomment-784985736](https://github.com/docker/for-linux/issues/264#issuecomment-784985736)<br>
Re: Add functionality of host.docker.internal for linux users?
Posted by Florian Micklich <mi...@apache.org>.
Hi Philipp,
are you using Linux with the CLI installation? How is here auto discovery possible?
So I need to set the SP_HOST variable in my environment or env file. This constantly results in a changed file in git that I have to pay attention every time I commit, push, ... so I don't accidentally commit it or clash into something else.
As it is written here [1] on Example walkthrough for pipeline-elements-jvm-all in IntelliJ:
-> For Linux user: You must change the environment variables stating host.docker.internal to you local machine's IP.
So I think this will increase compatibility for Linux users to use the default host.docker.internal.
```
> extra_hosts:<br>
> - "host.docker.internal:host-gateway"
```
Greetings
Florian
P.S. A short poll, who uses which operating system, would be interesting :-)
[1] https://cwiki.apache.org/confluence/display/STREAMPIPES/Extensions
This is a service that wraps all pipeline elements written with the standard Java wrapper, hence the the notion of "jvm-all".
You should automatically have a run configuration "all-pipeline-elements-jvm" as we ship in with the repository
(Optional) Modifiy environment variables depending on your OS (this is due to the dev setup of partly IDE and partly Docker, we need the local services to be reachable from within the Docker network)
For Windows/Mac user: you are good to go
For Linux user: You must change the environment variables stating host.docker.internal to you local machine's IP
Run the service
Navigate to "http://localhost:7023/" to see if the backend is running. You should see something similar to the following:
Am Mittwoch, dem 23.11.2022 um 16:50 +0000 schrieb Philipp Zehnder:
> Hi Florian,<br>
>
> what do you need “host.docker.internal” for?<br>
>
> I usually use the auto discovery of the IP address.<br>
> Sometimes, when I change the network, I must restart the backend, but then it will automatically find the correct IP address again.<br>
>
> Cheers,<br>
> Philipp<br>
>
> Von: Florian Micklich <[micklich@apache.org](mailto:micklich@apache.org)><br>
> Datum: Dienstag, 22. November 2022 um 18:03<br>
> An: StreanPipes Dev <[dev@streampipes.apache.org](mailto:dev@streampipes.apache.org)><br>
> Betreff: Add functionality of host.docker.internal for linux users?<br>
> Hi,<br>
>
> unlike Windows or Mac, the host.docker.internal command is not working by default in Linux.<br>
> A solution would be to add following code snippet the the docker-compose files [1].<br>
>
>
> ```<br>
> extra_hosts:<br>
> - "host.docker.internal:host-gateway"<br>
> ```<br>
>
> But this requires at least docker version 20.10+.<br>
> I think on our website the required docker version is 1.17.<br>
> Should we take this step and increase the version requirements to add this functionality?<br>
>
>
> Greetings<br>
> Florian<br>
>
> [1] [https://github.com/docker/for-linux/issues/264#issuecomment-784985736](https://github.com/docker/for-linux/issues/264#issuecomment-784985736)<br>
AW: Add functionality of host.docker.internal for linux users?
Posted by Philipp Zehnder <ph...@bytefabrik.ai>.
Hi Florian,
what do you need “host.docker.internal” for?
I usually use the auto discovery of the IP address.
Sometimes, when I change the network, I must restart the backend, but then it will automatically find the correct IP address again.
Cheers,
Philipp
Von: Florian Micklich <mi...@apache.org>
Datum: Dienstag, 22. November 2022 um 18:03
An: StreanPipes Dev <de...@streampipes.apache.org>
Betreff: Add functionality of host.docker.internal for linux users?
Hi,
unlike Windows or Mac, the host.docker.internal command is not working by default in Linux.
A solution would be to add following code snippet the the docker-compose files [1].
```
extra_hosts:
- "host.docker.internal:host-gateway"
```
But this requires at least docker version 20.10+.
I think on our website the required docker version is 1.17.
Should we take this step and increase the version requirements to add this functionality?
Greetings
Florian
[1] https://github.com/docker/for-linux/issues/264#issuecomment-784985736