You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@streampipes.apache.org by Philipp Zehnder <ze...@fzi.de> on 2019/11/16 08:57:18 UTC

Run and develop StreamPipes without configuring the IP address

Hi all,

we removed the requirement to define the servers IP address when StreamPipes is run completely in Docker. 
However, we still have to change several configs to ensure the new setting also works during development.

This change effects the following projects:

* streampipes 
  * Do we still need all of those configurations for the backend?
SP_COUCHDB_HOST=localhost
uri=http://localhost:8031/rdf4j-server
SP_KAFKA_HOST=host.docker.internal
SP_ZOOKEEPER_HOST=host.docker.internal
SP_JMS_HOST=host.docker.internal
SP_KAFKA_REST_HOST=host.docker.internal
SP_KAFKA_REST_PORT=8073
SP_BACKEND_HOST=host.docker.internal
SP_ELASTICSEARCH_HOST=host.docker.internal
SP_DATALAKE_HOST=host.docker.internal
SP_DATALAKE_PORT=9200
SP_INFLUX_HOST=host.docker.internal

* streampipes-pipeline-elements 
  * Suggestion of an example configuration for processing containers:

SP_PORT=6060
SP_HOST=host.docker.internal
SP_DEBUG=true

* streampipes-connect-adapters 
  * Suggestion of an example configuration for connect adapters

SP_KAFKA_HOST=localhost
SP_CONNECT_CONTAINER_HOST=localhost
SP_CONNECT_CONTAINER_MASTER_HOST=localhost
SP_CONNECT_CONTAINER_WORKER_HOST=localhost
SP_DATA_LOCATION=./test_data/
SP_DEBUG=true


Did I forget any configurations we have to change?

I am still not completely certain when to use localhost and when to use host.docker.internal during development. Currently I added host.docker.internal to my /etc/hosts file. 
Is there a way to remove this entry?

Additionally I suggest to remove all deployment folders (containing a docker-compose.yml file) in the projects and manage them centrally in the cli project.

Cheers,
Philipp

AW: Run and develop StreamPipes without configuring the IP address

Posted by Johannes Tex <jo...@web.de>.

Hi,



I think we don't need any more these configurations for the backend :

-        SP_DATALAKE_HOST

-        SP_DATALAKE_PORT



It was used for the first version of the data-lake with Elasticsearch , we
changed the data-lake database to Influx and removed the old data-lake API
with Elasticsearch last week.



Furthermore I think it would be useful to add the port configuration for
Influx:

SP_INFLUX_HOST=8086





Regards

Johannes







Von: Philipp Zehnder [mailto:zehnder@fzi.de]
Gesendet: Samstag, 16. November 2019 09:57
An: dev@streampipes.apache.org
Betreff: Run and develop StreamPipes without configuring the IP address



Hi all,



we removed the requirement to define the servers IP address when StreamPipes
is run completely in Docker.

However, we still have to change several configs to ensure the new setting
also works during development.



This change effects the following projects:



* streampipes

  * Do we still need all of those configurations for the backend?

SP_COUCHDB_HOST=localhost
uri=http://localhost:8031/rdf4j-server
SP_KAFKA_HOST=host.docker.internal
SP_ZOOKEEPER_HOST=host.docker.internal
SP_JMS_HOST=host.docker.internal
SP_KAFKA_REST_HOST=host.docker.internal
SP_KAFKA_REST_PORT=8073
SP_BACKEND_HOST=host.docker.internal
SP_ELASTICSEARCH_HOST=host.docker.internal
SP_DATALAKE_HOST=host.docker.internal
SP_DATALAKE_PORT=9200
SP_INFLUX_HOST=host.docker.internal



* streampipes-pipeline-elements

  * Suggestion of an example configuration for processing containers:



SP_PORT=6060
SP_HOST=host.docker.internal
SP_DEBUG=true



* streampipes-connect-adapters

  * Suggestion of an example configuration for connect adapters



SP_KAFKA_HOST=localhost

SP_CONNECT_CONTAINER_HOST=localhost

SP_CONNECT_CONTAINER_MASTER_HOST=localhost

SP_CONNECT_CONTAINER_WORKER_HOST=localhost

SP_DATA_LOCATION=./test_data/

SP_DEBUG=true





Did I forget any configurations we have to change?



I am still not completely certain when to use localhost and when to use
host.docker.internal during development. Currently I added
host.docker.internal to my /etc/hosts file.

Is there a way to remove this entry?



Additionally I suggest to remove all deployment folders (containing a
docker-compose.yml file) in the projects and manage them centrally in the
cli project.



Cheers,

Philipp