You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@streampipes.apache.org by "dshunter107 (via GitHub)" <gi...@apache.org> on 2023/09/28 23:01:37 UTC
[I] Program continually searches for Pipeline Elements in the "Install Pipeline Elements" tab
dshunter107 opened a new issue, #1975:
URL: https://github.com/apache/streampipes/issues/1975
### Apache StreamPipes version
0.92.0
### Affected StreamPipes components
Backend, Connect, Processing Elements
### What happened?
After failing to get the pipelines to show up through the developer tutorial, I decided to follow the user tutorial in YouTube. Here is the first YouTube video link: https://www.youtube.com/watch?v=80TlDLH8ayY&t=317s&ab_channel=YaserAliHusen. I would say the problem is severe because the user would not be able start tracking any real-time data.
I downloaded the Apache Streampipes from the website. I then launched docker as the Youtube video says. Afterwards, I went to the installer/compose folder to launch the program (docker-compose up -d) . Upon navigating to port 80 and going to Install Pipeline Elements, the search for pipeline elements never ends. This also happens to be the same problem that I had when I tried through the developer tutorial at this website: https://streampipes.apache.org/docs/extend-cli/
At the moment, I am a little stumped because the project is new to me. I also am unable to work on one of the issues because I would be unable to test the code. I was able to see that the server was returning an error code with number 500. I found a consistent error through Docker Desktop in the Streampipe service titled, "streampipes-extensions-all-jvm-1". Here is the error:
2023-09-28 18:57:38 2023-09-28T22:57:38.142Z ERROR 1 --- [ main] o.a.s.s.e.connect.ConnectRestClient : Could not register adapter at url - is a 'StreamPipes Core' service running?
2023-09-28 18:57:38
2023-09-28 18:57:38 org.apache.streampipes.commons.exceptions.SpHttpErrorStatusCode: 500 -
2023-09-28 18:57:38 at org.apache.streampipes.client.http.HttpRequest.executeRequest(HttpRequest.java:113)
2023-09-28 18:57:38 at org.apache.streampipes.client.api.AbstractClientApi.post(AbstractClientApi.java:46)
2023-09-28 18:57:38 at org.apache.streampipes.client.api.AdminApi.registerAdapters(AdminApi.java:36)
2023-09-28 18:57:38 at org.apache.streampipes.service.extensions.connect.ConnectRestClient.register(ConnectRestClient.java:38)
2023-09-28 18:57:38 at org.apache.streampipes.service.extensions.connect.ConnectWorkerRegistrationService.registerWorker(ConnectWorkerRegistrationService.java:35)
2023-09-28 18:57:38 at org.apache.streampipes.service.extensions.ExtensionsModelSubmitter.afterServiceRegistered(ExtensionsModelSubmitter.java:53)
2023-09-28 18:57:38 at org.apache.streampipes.service.extensions.StreamPipesExtensionsServiceBase.startExtensionsService(StreamPipesExtensionsServiceBase.java:77)
2023-09-28 18:57:38 at org.apache.streampipes.service.extensions.StreamPipesExtensionsServiceBase.init(StreamPipesExtensionsServiceBase.java:54)
2023-09-28 18:57:38 at org.apache.streampipes.service.extensions.StreamPipesExtensionsServiceBase.init(StreamPipesExtensionsServiceBase.java:44)
2023-09-28 18:57:38 at org.apache.streampipes.extensions.all.jvm.AllExtensionsInit.main(AllExtensionsInit.java:38)
2023-09-28 18:57:38 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2023-09-28 18:57:38 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
2023-09-28 18:57:38 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
2023-09-28 18:57:38 at java.base/java.lang.reflect.Method.invoke(Unknown Source)
2023-09-28 18:57:38 at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
2023-09-28 18:57:38 at org.springframework.boot.loader.Launcher.launch(Launcher.java:95)
2023-09-28 18:57:38 at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
2023-09-28 18:57:38 at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65)
2023-09-28 18:57:38
2023-09-28 18:57:38 2023-09-28T22:57:38.142Z INFO 1 --- [ main] s.s.e.c.ConnectWorkerRegistrationService : Retrying in 5 seconds
2023-09-28 18:57:43 2023-09-28T22:57:43.276Z ERROR 1 --- [ main] o.a.s.s.e.connect.ConnectRestClient : Could not register adapter at url - is a 'StreamPipes Core' service running?
2023-09-28 18:57:43
2023-09-28 18:57:43 org.apache.streampipes.commons.exceptions.SpHttpErrorStatusCode: 500 -
2023-09-28 18:57:43 at org.apache.streampipes.client.http.HttpRequest.executeRequest(HttpRequest.java:113)
2023-09-28 18:57:43 at org.apache.streampipes.client.api.AbstractClientApi.post(AbstractClientApi.java:46)
2023-09-28 18:57:43 at org.apache.streampipes.client.api.AdminApi.registerAdapters(AdminApi.java:36)
2023-09-28 18:57:43 at org.apache.streampipes.service.extensions.connect.ConnectRestClient.register(ConnectRestClient.java:38)
2023-09-28 18:57:43 at org.apache.streampipes.service.extensions.connect.ConnectWorkerRegistrationService.registerWorker(ConnectWorkerRegistrationService.java:35)
2023-09-28 18:57:43 at org.apache.streampipes.service.extensions.ExtensionsModelSubmitter.afterServiceRegistered(ExtensionsModelSubmitter.java:53)
2023-09-28 18:57:43 at org.apache.streampipes.service.extensions.StreamPipesExtensionsServiceBase.startExtensionsService(StreamPipesExtensionsServiceBase.java:77)
2023-09-28 18:57:43 at org.apache.streampipes.service.extensions.StreamPipesExtensionsServiceBase.init(StreamPipesExtensionsServiceBase.java:54)
2023-09-28 18:57:43 at org.apache.streampipes.service.extensions.StreamPipesExtensionsServiceBase.init(StreamPipesExtensionsServiceBase.java:44)
2023-09-28 18:57:43 at org.apache.streampipes.extensions.all.jvm.AllExtensionsInit.main(AllExtensionsInit.java:38)
2023-09-28 18:57:43 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2023-09-28 18:57:43 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
2023-09-28 18:57:43 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
2023-09-28 18:57:43 at java.base/java.lang.reflect.Method.invoke(Unknown Source)
2023-09-28 18:57:43 at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
2023-09-28 18:57:43 at org.springframework.boot.loader.Launcher.launch(Launcher.java:95)
2023-09-28 18:57:43 at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
2023-09-28 18:57:43 at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65)
2023-09-28 18:57:43
2023-09-28 18:57:43 2023-09-28T22:57:43.276Z INFO 1 --- [ main] s.s.e.c.ConnectWorkerRegistrationService : Retrying in 5 seconds
I am not entirely sure what it means, but I think this may help whoever is more knowledgeable than I to reproduce the error.
Thanks everyone.
### How to reproduce?
Download ApacheStreampipes 0.92.0 and navigate to the "Install Pipeline Elements"
### Expected behavior
The pipelines should have showed up after a brief search.
### Additional technical information
Windows 11, WSL, Docker (recently updated). Apache Streampipes 0.92.0
### Are you willing to submit a PR?
None
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampipes.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Program continually searches for Pipeline Elements in the "Install Pipeline Elements" tab
Posted by "dshunter107 (via GitHub)" <gi...@apache.org>.
dshunter107 commented on issue #1975:
URL: https://github.com/apache/streampipes/issues/1975#issuecomment-1751709118
{
"Id": "3cdc2f44768a6cd05c994133cc5468da5392214342a17df995f68e8a23df7a04",
"Created": "2023-10-07T12:47:54.314898372Z",
"Path": "consul",
"Args": [
"agent",
"-server",
"-bootstrap-expect=1",
"-data-dir=/consul/data",
"-node=consul-one",
"-bind={{ GetInterfaceIP \"eth0\" }}",
"-client=0.0.0.0",
"-enable-script-checks=true",
"-ui"
],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 5980,
"ExitCode": 0,
"Error": "",
"StartedAt": "2023-10-07T12:47:57.987159823Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "sha256:8b6c5f52aa821efcdcc3dbd909b75bba742e8ddc84df534842cb1f4b356f16ad",
"ResolvConfPath": "/var/lib/docker/containers/3cdc2f44768a6cd05c994133cc5468da5392214342a17df995f68e8a23df7a04/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/3cdc2f44768a6cd05c994133cc5468da5392214342a17df995f68e8a23df7a04/hostname",
"HostsPath": "/var/lib/docker/containers/3cdc2f44768a6cd05c994133cc5468da5392214342a17df995f68e8a23df7a04/hosts",
"LogPath": "/var/lib/docker/containers/3cdc2f44768a6cd05c994133cc5468da5392214342a17df995f68e8a23df7a04/3cdc2f44768a6cd05c994133cc5468da5392214342a17df995f68e8a23df7a04-json.log",
"Name": "/streampipes-consul-1",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": null,
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {
"max-file": "5",
"max-size": "12m"
}
},
"NetworkMode": "streampipes_spnet",
"PortBindings": {},
"RestartPolicy": {
"Name": "unless-stopped",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"ConsoleSize": [
0,
0
],
"CapAdd": null,
"CapDrop": null,
"CgroupnsMode": "host",
"Dns": null,
"DnsOptions": null,
"DnsSearch": null,
"ExtraHosts": [],
"GroupAdd": null,
"IpcMode": "private",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": false,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": null,
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"Isolation": "",
"CpuShares": 0,
"Memory": 0,
"NanoCpus": 0,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": null,
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": null,
"DeviceCgroupRules": null,
"DeviceRequests": null,
"MemoryReservation": 0,
"MemorySwap": 0,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": null,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"Mounts": [
{
"Type": "volume",
"Source": "streampipes_consul",
"Target": "/consul/data",
"VolumeOptions": {}
}
],
"MaskedPaths": [
"/proc/asound",
"/proc/acpi",
"/proc/kcore",
"/proc/keys",
"/proc/latency_stats",
"/proc/timer_list",
"/proc/timer_stats",
"/proc/sched_debug",
"/proc/scsi",
"/sys/firmware"
],
"ReadonlyPaths": [
"/proc/bus",
"/proc/fs",
"/proc/irq",
"/proc/sys",
"/proc/sysrq-trigger"
]
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/8cc0adaeb6f9ca7a65f2d6a01bfa06db7ef2b2cfac004dcc3b9a4f7472298a8d-init/diff:/var/lib/docker/overlay2/6ea10f1a3230f1e4a0e9bb4d95c368982a51d237e7802aef7aeb4557771c28b8/diff:/var/lib/docker/overlay2/c2799471936db975d6cba54def9b2f623ea67c686beb3196d042beb516b87a87/diff:/var/lib/docker/overlay2/57c50df5645c86c45e3272de1c134b9e54be8002e82d753e451af3a1a1e9ffa2/diff:/var/lib/docker/overlay2/5a74814e3472dfcb81d4a685ed2017202641c2073cc5b0e9841ec8e238cceb9a/diff:/var/lib/docker/overlay2/0271334f96572d1148a970332bf52117973baa06d319deb5f0fc5c98d0ecc3e2/diff:/var/lib/docker/overlay2/9227566736a7517381f9f4ef386c823bbf531e513bdbfa208008523d1480abe3/diff",
"MergedDir": "/var/lib/docker/overlay2/8cc0adaeb6f9ca7a65f2d6a01bfa06db7ef2b2cfac004dcc3b9a4f7472298a8d/merged",
"UpperDir": "/var/lib/docker/overlay2/8cc0adaeb6f9ca7a65f2d6a01bfa06db7ef2b2cfac004dcc3b9a4f7472298a8d/diff",
"WorkDir": "/var/lib/docker/overlay2/8cc0adaeb6f9ca7a65f2d6a01bfa06db7ef2b2cfac004dcc3b9a4f7472298a8d/work"
},
"Name": "overlay2"
},
"Mounts": [
{
"Type": "volume",
"Name": "streampipes_consul",
"Source": "/var/lib/docker/volumes/streampipes_consul/_data",
"Destination": "/consul/data",
"Driver": "local",
"Mode": "z",
"RW": true,
"Propagation": ""
}
],
"Config": {
"Hostname": "3cdc2f44768a",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": true,
"AttachStderr": true,
"ExposedPorts": {
"8300/tcp": {},
"8301/tcp": {},
"8301/udp": {},
"8302/tcp": {},
"8302/udp": {},
"8500/tcp": {},
"8600/tcp": {},
"8600/udp": {}
},
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"CONSUL_LOCAL_CONFIG={\"disable_update_check\": true}",
"CONSUL_BIND_INTERFACE=eth0",
"CONSUL_HTTP_ADDR=0.0.0.0",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"HASHICORP_RELEASES=https://releases.hashicorp.com"
],
"Cmd": null,
"Image": "consul:1.14.3",
"Volumes": {
"/consul/data": {}
},
"WorkingDir": "",
"Entrypoint": [
"consul",
"agent",
"-server",
"-bootstrap-expect=1",
"-data-dir=/consul/data",
"-node=consul-one",
"-bind={{ GetInterfaceIP \"eth0\" }}",
"-client=0.0.0.0",
"-enable-script-checks=true",
"-ui"
],
"OnBuild": null,
"Labels": {
"com.docker.compose.config-hash": "348b8b37dd6f34e6b208198389d4b71ebb157b4a2cdfbbb2c5197671345ec16f",
"com.docker.compose.container-number": "1",
"com.docker.compose.depends_on": "",
"com.docker.compose.image": "sha256:8b6c5f52aa821efcdcc3dbd909b75bba742e8ddc84df534842cb1f4b356f16ad",
"com.docker.compose.oneoff": "False",
"com.docker.compose.project": "streampipes",
"com.docker.compose.project.config_files": "C:\\users\\david\\Software Projects\\apache-streampipes-0.92.0\\installer\\compose\\docker-compose.yml",
"com.docker.compose.project.working_dir": "C:\\users\\david\\Software Projects\\apache-streampipes-0.92.0\\installer\\compose",
"com.docker.compose.service": "consul",
"com.docker.compose.version": "2.22.0",
"org.opencontainers.image.authors": "Consul Team <co...@hashicorp.com>",
"org.opencontainers.image.description": "Consul is a datacenter runtime that provides service discovery, configuration, and orchestration.",
"org.opencontainers.image.documentation": "https://www.consul.io/docs",
"org.opencontainers.image.source": "https://github.com/hashicorp/consul",
"org.opencontainers.image.title": "consul",
"org.opencontainers.image.url": "https://www.consul.io/",
"org.opencontainers.image.vendor": "HashiCorp",
"org.opencontainers.image.version": "1.14.3"
}
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "b30f9134bcc5309b8a9b5484a7823e32b32293d7ce6610f811c03eefc8a8d7dc",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"8300/tcp": null,
"8301/tcp": null,
"8301/udp": null,
"8302/tcp": null,
"8302/udp": null,
"8500/tcp": null,
"8600/tcp": null,
"8600/udp": null
},
"SandboxKey": "/var/run/docker/netns/b30f9134bcc5",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"streampipes_spnet": {
"IPAMConfig": {
"IPv4Address": "172.31.0.9"
},
"Links": null,
"Aliases": [
"streampipes-consul-1",
"consul",
"3cdc2f44768a"
],
"NetworkID": "4e27ccc0a6ae2d860b4e4af1a4ae1a8f3ca3cb4f09079e03a619035c41e43811",
"EndpointID": "08d3479ffee4eda4b4490a9378e7de2dd14a3ba09616f43fb4bdb4f2461f4e22",
"Gateway": "172.31.0.1",
"IPAddress": "172.31.0.9",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:ac:1f:00:09",
"DriverOpts": null
}
}
}
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Program continually searches for Pipeline Elements in the "Install Pipeline Elements" tab
Posted by "dominikriemer (via GitHub)" <gi...@apache.org>.
dominikriemer commented on issue #1975:
URL: https://github.com/apache/streampipes/issues/1975#issuecomment-1754463437
Hi @dshunter107 thanks for providing the logs.
I'm unfortunately still unsure what causes the error...I would advice the following for troubleshooting:
* When you clean the initial version, did you also remove all volumes? Maybe it's worth trying to run `docker-compose down -v` and then check if there is no volume present with `docker volume ls`.
* If the problem reoccurs after restarting StreamPipes (the logs of `streampipes-backend` should show some log output referring to the initial setup), I'd suggest to check if the individual containers can communicate in the Docker network.
* For that, you can attach to e.g., the extensions container and check if you can ping the backend container.
I hope this helps us to further track down the problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Program continually searches for Pipeline Elements in the "Install Pipeline Elements" tab
Posted by "dominikriemer (via GitHub)" <gi...@apache.org>.
dominikriemer commented on issue #1975:
URL: https://github.com/apache/streampipes/issues/1975#issuecomment-1747507754
Hi @dshunter107 thanks for providing the logs!
I'm not 100% sure what's the reason for the error you get, but it seems the extensions service uses a wrong hostname to connect to the backend.
I would suggest that you stop StreamPipes, clean the volumes (to make sure there is no clash between the CLI version and the docker-compose version since the CLI overrides some environment variables) and then try to start the plain docker-compose at first. This shouldn't cause any problem.
If you still get an error, I would suggest to check the registered IPs of the core and extensions service in Consul (check the UI running on port 8500).
I'm pretty sure we can get this resolved, sorry for the problems ;-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Program continually searches for Pipeline Elements in the "Install Pipeline Elements" tab
Posted by "dshunter107 (via GitHub)" <gi...@apache.org>.
dshunter107 closed issue #1975: Program continually searches for Pipeline Elements in the "Install Pipeline Elements" tab
URL: https://github.com/apache/streampipes/issues/1975
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Program continually searches for Pipeline Elements in the "Install Pipeline Elements" tab
Posted by "dshunter107 (via GitHub)" <gi...@apache.org>.
dshunter107 commented on issue #1975:
URL: https://github.com/apache/streampipes/issues/1975#issuecomment-1787452023
hello. You were correct. The problem is fixed. I needed to remove the volumes. Once I removed them, everything works as it is supposed to. Thanks. I can now visualize what apache streampipes is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Program continually searches for Pipeline Elements in the "Install Pipeline Elements" tab
Posted by "dshunter107 (via GitHub)" <gi...@apache.org>.
dshunter107 commented on issue #1975:
URL: https://github.com/apache/streampipes/issues/1975#issuecomment-1742253601
I was using the CLI before doing the docker-compose file. I was trying to work on the main branch at the time. I then download the 0.92.0 release version from the website and tried the docker-compose from those files.
Here is the beginnning of the log file for streampipes-backend:
2023-10-01 20:12:33 2023-10-02T00:12:32.978Z INFO --- [ main] o.a.s.commons.networking.Networking : Using auto-discovered IP: 172.31.0.7
2023-10-01 20:12:33 2023-10-02T00:12:33.025Z INFO --- [ main] o.a.s.commons.networking.Networking : Using default port: 8030
2023-10-01 20:12:33 2023-10-02T00:12:33.101Z INFO --- [ main] o.a.s.s.consul.ConsulProvider : Checking if consul is available on host consul and port 8500
2023-10-01 20:12:33 2023-10-02T00:12:33.354Z INFO --- [ main] o.a.s.s.consul.ConsulProvider : Successfully connected to Consul on host consul
2023-10-01 20:12:38 2023-10-02T00:12:38.672Z INFO --- [ main] o.a.s.s.consul.SpConsulServiceDiscovery : Successfully registered service at Consul: core-DwrAvK
2023-10-01 20:12:42 _______ __ ______ __
2023-10-01 20:12:42 | __| |_.----.-----.---.-.--------.| __ \__|.-----.-----.-----.
2023-10-01 20:12:42 |__ | _| _| -__| _ | || __/ || _ | -__|__ --|
2023-10-01 20:12:42 |_______|____|__| |_____|___._|__|__|__||___| |__|| __|_____|_____|
2023-10-01 20:12:42 |__|
2023-10-01 20:12:42 ** StreamPipes Core **
2023-10-01 20:12:42
2023-10-01 20:12:44 2023-10-02T00:12:44.041Z INFO 1 --- [ main] o.a.s.s.c.StreamPipesBackendApplication : Starting StreamPipesBackendApplication v0.92.0 using Java 17.0.7 with PID 1 (/streampipes-backend.jar started by root in /)
2023-10-01 20:12:44 2023-10-02T00:12:44.061Z INFO 1 --- [ main] o.a.s.s.c.StreamPipesBackendApplication : No active profile set, falling back to 1 default profile: "default"
2023-10-01 20:13:07 2023-10-02T00:13:07.273Z INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8030 (http)
2023-10-01 20:13:07 2023-10-02T00:13:07.466Z INFO 1 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
2023-10-01 20:13:07 2023-10-02T00:13:07.469Z INFO 1 --- [ main] o.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/10.1.8]
2023-10-01 20:13:08 2023-10-02T00:13:08.062Z INFO 1 --- [ main] o.a.c.c.C.[.[.[/streampipes-backend] : Initializing Spring embedded WebApplicationContext
2023-10-01 20:13:08 2023-10-02T00:13:08.076Z INFO 1 --- [ main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 22287 ms
2023-10-01 20:13:11 2023-10-02T00:13:11.776Z INFO 1 --- [ main] o.a.s.s.core.StreamPipesEnvChecker : Checking and updating environment variables...
2023-10-01 20:13:13 2023-10-02T00:13:13.141Z INFO 1 --- [ main] o.a.s.s.core.StreamPipesEnvChecker : No JWT signing mode provided (using default settings), consult the docs to learn how to provide JWT settings
2023-10-01 20:13:13 2023-10-02T00:13:13.146Z INFO 1 --- [ main] o.a.s.s.core.StreamPipesEnvChecker : Updating local auth config with signing mode HMAC
2023-10-01 20:13:13 2023-10-02T00:13:13.326Z INFO 1 --- [ main] o.a.s.s.c.utils.CouchDbViewGenerator : Checking if generic database genericstorage exists...
2023-10-01 20:13:14 2023-10-02T00:13:14.182Z INFO 1 --- [ main] o.a.s.s.c.utils.CouchDbViewGenerator : Database genericstorage already present
2023-10-01 20:13:14 2023-10-02T00:13:14.228Z INFO 1 --- [ main] o.a.s.s.c.migrations.MigrationsHandler : Checking for required migrations...
2023-10-01 20:13:16 2023-10-02T00:13:16.300Z INFO 1 --- [ main] o.a.s.s.c.migrations.MigrationsHandler : All migrations completed.
2023-10-01 20:13:16 2023-10-02T00:13:16.320Z INFO 1 --- [ main] o.a.s.s.c.StreamPipesBackendApplication : Pipeline health check will run every 60 seconds
2023-10-01 20:13:16 2023-10-02T00:13:16.362Z INFO 1 --- [ main] o.a.s.s.c.StreamPipesBackendApplication : Extensions logs will be fetched every 60 seconds
2023-10-01 20:13:19 2023-10-02T00:13:19.364Z INFO 1 --- [ main] o.s.b.a.w.s.WelcomePageHandlerMapping : Adding welcome page: class path resource [static/index.html]
2023-10-01 20:13:20 2023-10-02T00:13:20.237Z INFO 1 --- [ main] o.s.s.web.DefaultSecurityFilterChain : Will secure any request with [org.springframework.security.web.session.DisableEncodeUrlFilter@6ad6fa53, org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter@55ea2d70, org.springframework.security.web.context.SecurityContextHolderFilter@469d003c, org.springframework.security.web.header.HeaderWriterFilter@5a101b1c, org.springframework.web.filter.CorsFilter@5e742e4, org.springframework.security.web.authentication.logout.LogoutFilter@757529a4, org.apache.streampipes.rest.filter.TokenAuthenticationFilter@175acfb2, org.springframework.security.web.savedrequest.RequestCacheAwareFilter@57bd2029, org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter@203dd56b, org.springframework.security.web.authentication.AnonymousAuthenticationFilter@4a03bb68, org.springframework.security.web.session.SessionManagementFilter@
29f0802c, org.springframework.security.web.access.ExceptionTranslationFilter@1f12e153, org.springframework.security.web.access.intercept.AuthorizationFilter@69cac930]
2023-10-01 20:13:21 2023-10-02T00:13:21.305Z INFO 1 --- [pool-1-thread-1] o.a.s.s.c.StreamPipesBackendApplication : Checking for orphaned pipelines...
2023-10-01 20:13:21 2023-10-02T00:13:21.444Z INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8030 (http) with context path '/streampipes-backend'
2023-10-01 20:13:21 2023-10-02T00:13:21.476Z INFO 1 --- [pool-1-thread-1] o.a.s.s.c.StreamPipesBackendApplication : Found 0 orphaned pipelines
2023-10-01 20:13:21 2023-10-02T00:13:21.480Z INFO 1 --- [pool-1-thread-1] o.a.s.s.c.StreamPipesBackendApplication : Checking for gracefully shut down pipelines to be restarted...
2023-10-01 20:13:21 2023-10-02T00:13:21.523Z INFO 1 --- [pool-1-thread-1] o.a.s.s.c.StreamPipesBackendApplication : Found 0 pipelines that we are attempting to restart...
2023-10-01 20:13:21 2023-10-02T00:13:21.526Z INFO 1 --- [pool-1-thread-1] o.a.s.s.c.StreamPipesBackendApplication : No more pipelines to restore...
2023-10-01 20:13:21 2023-10-02T00:13:21.578Z INFO 1 --- [ main] o.a.s.s.c.StreamPipesBackendApplication : Started StreamPipesBackendApplication in 41.94 seconds (process running for 60.649)
After this errors were thrown. I believe that the backend should point to localhost, as I am not sure what the current address is. I think I saw on the cli that this was a common issue and that I need to manually change the ip address. I will look into the manual to see what I need to do. Any advise in the meantime would also be appreciated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Program continually searches for Pipeline Elements in the "Install Pipeline Elements" tab
Posted by "bossenti (via GitHub)" <gi...@apache.org>.
bossenti commented on issue #1975:
URL: https://github.com/apache/streampipes/issues/1975#issuecomment-1790117618
great to hear @dshunter107!
If you have any feedback or further questions feel free to approach us 🙂
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Program continually searches for Pipeline Elements in the "Install Pipeline Elements" tab
Posted by "dominikriemer (via GitHub)" <gi...@apache.org>.
dominikriemer commented on issue #1975:
URL: https://github.com/apache/streampipes/issues/1975#issuecomment-1742159080
Hi @dshunter107 thanks for reporting the issue!
Were you using the CLI before starting the standard `docker-compose` file?
The error you are reporting appears when the extension service can't find the core/backend service. When having a look at the beginning of the logs of the `streampipes-backend` container (there should be some info on the hostname/IP used to register the core) - is the address reported here some resolvable IP or something like `localhost`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Program continually searches for Pipeline Elements in the "Install Pipeline Elements" tab
Posted by "dshunter107 (via GitHub)" <gi...@apache.org>.
dshunter107 commented on issue #1975:
URL: https://github.com/apache/streampipes/issues/1975#issuecomment-1751708952
hey @dominikriemer. Thanks for responding. I am still having the problem. I used mvn clean on the eclipse development version, I also deleted that version from my hard drive and reinstalled it. Then for the 0.92.0 version, I fully stopped all the containers with docker-compose down. I even verified that the containers were down with docker desktop. Unfortunately, the problem is still there: the install pipeline elements will not load.
On my computer, the Consul is not on port 8500. I tried looking at netstat (with the options) to see if anything was running there, but nothing was running on port 8500. Surprisingly, with the command "netstat -a -n -o", I saw that a program with a name "com.docker.backend.exe" was running on two ports: (1) [0.0.0.0]:80 and (2) [172.0.0.1]:6443. I checked the logs for Consul, and I found the following at the start:
2023-10-07 08:48:01 ==> Starting Consul agent...
2023-10-07 08:48:01 Version: '1.14.3'
2023-10-07 08:48:01 Build Date: '2022-12-13 17:13:55 +0000 UTC'
2023-10-07 08:48:01 Node ID: '70e75ae4-42b9-07e1-a880-61ce407198bd'
2023-10-07 08:48:01 Node name: 'consul-one'
2023-10-07 08:48:01 Datacenter: 'dc1' (Segment: '<all>')
2023-10-07 08:48:01 Server: true (Bootstrap: true)
2023-10-07 08:48:01 Client Addr: [0.0.0.0] (HTTP: 8500, HTTPS: -1, gRPC: -1, gRPC-TLS: 8503, DNS: 8600)
2023-10-07 08:48:01 Cluster Addr: 172.31.0.9 (LAN: 8301, WAN: 8302)
2023-10-07 08:48:01 Gossip Encryption: false
2023-10-07 08:48:01 Auto-Encrypt-TLS: false
2023-10-07 08:48:01 HTTPS TLS: Verify Incoming: false, Verify Outgoing: false, Min Version: TLSv1_2
2023-10-07 08:48:01 gRPC TLS: Verify Incoming: false, Min Version: TLSv1_2
2023-10-07 08:48:01 Internal RPC TLS: Verify Incoming: false, Verify Outgoing: false (Verify Hostname: false), Min Version: TLSv1_2
2023-10-07 08:48:01
2023-10-07 08:48:01 ==> Log data will now stream in as it occurs:
2023-10-07 08:48:01
2023-10-07 08:48:01 2023-10-07T12:48:01.531Z [WARN] agent: BootstrapExpect is set to 1; this is the same as Bootstrap mode.
2023-10-07 08:48:01 2023-10-07T12:48:01.531Z [WARN] agent: bootstrap = true: do not enable unless necessary
2023-10-07 08:48:01 2023-10-07T12:48:01.531Z [WARN] agent: using enable-script-checks without ACLs and without allow_write_http_from is DANGEROUS, use enable-local-script-checks instead, see https://www.hashicorp.com/blog/protecting-consul-from-rce-risk-in-specific-configurations/
2023-10-07 08:48:01 2023-10-07T12:48:01.611Z [WARN] agent.auto_config: BootstrapExpect is set to 1; this is the same as Bootstrap mode.
2023-10-07 08:48:01 2023-10-07T12:48:01.611Z [WARN] agent.auto_config: bootstrap = true: do not enable unless necessary
2023-10-07 08:48:01 2023-10-07T12:48:01.611Z [WARN] agent.auto_config: using enable-script-checks without ACLs and without allow_write_http_from is DANGEROUS, use enable-local-script-checks instead, see https://www.hashicorp.com/blog/protecting-consul-from-rce-risk-in-specific-configurations/
2023-10-07 08:48:06 2023-10-07T12:48:06.159Z [INFO] agent.server.raft: initial configuration: index=1 servers="[{Suffrage:Voter ID:70e75ae4-42b9-07e1-a880-61ce407198bd Address:172.31.0.9:8300}]"
2023-10-07 08:48:06 2023-10-07T12:48:06.222Z [INFO] agent.server.raft: entering follower state: follower="Node at 172.31.0.9:8300 [Follower]" leader-address= leader-id=
2023-10-07 08:48:06 2023-10-07T12:48:06.417Z [INFO] agent.server.serf.wan: serf: EventMemberJoin: consul-one.dc1 172.31.0.9
2023-10-07 08:48:06 2023-10-07T12:48:06.438Z [WARN] agent.server.serf.wan: serf: Failed to re-join any previously known node
2023-10-07 08:48:06 2023-10-07T12:48:06.456Z [INFO] agent.server.serf.lan: serf: EventMemberJoin: consul-one 172.31.0.9
2023-10-07 08:48:06 2023-10-07T12:48:06.456Z [INFO] agent.router: Initializing LAN area manager
2023-10-07 08:48:06 2023-10-07T12:48:06.462Z [WARN] agent.server.serf.lan: serf: Failed to re-join any previously known node
2023-10-07 08:48:06 2023-10-07T12:48:06.523Z [INFO] agent.server: Adding LAN server: server="consul-one (Addr: tcp/172.31.0.9:8300) (DC: dc1)"
2023-10-07 08:48:06 2023-10-07T12:48:06.579Z [INFO] agent.server: Handled event for server in area: event=member-join server=consul-one.dc1 area=wan
2023-10-07 08:48:06 2023-10-07T12:48:06.581Z [INFO] agent.server.autopilot: reconciliation now disabled
2023-10-07 08:48:06 2023-10-07T12:48:06.638Z [INFO] agent.server.cert-manager: initialized server certificate management
2023-10-07 08:48:06 2023-10-07T12:48:06.734Z [INFO] agent: Started DNS server: address=0.0.0.0:8600 network=tcp
2023-10-07 08:48:06 2023-10-07T12:48:06.734Z [INFO] agent: Started DNS server: address=0.0.0.0:8600 network=udp
2023-10-07 08:48:06 2023-10-07T12:48:06.736Z [INFO] agent: Starting server: address=[::]:8500 network=tcp protocol=http
2023-10-07 08:48:06 2023-10-07T12:48:06.737Z [INFO] agent: Started gRPC listeners: port_name=grpc_tls address=[::]:8503 network=tcp
2023-10-07 08:48:06 2023-10-07T12:48:06.743Z [INFO] agent: started state syncer
2023-10-07 08:48:06 2023-10-07T12:48:06.743Z [INFO] agent: Consul agent running!
2023-10-07 08:48:07 2023-10-07T12:48:07.619Z [WARN] agent: Check is now critical: check=service:org.apache.streampipes.extensions.all.jvm-h4UVeS
2023-10-07 08:48:08 2023-10-07T12:48:08.448Z [WARN] agent: Check is now critical: check=service:org-apache-streampipes-sources-simulator-watertank-PeR5pW
2023-10-07 08:48:09 2023-10-07T12:48:09.935Z [WARN] agent: Check is now critical: check=service:core-Jz4rm8
2023-10-07 08:48:12 2023-10-07T12:48:12.254Z [WARN] agent: Check is now critical: check=service:org.apache.streampipes.extensions.all.jvm-pQdQJa
2023-10-07 08:48:13 2023-10-07T12:48:13.778Z [ERROR] agent.anti_entropy: failed to sync remote state: error="No cluster leader"
2023-10-07 08:48:13 2023-10-07T12:48:13.947Z [WARN] agent.cache: handling error in Cache.Notify: cache-type=connect-ca-leaf error="No cluster leader" index=0
2023-10-07 08:48:13 2023-10-07T12:48:13.949Z [ERROR] agent.server.cert-manager: failed to handle cache update event: error="leaf cert watch returned an error: No cluster leader"
2023-10-07 08:48:15 2023-10-07T12:48:15.601Z [WARN] agent.server.raft: heartbeat timeout reached, starting election: last-leader-addr= last-leader-id=
2023-10-07 08:48:15 2023-10-07T12:48:15.603Z [INFO] agent.server.raft: entering candidate state: node="Node at 172.31.0.9:8300 [Candidate]" term=22
2023-10-07 08:48:15 2023-10-07T12:48:15.667Z [INFO] agent.server.raft: election won: term=22 tally=1
2023-10-07 08:48:15 2023-10-07T12:48:15.668Z [INFO] agent.server.raft: entering leader state: leader="Node at 172.31.0.9:8300 [Leader]"
2023-10-07 08:48:15 2023-10-07T12:48:15.670Z [INFO] agent.server: cluster leadership acquired
2023-10-07 08:48:15 2023-10-07T12:48:15.676Z [INFO] agent.server: New leader elected: payload=consul-one
2023-10-07 08:48:16 2023-10-07T12:48:16.414Z [WARN] agent.cache: handling error in Cache.Notify: cache-type=connect-ca-leaf error="CA is uninitialized and unable to sign certificates yet: provider is nil" index=0
2023-10-07 08:48:16 2023-10-07T12:48:16.415Z [ERROR] agent.server.cert-manager: failed to handle cache update event: error="leaf cert watch returned an error: CA is uninitialized and unable to sign certificates yet: provider is nil"
2023-10-07 08:48:16 2023-10-07T12:48:16.734Z [INFO] agent.server.autopilot: reconciliation now enabled
2023-10-07 08:48:16 2023-10-07T12:48:16.747Z [INFO] agent.leader: started routine: routine="federation state anti-entropy"
2023-10-07 08:48:16 2023-10-07T12:48:16.747Z [INFO] agent.leader: started routine: routine="federation state pruning"
2023-10-07 08:48:16 2023-10-07T12:48:16.748Z [INFO] agent.leader: started routine: routine="streaming peering resources"
2023-10-07 08:48:16 2023-10-07T12:48:16.754Z [INFO] agent.leader: started routine: routine="metrics for streaming peering resources"
2023-10-07 08:48:16 2023-10-07T12:48:16.755Z [INFO] agent.leader: started routine: routine="peering deferred deletion"
2023-10-07 08:48:16 2023-10-07T12:48:16.774Z [WARN] agent.cache: handling error in Cache.Notify: cache-type=connect-ca-leaf error="CA is uninitialized and unable to sign certificates yet: no root certificate" index=0
2023-10-07 08:48:16 2023-10-07T12:48:16.774Z [ERROR] agent.server.cert-manager: failed to handle cache update event: error="leaf cert watch returned an error: CA is uninitialized and unable to sign certificates yet: no root certificate"
2023-10-07 08:48:16 2023-10-07T12:48:16.781Z [WARN] agent.cache: handling error in Cache.Notify: cache-type=connect-ca-leaf error="CA is uninitialized and unable to sign certificates yet: no root certificate" index=0
2023-10-07 08:48:16 2023-10-07T12:48:16.782Z [ERROR] agent.server.cert-manager: failed to handle cache update event: error="leaf cert watch returned an error: CA is uninitialized and unable to sign certificates yet: no root certificate"
2023-10-07 08:48:16 2023-10-07T12:48:16.792Z [INFO] connect.ca: initialized primary datacenter CA from existing CARoot with provider: provider=consul
2023-10-07 08:48:16 2023-10-07T12:48:16.792Z [INFO] agent.leader: started routine: routine="intermediate cert renew watch"
2023-10-07 08:48:16 2023-10-07T12:48:16.793Z [INFO] agent.leader: started routine: routine="CA root pruning"
2023-10-07 08:48:16 2023-10-07T12:48:16.793Z [INFO] agent.leader: started routine: routine="CA root expiration metric"
2023-10-07 08:48:16 2023-10-07T12:48:16.794Z [INFO] agent.leader: started routine: routine="CA signing expiration metric"
2023-10-07 08:48:16 2023-10-07T12:48:16.795Z [INFO] agent.leader: started routine: routine="virtual IP version check"
2023-10-07 08:48:16 2023-10-07T12:48:16.795Z [INFO] agent.leader: stopping routine: routine="virtual IP version check"
2023-10-07 08:48:16 2023-10-07T12:48:16.795Z [INFO] agent.leader: stopped routine: routine="virtual IP version check"
2023-10-07 08:48:17 2023-10-07T12:48:17.364Z [INFO] agent: Synced node info
2023-10-07 08:48:17 2023-10-07T12:48:17.627Z [WARN] agent: Check is now critical: check=service:org.apache.streampipes.extensions.all.jvm-h4UVeS
2023-10-07 08:48:18 2023-10-07T12:48:18.452Z [WARN] agent: Check is now critical: check=service:org-apache-streampipes-sources-simulator-watertank-PeR5pW
Afterwards, consul repeats that check is now critical. When I inspected streampipes-consul-1, I got the following
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org