You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@streampipes.apache.org by Florian Micklich <mi...@apache.org> on 2020/05/31 11:07:44 UTC
StreamPi
Hi Patrick,
I just read your awesome blog-post about using pi and k3s [1]
I am definitely thrilled and already planning to dust off my pi's.
I have so additional questions:
* Are you using a 64bit OS?
* Are you using a SD card and if so, do you think StreamPipes could
benefit from an external SSD?
* Pi-Foundation has just released 8 GB pi. Do you think that the
"normal" StreamPipes version could be possible in some way using
this little whopper (or whoppers in a cluster)? :)
* Can you describe how to setup the pi-cluster in general?
* Is this the goal? [2]. joking :-D
Greetings
Florian
[1]
https://streampipes.apache.org/docs/blog/2020/05/27/streampipes_on_k3s/
[2]
https://t3n.de/news/wp-content/uploads/2013/05/raspberry-pi-cluster-01.jpg
Re: StreamPi
Posted by Dominik Riemer <ri...@apache.org>.
Hi Florian,
it seems that the CouchDB database can't be reached - do you get any error in the logs or were you able to solve the problem?
Dominik
On 2020/07/18 10:47:27, Florian Micklich <fl...@disy.net.INVALID> wrote:
> Hi all,
>
> hurray. I have my own cluster running now.
>
> I am using 2 8GB Pi's running on an Ubuntu 64bit 20.04 LTS OS. I had to adjust the workflow to get the cluster run compared to this workflow [1].
>
>
> So now I started the cluster via
>
> helm install streampipes helm-chart
>
>
>
>
> and after a while the pods are running
>
>
> ubuntu@streampi-01:~$ kubectl get pods -o=custom-columns=NAME:.metadata.name,STATUS:.status.phase,NODE:.spec.nodeName
> NAME STATUS NODE
> zookeeper-6b4df686f5-49q2r Running streampi-02
> influxdb-5f749d85b9-8mw9k Running streampi-02
> consul-679d8df4d-pfr4w Running streampi-01
> sources-watertank-simulator-56946d58fd-98wqh Running streampi-02
> kafka-55d55bc7ff-59fq5 Running streampi-01
> couchdb-84d987b8b6-6zh7b Running streampi-01
> activemq-85b4fb77c7-frq87 Running streampi-01
> backend-7dd64c8ff6-zmfsc Running streampi-02
> connect-master-bf88d978f-26rsr Running streampi-01
> connect-worker-5496c5f5-cg7j4 Running streampi-02
> pipeline-elements-all-jvm-5d6b4b959f-62sgd Running streampi-01
> nginx-756bcf5ff8-zvktv Running streampi-02
>
>
> But after the setup page I get an exclamation mark for:
>
> preparing database "'users'...
>
> Prparing database 'connection'...
>
> preparing database 'notifications'...
>
> Installing pipeline elemets from http://pipeline-elements-all-jvm:8090
>
> Installing pipeline elemets from http:/source-watertank-simulator:8090
>
>
> If I want to login, I can't because of "The password you entered is incorrect. Please try again"
>
> On port 8090 and 5984 I get an connection error.
>
>
>
> So my first attempt is to use something like ./sp clean
>
> Is there something like this using helm?
>
>
> I am using the installer
>
> git clone -b dev --single-branch https://github.com/apache/incubator-streampipes-installer.git
>
> and
>
> ubuntu@streampi-01:~$ helm version
> version.BuildInfo{Version:"v3.1.2", GitCommit:"d878d4d45863e42fd5cff6743294a11d28a9abce", GitTreeState:"clean", GoVersion:"go1.13.8"}
>
> ubuntu@streampi-01:~$ kubectl version
> Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6+k3s1", GitCommit:"6f56fa1d68a5a48b8b6fdefa8eb7ead2015a4b3a", GitTreeState:"clean", BuildDate:"2020-07-16T20:44:09Z", GoVersion:"go1.13.11", Compiler:"gc", Platform:"linux/arm64"}
> Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6+k3s1", GitCommit:"6f56fa1d68a5a48b8b6fdefa8eb7ead2015a4b3a", GitTreeState:"clean", BuildDate:"2020-07-16T20:44:09Z", GoVersion:"go1.13.11", Compiler:"gc", Platform:"linux/arm64"}
>
>
> Next step is to try the full version mentioned in the Readme [2] after the light version is running.
>
>
> Greetings
>
> Florian
>
>
> [1] https://streampipes.apache.org/docs/blog/2020/05/27/streampipes_on_k3s/
>
> [2] https://github.com/apache/incubator-streampipes-installer/tree/dev/helm-chart
>
>
> Am 31.05.20 um 21:56 schrieb Patrick Wiener:
>
> Hi Florian,
>
> glad to hear that you liked it :) it was really fun to set it up, even tho it has not yet reached the size
> of the cluster you sent in the link ^^ (if ever).
>
> Regarding your questions:
>
>
>
> * Are you using a 64bit OS?
>
>
> I used the newest RPi 4 Model with 4 GB RAM. Haven’t tried on the old RPi 3 - however since we still
> use an average of around 1.8-2GiB of RAM for the lite version I doubt it’ll run and RPi 3 (currently).
>
>
>
> * Are you using a SD card and if so, do you think StreamPipes could
> benefit from an external SSD?
>
>
> used a plain SD card and don’t think it’ll give any more boost.
>
>
>
> * Pi-Foundation has just released 8 GB pi. Do you think that the
> "normal" StreamPipes version could be possible in some way using
> this little whopper (or whoppers in a cluster)? :)
>
>
> I saw that - really cool stuff. with „normal“ you mean the full version of the installer? If so, maybe you
> can try it out? :)
>
>
>
> * Can you describe how to setup the pi-cluster in general?
>
>
> You mean the hardware part? or software part? OS, Docker etc?
>
>
>
> * Is this the goal? [2]. joking :-D
>
>
> I guess no - even tho it would be kinda funny :)
>
>
> Cheers
> Patrick
>
>
>
>
> Am 31.05.2020 um 13:07 schrieb Florian Micklich <mi...@apache.org>:
>
> Hi Patrick,
>
> I just read your awesome blog-post about using pi and k3s [1]
> I am definitely thrilled and already planning to dust off my pi's.
>
> I have so additional questions:
>
> * Are you using a 64bit OS?
> * Are you using a SD card and if so, do you think StreamPipes could
> benefit from an external SSD?
> * Pi-Foundation has just released 8 GB pi. Do you think that the
> "normal" StreamPipes version could be possible in some way using
> this little whopper (or whoppers in a cluster)? :)
> * Can you describe how to setup the pi-cluster in general?
> * Is this the goal? [2]. joking :-D
>
>
> Greetings
> Florian
>
> [1]
> https://streampipes.apache.org/docs/blog/2020/05/27/streampipes_on_k3s/
>
> [2]
> https://t3n.de/news/wp-content/uploads/2013/05/raspberry-pi-cluster-01.jpg
>
>
>
>
>
>
>
> Disy Informationssysteme GmbH
> Florian Micklich
> Lösungsentwickler
> +49 721 16006 477, florian.micklich@disy.net
>
> Firmensitz: Ludwig-Erhard-Allee 6, 76131 Karlsruhe
> Registergericht: Amtsgericht Mannheim, HRB 107964
> Geschäftsführer: Claus Hofmann
>
> Bitte beachten Sie folgende Informationen für Kunden, Lieferanten und Bewerber
> - Datenschutz: www.disy.net/datenschutz
> - Informationspflichten: www.disy.net/informationspflichten
>
>
Re: StreamPi
Posted by Florian Micklich <fl...@disy.net.INVALID>.
Hi all,
hurray. I have my own cluster running now.
I am using 2 8GB Pi's running on an Ubuntu 64bit 20.04 LTS OS. I had to adjust the workflow to get the cluster run compared to this workflow [1].
So now I started the cluster via
helm install streampipes helm-chart
and after a while the pods are running
ubuntu@streampi-01:~$ kubectl get pods -o=custom-columns=NAME:.metadata.name,STATUS:.status.phase,NODE:.spec.nodeName
NAME STATUS NODE
zookeeper-6b4df686f5-49q2r Running streampi-02
influxdb-5f749d85b9-8mw9k Running streampi-02
consul-679d8df4d-pfr4w Running streampi-01
sources-watertank-simulator-56946d58fd-98wqh Running streampi-02
kafka-55d55bc7ff-59fq5 Running streampi-01
couchdb-84d987b8b6-6zh7b Running streampi-01
activemq-85b4fb77c7-frq87 Running streampi-01
backend-7dd64c8ff6-zmfsc Running streampi-02
connect-master-bf88d978f-26rsr Running streampi-01
connect-worker-5496c5f5-cg7j4 Running streampi-02
pipeline-elements-all-jvm-5d6b4b959f-62sgd Running streampi-01
nginx-756bcf5ff8-zvktv Running streampi-02
But after the setup page I get an exclamation mark for:
preparing database "'users'...
Prparing database 'connection'...
preparing database 'notifications'...
Installing pipeline elemets from http://pipeline-elements-all-jvm:8090
Installing pipeline elemets from http:/source-watertank-simulator:8090
If I want to login, I can't because of "The password you entered is incorrect. Please try again"
On port 8090 and 5984 I get an connection error.
So my first attempt is to use something like ./sp clean
Is there something like this using helm?
I am using the installer
git clone -b dev --single-branch https://github.com/apache/incubator-streampipes-installer.git
and
ubuntu@streampi-01:~$ helm version
version.BuildInfo{Version:"v3.1.2", GitCommit:"d878d4d45863e42fd5cff6743294a11d28a9abce", GitTreeState:"clean", GoVersion:"go1.13.8"}
ubuntu@streampi-01:~$ kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6+k3s1", GitCommit:"6f56fa1d68a5a48b8b6fdefa8eb7ead2015a4b3a", GitTreeState:"clean", BuildDate:"2020-07-16T20:44:09Z", GoVersion:"go1.13.11", Compiler:"gc", Platform:"linux/arm64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6+k3s1", GitCommit:"6f56fa1d68a5a48b8b6fdefa8eb7ead2015a4b3a", GitTreeState:"clean", BuildDate:"2020-07-16T20:44:09Z", GoVersion:"go1.13.11", Compiler:"gc", Platform:"linux/arm64"}
Next step is to try the full version mentioned in the Readme [2] after the light version is running.
Greetings
Florian
[1] https://streampipes.apache.org/docs/blog/2020/05/27/streampipes_on_k3s/
[2] https://github.com/apache/incubator-streampipes-installer/tree/dev/helm-chart
Am 31.05.20 um 21:56 schrieb Patrick Wiener:
Hi Florian,
glad to hear that you liked it :) it was really fun to set it up, even tho it has not yet reached the size
of the cluster you sent in the link ^^ (if ever).
Regarding your questions:
* Are you using a 64bit OS?
I used the newest RPi 4 Model with 4 GB RAM. Haven’t tried on the old RPi 3 - however since we still
use an average of around 1.8-2GiB of RAM for the lite version I doubt it’ll run and RPi 3 (currently).
* Are you using a SD card and if so, do you think StreamPipes could
benefit from an external SSD?
used a plain SD card and don’t think it’ll give any more boost.
* Pi-Foundation has just released 8 GB pi. Do you think that the
"normal" StreamPipes version could be possible in some way using
this little whopper (or whoppers in a cluster)? :)
I saw that - really cool stuff. with „normal“ you mean the full version of the installer? If so, maybe you
can try it out? :)
* Can you describe how to setup the pi-cluster in general?
You mean the hardware part? or software part? OS, Docker etc?
* Is this the goal? [2]. joking :-D
I guess no - even tho it would be kinda funny :)
Cheers
Patrick
Am 31.05.2020 um 13:07 schrieb Florian Micklich <mi...@apache.org>:
Hi Patrick,
I just read your awesome blog-post about using pi and k3s [1]
I am definitely thrilled and already planning to dust off my pi's.
I have so additional questions:
* Are you using a 64bit OS?
* Are you using a SD card and if so, do you think StreamPipes could
benefit from an external SSD?
* Pi-Foundation has just released 8 GB pi. Do you think that the
"normal" StreamPipes version could be possible in some way using
this little whopper (or whoppers in a cluster)? :)
* Can you describe how to setup the pi-cluster in general?
* Is this the goal? [2]. joking :-D
Greetings
Florian
[1]
https://streampipes.apache.org/docs/blog/2020/05/27/streampipes_on_k3s/
[2]
https://t3n.de/news/wp-content/uploads/2013/05/raspberry-pi-cluster-01.jpg
Disy Informationssysteme GmbH
Florian Micklich
Lösungsentwickler
+49 721 16006 477, florian.micklich@disy.net
Firmensitz: Ludwig-Erhard-Allee 6, 76131 Karlsruhe
Registergericht: Amtsgericht Mannheim, HRB 107964
Geschäftsführer: Claus Hofmann
Bitte beachten Sie folgende Informationen für Kunden, Lieferanten und Bewerber
- Datenschutz: www.disy.net/datenschutz
- Informationspflichten: www.disy.net/informationspflichten
Re: StreamPi
Posted by Patrick Wiener <wi...@apache.org>.
Hi Florian,
glad to hear that you liked it :) it was really fun to set it up, even tho it has not yet reached the size
of the cluster you sent in the link ^^ (if ever).
Regarding your questions:
> * Are you using a 64bit OS?
I used the newest RPi 4 Model with 4 GB RAM. Haven’t tried on the old RPi 3 - however since we still
use an average of around 1.8-2GiB of RAM for the lite version I doubt it’ll run and RPi 3 (currently).
> * Are you using a SD card and if so, do you think StreamPipes could
> benefit from an external SSD?
used a plain SD card and don’t think it’ll give any more boost.
> * Pi-Foundation has just released 8 GB pi. Do you think that the
> "normal" StreamPipes version could be possible in some way using
> this little whopper (or whoppers in a cluster)? :)
I saw that - really cool stuff. with „normal“ you mean the full version of the installer? If so, maybe you
can try it out? :)
> * Can you describe how to setup the pi-cluster in general?
You mean the hardware part? or software part? OS, Docker etc?
> * Is this the goal? [2]. joking :-D
I guess no - even tho it would be kinda funny :)
Cheers
Patrick
> Am 31.05.2020 um 13:07 schrieb Florian Micklich <mi...@apache.org>:
>
> Hi Patrick,
>
> I just read your awesome blog-post about using pi and k3s [1]
> I am definitely thrilled and already planning to dust off my pi's.
>
> I have so additional questions:
>
> * Are you using a 64bit OS?
> * Are you using a SD card and if so, do you think StreamPipes could
> benefit from an external SSD?
> * Pi-Foundation has just released 8 GB pi. Do you think that the
> "normal" StreamPipes version could be possible in some way using
> this little whopper (or whoppers in a cluster)? :)
> * Can you describe how to setup the pi-cluster in general?
> * Is this the goal? [2]. joking :-D
>
>
> Greetings
> Florian
>
> [1]
> https://streampipes.apache.org/docs/blog/2020/05/27/streampipes_on_k3s/
>
> [2]
> https://t3n.de/news/wp-content/uploads/2013/05/raspberry-pi-cluster-01.jpg
>