You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@syncope.apache.org by craig <cr...@gmail.com> on 2018/08/06 22:03:32 UTC
Kubernetes (Syncope Cannot talk to Postgres)
I am setting up syncope in Kubernetes. I will be happy to contribute my
yamls once I get it running as it doesn't seem to be a common setup for
Syncope. I am new to Syncope and I am having some issues.
I was able to get the docker-compose examples working just fine but having
connectivity issues when running in K8s. Without knowing Syncope (or even
postgres) that well I am struggling on where to start.
Issue: Syncope cannot connect to postgres:5432
20:49:13.640 ERROR
org.flowable.common.engine.impl.AbstractEngineConfiguration - Exception
while initializing Database connection
org.postgresql.util.PSQLException: Connection to postgres:5432 refused.
Check that the hostname and port are correct and that the postmaster is
accepting TCP/IP connections.
at
org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:245)
~[postgresql-42.2.2.jar:42.2.2]
at
org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
~[postgresql-42.2.2.jar:42.2.2]
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195)
~[postgresql-42.2.2.jar:42.2.2]
at org.postgresql.Driver.makeConnection(Driver.java:452)
~[postgresql-42.2.2.jar:42.2.2]
at org.postgresql.Driver.connect(Driver.java:254)
~[postgresql-42.2.2.jar:42.2.2]
at
com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:136)
~[HikariCP-3.2.0.jar:?]
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:369)
~[HikariCP-3.2.0.jar:?]
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:198)
~[HikariCP-3.2.0.jar:?]
at
com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:467)
~[HikariCP-3.2.0.jar:?]
at
com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:541)
~[HikariCP-3.2.0.jar:?]
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115)
~[HikariCP-3.2.0.jar:?]
at
com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
~[HikariCP-3.2.0.jar:?]
Things that I did:
1) Connect to database directly from postgres container command line "psql
-U syncope"
2) Connect to database from syncope conatiner command line "psql -U syncope
-h postgres"
3) Confirm that the port 5432 is open by running the command "telnet
postgres 5432" and it was open
4) Confirm that both a database named "syncope" and user named "syncope"
exist in the postgres database
5) Confirmed the configuration of K8s looks correct. The port appears to
be up, the replica sets look correct
Any help or guidance on things to look at would be helpful.
Craig
--
Sent from: http://syncope-user.1051894.n5.nabble.com/
Re: Kubernetes (Syncope Cannot talk to Postgres)
Posted by Craig Martin <cr...@gmail.com>.
Thank you all, I have a bit more information. It isn't working perfectly
yet but I am pretty close. The issue that I was having before appeared to
be related to running postgres as a statefulset in K8s (mulitple pods
working together) and setting up postgres with a master--> Slave model.
I was able to get syncope running just fine if I had a single primary
database. This is a good sign but not good enough for an actual
production install. There were two main issues:
1. No backups or replication which isn't realistic for a production
install
2. For some reason, if I do manually scale up (or restart) the database
then Syncope (and also console and enduser) crashes and burns. I had to
restart them as well for it to pick up the new database. This is
obviously an untenable solution as K8s is built around the idea of
ephemeral pods.
3. Syncope is very slow and a bit of a memory hog - I had to throw a
bunch of memory and CPU at it to get it to be performant. Not a big deal
but certainly not ideal to have such large docker containers.
None the less, I appear to be making progress. Any ideas would be helpful
but I will keep on pushing. I do have a holiday starting tomorrow so it
will be a couple weeks before I can pick this back up.
Thank you all for your help so far.
Craig
On Tue, Aug 7, 2018 at 1:16 AM Andrea Patricelli <
andreapatricelli@apache.org> wrote:
> Hi Craig,
>
>
> Il 07/08/2018 08:30, Francesco Chicchiriccò ha scritto:
> > On 07/08/2018 00:03, craig wrote:
> >> I am setting up syncope in Kubernetes. I will be happy to
> >> contribute my
> >> yamls once I get it running as it doesn't seem to be a common setup for
> >> Syncope.
> >
> > This sounds great, it would be a great addition, maybe to place right
> > after
> >
> >
> https://syncope.apache.org/docs/getting-started.html#docker-compose-samples
> >
> >
> >> I am new to Syncope and I am having some issues.
> >>
> >> I was able to get the docker-compose examples working just fine but
> >> having
> >> connectivity issues when running in K8s. Without knowing Syncope
> >> (or even
> >> postgres) that well I am struggling on where to start.
> >>
> >> Issue: Syncope cannot connect to postgres:5432
> >>
> >> 20:49:13.640 ERROR
> >> org.flowable.common.engine.impl.AbstractEngineConfiguration - Exception
> >> while initializing Database connection
> >> org.postgresql.util.PSQLException: Connection to postgres:5432 refused.
> >> Check that the hostname and port are correct and that the postmaster is
> >> accepting TCP/IP connections.
> >> [...]
>
> Did you set a password for the user "syncope" like explained at [1]?
>
> Is connectivity on protocol TCP allowed on your postgres instance?
> BTW I found something (maybe) useful for you at [2].
>
> >
> > Were the Syncope tables (SyncopeUser, for example) created, in the
> > database? Were the Flowable (e.g. ACT_*) and Quartz (e.g. QRTZ_*)
> > created as well?
> >
> > Did you setup any connection control on the postgresql container?
> >
> >> Things that I did:
> >>
> >> 1) Connect to database directly from postgres container command line
> >> "psql
> >> -U syncope"
> >> 2) Connect to database from syncope container command line "psql -U
> >> syncope
> >> -h postgres"
> >
> > Since this works, I cannot figure out why you get the exception above...
> >
> >> 3) Confirm that the port 5432 is open by running the command "telnet
> >> postgres 5432" and it was open
> >> 4) Confirm that both a database named "syncope" and user named
> >> "syncope"
> >> exist in the postgres database
> >> 5) Confirmed the configuration of K8s looks correct. The port
> >> appears to
> >> be up, the replica sets look correct
> >>
> >> Any help or guidance on things to look at would be helpful.
> >>
> >> Craig
> >
>
> HTH,
> Andrea
>
> [1] https://syncope.apache.org/docs/reference-guide.html#postgresql
> [2]
>
> https://blog.bigbinary.com/2016/01/23/configure-postgresql-to-allow-remote-connection.html
>
> --
> Dott. Andrea Patricelli
> Tel. +39 3204524292
>
> Developer @ Tirasa S.r.l.
> Viale D'Annunzio 267 - 65127 Pescara
> Tel +39 0859116307 / FAX +39 0859111173
> http://www.tirasa.net
>
> Apache Syncope PMC Member
>
>
Re: Kubernetes (Syncope Cannot talk to Postgres)
Posted by Andrea Patricelli <an...@apache.org>.
Hi Craig,
Il 07/08/2018 08:30, Francesco Chicchiriccò ha scritto:
> On 07/08/2018 00:03, craig wrote:
>> I am setting up syncope in Kubernetes. I will be happy to
>> contribute my
>> yamls once I get it running as it doesn't seem to be a common setup for
>> Syncope.
>
> This sounds great, it would be a great addition, maybe to place right
> after
>
> https://syncope.apache.org/docs/getting-started.html#docker-compose-samples
>
>
>> I am new to Syncope and I am having some issues.
>>
>> I was able to get the docker-compose examples working just fine but
>> having
>> connectivity issues when running in K8s. Without knowing Syncope
>> (or even
>> postgres) that well I am struggling on where to start.
>>
>> Issue: Syncope cannot connect to postgres:5432
>>
>> 20:49:13.640 ERROR
>> org.flowable.common.engine.impl.AbstractEngineConfiguration - Exception
>> while initializing Database connection
>> org.postgresql.util.PSQLException: Connection to postgres:5432 refused.
>> Check that the hostname and port are correct and that the postmaster is
>> accepting TCP/IP connections.
>> [...]
Did you set a password for the user "syncope" like explained at [1]?
Is connectivity on protocol TCP allowed on your postgres instance?
BTW I found something (maybe) useful for you at [2].
>
> Were the Syncope tables (SyncopeUser, for example) created, in the
> database? Were the Flowable (e.g. ACT_*) and Quartz (e.g. QRTZ_*)
> created as well?
>
> Did you setup any connection control on the postgresql container?
>
>> Things that I did:
>>
>> 1) Connect to database directly from postgres container command line
>> "psql
>> -U syncope"
>> 2) Connect to database from syncope container command line "psql -U
>> syncope
>> -h postgres"
>
> Since this works, I cannot figure out why you get the exception above...
>
>> 3) Confirm that the port 5432 is open by running the command "telnet
>> postgres 5432" and it was open
>> 4) Confirm that both a database named "syncope" and user named
>> "syncope"
>> exist in the postgres database
>> 5) Confirmed the configuration of K8s looks correct. The port
>> appears to
>> be up, the replica sets look correct
>>
>> Any help or guidance on things to look at would be helpful.
>>
>> Craig
>
HTH,
Andrea
[1] https://syncope.apache.org/docs/reference-guide.html#postgresql
[2]
https://blog.bigbinary.com/2016/01/23/configure-postgresql-to-allow-remote-connection.html
--
Dott. Andrea Patricelli
Tel. +39 3204524292
Developer @ Tirasa S.r.l.
Viale D'Annunzio 267 - 65127 Pescara
Tel +39 0859116307 / FAX +39 0859111173
http://www.tirasa.net
Apache Syncope PMC Member
Re: Kubernetes (Syncope Cannot talk to Postgres)
Posted by Francesco Chicchiriccò <il...@apache.org>.
On 07/08/2018 00:03, craig wrote:
> I am setting up syncope in Kubernetes. I will be happy to contribute my
> yamls once I get it running as it doesn't seem to be a common setup for
> Syncope.
This sounds great, it would be a great addition, maybe to place right after
https://syncope.apache.org/docs/getting-started.html#docker-compose-samples
> I am new to Syncope and I am having some issues.
>
> I was able to get the docker-compose examples working just fine but having
> connectivity issues when running in K8s. Without knowing Syncope (or even
> postgres) that well I am struggling on where to start.
>
> Issue: Syncope cannot connect to postgres:5432
>
> 20:49:13.640 ERROR
> org.flowable.common.engine.impl.AbstractEngineConfiguration - Exception
> while initializing Database connection
> org.postgresql.util.PSQLException: Connection to postgres:5432 refused.
> Check that the hostname and port are correct and that the postmaster is
> accepting TCP/IP connections.
> [...]
Were the Syncope tables (SyncopeUser, for example) created, in the
database? Were the Flowable (e.g. ACT_*) and Quartz (e.g. QRTZ_*)
created as well?
Did you setup any connection control on the postgresql container?
> Things that I did:
>
> 1) Connect to database directly from postgres container command line "psql
> -U syncope"
> 2) Connect to database from syncope container command line "psql -U syncope
> -h postgres"
Since this works, I cannot figure out why you get the exception above...
> 3) Confirm that the port 5432 is open by running the command "telnet
> postgres 5432" and it was open
> 4) Confirm that both a database named "syncope" and user named "syncope"
> exist in the postgres database
> 5) Confirmed the configuration of K8s looks correct. The port appears to
> be up, the replica sets look correct
>
> Any help or guidance on things to look at would be helpful.
>
> Craig
--
Francesco Chicchiriccò
Tirasa - Open Source Excellence
http://www.tirasa.net/
Member at The Apache Software Foundation
Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail
http://home.apache.org/~ilgrosso/
Re: Kubernetes (Syncope Cannot talk to Postgres)
Posted by Francesco Chicchiriccò <il...@apache.org>.
On 22/08/2018 17:22, Craig Martin wrote:
> Hi Francesco,
>
> Thanks for your help. No problem on committing the Asciidocs. Do
> you have any preference on where you think I should commit my helm
> charts? I was thinking right under the root "docker" folder but you
> know the code base much better than I do.
Hi Craig,
I think that under
https://github.com/apache/syncope/tree/master/docker/src/main/resources
will work; maybe you could also move the various docker-compose-* under
a docker-compose subfolder, and fix the references in obtain.adoc
Regards.
> For reference it is a set of folders:
>
> k8s
> -helm (Has a README)
> --postgres (holds custom values and chart)
> ---templates (holds the K8s YAMLs)
> --syncope (holds custom values and chart)
> ---templates (holds the K8s YAMLs)
>
> On Wed, Aug 22, 2018 at 12:53 AM Francesco Chicchiriccò
> <ilgrosso@apache.org <ma...@apache.org>> wrote:
>
> On 21/08/2018 20:33, Craig Martin wrote:
>> I have syncope now running on Kubernetes and created a simple
>> (two command) install with helm charts. Seems to work pretty
>> well but there is room for growth. I will add liveness probes,
>> readiness checks, and autoscaling for my own needs and would be
>> happy to commit that as well. I also created a simple
>> installation document to assist in the install.
>
> Hi,
> this is very nice to hear!
>
>> I tried to commit it but don't have access to the github repo.
>> Can someone grant me access (username: mornindew)? I was going
>> to create a feature branch off of master and let you all PR it,
>> is that correct?
>
> I think you could add it right after
>
> http://syncope.apache.org/docs/2.1/getting-started.html#docker-compose-samples
>
> e.g. in
>
> https://github.com/apache/syncope/blob/master/src/main/asciidoc/getting-started/obtain.adoc
>
> after line 463.
>
> Then you can check the results by building docs, as explained in
>
> http://syncope.apache.org/building#Building_documentation
>
> Regarding how to contribute, you can simply fork the repository on
> github and submit a PR - the full Git workflow is explained by
>
> https://cwiki.apache.org/confluence/display/SYNCOPE/Git+workflow
>
> Meanwhile, you could also send an ICLA:
>
> http://syncope.apache.org/contributing#How_do_I_become_a_contributor_or_a_committer.3F
>
> At The ASF, committership is "earned" by contribution - we're a
> do-cracy :-)
> You can read more about how The ASF works at
>
> https://www.apache.org/foundation/how-it-works.html
>
>> *Small comment and maybe feature request:*
>>
>> The Syncope Bootstrap is very coupled to the database (obviously)
>> but it doesn't handle very well if it cannot connect to the DB on
>> startup(e.g. retrys or circuit breaking). K8s is actually
>> pretty opinionated that you shouldn't create tight coupling like
>> this and it should auto heal itself. It would be nice if
>> Syncope-core was a bit more resiliant and could poll or retry.
>> This would make installation only one command (as opposed to two)
>> but more importantly could help it recover if there is a
>> connection issue.
>
> Syncope Core requires a DataSource, which is immediately accessed
> during application bootstrap for at least the following reasons:
>
> * create the JPA EntityManager (by OpenJPA; this will also create
> the required tables at first boot)
> * if tables are empty, load the default content from the relevant
> '*Content.xml' (by Syncope own code), indexes and views
> * create the Quartz tables (by Spring, simple )
> * create the Activiti / Flowable tables
> * create and store the Quartz jobs from the stored Tasks
>
> Once up and running, Syncope Core accesses the database through a
> connection pool, whose main scope is to retain valid connections
> for a whole lot of well-known good reasons; being unfamiliar with
> Kubernetes, I am not sure to understand how connection pools can
> work in general with it.
>
>> Thank you all for your stuff, I am happy to be digging deeper
>> into it. Cool stuff!!!
>
> Thanks to your interest and future contribution!
> Regards.
>
>> On Mon, Aug 6, 2018 at 4:03 PM craig <craigmarti@gmail.com
>> <ma...@gmail.com>> wrote:
>>
>> I am setting up syncope in Kubernetes. I will be happy to
>> contribute my
>> yamls once I get it running as it doesn't seem to be a common
>> setup for
>> Syncope. I am new to Syncope and I am having some issues.
>>
>> I was able to get the docker-compose examples working just
>> fine but having
>> connectivity issues when running in K8s. Without knowing
>> Syncope (or even
>> postgres) that well I am struggling on where to start.
>>
>> Issue: Syncope cannot connect to postgres:5432
>>
>> 20:49:13.640 ERROR
>> org.flowable.common.engine.impl.AbstractEngineConfiguration -
>> Exception
>> while initializing Database connection
>> org.postgresql.util.PSQLException: Connection to
>> postgres:5432 refused.
>> Check that the hostname and port are correct and that the
>> postmaster is
>> accepting TCP/IP connections.
>> at
>> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:245)
>> ~[postgresql-42.2.2.jar:42.2.2]
>> at
>> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
>> ~[postgresql-42.2.2.jar:42.2.2]
>> at
>> org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195)
>> ~[postgresql-42.2.2.jar:42.2.2]
>> at org.postgresql.Driver.makeConnection(Driver.java:452)
>> ~[postgresql-42.2.2.jar:42.2.2]
>> at org.postgresql.Driver.connect(Driver.java:254)
>> ~[postgresql-42.2.2.jar:42.2.2]
>> at
>> com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:136)
>> ~[HikariCP-3.2.0.jar:?]
>> at
>> com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:369)
>> ~[HikariCP-3.2.0.jar:?]
>> at
>> com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:198)
>> ~[HikariCP-3.2.0.jar:?]
>> at
>> com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:467)
>> ~[HikariCP-3.2.0.jar:?]
>> at
>> com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:541)
>> ~[HikariCP-3.2.0.jar:?]
>> at
>> com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115)
>> ~[HikariCP-3.2.0.jar:?]
>> at
>> com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
>> ~[HikariCP-3.2.0.jar:?]
>>
>>
>> Things that I did:
>>
>> 1) Connect to database directly from postgres container
>> command line "psql
>> -U syncope"
>> 2) Connect to database from syncope conatiner command line
>> "psql -U syncope
>> -h postgres"
>> 3) Confirm that the port 5432 is open by running the command
>> "telnet
>> postgres 5432" and it was open
>> 4) Confirm that both a database named "syncope" and user
>> named "syncope"
>> exist in the postgres database
>> 5) Confirmed the configuration of K8s looks correct. The
>> port appears to
>> be up, the replica sets look correct
>>
>> Any help or guidance on things to look at would be helpful.
>>
--
Francesco Chicchiriccò
Tirasa - Open Source Excellence
http://www.tirasa.net/
Member at The Apache Software Foundation
Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail
http://home.apache.org/~ilgrosso/
Re: Kubernetes (Syncope Cannot talk to Postgres)
Posted by Craig Martin <cr...@gmail.com>.
Hi Francesco,
Thanks for your help. No problem on committing the Asciidocs. Do you
have any preference on where you think I should commit my helm charts? I
was thinking right under the root "docker" folder but you know the code
base much better than I do. For reference it is a set of folders:
k8s
-helm (Has a README)
--postgres (holds custom values and chart)
---templates (holds the K8s YAMLs)
--syncope (holds custom values and chart)
---templates (holds the K8s YAMLs)
On Wed, Aug 22, 2018 at 12:53 AM Francesco Chicchiriccò <il...@apache.org>
wrote:
> On 21/08/2018 20:33, Craig Martin wrote:
>
> I have syncope now running on Kubernetes and created a simple (two
> command) install with helm charts. Seems to work pretty well but there is
> room for growth. I will add liveness probes, readiness checks, and
> autoscaling for my own needs and would be happy to commit that as well. I
> also created a simple installation document to assist in the install.
>
>
> Hi,
> this is very nice to hear!
>
> I tried to commit it but don't have access to the github repo. Can
> someone grant me access (username: mornindew)? I was going to create a
> feature branch off of master and let you all PR it, is that correct?
>
>
> I think you could add it right after
>
>
> http://syncope.apache.org/docs/2.1/getting-started.html#docker-compose-samples
>
> e.g. in
>
>
> https://github.com/apache/syncope/blob/master/src/main/asciidoc/getting-started/obtain.adoc
>
> after line 463.
>
> Then you can check the results by building docs, as explained in
>
> http://syncope.apache.org/building#Building_documentation
>
> Regarding how to contribute, you can simply fork the repository on github
> and submit a PR - the full Git workflow is explained by
>
> https://cwiki.apache.org/confluence/display/SYNCOPE/Git+workflow
>
> Meanwhile, you could also send an ICLA:
>
>
> http://syncope.apache.org/contributing#How_do_I_become_a_contributor_or_a_committer.3F
>
> At The ASF, committership is "earned" by contribution - we're a do-cracy
> :-)
> You can read more about how The ASF works at
>
> https://www.apache.org/foundation/how-it-works.html
>
> *Small comment and maybe feature request:*
>
> The Syncope Bootstrap is very coupled to the database (obviously) but it
> doesn't handle very well if it cannot connect to the DB on startup(e.g.
> retrys or circuit breaking). K8s is actually pretty opinionated that you
> shouldn't create tight coupling like this and it should auto heal itself.
> It would be nice if Syncope-core was a bit more resiliant and could poll or
> retry. This would make installation only one command (as opposed to two)
> but more importantly could help it recover if there is a connection issue.
>
>
> Syncope Core requires a DataSource, which is immediately accessed during
> application bootstrap for at least the following reasons:
>
> * create the JPA EntityManager (by OpenJPA; this will also create the
> required tables at first boot)
> * if tables are empty, load the default content from the relevant
> '*Content.xml' (by Syncope own code), indexes and views
> * create the Quartz tables (by Spring, simple )
> * create the Activiti / Flowable tables
> * create and store the Quartz jobs from the stored Tasks
>
> Once up and running, Syncope Core accesses the database through a
> connection pool, whose main scope is to retain valid connections for a
> whole lot of well-known good reasons; being unfamiliar with Kubernetes, I
> am not sure to understand how connection pools can work in general with it.
>
> Thank you all for your stuff, I am happy to be digging deeper into it.
> Cool stuff!!!
>
>
> Thanks to your interest and future contribution!
> Regards.
>
> On Mon, Aug 6, 2018 at 4:03 PM craig <cr...@gmail.com> wrote:
>
>> I am setting up syncope in Kubernetes. I will be happy to contribute my
>> yamls once I get it running as it doesn't seem to be a common setup for
>> Syncope. I am new to Syncope and I am having some issues.
>>
>> I was able to get the docker-compose examples working just fine but having
>> connectivity issues when running in K8s. Without knowing Syncope (or
>> even
>> postgres) that well I am struggling on where to start.
>>
>> Issue: Syncope cannot connect to postgres:5432
>>
>> 20:49:13.640 ERROR
>> org.flowable.common.engine.impl.AbstractEngineConfiguration - Exception
>> while initializing Database connection
>> org.postgresql.util.PSQLException: Connection to postgres:5432 refused.
>> Check that the hostname and port are correct and that the postmaster is
>> accepting TCP/IP connections.
>> at
>>
>> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:245)
>> ~[postgresql-42.2.2.jar:42.2.2]
>> at
>>
>> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
>> ~[postgresql-42.2.2.jar:42.2.2]
>> at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195)
>> ~[postgresql-42.2.2.jar:42.2.2]
>> at org.postgresql.Driver.makeConnection(Driver.java:452)
>> ~[postgresql-42.2.2.jar:42.2.2]
>> at org.postgresql.Driver.connect(Driver.java:254)
>> ~[postgresql-42.2.2.jar:42.2.2]
>> at
>>
>> com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:136)
>> ~[HikariCP-3.2.0.jar:?]
>> at
>> com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:369)
>> ~[HikariCP-3.2.0.jar:?]
>> at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:198)
>> ~[HikariCP-3.2.0.jar:?]
>> at
>> com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:467)
>> ~[HikariCP-3.2.0.jar:?]
>> at
>> com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:541)
>> ~[HikariCP-3.2.0.jar:?]
>> at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115)
>> ~[HikariCP-3.2.0.jar:?]
>> at
>>
>> com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
>> ~[HikariCP-3.2.0.jar:?]
>>
>>
>> Things that I did:
>>
>> 1) Connect to database directly from postgres container command line
>> "psql
>> -U syncope"
>> 2) Connect to database from syncope conatiner command line "psql -U
>> syncope
>> -h postgres"
>> 3) Confirm that the port 5432 is open by running the command "telnet
>> postgres 5432" and it was open
>> 4) Confirm that both a database named "syncope" and user named "syncope"
>> exist in the postgres database
>> 5) Confirmed the configuration of K8s looks correct. The port appears
>> to
>> be up, the replica sets look correct
>>
>> Any help or guidance on things to look at would be helpful.
>
> --
> Francesco Chicchiriccò
>
> Tirasa - Open Source Excellencehttp://www.tirasa.net/
>
> Member at The Apache Software Foundation
> Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMailhttp://home.apache.org/~ilgrosso/
>
>
Re: Kubernetes (Syncope Cannot talk to Postgres)
Posted by Francesco Chicchiriccò <il...@apache.org>.
On 21/08/2018 20:33, Craig Martin wrote:
> I have syncope now running on Kubernetes and created a simple (two
> command) install with helm charts. Seems to work pretty well but
> there is room for growth. I will add liveness probes, readiness
> checks, and autoscaling for my own needs and would be happy to commit
> that as well. I also created a simple installation document to assist
> in the install.
Hi,
this is very nice to hear!
> I tried to commit it but don't have access to the github repo. Can
> someone grant me access (username: mornindew)? I was going to create
> a feature branch off of master and let you all PR it, is that correct?
I think you could add it right after
http://syncope.apache.org/docs/2.1/getting-started.html#docker-compose-samples
e.g. in
https://github.com/apache/syncope/blob/master/src/main/asciidoc/getting-started/obtain.adoc
after line 463.
Then you can check the results by building docs, as explained in
http://syncope.apache.org/building#Building_documentation
Regarding how to contribute, you can simply fork the repository on
github and submit a PR - the full Git workflow is explained by
https://cwiki.apache.org/confluence/display/SYNCOPE/Git+workflow
Meanwhile, you could also send an ICLA:
http://syncope.apache.org/contributing#How_do_I_become_a_contributor_or_a_committer.3F
At The ASF, committership is "earned" by contribution - we're a do-cracy :-)
You can read more about how The ASF works at
https://www.apache.org/foundation/how-it-works.html
> *Small comment and maybe feature request:*
>
> The Syncope Bootstrap is very coupled to the database (obviously) but
> it doesn't handle very well if it cannot connect to the DB on
> startup(e.g. retrys or circuit breaking). K8s is actually pretty
> opinionated that you shouldn't create tight coupling like this and it
> should auto heal itself. It would be nice if Syncope-core was a bit
> more resiliant and could poll or retry. This would make installation
> only one command (as opposed to two) but more importantly could help
> it recover if there is a connection issue.
Syncope Core requires a DataSource, which is immediately accessed during
application bootstrap for at least the following reasons:
* create the JPA EntityManager (by OpenJPA; this will also create the
required tables at first boot)
* if tables are empty, load the default content from the relevant
'*Content.xml' (by Syncope own code), indexes and views
* create the Quartz tables (by Spring, simple )
* create the Activiti / Flowable tables
* create and store the Quartz jobs from the stored Tasks
Once up and running, Syncope Core accesses the database through a
connection pool, whose main scope is to retain valid connections for a
whole lot of well-known good reasons; being unfamiliar with Kubernetes,
I am not sure to understand how connection pools can work in general
with it.
> Thank you all for your stuff, I am happy to be digging deeper into
> it. Cool stuff!!!
Thanks to your interest and future contribution!
Regards.
> On Mon, Aug 6, 2018 at 4:03 PM craig <craigmarti@gmail.com
> <ma...@gmail.com>> wrote:
>
> I am setting up syncope in Kubernetes. I will be happy to
> contribute my
> yamls once I get it running as it doesn't seem to be a common
> setup for
> Syncope. I am new to Syncope and I am having some issues.
>
> I was able to get the docker-compose examples working just fine
> but having
> connectivity issues when running in K8s. Without knowing Syncope
> (or even
> postgres) that well I am struggling on where to start.
>
> Issue: Syncope cannot connect to postgres:5432
>
> 20:49:13.640 ERROR
> org.flowable.common.engine.impl.AbstractEngineConfiguration -
> Exception
> while initializing Database connection
> org.postgresql.util.PSQLException: Connection to postgres:5432
> refused.
> Check that the hostname and port are correct and that the
> postmaster is
> accepting TCP/IP connections.
> at
> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:245)
> ~[postgresql-42.2.2.jar:42.2.2]
> at
> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
> ~[postgresql-42.2.2.jar:42.2.2]
> at
> org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195)
> ~[postgresql-42.2.2.jar:42.2.2]
> at org.postgresql.Driver.makeConnection(Driver.java:452)
> ~[postgresql-42.2.2.jar:42.2.2]
> at org.postgresql.Driver.connect(Driver.java:254)
> ~[postgresql-42.2.2.jar:42.2.2]
> at
> com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:136)
> ~[HikariCP-3.2.0.jar:?]
> at
> com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:369)
> ~[HikariCP-3.2.0.jar:?]
> at
> com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:198)
> ~[HikariCP-3.2.0.jar:?]
> at
> com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:467)
> ~[HikariCP-3.2.0.jar:?]
> at
> com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:541)
> ~[HikariCP-3.2.0.jar:?]
> at
> com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115)
> ~[HikariCP-3.2.0.jar:?]
> at
> com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
> ~[HikariCP-3.2.0.jar:?]
>
>
> Things that I did:
>
> 1) Connect to database directly from postgres container command
> line "psql
> -U syncope"
> 2) Connect to database from syncope conatiner command line "psql
> -U syncope
> -h postgres"
> 3) Confirm that the port 5432 is open by running the command "telnet
> postgres 5432" and it was open
> 4) Confirm that both a database named "syncope" and user named
> "syncope"
> exist in the postgres database
> 5) Confirmed the configuration of K8s looks correct. The port
> appears to
> be up, the replica sets look correct
>
> Any help or guidance on things to look at would be helpful.
>
--
Francesco Chicchiriccò
Tirasa - Open Source Excellence
http://www.tirasa.net/
Member at The Apache Software Foundation
Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail
http://home.apache.org/~ilgrosso/
Re: Kubernetes (Syncope Cannot talk to Postgres)
Posted by Craig Martin <cr...@gmail.com>.
I have syncope now running on Kubernetes and created a simple (two command)
install with helm charts. Seems to work pretty well but there is room for
growth. I will add liveness probes, readiness checks, and autoscaling for
my own needs and would be happy to commit that as well. I also created a
simple installation document to assist in the install.
I tried to commit it but don't have access to the github repo. Can
someone grant me access (username: mornindew)? I was going to create a
feature branch off of master and let you all PR it, is that correct?
*Small comment and maybe feature request:*
The Syncope Bootstrap is very coupled to the database (obviously) but it
doesn't handle very well if it cannot connect to the DB on startup(e.g.
retrys or circuit breaking). K8s is actually pretty opinionated that you
shouldn't create tight coupling like this and it should auto heal itself.
It would be nice if Syncope-core was a bit more resiliant and could poll or
retry. This would make installation only one command (as opposed to two)
but more importantly could help it recover if there is a connection issue.
Thank you all for your stuff, I am happy to be digging deeper into it.
Cool stuff!!!
Craig
On Mon, Aug 6, 2018 at 4:03 PM craig <cr...@gmail.com> wrote:
> I am setting up syncope in Kubernetes. I will be happy to contribute my
> yamls once I get it running as it doesn't seem to be a common setup for
> Syncope. I am new to Syncope and I am having some issues.
>
> I was able to get the docker-compose examples working just fine but having
> connectivity issues when running in K8s. Without knowing Syncope (or even
> postgres) that well I am struggling on where to start.
>
> Issue: Syncope cannot connect to postgres:5432
>
> 20:49:13.640 ERROR
> org.flowable.common.engine.impl.AbstractEngineConfiguration - Exception
> while initializing Database connection
> org.postgresql.util.PSQLException: Connection to postgres:5432 refused.
> Check that the hostname and port are correct and that the postmaster is
> accepting TCP/IP connections.
> at
>
> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:245)
> ~[postgresql-42.2.2.jar:42.2.2]
> at
>
> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
> ~[postgresql-42.2.2.jar:42.2.2]
> at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195)
> ~[postgresql-42.2.2.jar:42.2.2]
> at org.postgresql.Driver.makeConnection(Driver.java:452)
> ~[postgresql-42.2.2.jar:42.2.2]
> at org.postgresql.Driver.connect(Driver.java:254)
> ~[postgresql-42.2.2.jar:42.2.2]
> at
>
> com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:136)
> ~[HikariCP-3.2.0.jar:?]
> at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:369)
> ~[HikariCP-3.2.0.jar:?]
> at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:198)
> ~[HikariCP-3.2.0.jar:?]
> at
> com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:467)
> ~[HikariCP-3.2.0.jar:?]
> at
> com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:541)
> ~[HikariCP-3.2.0.jar:?]
> at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115)
> ~[HikariCP-3.2.0.jar:?]
> at
> com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112)
> ~[HikariCP-3.2.0.jar:?]
>
>
> Things that I did:
>
> 1) Connect to database directly from postgres container command line "psql
> -U syncope"
> 2) Connect to database from syncope conatiner command line "psql -U
> syncope
> -h postgres"
> 3) Confirm that the port 5432 is open by running the command "telnet
> postgres 5432" and it was open
> 4) Confirm that both a database named "syncope" and user named "syncope"
> exist in the postgres database
> 5) Confirmed the configuration of K8s looks correct. The port appears to
> be up, the replica sets look correct
>
> Any help or guidance on things to look at would be helpful.
>
> Craig
>
>
> --
> Sent from: http://syncope-user.1051894.n5.nabble.com/
>