Flink with Hibernate ORM doesn't release memory after job is canceled

[Sendoh <un...@gmail.com>]

Hi Flink users,

Does anyone use Flink with Hibernate ORM and have some examples?

We have a job which reads a list of topics from Postgres accessed by
Hibernate ORM as parameter for Kafka consumer. The Mapper also reads
information from Postgres with Hibernate ORM, meaning, both master and
worker access Postgres.

I found out that this Flink job doesn't release memory when the Flink job is
canceled, both master and worker. 
(There is another issue that in this Flink job use lots of memory even with
all possible performance tuning of Hibernate ORM, e.g.
c3p0 connection pool, ehcache size limit, batch size and so on). 

The data accessed by this Flink job is quite small. I would estimate it to
be no more than 50 MB, and each node has 2 GB RAM. Is there any suggest I
could work on?

Best,

Sendoh



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-with-Hibernate-ORM-doesn-t-release-memory-after-job-is-canceled-tp13033.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Re: Flink with Hibernate ORM doesn't release memory after job is canceled

[Aljoscha Krettek <al...@apache.org>]

Hi Sendoh,

Could you please go into some details about your job. Maybe show some representative code snippets or how the ORM is used exactly.

Best,
Aljoscha
> On 5. May 2017, at 18:10, Sendoh <un...@gmail.com> wrote:
> 
> Hi Flink users,
> 
> Does anyone use Flink with Hibernate ORM and have some examples?
> 
> We have a job which reads a list of topics from Postgres accessed by
> Hibernate ORM as parameter for Kafka consumer. The Mapper also reads
> information from Postgres with Hibernate ORM, meaning, both master and
> worker access Postgres.
> 
> I found out that this Flink job doesn't release memory when the Flink job is
> canceled, both master and worker. 
> (There is another issue that in this Flink job use lots of memory even with
> all possible performance tuning of Hibernate ORM, e.g.
> c3p0 connection pool, ehcache size limit, batch size and so on). 
> 
> The data accessed by this Flink job is quite small. I would estimate it to
> be no more than 50 MB, and each node has 2 GB RAM. Is there any suggest I
> could work on?
> 
> Best,
> 
> Sendoh
> 
> 
> 
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-with-Hibernate-ORM-doesn-t-release-memory-after-job-is-canceled-tp13033.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.