You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by nragon <nu...@wedotechnologies.com> on 2017/04/03 16:55:30 UTC

Re: In-Memory data grid

Just to add a scenario.

My current arquitecture is the following:
I've deployed 4 ignite node in yarn and 5 task managers with 2G and 2 slots
each.
As cache on ignite I have on record in key/value (string, object[])
My thoughput without ignite is 30k/sec when I add the lookup i get 3k/sec

My question is, has anyone used ignite as fast lookup or any other in memery
data grid? If so, did you have any major performance impact?

Thanks



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/In-Memory-data-grid-tp12494p12510.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Re: In-Memory data grid

Posted by nragon <nu...@wedotechnologies.com>.

My concern and my requirements are that the cache must be shared with other
jobs or any other application.
If a user or job changes the value in ignite, it must be updated in every
flink job it uses that cache.

Thanks



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/In-Memory-data-grid-tp12494p12514.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Re: In-Memory data grid

Posted by Steve Jerman <st...@kloudspot.com>.

I had a DB look up to do and  used com.google.common.cache.Cache with a 10 sec timeout:

private static Cache<String, String> locationCache = CacheBuilder.newBuilder().maximumSize(1000).expireAfterAccess(10, TimeUnit.SECONDS).build();

Seemed to help a lot with throughput….

Steve

On Apr 3, 2017, at 10:55 AM, nragon <nu...@wedotechnologies.com>> wrote:

Just to add a scenario.

My current arquitecture is the following:
I've deployed 4 ignite node in yarn and 5 task managers with 2G and 2 slots
each.
As cache on ignite I have on record in key/value (string, object[])
My thoughput without ignite is 30k/sec when I add the lookup i get 3k/sec

My question is, has anyone used ignite as fast lookup or any other in memery
data grid? If so, did you have any major performance impact?

Thanks



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/In-Memory-data-grid-tp12494p12510.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com<http://Nabble.com>.