You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Denis Magda <dm...@gridgain.com> on 2015/12/09 18:15:00 UTC
Re: Need to effect cache at time of changes made in database
Hi Kamesh,
There is no way to trigger Ignite cache when an update happens at a database
layer.
You should either
- periodically use some predicate in conjunction with IgniteCache.loadAll in
order to load data from DB; or
- make the other app to do updates through the cache.
Does any of suggestions above work for you?
Regards,
Denis
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Need-to-effect-cache-at-time-of-changes-made-in-database-tp2182p2188.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Need to effect cache at time of changes made in database
Posted by vkulichenko <va...@gmail.com>.
Kamesh,
Unfortunately, there is no dedicated example for this store (probably worth
adding). But you can take a look at GridCacheLoadOnlyStoreAdapterSelfTest
class [1] which is a unit test for it.
[1]
https://github.com/gridgain/gridgain/blob/master/modules/core/src/test/java/org/gridgain/grid/cache/store/GridCacheLoadOnlyStoreAdapterSelfTest.java
-Val
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Need-to-effect-cache-at-time-of-changes-made-in-database-tp2182p2229.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Need to effect cache at time of changes made in database
Posted by kamesh <bk...@gmail.com>.
Dear vkulichenko,
Thank you for replying.
1) can you give me any example link for CacheLoadOnlyStoreAdapter. I didn't
find any example for CacheLoadOnlyStoreAdapter in downloaded Ignite zip
file.
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Need-to-effect-cache-at-time-of-changes-made-in-database-tp2182p2218.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Need to effect cache at time of changes made in database
Posted by vkulichenko <va...@gmail.com>.
Hi Kamesh,
Here are my answers.
1. You should try loading the data in multithreaded fashion. The easiest way
to do this is to extend CacheLoadOnlyStoreAdapter instead of
CacheStoreAdapter and implement inputIterator and parse methods instead of
loadCache method. This adapter will parallelize loading process
automatically. Also if you have several nodes, you can speed up the process
with partition-aware approach which is described in [1].
2. Can you please clarify what you mean here? How objects are linked to each
other? What behavior do you expect when you change one of them?
3. With Ignite you can execute any code on server nodes using Compute Grid
[2]. Your tasks and/or closures can access data, so you will be able to do
any required processing. Will this work for you?
4. No. Core functionality is the same in all editions, including third-party
builds provided by GridGain.
[1]
https://apacheignite.readme.io/docs/data-loading#section-partition-aware-data-loading
[2] https://apacheignite.readme.io/docs/compute-grid
-Val
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Need-to-effect-cache-at-time-of-changes-made-in-database-tp2182p2207.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Need to effect cache at time of changes made in database
Posted by kamesh <bk...@gmail.com>.
Hi Denis,
Thank you for replying.
I have following quires,
1) I have a performance issue. I uses above code for loading 4,00,00,000 of
records into cache it takes nearly 6 hrs. of time. can you give me solution
for increase performance.
2) I want in-memory computation means, I have a tables (means cache) which
are interlinked to each other. while changing values in one table(cache) it
will effect on remaining ones. Can you help me for this scenario.
3) Is ignite supporting stored procedures? If it is, it will help more to
me.
4)Right now I am using community edition is it impact on performance?
can you please help me.
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Need-to-effect-cache-at-time-of-changes-made-in-database-tp2182p2202.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Need to effect cache at time of changes made in database
Posted by Denis Magda <dm...@gridgain.com>.
Hi Kamesh,
I general you solution should work fine. I've not noted any issue in your
code snippet.
Optionally, you may want try to reuse CacheLoadOnlyStoreAdapter. It should
work faster than your implementation.
There are no any special examples for CacheLoadOnlyStoreAdapter. However,
you can refer to the following tests located in Ignite sources to see how
it's used in practice
-org.apache.ignite.cache.store.GridCacheLoadOnlyStoreAdapterSelfTest.
In addition there was a discussion around this cache adapter on this user
list:
http://apache-ignite-users.70518.x6.nabble.com/Data-Loading-Performance-Issue-td1958.html#a2012
--
Denis
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Need-to-effect-cache-at-time-of-changes-made-in-database-tp2182p2197.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Need to effect cache at time of changes made in database
Posted by kamesh <bk...@gmail.com>.
Thanks for giving reply.
I need suggestions from you. I am adding how I am caching my table into
IgniteCache. Let me know is it correct way or not. If not give me right
path.
User.java (Pojo class)
---------
public class User {
@QuerySqlField(index = true)
private Long userId;
@QuerySqlField
private String name;
@QuerySqlField
private String windowsLogin;
@QuerySqlField
private Long readAccess;
@QuerySqlField
private Long writeAccess;
@QuerySqlField
private Long taxonomy;
@QuerySqlField
private Long versionLock;
@QuerySqlField
private Long admin;
@QuerySqlField
private Long numbers;
@QuerySqlField
private Long superUser;
@QuerySqlField
private Long importAccess;
@QuerySqlField
private Long exportAccess;
@QuerySqlField
private Date lastModifiedOn;
@QuerySqlField
private Long lastModifiedBy;
//accesser methods
}
Cache Store Class
------------------
public class UserCacheStore extends CacheStoreAdapter<Long,User> {
@CacheStoreSessionResource
private CacheStoreSession ses;
@Override
public User load(Long key) throws CacheLoaderException {
// TODO Auto-generated method stub
return null;
}
@Override
public void delete(Object arg0) throws CacheWriterException {
// TODO Auto-generated method stub
}
@Override
public void write(Entry<? extends Long, ? extends User> userEntry)
throws CacheWriterException {
// TODO Auto-generated method stub
Long key=userEntry.getKey();
User user=userEntry.getValue();
//Connection con=ses.attachment();
/*try(PreparedStatement ps=con.prepareStatement("")){
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}*/
}
/** {@inheritDoc} */
@Override public void loadCache(IgniteBiInClosure<Long, User> clo,
Object... args) {
if (args == null || args.length == 0 || args[0] == null)
throw new CacheLoaderException("Expected entry count parameter
is not provided.");
final int entryCnt = (Integer)args[0];
System.out.println("comming..........");
Connection conn = ses.attachment();
try (PreparedStatement stmt = conn.prepareStatement("select * from
USERS")) {
// stmt.setInt(1, entryCnt);
ResultSet rs = stmt.executeQuery();
int cnt = 0;
while (rs.next()) {
User user = new User(rs.getLong("USERID"),
rs.getString("NAME"),
rs.getString("WINDOWS_LOGIN"),
rs.getLong("READACCESS"),
rs.getLong("WRITEACCESS"),
rs.getLong("TAXONOMY"),
rs.getLong("VERSIONLOCK"),
rs.getLong("ADMIN"),
rs.getLong("NUMBERS"),
rs.getLong("SUPERUSER"),
rs.getLong("IMPORTACCESS"),
rs.getLong("EXPORTACCESS"),
rs.getDate("LASTMODIFIEDON"),
rs.getLong("LASTMODIFIEDBY"));
clo.apply(user.getUserId(), user);
cnt++;
}
System.out.println(">>> Loaded " + cnt + " values into cache.");
}
catch (SQLException e) {
throw new CacheLoaderException("Failed to load values from cache
store.", e);
}
}
}
Example Class
--------------
public class UserCacheJdbcExmaple {
private static final String CACHE_NAME =
UserCacheJdbcExmaple.class.getSimpleName();
/** Heap size required to run this example. */
public static final int MIN_MEMORY = 1024 * 1024 * 1024;
public static void main(String[] args) {
// TODO Auto-generated method stub
try (Ignite ignite =
Ignition.start("com/wallet/grigain/cache/store/example-ignite.xml")) {
System.out.println();
System.out.println(">>> User Cache store example started.");
CacheConfiguration<Long, User> ccfg=new CacheConfiguration<Long,
User>("userCache");
ccfg.setAtomicityMode(TRANSACTIONAL);
// Configure JDBC store.
ccfg.setCacheStoreFactory(FactoryBuilder.factoryOf(UserCacheStore.class));
// Configure JDBC session listener.
ccfg.setCacheStoreSessionListenerFactories(new
Factory<CacheStoreSessionListener>() {
@Override public CacheStoreSessionListener create() {
CacheJdbcStoreSessionListener lsnr = new
CacheJdbcStoreSessionListener();
lsnr.setDataSource(ConnectionUtil.DATA_SRC);
return lsnr;
}
});
ccfg.setIndexedTypes(Long.class,User.class);
//ccfg.setTypeMetadata(typeMetadata());
ccfg.setReadThrough(true);
ccfg.setWriteThrough(true);
IgniteCache<Long, User> cache=ignite.getOrCreateCache(ccfg);
cache.loadCache(null, 99_99_99_999);
}
}
}
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Need-to-effect-cache-at-time-of-changes-made-in-database-tp2182p2191.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.